Transcript Integrating Assessment into the Curriculum

Integrating Assessment
to the Curriculum
Marta Colón de Toro, SPHR
Assessment Coordinator
College of Business Administration
UPR-Mayagüez
February 21, 2006 – UPR-Cayey
5 Conversations:
1. Teaching and Learning
2. Assessment and Curriculum
3. AACSB’s expectations for assurance of
learning
4. Planning & Implementing Assessment
5. Managing Results
2
Learning Objectives
After completing today’s workshop you should be
able to:
1. Converse about the transformation power of teaching
and learning.
2. Identify assessment integration strategies.
3. Discuss AACSB’s assurance of learning standards.
4. Draft a plan to assess learning across the curriculum.
5. Design and apply assessment interventions in your
programs and courses.
6. Explain how to use assessment results to improve
programs and courses.
3
Stages of Grieving





Denial
Anger
Bargaining
Depression
Acceptance
On Death and Dying
by Elisabeth Kubler-Ross
4
Objective #1
The Transformation Process
Input
Incoming
Student
• Demographics
•Academics
• Experience
•Talents
Teaching-Learning
Curriculum
Seminars and Competitions
Internship/Coop Education
Student Associations
Extracurricular Experiences
Output
Completing
Student
• Knowledge
•Skills
•Abilities
•Experience
•Attitudes /Values
5
Objective #1
Making Sure Students Learn
Market Needs
Vision/Mission
Design Curriculum
and Set Goals
Improve
Continuously
Enable Learning
Assure
Learning
6
Objective #2
Purpose of Assessment
Assessment
Intended Outcomes
Actual Outcomes
1. Act ethically
1.
2. Quantitative Analysis
2.
3. Apply models
3.
4. Decision making
4.
Reveals the
Gap
7
Objective #2
Assess Learning to:



Provide feedback and guidance to
individual students.
Assist the school and faculty members to
improve courses and programs.
Assure external constituents that the
institution meets its goals.
8
Objective #2
Levels of Assessment
Institutional Learning Goals
Program Learning Goals
Course Goals
Lesson
Objectives
9
Objective #2
When to Assess

Entry
• Profile incoming students
• Demographics
• Academic achievement
• Previous Business KSAOs

Midpoint

Exit
• Measure progress/added value
• Standardized test
• Alumni performance
10
Effective curricula is…
Objective #2
…pertinent
…coherent
…strategic
…dynamic
…assessable
…intentionally designed!
11
UPR-Cayey’s Curriculum
Objective #2
BACHELOR IN GENERAL BUSINESS ADMINISTRATION
General Education
63 crds.
Core courses (Arts)
39
Core courses (Sciences)
12
Departmental Requirements
12
Concentration Requirements
43
Professional Electives
15
Free Electives
12
Total 133 crds.
12
Objective #3
AACSB Standards




AACSB only requires assessment at the
program level.
Assessments using samples are allowed.
Assessment must use direct measures of
achievement
Indirect measures may be used to supplement
direct measures of achievement.
13
AACSB: Learning Expectations Objective #3
at the Bachelor’s Level




Educate students in a broad range of knowledge and
skills as a basis for careers in business.
Build on the students’ pre-collegiate educations to
prepare them to enter and sustain careers in the
business world and contribute positively in the larger
society.
Provide knowledge and skills for successful performance
in a complex environment requiring intellectual ability to
organize work, make and communicate sound decisions,
and react successfully to unanticipated events.
Develop learning abilities suitable to continue higherlevel intellectual development.
14
Objective #3
Standard 15


The school uses well documented, systematic processes
to develop, monitor, evaluate, and revise the substance
and delivery of the curricula of degree programs and to
assess the impact of the curricula on learning.
Curriculum management includes inputs from all
appropriate constituencies which may include faculty,
staff, administrators, students, faculty from non-business
disciplines, alumni, and the business community served
by the school.
15
Objective #3
Standard 16

Adapting expectations to the school’s
mission and cultural circumstances, the
school specifies learning goals and
demonstrates achievement of learning
goals for key general, managementspecific, and/or appropriate disciplinespecific knowledge and skills that its
students achieve in each undergraduate
degree program.
16
Objective #3
Standard 17

The bachelor’s or undergraduate level
degree programs must provide sufficient
time, content coverage, student effort, and
student-faculty interaction to assure that
the learning goals are accomplished.
17
Intent of Assurance of
AACSB’s Learning Standards



Objective #3
Evaluate how well the school
accomplishes the educational claims at
the core of its activities.
Do students achieve learning
appropriate to the programs in which
they participate?
Do they have the knowledge and skills
appropriate to their earned degrees?
18
Objective #3
Learning Goals
Specific
Measurable
Achievable
Realistic
Timely
19
Objective #3
Examples of Measures of Achievement
Example 1
 School A has defined a learning goal in ethical reasoning for each of
its four undergraduate majors. Student achievement on this goal is
relevant to demonstrating satisfaction of Standard 16. The school’s
faculty has defined the goal.
Learning Goal
 “Each student can recognize and analyze ethical problems and
choose and defend resolutions for practical situations that occur in
accounting, human resource management, and marketing.”
Demonstration of Achievement
 The school uses course-embedded exercises in three required
introductory-level courses. Faculty in the three disciplines have
developed different methods for instructing and assessing
achievement toward this learning goal.
20
Objective #3
Examples of Measures of Achievement
In Accounting



A two-week module near the end of the introductory
course is devoted to “Ethical standards and fraud in
accounting.”
A topic outline has been developed by faculty
members to structure an exam on the materials of
this module, and a standard set of expectations has
been created for grading the exam.
In addition to this exam’s contribution to the course
grade, it provides a pass/fail indication on the
learning goal.
21
Objective #3
Examples of Measures of Achievement
In Human Resource Management


Students must provide four written analyses of
problem situations during the course. On three of
these analyses (on the topics of selection, reward
systems, and job design), students are asked to
respond to ethical issues.
A standard scoring key on the ethical component
provides evaluation toward the course grade and a
pass/fail indication on the learning goal.
22
Objective #3
Examples of Measures of Achievement
In Marketing



Each student must compose a term paper analyzing
a current national or international marketing
campaign.
The analysis must include a specified set of
components, and ethical issues that have been
presented in lectures are among the required
components.
In addition to the overall grade of the paper, each
student receives a pass/fail indicator on the ethics
component.
23
Objective #3
Examples of Measures of Achievement
In addition to reporting course grades



Each instructor of these three courses
provides a checklist of all of those students
who successfully completed the ethics
expectation.
This information is a part of each student’s
record and all three parts of the learning goal
must be achieved before graduation.
Students who fail the ethics evaluation while
passing the course repeat the evaluation
exercise or ethics module until they are
successful.
24
Objective #3
Examples of Measures of Achievement
Example 2: Communication skills
 School B has a communications learning goal that is a part of its expectations for
all undergraduate degrees. Student achievement on this goal is relevant to
demonstrating satisfaction of Standard 16. The school’s faculty has defined the
goal.
Learning Goal
 “Each student can conceptualize a complex issue into a coherent written
statement and oral presentation.”
Demonstration of Achievement
 The Strategic Management course required of each student in the final year of the
program includes among its course evaluations a written analysis of a multifunctional case study and an oral presentation on an industry-wide analysis.
 A faculty task force has developed a standardized scoring key for use with these
two exercises. Using dimensions agreed to by the faculty, each student’s
performance on these exercises is evaluated.
 Students must repeat the exercises until they have satisfactorily accomplished
minimum levels of performance.
25
Objective #4
Integrating Assessment: The Plan
Study Vision and Mission Statements
2. State rationale and purpose of the plan
3. Identify your Audiences
4. Define Outcomes-based Learning Goals
5. Determine enabling activities for each goal
6. Select assessment instruments and measures
7. Assess and collect data
8. Analyze, interpret and report results
9. Introduce change for improvement
10. Assess the plan
1.
26
Objective #5
Program Assessment Instruments

Direct
•
•
•
•
•
•

Diagnostic test to incoming
students on learning goals
(entry/midpoint/exit)
Course-embedded
assessment
•
•
•
•
Capstone courses
Projects
Seminar
Writing assignments
Indirect
•
•
•
•
•
Employer Reports on
Alumni
Exit Surveys
Focus groups
Interviews
Student evaluation of
courses
Internships/Coop
Proficiency exams
Professional exams
Student Portfolio
27
Objective #5
Classroom Assessment Techniques







Learner-Centered
Teacher-Directed
Mutually Beneficial
Formative
Context-Specific
Ongoing
Rooted in Good Teaching Practice
28
Objective #5
Classroom Assessment Techniques
Assumptions:
 The quality of student learning is directly, although not exclusively, related to the
quality of teaching.
 To improve their effectiveness, teachers need first to make their goals and
objectives explicit and then to get specific, comprehensible feedback on the
extent to which they are achieving those goals and objectives.
 To improve their learning, students need to receive appropriate and focused
feedback early and often; they also need to learn how to assess their own
learning.
 The type of assessment most likely to improve teaching and learning is that
conducted by faculty to answer questions they themselves have formulated in
response to issues or problems in their own teaching.
 Systematic inquiry and intellectual challenge are powerful sources of motivation,
growth, and renewal for college teachers, and Classroom Assessment can
provide such challenge.
 Classroom Assessment does not require specialized training; it can be carried out
by dedicated teachers from all disciplines.
 By collaborating with colleagues and actively involving students in Classroom
Assessment efforts, faculty (and students) enhance learning and personal
satisfaction.
29
Objective #5
Classroom Assessment Techniques
Starting small involves three steps:
Step 1: Planning –
•Select one, and only one, of your classes in which to try out the
Classroom Assessment. Decide on the class meeting and select a
Classroom Assessment Technique. Choose a simple and quick one.
Step
2: Implementing
•Make sure the students know what you are doing and that they clearly
understand the procedure. Collect the responses and analyze them as
soon as possible.
Step
3: Responding
•To capitalize on time spent assessing, and to motivate students to
become actively involved, "close the feedback loop" by letting them
know what you learned from the assessments and what difference that
information will make.
30
Objective #5
Classroom Assessment Techniques
Five suggestions for a successful start:
 If a Classroom Assessment Techniques does not appeal
to your intuition and professional judgment as a teacher,
don't use it.
 Don't make Classroom Assessment into a self-inflicted
chore or burden.
 Don't ask your students to use any Classroom
Assessment Technique you haven't previously tried on
yourself.
 Allow for more time than you think you will need to carry
out and respond to the assessment.
 Make sure to "close the loop." Let students know what
you learn from their feedback and how you and they can
use that information to improve learning.
31
Objective #5
Classroom Assessment Techniques












1 Minute paper
Muddiest point
One-sentence summary
Direct paraphrasing
Application cards
Concept maps
Empty outlines
Pro-con grids
Class journals
Portfolios
What’s the principle
Rubrics
32
Objective #6
Using assessment results to improve




Collect data
Interpret it
Report it
Act on it
33
Objective #6
Examples of Improvements




Course pre-requisites were reviewed after two
years of teaching it.
A specific rubric was developed to guide
students in completing an assignment on
questionnaire development for research.
In accounting, more class time is being
dedicated to bank account reconciliation topic
after assessing student performance.
Curricular revision is being based on
assessment results.
34
In Closing








Create awareness
Educate and train
KISS
Procure support
Recognize the stages of grieving
Do it for you!
Persist
Use results!!!
35
[email protected]
36