Transcript TRAINING MODULE OF OUTCOME BASED EDUCATION

TRAINING ON OUTCOME-BASED EDUCATION
Year 2011
INTRODUCTION TO
OUTCOME-BASED ASSESSMENT
Year 2011
LEARNING OUTCOME
At the end of this session, participants should be
able to:
• describe the basic concept of outcome based
assessment and its application (C3,A4)
• explain assessment terms used in outcome
based education. (C3,A3)
• describe various methods of assessment in
accordance with the course and learning
outcome.(C3, A3)
• design rubrics for assessing learning outcomes
(C3,P2)
3
ASSESSMENT IS…
a way of finding out what a person knows,
understands and thinks, and what he or she can do.
the systematic collection and analysis of information
to monitor and improve student learning.
is not a one-time event, but a dynamic on-going
process.
4
OUTCOME BASED ASSESSMENT
Focuses on providing students with multiple
opportunities to practice what they need to do, and
sets the provision of feedback on that practice
(formative). Eventually, students must demonstrate
achievement (summative).
Currently, most assessment tasks encourage the
reproduction of content, whereas we should be
looking for the production of new knowledge (analyze,
evaluate and create parts of Bloom’s top level cognitive
processes).
5
OUTCOME
BASED
EDUCATION
OUTCOME BASED
ASSESSMENT
6
PURPOSE OF ASSESSMENT
Assessment For Learning - constructive feedback
to be given to students for use in their future
learning.
Assessment Of Learning - assessing students’
achievements for grading and certification
Assessment As Learning - students are capable
of becoming independent in their learning
7
PRINCIPLES OF ASSESSMENT
VALIDITY
FLEXIBILITY
PRINCIPLES
OF
ASSESSMENT
REALIBILITY
FAIRNESS
8
ASSESSMENT LEVELS
Program AIMS (PAI)
Programme Learning Outcomes (PLO)
Course Learning Outcomes (CLO)
Topic Learning Outcomes
Few years after
Graduation – 3 to 5
years
Upon
graduation
Upon subject
completion
Upon
topic completion
9
REFLECTION 1
• What do you do when your class performs
poorly in quiz/test/practical work, etc?







Adjust the scores upward
Grade on a curve
Adjust the difficulty level of the next assessment
Give advice on study strategies
Provide more practice problems, review, etc
Spend more time on the materials in the future
Change your teaching materials or approach
10
Assessment Task
Assessment task refers
to summative and
formative assessment or criterion referred
assessment. Examples are case study, critique,
debate, demonstration, drawing, sketch, essay,
exhibition, interview, journal, laboratory/practical,
literature
review,
logbook,
model,
oral
examination, peer review, presentation, portfolio,
practicum, problem solving, projects, reflection,
reports, self assessment, research paper, thesis
and workshop
11
ASSESSMENT TERMS
Assessment Criteria
• A list of requirements that describes how well a student is able to achieve
the learning outcome in order to be awarded a particular grade.
Read more: http://www.answers.com/topic/assessmentcriteria#ixzz1DjoBm8oF
Continuous Assessment
Continuous Assessment is an assessment that takes place over a period of
time. In other words you will be assessed right through your learning
process and not only after the learning process. By having continuous
assessment you can track the improvement (if any) of the learner, you will
be able to give more support and guidance and the learner will have more
opportunities to improve.
http://wiki.answers.com/Q/What_is_continuous_Assessment#ixzz1DjnPB
HWy
12
ASSESSMENT TERMS
Evaluation
A process that enables us to measure, compare, and
judge the quality of students’ work, schools, or
specific educational programs.
Measurement
The process by which attributes or dimensions of
some physical objects, process or opinions are
determined and carried out using questionnaires and
standardized test etc.
13
DOMAINS OF EDUCATIONAL GOALS
Cognitive
The Head
Affective
The Heart
Psychomotor
The Hand
Adapted from DrJJ. UiTM.
3H
14
BLOOM’S TAXONOMY
Learning outcomes
Affective
Psychomotor / skills
Evaluation (C6)
Internalizing (A5)
Synthesis (C5)
Analysis (C4)
Application (C3)
Comprehension
Lower order
(C2)
Knowledge (C1)
Organisation
(A4)
Valuing (A3)
Responding (A2)
Naturalisation
(P5)
Articulation (P4)
Precision (P3)
Manipulation
(P2)
Imitation (P1) 15
DOMAINS
Higher
order
Cognitive
Receiving (A1)
LEARNING DOMAINS
DOMAIN OF LEARNING OUTCOMES (MQF)
Knowledge
of
Discipline
Areas
Information
Management
& Lifelong
Learning Skills
Managerial &
Entrepreneu
rial Skills
Practical
Skills
Learning
Outcomes
Problem
Solving &
Scientific
Skills
GENERIC STUDENT ATTRIBUTES (MOHE)
Social Skills &
Responsibilities
Values,
Attitudes &
Professionalism
Communication
Leadership &
Team Skills
CRITICAL THINKING &
PROBLEM SOLVING
SKILLS
ENTREPRENEURSHIP
SKILLS
COMMUNICATION
SKILLS
INFORMATION
MANAGEMENT SKILLS &
CONTINUOUS SKILLS
MORAL &
PROFESSIONAL SKILLS
TEAMWORK SKILLS
LEADERSHIP SKILLS
16
RELATIONSHIPS BETWEEN CLO’S, PLO’S, PAI’S and
COMPLIANCE TO THE STAKEHOLDERS
STAKEHOLDERS
Regulatory
/Professional
Bodies
KPT/MQA
IHL
requirements
Industrires
students…
….. etc
PRORGRAME AIMSPROGRAMME
LEARNING
OUTCOMES
COURSE
LEARNING
OUTCOMES
PAI-2
PAI-3
PAI-4
PAI-5 ….. etc
PLO-1
PLO-2
PLO-3
PLO-4
PLO-5 ….. etc
Subject
CLO-1
Subject
CLO-2
Subject
CLO-3
PAI-1
Subject
CLO-4
Subject
CLO-5
Management support and commitment
17
FORMATIVE ASSESSMENT
Formative assessment is generally carried out
throughout a course or project, in which the
feedback from the learning activities are used to
improve the process of learning.
Formative
assessment is also referred to as "educative
assessment,“ to aid learning. In an educational
setting, formative assessment might be a lecturer or
the learner, providing feedback on a student's work,
and this would not necessarily be used for grading
purposes. Formative assessments are diagnostic in
nature.
18
SUMMATIVE ASSESSMENT
Summative assessment is generally carried out at the
end of a learning programme. Summative
assessments are typically used to assign students a
course grade. Summative assessments are evaluative
and often in the form of a report for a third party e.g.
the Head of Department or parents.
“when the cook tastes the soup, that’s formative, when
the guests taste the soup, that’s summative.”
Albert Stake
19
DIAGNOSTIC ASSESSMENT
Diagnostic Assessment occurs at the beginning of the
teaching/learning cycle. It involves making
judgments as to how a student is performing against
a predetermined set of criteria.
Diagnostic
assessment is also a strategy to prepare students for
the course requirements. It can be used as a form of
orientation or induction.
20
DIRECT ASSESSMENT
Direct assessment of learning: gathers evidence, based on
students’ performance, which demonstrates the learning itself.
Examples: quiz, test, final exam, assignments, projects, Final
year project, PBL, Laboratory Experiments, Exit Interview etc
Indirect assessment of learning: gathers reflection about the
learning or secondary evidence of its existence.
Examples: industrial training, external examiner, alumni
surveys, employer surveys, student exit survey etc.
21
DIRECT VS INDIRECT ASSESSMENT
DIRECT ASSESSMENT
INDIRECT ASSESSMENT
• Quizzes
• Tests
• Final Examination
• Assignments
• Projects
• Final Year Project
• Lab Experiments
• PBL/POPBL
• Portfolio
•
•
•
•
•
•
•
•
•
Alumni survey
Employer survey
Student Surveys
Exit Interviews with Graduates
Report and minute of meeting with
Industrial Advisor
Report and minute of meeting with
External Examiner
Percentage of students who go on to
undergraduate school
Retention and Transfer Studies
Job Placement Statistics
22
TYPES OF ASSESSMENT
PEER
ASSESSMENT
SELF
ASSESSMENT
AUTHENTIC
ASSESSMENT
23
NORM REFERENCED ASSESSMENT
• Norm-referenced assessment compares students to
that or those of a larger group. or "norm group"
which is a national sample representing a wide and
diverse cross-section of students. Students are
assigned grades according to their standing relative
to other students and it normally follows the normal
curve (grading on the curve). NRA is usually used to
sort students and not to measure achievement
towards some criteria of performance For this
reason, there are strong moves internationally to
move towards CRA.
24
CRITERION-REFERENCED ASSESSMENT
In criterion-referenced assessment, students'
performances are compared to a specific
learning objective or performance standard
and not to the performance of other students
For CRA to be beneficial to learning, explicit
criteria are essential which provide clear
learning goals for students.
25
REFLECTION 2
The following are some of the issues in assessment.
Discuss in pairs some of these issues.
1.
2.
3.
4.
5.
6.
Overload of students and staff
Too many assignments with the same deadline
Insufficient time for students to do the assignments
Insufficient time for staff to mark the assignments
Inadequate or superficial feedback provided to students.
Overuse of one mode of assessment such as written
examinations, essays or closed problems
26
FEEDBACK
FEEDBACK
Learning
Assessment
Feedback is the bridge between learning and
assessment
27
FEEDBACK
Some useful guidelines for giving feedback
Return all assessment scripts to students
Clear and precise comments to individual student ‘s work
Include both positive and critical comments
Be constructive, give advice on how to improve
Be organised in giving feedback e.g. separate / structure feedback sheet in relation to
assessment criteria involved
Open discussion and clarification of common mistakes or misconceptions
Appropriately identify good performance for students to learn from each other
28
RUBRICS
29
RUBRIC
A rubric is a set of criteria and standards for assessing
students’ work or performance. . For each criterion and
standard, a rubric defines the specific attributes that will
be used to score or judge the student’s performance and
to differentiate between different levels of
performances. Scoring rubrics include one or more
dimensions on which performance is rated, definitions
and examples that illustrate the attribute(s) being
measured and a rating scale for each dimension.
Dimensions are generally referred to as criteria, the
rating scales as levels, and definitions as descriptors.
30
TYPES OF RUBRIC
Holistic rubrics provide a single score based on an
overall impression of a student’s performance on a task.
Analytic rubrics provide specific feedback along several
dimensions
General rubrics contain criteria that are general across
tasks.
Task specific rubrics are unique to a specific task.
31
HOLISTIC RUBRIC
Description
Score
Demonstrates complete understanding of the problem. All
requirements of task are included in response.
5
Demonstrates considerable understanding of the problem. All
requirements of task are included
4
Demonstrates partial understanding of the problem. Most
requirements of task are included.
3
Demonstrates little understanding of the problem. Many requirements
of task are missing
2
Demonstrates no understanding of the problem
1
No response/task not attempted
0
32
ANALYTIC RUBRIC
Needs
Improvement
(1)
Clarity (Thesis
The purpose of the
supported by relevant student work is not
information and
well-defined. Central
ideas.)
ideas are not focused
to support the thesis.
Thoughts appear
disconnected.
Organization
(Sequencing of
elements/ideas)
Developing
(2)
Sufficient
(3)
Above Average
(4)
The central purpose
of the student work is
identified. Ideas are
generally focused in a
way that supports the
thesis.
The central purpose The central purpose
of the student work is of the student work is
clear and ideas are
clear and supporting
almost always focused ideas always are
in a way that supports always well-focused.
the thesis. Relevant Details are relevant,
details illustrate the enrich the work.
author’s ideas.
Information and ideas Information and ideas Information and ideas Information and ideas
are poorly sequenced are presented in an are presented in a
are presented in a
(the author jumps
order that the
logical sequence
logical sequence
around). The
audience can follow which is followed by which flows naturally
audience has difficulty with minimum
the reader with little and is engaging to the
following the thread difficulty.
or no difficulty.
audience.
of thought.
33
ANALYTIC RUBRIC
• Determines your performance levels.
• Examples of performance levels:Excellent/ Good/Moderate/Poor/ Very Poor or
Excellent/Good/ Satisfactory/Needs
Improvement
• Numbers/numerical scores – 4/3/2/1
34
STEPS TO CREATE A RUBRIC
List the criteria that will be used to assess performance.
Determine your peformance levels. Examples of
performance levels. Excellent/ Good/Moderate/Poor/ Very
Poor or Excellent/Good/ Satisfactory/Needs Improvement .
Write a description for each performance level.
Evaluate and revise rubric as needed.
35
ASSESSMENT TASK
36
SELECT ASSESSMENT METHODS
No.
Category of Learning Outcome
Sample of Assessment Methods
1
Thinking critically and
making judgments
essay, report, journal , present a case for an
interest group ,Book review (or article) for a
particular journal and write a newspaper
article
2.
Solving problems and
developing plans
group work, work-based problem, analyse a
case and conference paper (or notes for a
conference paper plus annotated
bibliography)
3.
Performing procedures and Demonstration, Role Play, Make a video
demonstrating techniques (write script and produce/make a video),
Produce a Poster, Lab report
4.
Managing and developing
oneself
Journal , Portfolio, Learning Contract and
Group work
37
SELECT ASSESSMENT METHODS
No.
Category of Learning Outcome
Sample of Assessment Methods
5
Accessing and managing
information
Annotated bibliography , Project, Applied
task ,Applied problem
6.
Demonstrating knowledge
and understanding
Written examination, Oral examination, Essay
, Report Short answer questions: True/False/
Multiple Choice Questions (paper-based or
computer-aided-assessment)
7.
Designing, creating,
performing
Portfolio, Performance , Presentation,
Projects
8.
Communicating
Written presentation (essay, report,
reflective paper etc.), Oral presentation.
Group work, discussion/debate/role play,
observation of real or simulated professional
practice
38
ASSESSMENT TASKS
Objective Tests measure the learners’ ability to
remember facts and figures as well as their
comprehension of course materials. Common
variations include multiple-choice (MCQ), true-false,
and matching items.
39
ASSESSMENT TASKS
Case Studies describes typical issues or
problems. It gives students the opportunity to
place themselves in the position of the
decision maker in a specific simulated
situation. The scenarios are usually taken from
real life and presented from the standpoint of
the decision maker. A successful case is one
which offers debate on alternative courses of
action, rather than offering a single settled
outcome or solution
40
ASSESSMENT TASKS
Essay Questions are the most commonly-used
assessment methods in polytechnics. There are
two main varieties:
• Unstructured questions, also known as free-response
questions. Students have maximum freedom for
discussion.
• Structured or restricted-response
questions. The
student is given far less freedom to determine the
nature and scope of the response. Often, the questions
guide both the structure and the content of the
expected response.
41
ASSESSMENT TASKS
• Capstone Assessment is often in final courses or
internships in a program. It provides an excellent
opportunity to identify whether students have
acquired the central knowledge and skills that are
the carefully selected outcomes expected of the
program. Capstone assessment may also want to
assess broader skills important to our graduates such
as problem solving, computer literacy, team work,
communication skills, and even the reading of
professional
materials.
(www.indstate.edu/
academicaffairs /assessment/capstone_assessmentintro.pp/)
42
ASSESSMENT TASKS
• End-of-chapter problems are used to reinforce
and apply some concepts and skills learned in
the classroom. As an example when a
mathematical procedure is presented in the
classroom, students are given numerical
practice problems to solve in order, after class.
These numerical problems are usually oversimplified real-life problems calling for direct
substitution of values into formulae.
43
ASSESSMENT TASKS
• Reflective journal is a piece of writing which
allows students to record thoughts and
insights about their own learning experience.
This can be writing about what and how they
have learned and understood a topic. It can
also be a review of their learning process, selfevaluation of their performance and planning
for future learning based on past learning
experience, etc
44
ASSESSMENT TASKS
• Seminar Presentation. Student works
individually, or in teams, to investigate a topic
relevant to the course and present their
findings in the form of a seminar. There are
other variations of the seminar presentation.
45
ASSESSMENT TASKS
• A portfolio is a collection of course-related
work performed by the students. It is also a
written reflection in which students evaluate
their own learning.. Traditionally, portfolios
are used to assess learning in the arts and
humanities. However, they can be equally
useful in just about any discipline.
46
ASSESSMENT TASKS
• Projects are popular methods of assessment
in Polytechnics today. Projects can be based
on literature and/or empirical research on a
relevant problem. They can also be applied
projects – solving real-life or simulated
problems, producing technical prototypes, or
preparing a business plan.
47
ASSESSMENT TASKS
• Examinations are used primarily for grading or
selection purposes. We call this activity, in
educational terms, summative assessment. It can use
many of the methods we discussed above. Since
examinations can take on many variations, it is
important to understand the range of possibilities in
order to design examinations that are compatible
both with outcome-based and student-centred
education.
Adapted from http://www.polyu.edu.hk/obe/
Check also Enhanced Learning and Teaching, Hong Kong University of Science and Technology
at http://celt.ust.hk/ideas/afl/MExam/index.html#see
48
CONCLUSION
• The OBE implementation requires a lot of
coordination in planning which involves a lot of
assessment. A well designed quality management
system is needed to ensure that our academic
programme , delivery system, assessing methods and
our graduates are at high quality. A framework of
continuous improvement with the involvement of
the academic staff, external examiner, students
participation and other stakeholders will probably
contribute to the successful implementation of
outcome-based assessment in polytechnics.
49
ACTIVITIES
50
ACTIVITY 1
• Designing Test Items.
This exercise offers you some experience in
constructing test items.
By referring to the JSP and JSI, outline the CLO
to be covered by the test.
Use the Exam Mapping Table as a guide to
construct the test items.
51
EXAM MAPPING TABLE
Exam
Content
Type
CLO
(Essay/Calc
/Sketch)
PLO
Bloom
Taxonomy
Question 1
a.
b.
c.
Question 2
a.
b.
c.
52
ACTIVITY 2
Think of your favourite assessment when you were an
undergraduate. In groups of three or four, discuss:
What makes a good assessment from your point of view?
What does it reveal about the tasks that you like doing?
Solving problems? Learning, discovering, finding out,
researching? Reading? Writing?
53
ACTIVITY 3
Please refer to the handout given to you.
54
ACTIVITY 4
The first step when creating a rubric is to
determine what you want students to learn
from the assignment.
In a groups of 4 , create a Task Specific Rubric
for student assessment.
55
56