Transcript Outcomes-based Teaching and Learning Pre
Outcomes-based Teaching and Learning Pre-workshop 3
Designing
Assessment Tasks and Grading Criteria
Intended Outcomes of Pre-workshop Exercises for members of each group
1. Review existing assessment tasks (ATs) in relation to alignment with subject ILOs.
2. Identify areas requiring changes.
3. Revise or re-design ATs to align to subject ILOs.
4. Discuss and share your ideas with your group members.
Before the Workshop
Refer to the Pre-workshop reference provided and the revised subject ILOs in Pre-workshop Exercise 1, conduct Pre workshop Exercises 4, 5 & 6 to: a. design assessment tasks to address the revised subject IlOs. Consider if the ATs require the students to engege in the ILO verbs? Do the weightings reflect the relative importance of the ILOs?
b. design grading criteria for either the ILOs or the assessment tasks.
Discuss with your group members when doing the above exercises. Complete the exercises to be ready for discussion and sharing in the workshops on 2 December 2006.
Outcomes-based Teaching and Learning (OBTL) using Constructive Alignment
Teaching: Engaging the student in the verb in the ILO ILO: What the student has to learn Assessment: How well the student has met the ILO
Designing Assessment Tasks (ATs)
Steps: 1. Selecting a practicable task that embodies the target ILO verb.
2. Making a judgment on how well the ILO has been met by the students' performance on the ATs – developing grading criteria.
Assessment Tasks (ATs)
.
Provide students the opportunity to demonstrate whether or not they have achieved the ILOs and what level their performance is in those ILOs. .
Provide the evidence allowing teachers to make a judgment about the level of a student’s performance against the ILOs and to award a final grade. .
Should be appropriately designed or selected to address the ILOs that we want to assess.
.
Different assessment tasks address different ILOs.
There would therefore be different kinds of assessment task.
Selection of Assessment Tasks
A range of different assessment tasks may be required to address the range of ILOs of a subject.
Questions to be asked in selecting assessment tasks: 1. Are the assessment tasks able to address all ILOs?
2. Are the assessment tasks aligned to their appropriate ILOs? Are the students required to engage in the verbs identified in the ILOs?
3. Are the assessment tasks practicable with respect to available time and resources?
4. Do the assessment tasks reflect the relative importance of the subject ILOs?
5. Is the assessment workload realistic for teachers and students?
Assessment Tasks Subject ILOs
Constructive Alignment of ILOs and Assessment Tasks
AT 1 AT 2 Subject ILO 1 AT 3 Subject ILO 2 Subject ILO 3 Subject ILO 4 Subject ILO 5 Consider if : all ILOs are being addressed?
there is a balanced coverage of the ILOs? the more important ILOs are given appropriate assessment emphasis.
Common ILOs
Describe Explain Integrate Analyse Apply Solve problem Design, create Reflect Communicate
Possible Assessment Tasks
assignment, essay question exam assignment, essay question exam, oral, letter-to-a-friend project, assignment case study, assignment project, case study, experiment case study, project, experiment project, experiment reflective diary, portfolio, self-assessment a range of oral, writing or listening tasks addressing the ILOs, e.g. presentation, debate, role play, reporting, assignment, precis, paraphasing, answering questions etc.
Assessment tasks for the sample subject ILOs
1. Explain different communicative strategies.
AT: Write an assignment or answer exam questions explaining the four most important strategies used in report writing. 2.
Apply appropriate strategies in different modes of communication.
3. AT: Write a report on a tutorial case study discussion on a selected topic.
Reflect and improve own communicative strategies.
AT: Self-assessment of the report, identifying strengths and areas needing improvement. Revise the report based on self-assessment.
The three ILOs could be assessed through one assessment task – writing a report on a tutorial case study discussion followed by a self-assessment and revised report.
a.
b.
c.
d.
Write a report on a tutorial group discussion; Explain the strategies used in writing the report; Self-assess the report, identifying strengths and strategies to be improved.
Revise the report based on c. and submit the revised report together with the first version.
Examination is a very commonly used assessment task especially for large classes. We need to consider if 1. examinations involving answering essay type of questions under invigilated conditions is able to assess students' performance in some high level ILOs, e.g. apply, reflect, create etc.
2. there are other alternative assessments tasks which will more appropriately addess those high level ILOs.
If we must have an invigilated “final exam”, there are far better formats than the conventional written essays; for example gobbets, critical incidents (e.g. comment on a video segement), letter-to-a-friend, and so on.
If we must have exams, consider if the weighting (usually 50% or more) is appropriate to reflect the relative importance of the ILOs being addressed.
Exercise 4 - Designing Assessment Tasks
Consider the intended learning outcomes of your subject in Exercise 1, suggest appropriate assessment task(s) to address these ILOs. For each of the tasks, explain what the students are expected to do in completing the task. Are the students to enact the target ILO verbs? Discuss with your group required members.
Subject ILOs Assessment Tasks Student activities in completing the task
After deciding on the assessment tasks, we have to consider the following issues.
1. Specification of the tasks Especially when continuous assessment is used, the different assessment tasks making up the continuous assessment will need to specified. 2. Assessment weightings If assessment tasks are given different assessment weightings, ensure that these weightings relfect the relative importance of the ILOs. Take the four sample subject ILOs – explain, apply reflect and improve. Are they of equal importance? If not, then the more important ILOs should be given heavier assessment weighting.
Exercise 5 – Assessment Weighting
Go back to the ATs designed in Exercise 4, now consider assessment weightings to ensure that they reflect the relative importance of the ILOs. Discuss with your group members.
Subject ILOs Assessment Tasks Student activities in Wt completing the task Does the weighting of the ATs reflect the relative importance of the ILOs they address?
Grading Criteria
Student performance in assessment tasks should be judged against a set of clear grading criteria defining the quality of performance expected of each of the grades.
Grading ILOs
Direct (Grading the ILOs) Grading criteria (rubrics) of individual ILOs Derive final grade Indirect (Grading the assessment tasks which are aligned with the ILOs) Grading criteria (rubrics) of individual assessment tasks Derive final grade
Example of Direct Grading Criteria of ILOs Grade point/unit Marginal Pass D 1.00
Satisfactory C C C+ 1.70 2.00 2.30
B Good Excellent B B+ A A 2.70 3.00 3.30 3.70 4.00
ILOs Explain Able to identify and briefly write about limited points.
Able to identify a number relevant points with some Able to identify a full As in “Good” but range of relevant provides views on Very little evidence of details. Uses these points using these points to Points with details. to provide a fair reasoning possible alternative Supported by relevant causes and/or results provide reasoning to why they are inter or causality. No evidence of a comprehensive literature. Points are under changing organized to provide a conditions. Able to related. overview of reasoning or causality.
comprehensive and cohesive reasoning or link current reasoning to causality. situations in real life professional contexts.
Reflect Able to use available information to self Able to use available information to self Able to use available information to self As in “Good”. Able to generalize self evaluate and identify evaluate and identify limited aspects of own more aspects of own evaluate and identify evaluation to beyond the full range of owm existing context. strengths and weaknesses strengths and weaknesses in a general sense. No in a general sense. Little evidence of suggestions application of theory in of ways to improve self-evaluation and limited strengthes and weak Suggest ways of nesses. Self-evaluation improving perform is based on theory. ance to real-life Increasingly able to professional performance. No evidence suggestions of ways to suggest ways to of theory being used in improve performance. improve performance contest. self-evaluation. in a specific context.
Example of Indirect Grading Criteria of Assessment Tasks in a Portfolio Marginal Pass D 1.00
Satisfactory C C C+ 1.70 2.00 2.30
B Good Excellent B B+ A A 2.70 3.00 3.30 3.70 4.00
The pieces of evidence The evidence is relevant, are relevant and accurate and covers The evidence presents a As in “B” but with good appreciation of higher degree of accurate, but are several aspects of the the general thrust of the originality and isolated, addressing subject. Little evidence of subject. Good coverage evidence of inter one aspect of the an overall view of the with relevant and nalization into subject. Demonstration subject. Demonstrates accurate support. A clear personalized model of understanding in a declarative understanding view of how various of practice. Good minimally acceptable of a reasonable amount of way. Poor coverage, no content. Able to discuss aspects of the subject evidence of reflect integrate to form a ion on own originality, weak content meaningfully. thrust or purpose. performance based justification of portfolio Good coverage but little Good evidence of on theory.
items. Inappropriate Application or integration. application of subject Generalizes subject self-evaluation.
Fair justification of items.
Attempted realistic self Content to practice. content to new and Portfolio items well unfamiliar real evaluation justified. Realistic self-evaluation. life contexts.
Departments need to decide whether to use direct or indirect grading of the ILOs.
A holistic way of deriving a final grade from multiple grades
ILOs 1. Relate 2. Apply 3. Identify AT1 √ AT2 √ AT3 √ √ √ AT4 4. Explain 5. Communicate Grade A – if all ILOs well achieved but 1 & 2 excellent B – if all are well achieved √ √ C – if 3, 4 & 5 are well achieved D – if marginal or only 2 ILOs are achieved
An example of a holistic way of deriving final grade
Curriculum and Instruction: A subject in a course for Ed. Psychlsts.
Grading will be based on your attaining the following ILOs 1. Apply the principles of good teaching and assessment to chosen contexts.
2. Relate selected aspects of curriculum design and management to the educational system in Hong Kong.
3. Apply the content and experiences in this subject to enhance your effectiveness as an educational psychologist.
4. Show examples of your reflective decision-making as an educational psychologist.
Final grades will depend on how well you can demonstrate that you have met all the ILOs: A Awarded if you have clearly met all the ILOs, provide evidence of original and creative thinking, perhaps going beyond established practice.
B Awarded when all ILOs have been met very well and effectively.
C Awarded when the ILOs have been addressed satisfactorily, or where the evidence is strong in some ILOs, weaker but acceptable in others.
F Less than C, work plagiarised, not submitted.
Example of Holistic Grading of A
ssessment Tasks in a Portfolio
Marginal D Adequate C C C+ 1.00 1.70 2.00 2.30 Good B Excellent B B+ A 2.70 3.00 3.30 3.70 4.00
A The pieces of evidence The evidence is relevant, The evidence presents a As in “B” but with are relevant and accurate and covers good appreciation of higher degree of accurate, but are several aspects of the the general thrust of the originality and isolated, addressing course. Little evidence of course. Good coverage evidence of inter one aspect of the an overall view of the with relevant and nalization into course. Demonstration course. Demonstrates accurate support. A clear personalized model of understanding in a declarative understanding view of how various of practice. Good minimally acceptable of a reasonable amount of aspects of the course evidence of reflect way. Poor coverage, no content. Able to discuss integrate to form a ion on own originality, weak content meaningfully. Thrust or purpose. Performance based justification of portfolio Good coverage but little Good evidence of on theory.
items. Application or integration. application of course Generalizes course Fair justification of items. Content to practice. content to new and Portfolio items well Unfamiliar real justified.
life contexts.
Exercise 6 – Developing Grading Criteria
Consider the intended learning outcomes and assessment tasks you have identified in Exercise 4, suggest appropriate grading criteria addressing either (a) the ILOs or (b) the assessment tasks. Discuss with your group members.
Subject ILO and AT: Grade Grading criteria
Help our students understand the grading
criteria
. Remind students of the ILOs to be assessed. . Explain to students the standards expected of each grade.
. Illustrate with previous assessment tasks on
How the grading criteria have been applied. How grades have been awarded.
How a better grade could have been achieved.
. Clarify any problems that students may have.