Transcript Slide 1

Enhancing assessment and
feedback: an evidence-based
response
Chris Rust
Deputy Director
ASKe Centre for Excellence in Teaching and Learning
(Assessment Standards Knowledge exchange)
Head, Oxford Centre for Staff & Learning Development
Oxford Brookes University
ASKe Directorate:
Margaret Price, Jude Carroll, Berry O’Donovan and Chris Rust
Programme
1.15
1.30
1.35
1.50
2.10
2.30
2.45
2.55
3.05
3.20
3.40
4.10
4.20
4.30
Coffee
Welcome and introduction
Activity
Tacit knowledge & the social-constructivist assessment
process model
Student engagement with criteria
Feedback - the issues
Activity - 7 conditions
Improving feedback - preparing students
Activity
Improving feedback - making it fit for purpose
Activity
The rest of the cycle
Summation & evaluation
End
Assessment Standards Knowledge exchange
Three main points from the literature
 Assessment and feedback are key drivers of student
learning
 Current assessment and feedback processes are
often not effective
 There is lots of useful and evidence-based meta-level
advice in the literature plus a plethora of articles on
successful practice.
Assessment Standards Knowledge exchange
Focus of ASKe
 Communicating and engaging students with assessment
requirements: standards, criteria and feedback
 A key issue in assessment is that students often do not
understand what is a better piece of work and do not understand
what is being asked of them particularly in terms of standards and
criteria (O’Donovan et al., 2001)
 Feedback is also often not understood (Lea and Street, 1998),
often considered too vague (Higgins, 2000) and the language
complex and subject to interpretation (Lea and Street, 1998;
Ridsdale 2003)
Assessment Standards Knowledge exchange
Activity
Two students are given the same written assessment
task (e.g. an essay in your discipline). In the tutor
feedback on the task both students are congratulated
on producing a ‘highly analytical’ piece of work. One
student is a postgraduate, the other a first year
undergrad.
Does the term ‘highly analytical’ mean something
different at these two levels? And, if so, what is the
difference? How do you explain to students the
standard of work expected?
Assessment Standards Knowledge exchange
Making meaning requires explicit and tacit knowledge
 Meaningful understanding of standards requires both tacit and
explicit knowledge (O’Donovan, B., Price, M., & Rust, C., 2004)
 “we can know more than we can tell” (Polanyi, reprinted 1998,
p.136).
 Verbal level descriptors are inevitably ‘fuzzy’ (Sadler 1987)
 There is a cost (in terms of time and resources) to codifying
knowledge which increases the more diverse an audience’s
experience and language (Snowdon, 2002).
 Tacit knowledge is experience-based and can only be revealed
through the sharing of experience – socialisation processes
involving observation, imitation and practice (Nonaka, 1991)
 ‘making sense of the world’ is seen as a social and collaborative
activity (Vygotsky, 1978).
Assessment Standards Knowledge exchange
So what are we bringing to the party?
Social-constructivist view of assessment
the social-constructivist view of learning argues that knowledge
is shaped and evolves through increasing participation within
different communities of practice
the social-constructivist process model of assessment argues that
students should be actively engaged with every stage of the
assessment process in order that they truly understand the
requirements of the process, and the criteria and standards being
applied, and should subsequently produce better work
(Rust C., O’Donovan, B., & Price, M., 2005)
Assessment Standards Knowledge exchange
Assessment Standards Knowledge exchange
Assessment Standards Knowledge exchange
Marking exercise
Immediate results
Cohort 1 (99/00)
Cohort 2 (00/01)
Cohort 3 (01/02
participants av. mk
non participants av. mk.
59.78
59.86
55.7
54.12
52.86
49.7
57.91
56.4
51.3
51.7
Results 1 year later
Cohort 1
Cohort 2
Rust, C., Price, M & O’Donovan, B.(2003) "Improving students’ learning by
developing their understanding of assessment criteria and processes”
Assessment and Evaluation in Higher Education, Vol. 28, No. 2
Assessment Standards Knowledge exchange
Assessment Standards Knowledge exchange
Potential of feedback
Feedback is the most powerful single influence that makes a
difference to student achievement
Hattie (1987) - in a comprehensive review of 87 meta-analyses of
studies
Feedback has extraordinarily large and consistently positive effects
on learning compared with other aspects of teaching or other
interventions designed to improve learning
Black and Wiliam (1998) - in a comprehensive review of formative
assessment
Students are hungry for feedback to develop their learning
(Higgins et al, 2002)
Assessment Standards Knowledge exchange
Retention and feedback
The number of opportunities available for feedback is an
important variable in non-completion of students in the
early years of study
(Yorke, 1999)
Where students are uncertain about their ability to
succeed, formative feedback is of particular importance
(Yorke & Longden, 2004)
Assessment Standards Knowledge exchange
Feedback problems
•
•
•
•
•
•
•
•
Unhelpful feedback (Maclellan, 2001)
Too vague (Higgins, 2000)
Subject to interpretation (Ridsdale, 2003)
Not understood (e.g. Lea and Street, 1998)
Don’t read it (Hounsell, 1987)
Damage self-efficacy (Wotjas, 1998)
Has no effect (Fritz et al, 2000)
Seen to be too subjective (Holmes & Smith,
2003)
Assessment Standards Knowledge exchange
11 conditions under which assessment
supports learning 1 (Gibbs and Simpson, 2002)
1. Sufficient assessed tasks are provided for students to
capture sufficient study time (motivation)
2. These tasks are engaged with by students, orienting
them to allocate appropriate amounts of time and effort
to the most important aspects of the course (motivation)
3. Tackling the assessed task engages students in
productive learning activity of an appropriate kind
(learning activity)
4. Assessment communicates clear and high expectations
(motivation)
Assessment Standards Knowledge exchange
11 conditions under which assessment
supports learning 2 (Gibbs and Simpson, 2002)
5 Sufficient feedback is provided, both often enough and in enough
detail
6 The feedback focuses on students’ performance, on their learning
and on actions under the students’ control, rather than on the
students themselves and on their characteristics
7 The feedback is timely in that it is received by students while it still
matters to them and in time for them to pay attention to further
learning or receive further assistance
8 Feedback is appropriate to the purpose of the assignment and to its
criteria for success.
9 Feedback is appropriate, in relation to students’ understanding of
what they are supposed to be doing.
10 Feedback is received and attended to.
11 Feedback is acted upon by the student
Assessment Standards Knowledge exchange
Improving feedback - prepare students (in Yr 1 esp.)
•
•
•
•
•
Aligning expectations (of staff & students, & between teams of markers)
- often a mismatch of expectations e.g correcting errors, advice for the
future, diagnosis of general problems, comments specific only to that piece
of work. These mismatches occur frequently with no particular pattern
about who holds which view/perspective but problems arise when the the
two don't coincide. Purpose of feedback may vary from assignment to
assignment so would need to be clarified each time. (Freeman & Lewis, 1998)
Identifying all feedback available
Model the application of feedback
- e.g. using previously-marked assignments to show how feedback was
used to improve later assignments
Encourage the application of feedback
- e.g. in a subsequent piece of work the student is required to show how
they have used prior feedback to try to improve their work and some marks
it is the interaction between both believing in selfallocated for this.
responsibility and using assessment formatively that
Require and develop self-assessment leads to greater educational achievements
(Brown & Hirschfeld, 2008)
Assessment Standards Knowledge exchange
Activity
In 3’s, discuss:
• How do you currently prepare students to
understand and engage with feedback?
• Which of these ideas could you introduce, or
develop further, and how?
Assessment Standards Knowledge exchange
Improving feedback - ensure it is fit for purpose
•
•
•
•
•
•
•
•
•
Ensure students have MOM - Motive, Opportunity, Means (Angelo, 2007)
Draft-plus-rework - feedback effort (for markers and students) is located at
the draft stage, and possibly only a summative grade is given for the final
submission
Improve the linkage of assessment strategies across programmes and
between modules/units
Increase student engagement and understanding through dialogue - inclass discussion of exemplars, peer-review discussions supported by tutors,
learning-sets, etc.
Identify what is feasible in a given assessment context - written feedback
can often do little more than ‘diagnose’ development issues and then direct
students to other resources for help and support
Ensure it is timely - ‘quick and dirty’ generic feedback, feedback on a draft,
MCQs & quizzes, etc. (using technology may help)
Consider the role of marks - they obscure feedback
Reduce over-emphasis on written feedback - oral can be more effective
(McCune, 2004). But individual F2F can be resource intensive
Review resource allocations (N.B. OU 60%)
Assessment Standards Knowledge exchange
Activity
Individually:
Choose one or more specific ideas to improve
feedback that you think you could use. In as much
detail as possible, identify how you would put the
idea/s into practice.
In pairs:
Take it in turns to explain your plans to your partner.
The job for the listener is to be a friendly and
constructive critic
Assessment Standards Knowledge exchange
7 principles of good feedback practice
(Nicol and Macfarlane-Dick, 2006)
Good feedback practice:
1. helps clarify what good performance is (goals, criteria, expected
standards)
2. facilitates the development of reflection and self-assessment in
learning
3. delivers high-quality information to students about their learning
4. encourages teacher and peer dialogue around learning
5. encourages positive motivational beliefs and self-esteem
6. provides opportunities to close the gap between current and desired
performance
7. provides information to teachers that can be used to help shape the
teaching
Assessment Standards Knowledge exchange
Assessment Standards Knowledge exchange
Rust C.,O’Donovan B & Price., M (2005)
Assessment Standards Knowledge exchange