Transcript Transforming assessment through the tenets

Engaging students with formative
assessment (and feedback)
Margaret Price
10 December 2013
This session aims to :
• To examine the factors that influence levels of
student engagement with formative assessment and
feedback
• To critically evaluate approaches to increasing levels
of engagement with formative assessment
• Emphasize a programme approach and link with
session programme-focused assessment
2
Structure
Engagement with assessment
• What causes non engagement
• How can these be addressed
Engagement with feedback
• Research findings
• Relevance of the whole learning experience
• Resources and feedback
• Approaches to supporting engagement with
feedback
3
Sources of problem
Students
How to help the students engage ( overload? partying?
boredom?)
Check assumptions. Setting expectations, modelling and
influencing culture (tolerance of ‘laziness’)
Assessment design
Audit - relevance, how interesting, have students got a
chance of developing expertise
Staff expertise
Downsides of engagement methods (surface learning)
Level of assessment literacy e.g peer assessment
4
Why don’t students engage with
formative assessment
Each group generate as many reasons as you can in 5
minutes.
• Which do you think are the 3 most impactful reasons
• Choose the reasons that students would give
(assuming an honest reflective answer)?
• What are the reasons that most staff attribute to the
problem of non engagement? (the culture of the
organisation)?
5
Ways to get students to engage?
Each group list as many ways as you can in 5
minutes
• Which are the 3 most commonly used methods?
• Which are the 3 most effective methods?
• Which are the 3 that would meet most
resistance?
6
Perspectives on engagement with
assessment task
•
•
•
•
Compulsion
Incentivising
Nudging
Developing self motivation through assessment
literacy
• Self motivated
• Intrinsic engagement
Prevention is better than cure
Use the cards to create effective ‘formulae’ that
lead to student engagement.
Cards available:
Engagement problem
Actions to take
Alert cards (red) highlight further issues to be
addressed.
8
Engagement with Feedback
Don’t start here – part of the assessment and learning package.
Plan
• Research findings
• Relevance of the whole learning experience
• Resources and feedback
• Approaches to supporting engagement with
feedback
10
Look beyond feedback product
 Clarity of purpose
 Learning effectiveness (and student engagement) is
strongly influenced by opportunity to apply
feedback to future performance This relies on:
ability to understand feedback (legibility and
interpretation)
expectations of the utility of feedback
perception of self efficacy
 The relational dimension of feedback is key to
student engagement
 Dialogue supports understanding and engagement)
(Price et al 2010)
Conceptual shifts
1.Self and peer assessment need to be
seen as essential graduate attributes
(i.e. learning outcomes themselves, rather
than simply processes)
2.Feedback needs to be seen as a dialogue
(rather than a monologue)
12
Feedback moments
 Where there is a clear need to apply feedback
 Pre assessment
 Reflection points
Identify them within each programme
Feedback moments?
Learning development
Radical movement involving zones of discomfort, ‘threshold concepts and
troublesome knowledge’ (Meyer and Land, 2006)
Changes in ‘epistemology and knowledge structures’ (Basil Bernstein in
Moore et al, 2006)
Phase 1
Phase 2
Phase 3
Sources of feedback
Staff: Traditional – personal, generic.
Feedback on Self Assessment
e.g. 5mins oral feedback for 140 students
Personal tutor consultation
Peers: Peer review and peer assessment
Peer assisted learning
Self:
Require self assessment
Peer marking using model answers
(Forbes & Spence, 1991)
Scenario:
• Engineering students had weekly maths problem sheets
marked and problem classes
• Increased student numbers meant marking impossible and
problem classes big enough to hide in
• Students stopped doing problems
• Exam marks declined (Average 55%>45%)
Solution:
• Course requirement to complete 50 problem sheets
• Peer assessed at six lecture sessions but marks do not count
• Exams and teaching unchanged
Outcome: Exam marks increased (Av. 45%>80%)
16
Peer feedback - Geography (Rust, 2001)
Scenario
 Geography students did two essays but no apparent improvement from
one to the other despite lots of tutor time writing feedback
 Increased student numbers made tutor workload impossible
Solution:
 Only one essay but first draft required part way through course
 Students read and give each other feedback on their draft essays
 Students rewrite the essay in the light of the experience
 In addition to the final draft, students also submit a summary of how the
2nd draft has been altered from the1st in the light of the feedback
Outcome: Much better essays
Peer feedback - Computing (Zeller, 2000*)
The Praktomat system allows students to read, review, and assess each
other’s programs in order to improve quality and style. After a successful
submission, the student can retrieve and review a program of some fellow
student selected by Praktomat. After the review is complete, the student
may obtain reviews and re-submit improved versions of his program. The
reviewing process is independent of grading; the risk of plagiarism is
narrowed by personalized assignments and automatic testing of submitted
programs.
In a survey, more than two thirds of the students affirmed that reading each
other’s programs improved their program quality; this is also confirmed by
statistical data. An evaluation shows that program readability improved
significantly for students that had written or received reviews.
[*Available at: http://www.infosun.fim.unipassau.de/st/papers/iticse2000/iticse2000.pdf]
Resources and effectiveness
• Review resource allocations in total (OU resourcing model)
• Feedback methods:
• Oral(F2F)/audio rather than written (see the Sounds Good website
at: http://sites.google.com/site/soundsgooduk/)
• Exemplars/model answers including modelling improvement
following feedback
• Generic (including ‘quick and dirty’), feedback on a draft, MCQs &
quizzes, etc.
• Feedback workshop
• Require students to demonstrate how they have used feedback in
subsequent work.
• Target resources where most needed in programme
• Use self and peer methods more frequently
• Target feedback moments
Figure 1: Peer-review as a method of encouraging
students to discuss and compare their
understanding of assessment criteria
IN-CLASS
ACTIVITY
OUT OF CLASS
ACTIVITY
Figure 1
2. Students
bring draft
individual
assignments
for peer
review
1. Tutor leads
discussion on
assessment
criteria and
process of peer
review
4. Students
rewrite and
submit
individual
assignments
3. In-class
discussions
between student
groups as they
review each other’s
work, monitored by
tutor.
5. Tutor
marks
assignments
and prepares
feedback
6. Tutor
hands back
assignments
and leads
discussion on
feedback
MODULE TIMELINE
Week1
Week12
Assignment
point
Figure 2: the use of 'exemplars' as a
mechanism for encouraging dialogue
about assessment criteria
IN-CLASS
ACTIVITY
OUT OF CLASS
ACTIVITY
Figure 2
2. Students
write and
submit
individual
assignments
1. Tutor leads
discussion of
previouslymarked
exemplars
annotated with
feedback
3. Tutor
marks
assignments
and prepares
feedback
4. Tutor
hands back
assignments
and leads
discussion on
feedback
MODULE TIMELINE
Week1
Week12
Assignment
point
Figure 3: Generic feedback
and self critique
IN-CLASS
ACTIVITY
OUT OF CLASS
ACTIVITY
Figure 3
1. Students
draft and
submit
individual
assignments
2. Tutor marks
sample of
assignments
and prepares
generic
feedback
4. Students rewrite and
submit individual
assignments with
reflective commentary
on how they have
incorporated the
generic feedback
3. In-class
discussion of
generic cohort
feedback
based on
coursework
sample
5. Tutor
grades
assignments
6. Tutor
hands back
assignments
with grade
only
MODULE TIMELINE
Week1
Week12
Assignment
point
Activity
In groups:
Anticipate arguments or factors that will militate
against change – how will you counter them
Begin to develop a checklists for programme
teams to use in the process of developing
assessment strategies at programme and
module level.
23