Slide 1

Assessment for learning: the benefits
of generating feedback

David Nicol
Professor of Higher Education
Centre for Academic Practice and Learning Enhancement (CAPLE]
Director, Peer project (www.reap.ac.uk)
University of Strathclyde, Scotland
University of Cadiz, March 9-11, 2011


Slide 2

Plan
Background
Re-engineering Assessment Practices
(REAP) project
www.reap.ac.uk/reap/index.html
Concepts and example of practice
Institutional considerations
PEER project www.reap.ac.uk/peer.aspx
Discussion


Slide 3

Background
Departments and faculties: educational improvement
projects, including REAP project
Policy/strategy: development of educational policy
and principles of assessment & feedback (based on
REAP)
Students: ‘Feedback as dialogue’ campaign
Quality procedures:– redesigning course
documentation and review procedures
PEER project – developing students ability to evaluate
the quality of each other’s work
See www.reap.ac.uk


Slide 4

REAP : Re-engineering Assessment Practices
Scottish Funding Council for Universities (£1m)
3 Universities - Strathclyde, Glasgow & Glasgow
Caledonian
Large 1st year classes (160-600 students)
A range of disciplines (19 modules ~6000 students)
Many technologies: online tests, simulations, discussion
boards, e-portfolios, e-voting, peer/feedback software,
VLE, online-offline
Learning quality and teaching efficiencies
Assessment for learner self-regulation
www.reap.ac.uk


Slide 5

Background (1)
Gibbs, G. & Simpson, C (2004) Conditions under which
assessment supports students’ learning, Learning and
Teaching in Higher Education, 1, 3-31.
See:

Formative Assessment in Science Teaching (FAST)
project at: http://www.open.ac.uk/science/fdtl


Slide 6

Gibbs and Simpson (2004)
Assessment tasks [Conditions 1-4]
1. Capture sufficient study time (in and out of class)
2. Are spread out evenly across timeline of study
3. Lead to productive activity (deep vs surface)
4. Communicate clear and high expectations
i.e concern here is with ‘time on task’ how much work
students do - their active engagement in study


Slide 7

Background (2)

Literature Review
Nicol, D. & Macfarlane-Dick, D. (2006). Formative
assessment and self-regulated learning: A model and
seven principles of good feedback practice. Studies in
Higher Education, 34 (1), 199-218
Background
Student Enhanced Learning through Effective Feedback
[SENLEF] project funded by HE Academy
REAP project: www.reap.ac.uk


Slide 8

Rethinking assessment and feedback
1. Consider self and peers as much as the teacher as sources
of assessment and feedback
 Tap into different qualities than teacher can provide
 Saves time
 Provides considerable learning benefits (lifelong learning)
2. Focus on every step of the cycle:
Understanding the task criteria
Applying what was learned in action
3. Not just written feedback:
 Also verbal, computer, vicarious, formal and informal


Slide 9

Seven principles of good feedback
Good feedback:
1. Clarifies what good performance is (goals, criteria,
standards).
2. Facilitates the development of reflection and selfassessment in learning
3. Delivers high quality information to students: that
enables them to self-correct
4. Encourages student-teacher and peer dialogue around
learning
5. Encourages positive motivational beliefs & self esteem
6. Provides opportunities to act on feedback
7. Provides information to teachers that can be used to
help shape their teaching (making learning visible)
Source: Nicol and Macfarlane-Dick (2006)


Slide 10

Principle 1: Clarify what good performance is (the context of dialogue)
EMPOWERMENT/
SELF-REGULATION

• Students create criteria
• Students add own criteria
• Students identify criteria
from samples of work
• Exemplars of different
performance levels
provided
• Students rephrase
criteria in own words
• Provide document with
criteria
ENGAGEMENT


Slide 11

Principle 2: Facilitate reflection and self-assessment in learning
EMPOWERMENT/
SELF-REGULATION

• Students create MCQs
including feedback for
right and wrong
(distractors) answers
• Students self-assess
using MCQs and
confidence ratings
• Students self-assess own
performance using online
MCQs
ENGAGEMENT


Slide 12

Two meta principles
1. Meta-PRINCIPLE 1: time and effort on task
(structured engagement) i.e. steers on how much
work to do and when – Gibbs and Simpson 4
conditions
2. Meta-PRINCIPLE 2: developing learner selfregulation (empowerment/self-regulation) i.e
steers to encourage ownership of learning – the
seven principles discussed above.
Key task for teacher is to balance 1 and 2


Slide 13

Example:
Psychology


Slide 14

Psychology

560 first year students
6 topic areas (e.g. personality, classical
conditioning), 48 lectures, 4 tutorials, 12
practicals
Assessment; 2 x MCQs (25%), tutorial
attendance (4%), taking part in experiment
(5%), essay exam (66%)


Slide 15

Problems identified
No practice in writing skills but required in
the exam
More detail provided in lectures than
mentioned in exams (not enough
independent reading)
No feedback except on Multiple Choice
Questions (percent correct)
Didn’t want to increase staff workload


Slide 16

Psychology Redesign
Discussion board in Learning Management
System
Students in 85 discussion groups of 7-8, same
groups throughout year
Also open discussion board for class
Friday lectures cancelled – discover material
themselves
Series of online tasks


Slide 17

Structure of group tasks
6 cycles of 3 weeks (one cycle x major course topic)
• First week: ‘light’ written task (e.g. define terms) = 7
short answers (all answer)
• Second week = guided reading
• Week three: ‘deep’ written task: students collaborate
in writing a 700-800 word essay on the same topic.
Within each week:
• The Monday lecture – introducing material
• Immediately after lecture, task posted online – for
delivery the following Monday
• Model answers (selected from students) posted for
previous week’s task


Slide 18

The teaching role
Participation in the discussions was compulsory but
not marked (in subsequent years there was 2% mark
for participation)
Course leader provided general feedback to the
whole class – often motivational
He encouraged students to give each other feedback
The group discussions were not moderated but
monitored for participation


Slide 19

An example of ‘deep’ task
The Task – 800 word essay:
Assess the strengths and weaknesses of Freud’s and
Eysenck’s theories of personality. Are the theories
incompatible?
readings suggested
questions provided which all students should try


Slide 20

Relation to the Gibbs & Simpson’s
four assessment conditions
1. Tasks require significant study out of class (condition

1)
2. Tasks are distributed across topics and weeks
(condition 2)
3. They move students progressively to deeper levels of
understanding (condition 3)
4. There are explicit goals and progressive increase in
challenge (condition 4)


Slide 21

Relation to 7 feedback principles
1. Standard format and model answers provide progressive
2.
3.
4.

5.
6.
7.

clarification of expectations (principle 1)
Students encouraged to self-assess against model answer
(principle 2)
Course leader provides motivational and meta-level
feedback and selects model answers (principle 3)
Online peer discussion aimed at reaching consensus is core
feature of design about response (principle 4)
Focus on learning not just marks, sense of
control/challenge enhanced motivation (principle5)
Repeated cycle of topics and tasks provide opportunities to
act on feedback (principle 6)
VLE captures all interactions allowing course leader to
monitor progress and adapt teaching (principle 7)


Slide 22

Benefits
Students worked exceptionally hard
Written responses of exceedingly high standard
Students took responsibility for learning
High levels of motivation: atmosphere in class
improved
Online interactions showed powerful ‘scaffolding’ and
community building
Feedback with 560 students through peer and selffeedback (model answers)
Easy for tutors to monitor participation
Improved mean exam performance (up from 51-59%,
p<0.01) weaker students benefit most


Slide 23

Has it worked?
I read more about Psychology and read it earlier
in each semester than I would have done without
the online projects
60
Numbers

50
40
30
20
10
0
1

2

3
agreed-disagreed

4

5


Slide 24

numbers

I found that reading other people's contributions
helped me understand Psychology
80
70
60
50
40
30
20
10
0

Series1

1

2

3
agreed-disagreed

4

5


Slide 25

I learned more in Psychology because of online
projects than I did in my other subjects
60

numbers

50
40
30

Series1

20
10
0
1

2

3
agreed-disagreed

4

5


Slide 26

Guidelines for Implementation
1.
2.

3.
4.
5.
6.

A single principle or many?
Tight-loose – maintain fidelity to the principles (tight)
but encourage disciplines to develop their own
techniques of implementation (loose)
Balance teacher feedback with peer and selfgenerated feedback
Focus on developing students’ own ability for critical
evaluation
Create opportunities for ‘learning communities’ to
emerge
The more actively engaged students are, the better
the course design


Slide 27

Developments since REAP

1. Principles of Assessment and Feedback approved by
2.
3.
4.
5.

University Senate and embedded in policy (2008)
Use of principles to inform curriculum renewal and
Quality Assurance processes
‘Feedback as Dialogue’ campaign to gain
commitment of students
PEER Project (Peer Evaluation in Education Review)
Work on the way we document our courses and
programmes


Slide 28


Slide 29


Slide 30

Peer Review in Education Evaluation [PEER]
The aims of the PEER project are to:
Review evidence base for peer review
Develop educational designs for peer review (and
self-review)
Identify software support for peer review
Pilot implementations of peer review with large
student numbers
Produce guidelines for higher education – why do it,
how to do it, pitfalls and solutions and software
possibilities.
see http://www.reap.ac.uk/peer.aspx


Slide 31

PEER Project: Core Ideas
All graduate development in higher education requires
that:
1. Students learn to evaluate critically the quality and

impact of their own work (e.g. academic texts, problem
solutions, designs)

2. Students learn to evaluate critically the work of others

(e.g. peer review and peer feedback)

Ref: Nicol, D (2010) The foundation for graduate attributes: developing selfregulation through self and peer assessment. www.reap.ac.uk/peer.aspx


Slide 32

The research on peer feedback
Mainly about peers marking each others work
Where peer feedback is the focus it is usually about
peers augmenting teacher feedback by increasing the
quantity and variety of feedback


Slide 33

The focus of PEER project
Scenarios where students make evaluative judgements
about the work of peers and provide a feedback
commentary, usually written
Not talking about
.....collaborative tasks where students give each other
informal feedback
.....scenarios where the focus is on students evaluating
each other’s performance in group working
.....scenarios where the focus is on students
grading/marking each other’s work although some
rating of performance might be part of the peer design


Slide 34

Benefits of feedback construction (1)
Constructivist rather than transmission perspective
Producing is cognitively more demanding than reading
feedback from others: students cannot be passive
Students actively exercise assessment criteria
(process/reprocess) from many different perspectives
See how others tackle assignments and learn that
quality can be produced in different ways
Develop writing skills through commentaries
Deepens critical thinking in the discipline


Slide 35

Benefits of feedback construction (2)
Learn to assess own work, to monitor and evaluate
their own productions - same skills involved
Develops self-regulation – for life beyond university.
Changes the power relationship in class (shares
responsibility across students).
Can develop learning communities


Slide 36

Giving and receiving feedback
Giving and receiving feedback within same domain
enriches the whole process.
Students are able to compare the feedback they
receive with the feedback they produce
And learn how different reviewers perceive their work –
a single source of feedback is never satisfactory
Can develop a sense of shared responsibility across
students for learning


Slide 37

Example: peer feedback
•
•
•

•
•
•
•
•

Students write essay on one topic from three
Each student provides feedback on three essays in another topic
anonymously using rubric
The rubric: write a short summary of the essay, comment on and
rate (four point scale) the structure, arguments, evidence, writing,
suggest ways of improving the essay.
Students receive peer reviews of their own essays
Then review, comment on and rate their own essay using same
rubric.
Graded: for participating in the task, for their own essay and for
their review of it.
Finally students rate 3 reviews (on others’ work) and comment on
how useful they think they would be to author.
Software used to support administration


Slide 38

Six learning opportunities in peer review
Producing the target assignment
Reviewing and producing peer feedback
Receiving feedback from peers
Responding to peer reviews (on own reviews of those
given to others)
5. Self-reviewing one’s own work (e.g. after peer
review)
6. Receiving feedback from the teacher – e.g. on the
assignment, peer review or self-review
1.
2.
3.
4.

1-3 comprise the essentials of peer review while 4-6
are further options.


Slide 39

Principles of effective peer review
Peer review should
1. Engage students in active use of criteria and standards
2. Involve students in constructing commentaries in relation to peer
3.
4.
5.
6.
7.

judgements, not just marks
Lead to practice in both analytic (componential) and holistic
(configurational) judgements about quality
Facilitate dialogue around the object and quality of the review
Ensure an atmosphere of mutual trust and respect
Integrate self-review activities within peer review designs
Ensure that making peer judgements is a regular activity and not
a one-off event

www.reap.ac.uk/PEER.aspx


Slide 40

Some of my Publications
Nicol, D (2010) The foundation for graduate attributes: developing self-regulation thorugh self
and peer assessment, QAA Enhancement Themes, Scotland
Nicol, D (2010) From monologue to dialogue: Improving written feedback in mass higher
education. Assessment and Evaluation in Higher Education. 35(5), 501 -517
Nicol, D and Draper, S (2010) A blueprint for transformational organisational change in HE: REAP
as a case study, Published by the Higher Education Academy, UK (see reap.ac.uk website)
Nicol, D (2009) Transforming assessment and feedback: Enhancing integration and empowerment
in the first year, Published by Quality Assurance Agency, Scotland
(http://www.enhancementthemes.ac.uk/documents/firstyear/FirstYear_TransformingAssess.pdf
Nicol, D (2009), Assessment for learner self-regulation: Enhancing achievement in the first year
using learning technologies, Assessment and Evaluation in Higher Education, 34(3), 335-352
Nicol, D (2007), Laying the foundation for lifelong learning: cases studies of technology
supported assessment processes in large first year classes, British Journal of Educational
Technology, 38(4), 668-678
Nicol, D (2007) E-assessment by design: using multiple-choice tests to good effect, Journal of
Further and Higher Education.31(1), 53-64.
Nicol, D. & Milligan, C. (2006), Rethinking technology-supported assessment in relation to the
seven principles of good feedback practice. In C. Bryan and K. Clegg, Innovations in Assessment,
Routledge.
Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A
model and seven principles of good feedback practice, Studies in Higher Education, 31(2), 199218.
See www.reap.ac.uk/resources.aspx for copies.