Transcript The seven principles pf goodfeedback: supporting student

Re-engineering Assessment Practices
[REAP] in Higher Education
David Nicol, Project Director, REAP
Deputy-Director (Research & Development in e-learning)
Centre for Academic Practice and Learning Enhancement
(CAPLE), University of Strathclyde
Jenny Booth, Learning Technology Adviser, REAP.
(www.reap.ac.uk)
CAA Conference, Loughborough, 4th July 2006
Plan
What is the REAP project?
 Case study – assessment redesign in first year class
 Pedagogical focus
 Promoting transformational change in assessment
at institutional level

SFC Transformation programme






Re-engineering Assessment Practices
[REAP] project is one of the six funded
E-learning transformational projects
New approaches to teaching/ learning
Embedding and substitution
Measurable benefits to institution/sector
Partnerships with other HEIs/FEIs
Rationale for the REAP proposal



Assessment – key driver of student learning
Assessment is a major cost in HE: economies of
scale limited
Assessment influences a wide range of
organisational, pedagogical and business
processes in HE
The REAP Project
3 HEIs (Strathclyde, Glasgow Caledonian Business
School, Glasgow University)
 Focus is on large 1st year classes
 Assessment for learner self-regulation
 Range of technologies: online tests, simulations,
discussion boards, e-voting, e-portfolios, peer/feedack
software, admin systems, VLEs, offline-online
 Learning quality and efficiencies
 Outputs: re-design case studies: advice/tools to support
strategic change in institution (transformation), sharing

Case Example
Two recent literature reviews on assessment
(synthesis)
 Case study of first year assessment redesign (drawn
from REAP project)
 Analysis of a case study in relation to assessment
principles
 Discuss issues

First Year: The academic experience
What is important?
 Coping with transition
 Understanding what is required
 Engagement with academic programmes
 Support and feedback
 Experiences of success
 Agents of own learning
 Belief in self (ability) and motivation
 Social dynamics of learning (belonging)
Mantz Yorke (UK) and Vincent Tinto’s (US) research
Gibbs and Simpson (2004)
Literature review:
Gibbs, G. & Simpson, C (2004) Conditions under
which assessment supports students learning,
Learning and Teaching in Higher Education, 1, 3-31.
See: Formative Assessment in Science Teaching
(FAST) project at: http://www.open.ac.uk/science/fdtl/
Gibbs and Simpson (2004)
Assessment tasks [Conditions 1-4]
1. Capture enough study time (in and out of class)
2. Are spread out evenly across timeline of study
3. Lead to productive activity (deep vs surface)
4. Communicate clear and high expectations
Gibbs and Simpson (2004)
Feedback [Conditions 5-11]
5.
Is sufficient (in frequency; detail)
6.
Is provided quickly enough to be useful
7.
Focuses on learning rather than on marks
8.
Is linked to assessment criteria/expected outcomes
9.
Makes sense to students
10. Is received by students and attended to
11. Is acted upon, to improve work and/or learning
Nicol and Macfarlane-Dick (2006)

Literature review:
Nicol, D. & Macfarlane-Dick, D. (2006). Formative
assessment and self-regulated learning: A model and
seven principles of good feedback practice. Studies in
Higher Education, 34 (1), 199-218
See:
 Student Enhanced Learning through Effective
Feedback [SENLEF] project funded by HE Academy
 REAP project: www.reap.ac.uk
Research on Assessment in HE
Teaching/learning paradigm
Constructivist
Transmission
[student-centred]
Assessment paradigm
Transmission
[teacher-centred]
Some key research findings




Students are always engaged in self-assessment/selfregulation of their own learning (Black & Wiliam,
2005). Logically entailed by constructivist ideas
(Winne, 2005)
The act of using teacher feedback implies that selfassessment must be present (Sadler, 1983, Nicol and
Macfarlane-Dick, 2006)
Feedback in HE is being significantly reduced so how
are students still learning?
The question is: how can we scaffold students
learning so they become better at self-regulation?
(Lajoie, 2005)
Scaffolding self regulation: 7 principles of good feedback
1.
2.
3.
4.
5.
6.
7.
Clarify what good performance is (goals, criteria,
standards).
Facilitate reflection and self-assessment in learning
Deliver high quality feedback to students: feedback
that enables students to monitor and self-correct
Encourage peer and tutor dialogue around learning
Encourage positive motivational beliefs & self esteem
through assessment
Provide opportunities to close the feedback loop
Use feedback information to shape teaching
Source: Nicol and Macfarlane-Dick (2006) Formative assessment and self-regulated learning: A model and seven
principles of good feedback practice, Studies in Higher Education
Re-engineering Assessment Practices [REAP]
Psychology Case Study
Context:
 560 first year students
 Mixture of psychology majors (130) and those
taking psychology only for one year (430)
 6 topic areas, 48 lectures, 4 tutorials, 12 practicals
 Assessment; 2 x MCQs (25%), tutorial attendance
(4%), taking part in experiment (5%), essay exam
(66%)
Psychology Case Study
Problems identified with the course:
 Students got no practice in writing skills but
required in the exam
 No feedback except on MCQs (percent correct)
 Didn’t want to increase staff workload
 Wanted to improve overall exam marks
 And standard of entrant to second year
Psychology re-design
Stage 1: Question 1: moderate difficulty (50 words)
Individual response – post it –discuss answer in groups (of 6)
Timed release: model answer to self-evaluate their response
Stage 2: Question 2: difficult (100 words)
Group response – discuss (online) – agree – post response
Model answer released for stage 2
Stage 3: Question 3: complex (300 word essay)
Group response – discuss (online) – agree – post
Model answer released for stage 3
Psychology



Task 1: Define and describe structural encoding,
phonological encoding and semantic encoding.
Provide an example of each construct. (50 words,
individual)
Task 2: Describe the serial position effect and its
two separate components. Discuss the specific
structural components of memory that are
responsible. (100 word, group response)
Task 3: Summarise the ‘stage theory’ of memory.
To what extent does it provide an adequate theory
of memory? (300 word, group response)
Psychology re-design: key features








Writing tasks related to current lecture delivery (on
human memory).
78 volunteers – induction task
Question of increasing difficulty and length of
answer (50 words, 100 words and 300 words)
Moving from individual to group responses.
‘Scaffolding’ through peer discussion
Cognitive growth thro conflict resolution
Comparing work against model answers
Use of assignment/ discussion tools in (WebCT)
Planned developments






Replace half the lectures with discussion tasks
Tasks will become progressively more demanding
within and across topics as the year progresses.
Participation will be compulsory and monitored
Model answers selected from students reponses
Once a final group response has been agreed each
student will submit a copy to VLE (WebCT)
Purpose is to check on participation rates and
ensure that all students have at least ‘had sight’ of
the group response.
Source: Jim Baxter, Psychology, Strathclyde University
The seven feedback principles






Standard format and model answers provide progressive
clarification of expectations (clear goals, principle 1)
Students encouraged to self-assess against model
answer (self-assessment, principle 2)
Online peer discussion aimed at reaching consensus
about response (dialogue, principle 4)
Staged complexity and focus on learning rather marks
(motivation, principle 5)
Repeated cycle of topics and tasks (closing gap,
principle 6)
Tutors can monitor progress and adapt (shaping
teaching, principle 7)
Gibbs & Simpson’s four assessment conditions




The individual and group responses require regular
study out of class (assessments capture sufficient study
time, condition 1)
Tasks are distributed across topics and weeks (are
spread out evenly, condition 2)
The staged questions require progressively deeper levels
of understanding (productive/ deep learning, condition
3)
There are explicit goals and a progressive increase in
challenge (communicates clear and high expectations,
condition 4)
What can we learn from this case study?






Use of a simple technology (discussion board)
Considerable thought gone into the pedagogical
design [which is transferable].
Supported by robust assessment/learning
principles
Coordinated approach within the department
Evaluation showed a lot of supportive
socialisation during induction task
Intention to compare year on year examination
performance.
Bigger Picture




In the early years there is a need to balance
opportunities for self-regulation within organised
academic structures and activities.
Transformation requires whole course redesign
Design for and evaluation of pedagogical
effectiveness.
Transformational change = institutional strategic
level - economic models for first year teaching, (full
economic), cost to change, management of change
and its evaluation (benchmarking e-learning).
Questions and discussion
The context of use of online MCQ tests
P1. Students construct MCQs (creating criteria)
P2. Open book scenario with MCQs or confidence
ratings in MCQs (self-correction/ reflection)
P3. Results of tests discussed in tutorials (enhanced
teacher feedback)
P4. Unique MCQs per student with sharing (peer
dialogue)
P5. Repeated opportunities to take MCQ tests leading
to final summative test (motivating)
P6. Students repeating MCQs (closing loop)
P7. MCQs before lecture (just in time teaching)
Other relevant papers
Nicol, D (accepted for publication), Laying the
foundation for lifelong learning: cases studies of
technology supported assessment processes in large
first year classes, British Journal of Educational
Technology.
Nicol, D. & Milligan, C. (2006), Rethinking
technology-supported assessment in relation to the
seven principles of good feedback practice. In C. Bryan
and K. Clegg, Innovations in Assessment, Routledge.