Transcript Slide 1

Assessment which works:
Enhancing student learning
without killing the tutors
Prof. Sue Bloxham
[email protected]
Some
assessment
challenges
Low satisfaction rates;
More students – more marking;
Anxiety about innovation in
assessment (extra work, QA);
Difficulty engaging students in
formative assessment, peer
assessment.
Focus today on
pragmatic ways of transforming
our assessment practice
Certification
to identify and discriminate
between different levels of
achievement, and
between students;
providing a license to
practice in the case of
professional programmes;
enabling selection of
students for further study
and employment.
This is assessment of learning.
Quality assurance
to provide evidence for relevant
stakeholders (for example, external
examiners, QAA, professional bodies);
to enable them to judge the
appropriateness of standards on the
programme.
This is assessment of learning.
Student Learning
to promote effective learning;
formative and diagnostic;
steering students’ approach to studying;
giving the tutor useful information to inform
changes in teaching strategies.
This is assessment for learning.
Lifelong learning: sustainable assessment
to achieve an understanding of standards;
to learn how to make judgments;
to be able to use criteria;
to be able to tell when you really
understand something.
This is assessment
as learning.
The unbalanced purposes
of assessment
Assessment for
and as learning
Certification & QA
Assessment for and as
learning can:
Be immediate;
Regular;
In-class;
On-line;
Involve students in their
own assessment ;
Uses peer assessment.
Quick, cheap & low stakes
Assessment for
certification & QA
involves:
Formal feedback;
2nd marking;
Moderation;
External examiners;
Assessment boards.
Slow, costly & high
stakes
Characteristics of assessment which
promote learning and employability
•
•
•
•
•
•
•
•
•
•
Formative, involving dialogue;
Demands higher order learning;
Learning and assessment are integrated;
Students are involved in assessment;
It promotes thinking about the learning process;
Assessment expectations should be made clear;
Involves active engagement of students, developing
independent learning;
Tasks should be authentic and involve choice;
Tasks align with important learning outcomes;
Assessment should be used to evaluate teaching.
Exams & tests
•
•
•
•
•
•
They often come at the
end of a course;
They are not integrated
into the learning;
The criteria are oblique;
Tend to assess factual &
conceptual knowledge
only.
They rarely result in useful feedback (sometime usefully
programmed into a test);
When students do badly we tend to blame the students
rather than use the results to diagnose problems with our
teaching.
Essays
have the potential to meet
many characteristics but:
• Often no formative element;
• Questions may ask students to ‘evaluate’ or
‘critically assess’ a topic but if students can pass
adequately by regurgitating others’ evaluation or
criticism (from lectures or reading), they may avoid
higher order learning;
• Students not involved in assessment or thinking
about learning;
• Rarely authentic;
• Criteria often oblique.
Discussion
• Briefly discuss current practice with those sitting
near you:
• How well balanced are the different purposes
of assessment?
• How much are students engaged in their
learning?
• How satisfied are students with assessment
and feedback?
Psychology Redesign
e.g.’Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of
personality. Are the theories incompatible?
• Guidance provided for tackling the question and
working in a group;
• Best essays posted on VLE as feedback;
• Students used familiar language to discuss
academic concepts – Dialogue.
Nicol 2008
13
• 560 students in groups of 7-8;
• 3 week cycle culminating in 700-800 word essay
Field-based enquiry
•
•
•
•
•
integrated with the
learning
encourages independent
and active learning
involves students in the
assessment process
(avoiding grades in the
early stages)
higher order skills,
complexity
authenticity, choice
•
•
•
includes formative stages,
students can get help and
feedback in a low stakes
way
Expectations available to
students both in written
criteria and embedded in
feedback
Potential to integrate
learning from university
with learning from other
contexts
Peer assessment in lab reports
• Formative
• Students develop understanding of quality in analysing and
reporting science
• Higher order learning, focus on the science rather than a
description of the process
• Students involved in assessment, developing skills of
evaluation
Assessment tasks for engagement
& independent learning
• Patchwork texts (students have regular tasks which they bring to
class for formative feedback – tutor marks a final, reflective
synthesis of these tasks – see P. Ovens for example in Science
education)
• Tutor sets readings with questions each week. Students bring draft
answers for discussion with peers to class, groups can ask the tutor
if they are stuck, then individuals submit revised answers (4-500
wds). Tutor posts generic feedback on VLE. Portfolio of final answers
submitted. Tutor only marks week 4, 7, 9 for feedback & grade
(selects the weeks after submission).
• Students submit draft assignments on line for anonymous peer
assessment. Students submit revised assignment with copies of
their reviews and statement on how they responded to their
reviewers (Liverpool example).
Key feature is that the students see this formative work as directly
contributing to summative work. Don’t call it formative!!
Caution
• Multiple methods have been associated
with negative learning outcomes for
students
• Extra workload for students (perhaps this
is a good thing?)
Engaging students in formative
assessment – key requirements
• It clearly feeds into summative assessment
tasks;
• The students must submit it in some way
(bring to class, post on line, hand it in) and
action is taken if they don’t;
• Students receive useful feedback
on it;
• It is not contaminated by
summative purposes.
Selling peer assessment
Evidence shows students find their peers a useful and more
approachable source of help with assignments but we need to
stress the main value in peer assessment is standing in the
shoes of the assessor – not being assessed – because:
• learning about standards – absolutely crucial to making
progress and understanding feedback
• Seeing other ways of going about the task – develops
strategies for taking action
• Key employability skill – being able to judge own performance
and assessing and giving feedback to others
• More opportunity for dialogue
• Chance for more formative feedback
Peer assessment needs to become a regular feature of programmes so that it is
taken seriously and taken for granted as part of learning at this level.
Staff work load
A heavy staff workload in assessment is
not necessarily helpful to student
learning. Improving assessment needs
to be accompanied by a shift in effort
from tutors to students:
• more student activity and engagement
(doing tasks, reviewing progress, selfregulation) and less staff activity
(marking)
• Formative assessment has the
potential to achieve both of these and
is most likely to increase achievement.
Tutor friendly formative
feedback
• Tutor posts good examples or model answers on
VLE;
• Students peer assess tasks using assessment
criteria;
• Tutors give feedback on posters, presentations in
class;
• Oral/ on-line feedback to group after marking a
sample
• Tasks done on-line (e.g EMQs, MCQs), auto
marked and give immediate feedback;
• Tutors put main effort into marking drafts (agreed
with examiner), just checking for change and
putting mark on final piece.
In-class activity
Out of class activity
Example of module-level approaches:
the use of exemplars annotated with feedback to
encourage dialogue about assessment criteria
Students
write and
submit
individual
assignments
Tutor leads
discussion of
exemplars
previously
marked and
annotated
with
feedback
Module timeline
From Oxford Brookes
FDTL project on
feedback
Tutor
assesses
assignments and
prepares
feedback
Tutor hands
back
assignments
and leads
discussion on
feedback
Submission point
22
From Oxford Brookes
FDTL project on
feedback
1.
Students
draft &
submit
individual
assignments
In-class activity
Out of class activity
Example of module-level approaches:
Generic (non-personalised) feedback on drafts
plus reflective commentary
2. Tutor marks
sample of
assignments
and prepares
generic
feedback
4. Students rewrite
and submit
assignments with
reflective
commentary on
how they have
incorporated the
generic feedback
5. Tutor
assesses
assignments
6. Tutor
hands back
assignments
with minimal
formative
feedback
3. In-class
discussion of
generic
feedback
Module timeline
Submission point
23
Conclusion
Addressing student dissatisfaction and staff workloads requires
us to:
- Balance the different purposes of assessment;
- Encourage and reward student engagement;
- Ensure that students’ study efforts are directed towards
meaningful learning;
- Provide sufficient low stakes, informal assessment and
feedback for students to grasp both the standards required
and the strategies needed to achieve them.
- Use tutor time most effectively for learning and limit the time
absorbed by QA to essential summative items;
Additional slides which
delegates may find useful
Prof. Sue Bloxham
[email protected]
Dealing with concerns regarding
assessment which isn’t so easily quality
assured for certification
For example, peer, group and self assessment, in class assessment,
presentations, feedback at the draft stage
Potential ways forward
• Take a programme approach to assessment design
• Make greater use of assessment methods which combine
different purposes (e.g. inquiry tasks,)
• Use peer & self assessment methods in low stakes ways,
e.g. for formative assessment and feedback.
• Make sure you have a robust method for dividing group
marks (See University assessment guide)
Benefits to students of moving to inclass, on-line, ongoing assessment and
feedback
• Immediate feedback
• More feedback
• Assessment & teaching/learning are
integrated
• Students involved in assessment – gaining
better understanding of standards and own
performance
• Potential for greater student engagement
throughout modules
• More independent study
• integration with work
experience
• Raise expectations regarding
study workload
Quicker, cheaper and low stakes
Feedback from different sources
collaborative tasks
case discussions
team assessment
peer assessment
self feedback
work-based mentors, etc
These methods allow students to check out their understanding and
practice judgements in relative safety. Seeing how other students go
about things is a key source of feedback on their own work and helps
them find ways to fill the ‘gap’ between current performance/
knowledge and what is expected.
28
Feedback
1. Be clear about the purpose of the feedback ie what
should students expect to get from it in particular
instances.
2. Ensure students have sufficient support in
understanding criteria and standards to relate to the
feedback provided.
3. Require students to self assess against generic/cohort
feedback.
4. Model the process of feeding forward for the students.
5. Require students to demonstrate how they have used
feedback in subsequent work.
Some other assessment methods
•
•
•
•
•
•
•
•
•
Evaluation of journal article or other paper
Film or radio programmes about a specific topic
Research Grant applications
Lay commentary on specialist material, e.g. journal article
Poster – presenting information clearly & concisely
Presentation – oral communication
Problems and case study analysis
Reflective Journals, Diaries & learning logs
Writing tasks: newspaper articles, press releases, executive
summaries, information sheets.
• Wikipedia entry
Shorter pieces can demand just as much student time but
involve significantly less staff marking time.