Transcript Document

Technology-enhanced assessment and feedback: from challenge to change
Overview
 Assessment and feedback challenges
 Technology-enhanced assessment, some examples
 From challenge to change
 Resources
20/07/2015
Changing the Learning Landscape
slide 2
Assessment and feedback challenges
Section subtitle
20/07/2015
Changing the Learning Landscape
slide 3
Assessment and feedback: challenges
What areas are uppermost in your mind as challenges relating to
assessment and feedback in your context?
20/07/2015
Changing the Learning Landscape
slide 4
What the programme is telling us
 A snapshot of the state of play at 10 UK institutions, working
with eight institutional change projects across the UK
www.jisc.ac.uk/assessmentandfeedback
 Why baseline?
– Programme and project level
 Sources of evidence
– Strategy documents, QAA reports, rich pictures,
questionnaires and surveys, course evaluations.....
20/07/2015
Changing the Learning Landscape
slide 5
Issues: strategy and policy
 Formal strategy/policy documents lag behind current thinking
 Educational principles are rarely enshrined in strategy/policy
 Devolved responsibility makes it difficult to achieve parity of
learner experience
20/07/2015
Changing the Learning Landscape
slide 6
Issue: stakeholder engagement
 Learners are not often actively engaged in developing practice
 Assessment and feedback practice does not reflect the reality
of working life
 Administrative staff are often left out of the dialogue
20/07/2015
Changing the Learning Landscape
slide 7
Issue: assessment and feedback practice
 Traditional forms such as essays/exams still predominate
 Timeliness of feedback is an issue
 Curriculum design issues inhibit longitudinal development
20/07/2015
Changing the Learning Landscape
slide 8
Technology-enhanced assessment and feedback
20/07/2015
Changing the Learning Landscape
slide 9
Technology-enhanced assessment and feedback
‘The wide range of ways in which technology
can be used to support assessment and
feedback.’
These technologies may be generic (such as VLEs, wikis,
podcasts, e-portfolio systems) or purpose-built (such as onscreen assessment systems and tools to support peer review)
The focus is therefore broader than was has traditionally been
described as e-assessment
‘Technology doesn’t solve everything but it enables me to do
something about it’
Prof. Mark Russell, University of Hertfordshire
20/07/2015
Changing the Learning Landscape
slide 10
Technology to support......
20/07/2015
Changing the Learning Landscape
slide 11
Challenge
Engaging learners with feedback, and providing
opportunities for ‘feedforward’
20/07/2015
Changing the Learning Landscape
slide 12
Audio feedback
 Reflections from audio feedback projects
– All suggest students find benefits over written feedback, more personal,
understandable, and clear. Students were more attentive. Although
one found low performing learners prefer written (disputed elsewhere),
and length can be an issue
– One study found tutors found time taken to be less or the same; another
found all tutors struggled with the admin burden. One found the time
taken was outweighed by the convenience of not repeating key points
– Learners preferred some visual (asynchronous or synchronous) rather
than audio alone
– Type of audio feedback should be tailored to learner type and
assessment type
– Distance learners found audio feedback establishes a relationship with
tutors – changes in tone of voice, intonation and humour help
– Useful when combined with face-to-face follow up to ensure feedforward
20/07/2015
Changing the Learning Landscape
slide 13
DUCKLING
Approach
– Explored podcasts for
the delivery of feedback
 Benefits
‘The results from the pilot action
research showed that podcasting
and voice board trials were the lowest
cost interventions with the highest
impact on the learning. They resulted
in a well-evidenced increase in learner
benefits in terms of personalisation,
flexibility, reduced isolation and
increased engagement, at a marginal cost in tutor time.’
20/07/2015
Changing the Learning Landscape
slide 14
Making Assessment Count
 Approach
 Benefits
– qualitative evidence
from students and staff
‘It has helped I think because since then my marks have shot
up.”’
20/07/2015
Changing the Learning Landscape
slide 15
InterACT
Feedback is.........
http://www.flickr.com/photos/khalidalbaih/5653817859
20/07/2015
Changing the Learning Landscape
slide 16
Challenge
Peer review and assessment
20/07/2015
Changing the Learning Landscape
slide 17
WebPA
 Benefits of peer review
– Process of reviewing another's work is cognitively more challenging than
receiving feedback, prompting deeper learning (Nicol, 2013)
– Students gain a better understanding of the assessment criteria
– Develop key skills such as critical thinking and self-evaluation
– Students reflect on their own assignment as a result
 Peer assessment: approach
– Assessing group work, and in particular individuals contributions to it
can be a challenge
– WebPA – online tool supporting peer assessment of groupwork
• Helps to surface the process as well as the product of learning
20/07/2015
Changing the Learning Landscape
slide 18
Challenge
Employability
20/07/2015
Changing the Learning Landscape
slide 19
Springboard TV
 Approach
– Looked to address issues of motivation and retention on media courses
– Redesigned the curriculum around
production activities for an internet
TV station
– College uses the station as core to
its communication channels and is
Core to its strategy
 Benefits
- ‘88% of learners have told us that being able to upload work to the project
website has led to an improvement in their motivation. They work harder,
show more pride in their work and show an active desire to compete with
one another to improve their work.’
20/07/2015
Changing the Learning Landscape
slide 20
COLLABORATE
 Approach
20/07/2015
Changing the Learning Landscape
slide 21
Challenge
Programme level / holistic assessment
20/07/2015
Changing the Learning Landscape
slide 22
Personally owned, institutionally supported....
‘It has often been reflected upon by staff and students alike that they have the skills
and attributes that employers are seeking but they just don’t know that they have
these skills or attributes, so are unable to articulate or evidence them’
University of Exeter, COLLABORATE
 e-Portfolio (systems)...
– Student-owned spaces which, when used appropriately, can support the
development of reflective, lifelong learners
– Can help prepare learners for employability, both work-competencies
and ‘softer’ transferable skills. Through increased
opportunities for dialogue around, reflection on, and
articulation of skills for a range of purposes
– Provide opportunities for holistic and integrative
approaches to assessment
– Help to surface the process of learning
– E.g. Presentation and evidencing graduate attributes
 See case studies at: www.jisc.ac.uk/eportimplement
20/07/2015
Changing the Learning Landscape
slide 23
Assessment management
20/07/2015
Changing the Learning Landscape
slide 24
Evaluation of assessment management strategies
 Approach
– EBEAM conducted an evaluation of the use of electronic assessment
management strategies
 Benefits
– Students
‘There is strong evidence to suggest that not only is electronic
assessment management their preference, but that they are very
much of the mind that it is their entitlement’
EBEAM draft final report
– Increased control and agency
– Reduced anxiety
– Improved privacy and security
– Increased efficiency and convenience
– Feedback which is clearer and easier to engage
with, understand and store for later use
20/07/2015
Changing the Learning Landscape
slide 25
Evaluation of assessment management strategies
 Key points
– Policies and buy in needed around assessment and feedback
turnaround policies
– Rubrics providing criteria for marks and feedback need to be clear
– Findings suggest the issue is less that students aren’t engaging with
their feedback, and more that students are finding it hard to interpret
– Need to consider how long students have access to marks and
feedback
– Consider supporting staff professional development in the writing of
feedback that supports dialogue and feedforward
– Analytics can be a positive motivator and is worth pursuing
– AND need to remember that for some students, paper-based formats for
assessment and feedback remain important , and need to be considered
20/07/2015
Changing the Learning Landscape
slide 26
Evaluation of the time savings from paperlessness
20/07/2015
Changing the Learning Landscape
slide 27
Summary: what technology offers
 Authenticity
 Improved learning engagement, e.g. Through formative assessments with
adaptive feedback
 Choice in the timing and location of assessments
 Capture of wider skills and attributes not easily assessed by other means
 Surfacing the process of learning, not just the end product
 Efficient submission, marking, moderation and data storage processes
 Consistent, accurate results
 Immediate feedback
 Increased opportunities for learners to act on feedback, e.g. through
reflection in e-portfolios
 Innovative approaches based around use of creative media and online
peer and self-assessment
 Accurate, timely and accessible evidence on the effectiveness of
curriculum design and delivery
20/07/2015
Changing the Learning Landscape
slide 28
From principles to practice
20/07/2015
Changing the Learning Landscape
slide 29
REAP principles of good assessment and feedback
Good assessment and feedback should:
1. Clarify what good performance is (goals, criteria, standards).
2. Facilitate the development of reflection and self-assessment
3.
4.
5.
6.
7.
in learning
Deliver high quality feedback to students: that enables them
to self-correct
Encourage peer and student-teacher dialogue around
learning
Encourage positive motivational beliefs & self esteem
through assessment
Provide opportunities to act on feedback
Provide information to teachers that can be used to help
shape their teaching (making learning visible)
Nicol and Macfarlane-Dick (2006)
20/07/2015
Changing the Learning Landscape
slide 30
Principle-led change
 Viewpoints
approach
20/07/2015
Changing the Learning Landscape
slide 31
Principle-led change
Viewpoints
 Evaluation of the Viewpoints project and its effects at the
University of Ulster
 Workshop process highly successful – ideas have spread
beyond the workshop and across the institution
 Principles established as policy and embedded in quality
procedures
 Model of change extrapolated – ‘principles-based discourse
model’
20/07/2015
Changing the Learning Landscape
slide 32
Principle-led change
 e-Affect, University of Belfast
– Developing a set of principles, with a decision tree showing examples of
how to achieve this, with what technologies
– E.g. ‘Provide opportunities to act on feedback’
20/07/2015
Changing the Learning Landscape
slide 33
Principle-led change
 e-Affect, University of Belfast
– Created assessment designs before and after, colour coded to each
principle:
20/07/2015
Changing the Learning Landscape
slide 34
Resources
20/07/2015
Changing the Learning Landscape
slide 35
http://bitly/jiscdsaf
Follow us
#jiscassess
www.netvibes.com/jiscassess
elearning.jiscinvolve.org
Webinar series
bit.ly/afwebinars
www.youtube.com/jisccdd
© HEFCE 2012
The Higher Education Funding Council for England, on behalf of JISC, permits reuse
of this presentation and its contents under the terms of the Creative Commons
Attribution-Non-Commercial-No Derivative Works 2.0 UK England & Wales Licence.
http://creativecommons.org/licenses/by-nc-nd/2.0/uk
20/07/2015
Changing the Learning Landscape
slide 38