Transcript Course Review on Assessment Professor Brenda Smith THE TESTA MODEL – FOR ENHANCEMENT 10 steps to auditing assessment across a programme • It is intended to.

Course Review on
Assessment
Professor Brenda Smith
THE TESTA MODEL – FOR
ENHANCEMENT
10 steps to auditing assessment across a
programme
• It is intended to map assessment patterns across a
programme
• It looks back at modules over a 3 year degree
course to chart a typical students experience
TESTA: Transforming the experience of
students through assessment
• Funded by the National Teaching Fellowship
Scheme
• Seeks to develop course assessment
interventions and to evaluate them
• Involves 4 universities and led by Graham Gibbs
TESTA Model
It maps:
• Volume of summative & formative assessment
• Variety of assessment methods
• Percentage from coursework and exams
• Volume of written and oral feedback
• How criteria contribute to student clarity about goals
and standards
Caveats…
• Gives an overview from a course leader or course
team perspective
• There is often a gap between documentation &
the reality of the student experience
• It works best when triangulated with other data
like focus groups with students and the
Assessment Experience Questionnaire (AEQ)
TEN STEPS - STEP 1
Select a named programme
What is the typical student cohort
• Number of students in each year
• What is their previous educational experience?
• What are students career aspirations?
Year 1
Year 2
Year 3
STEP 2
• How many modules are there in the first year of the
programme?
• What is their credit weighting?
20
Credits
40
Credits
Business Studies
Year 1
•
•
•
•
•
•
Management Accounting
Contract Law
Foundations of Marketing
Economics for Business
Managing People
Quantitative Business Methods
STEP 3
Describe the nature of assessment in each
module
• Type of assessment
• Is it formative and summative?
Now tally the summative, formative and
variety of assessment methods
STEP 4
• Repeat for the second and third years
• Tally the number of summative, formative and the
variety
• Tally the number of assessments by exam, and
work out the relative percentage of course work
to exams
STEP 5
• For each assessed task estimate how long it takes
from the submission of an assessed task to the
return of feedback and marks
• Are feedback and marks separated?
• What is the system for the collection of assessed
work?
• Is certain feedback and the mark returned faster
than others? E.g presentations
STEP 5 continued….
• Are feedback and marks released as provisional?
• Is formative feedback returned more quickly than
summative feedback?
• Are there different variables e.g. 80% of written
work is returned after 21 days, & 20% is
presentation, returned within 1 day?
• WORK OUT THE OVERALL RETURN FACTOR
STEP 6
Work out as best you can how much oral feedback a student is
likely to receive each year and over the degree
• Tutorial provision before & after assessment points, plus
estimated take up of these % wise & length of tutorial
• You may need to explore how assessment is handed back
• May involve looking at numbers of students called back after
a failed submission
• May involve quantifying proportion of oral feedback given
through digital technology- (MP3 files & smart phones) &
proportion who listen, oral feedback given in whole class
sessions and on placements etc
STEP 7
Look at explicit assessment criteria and learning outcomes in
the programme documentation
• How do written statements appear in mapping from
programme to module to task?
• Is there alignment between tasks & intended outcomes?
• How do students know what ‘good’ is in relation to
assessment tasks
STEP 7 continued…
• Does the programme have a system wide
approach to developing the student ‘nose’ for
what constitutes a good essay, presentation,
poster etc?
• How do feedback processes clarify goals and
standards?
STEP 8
• Using the sequence guide (given as a separate
sheet) map the sequence of different assessment
tasks across modules and levels.
• Describe how assessment articulates across the
programme , feeding forward into the next
assessment, providing practice and allowing
students to develop their performance
STEP 9
Gather together 15-20 cover sheets and if possible scripts with
comments on them from each year – 1st, 2nd & 3rd.
• Count the number of words on the cover sheets and scripts
• Add 1st, 2nd & 3rd year total words
• Divide the total by the number of student cover sheets
• Multiply the average number of words of written feedback
by the total volume of summative feedback
• This is the total volume of written work a student might
expect over the course of a 3 year programme
STEP 10
• Put all the data together in a document
• Now working with the course team what are you really
pleased about?
• What might you want to do differently?
• How might you run some student focus groups to
triangulate this information?
Ref: De Tansy Jessop, TESTA
Table 2 Range of characteristics of assessment environments
found in different degree programmes
Characteristic of assessment environment
Minimum
Maximum
Percentage of degree marks derived from examinations
17%
100%
Percentage of degree marks derived from coursework
0%
83%
Total number of times work marked per student
11
95
Variety of assessment methods
1
18
Total number of formative-only assessments per student
2
134
Total number of words of written feedback per student
2,700
10,350
Total number of hours of oral feedback per student
3
68
1
28
Average number of days between submission of
assignment and feedback
REF: Graham Gibbs “Using assessment to support student learning”
Leeds Metropolitan University
Positive student
learning responses
The following programme level assessment characteristics have been
found to be associated with a variety of positive student learning
responses:
• a high volume of formative-only assessment
• a high volume of oral feedback
• timely feedback
… and to a lesser extent:
• a high volume of written feedback
• a low proportion of marks from coursework
Negative student learning
responses
The following programme level assessment characteristics
have been found to be associated with a variety of
negative student learning responses:
• a high volume of summative assessment
• a wide variety of types of assessment
• highly explicit criteria and standards
• highly ‘aligned’ assessment (with different assessments
and criteria associated with each learning outcome)
Figure 1: A student’s estimate of his weekly study effort
(in hours) on a programme with no assignments and an exam
in week 12. Total study effort = 68 hours
Figure 2: A student’s estimate of her weekly study effort (in
hours) on a programme with three assignments due in weeks
4, 7 and 10 and an exam in week 12. Total study effort = 78 hours
Ref: Using assessment to support student learning (2010)
By Graham Gibbs ISBN 978-1-907240-06-5
TESTA Programme Portrait
Wombledown University, Creative Arts Degree (CAD)
TESTA Programme Portrait
Wombledown University, Creative Arts Degree (CAD)
• On average, a CAD student received 214 words of feedback
on each summative assessment, low oral feedback from
tutors, and no formative assessment was formalised in the
documentation. The programme was high in variety, and it
was the quickest on return of assessed work
Focus Group Data
USE OF FEEDBACK
•
•
•
•
•
•
•
Students use feedback to feed-forward
For some module feedback does not feed forward
Peer feedback is highly valued & integral to the program
It takes time and practice to give good feedback
Low stakes, informal feedback is valued
Comments without marks increase use of feedback
Marks without feedback diminish use of feedback