Developing an effective assessment strategy

Peter Hartley, Professor of Education Development University of Bradford [email protected]

2

This session

• introduction • what’s the problem?

• introduce current projects and initial outcomes: the PASS and TESTA projects • identify important issues, suggestions and implications

3

Introduction

• myself • good and bad news!

4

Introduction

• myself • good and bad news!

– The bad news – the jury is out!

– The good news – growing evidence and examples from new projects and initiatives with the opportunity to build on and extend others’ experience

5

What’s the problem? #1

• Consider the issues buried in the title of this paper – the implications of ‘developing’ – effective ‘for whom’?

– clarifying the purposes of ‘assessment’ – the notion of ‘strategy’

What’s the problem? #2

6 • See the PASS Issues Paper – Please comment/feedback and use • Would highlight: – Assessment ‘drives and channels’ – What/why are we measuring: the ‘slowly learnt’ – Limitations of grading systems (e.g. marks are not numbers) – Implications for course structures/regulations

7

Introducing the projects PASS project #1

• NTFS group project over 3 years – One year of development and investigation and two years of implementation • Consortium – Led by Bradford – 2 CETLs – ASKE and AfL – Plus Exeter, Plymouth and Leeds Met.

8

PASS project #2

• Outcomes and outputs – approaches to PBA – ‘choice and consequence’ guides – workshop and resources for local implementation – case studies from different disciplines

9

TESTA project , #1

• NTFS group project with 4 partner universities • ‘aims to improve the quality of student learning through addressing programme level assessment.’

10

TESTA project # 2

• starting from audit of current practice, so far looked at nine programmes • surveyed students using focus groups and AEQ – Assessment Experience Questionnaire – Graham Gibbs et al • also using tool to identify programme level ‘assessment environments’ (Gibbs)

11

Issues to disentangle include:

• Defining assessment • Assessment environments and their impact • Defining ‘programme-based’ assessment • Student perceptions and expectations • The need for a strategic approach • Grading and credit

12

Defining assessment: a challenge

– program outcomes “need to be assessed in complex, multidimensional student performances” – “Multidimensional performance entails the whole dynamic nexus of the individual’s intentions, thoughts, feelings, and construals in a dynamic line of action and his or her entanglement in an evolving situation and its broader context. Such a context may be within or across work, family, civic, or other settings.” – (Rogers, Mentkowski, & Reisetter Hart, 2006, p. 498).

13

Assessment environment and impact

• Interim findings from TESTA – variety of assessments can cause problems – Issues over understanding assessment criteria, marker variation, and feedback – variation across programmes – QA ‘myths and traditions’ can get in the way

14 Starting to define PBA

15

Typical student perceptions and concerns (based on PASS)

• perceptions of ‘the course’ are variable.

• assessment experienced as ‘fragmented’.

• anxieties re move to more integrated assessment – perceived risk in terms of performance.

• Concerns about feedback and timing.

16

The need for strategy

• An example finding from Gibbs – ‘greater explicitness of goals and standards was not associated with students experiencing the goals and standards to be clearer’ • And what did make a difference?

The need for strategy

17 • An example finding from Gibbs – ‘greater explicitness of goals and standards was not associated with students experiencing the goals and standards to be clearer’ • And what did make a difference?

• Formative-only assessment • More oral feedback • Students ‘came to understand standards through many cycles of practice and feedback’

18

An example: Peninsula Medical School

• NB Case study forthcoming from PASS • Includes: • four assessment modules that run through the 5 year undergraduate medical programme and are not linked directly to specific areas of teaching • focus on high-quality learning (Mattick and Knight, 2007)

19

Issues re grading and credit

• Teaching/assessment links • The assessment/credit link • Credit cf accreditation • Threshold cf grades in performance

20

And back to ‘effective assessment strategy’

21

And back to ‘effective assessment strategy’

• Will it explain to staff, students and external agencies: – How the course/programme assesses the main outcomes?

– How assessment and teaching are linked?

– How assessment both supports ‘high-quality learning’ and develops it over the course?

22

And finally …

• Other initiatives and events, e.g.

– EARLI conference later this year – Growth of work on feedback (e.g. use of audio as in ASEL)

23

And absolutely finally …

Contacts for PASS • The project website • Project Director: Peter Hartley [email protected]

• Project Manager: Ruth Whitfield [email protected]