Assessment

Lecture for PGCE September 2012 Prof Robert Coe School of Education Tel: (+44 / 0) 191 334 4184 Fax: (+44 / 0) 191 334 4180 E-mail: [email protected]

Definition of a grade

‘An inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material.’

2 Dressell (1983)

Assessment

    What different types of assessment are there?

Problematising assessment: assessment criteria; assessment for accountability; teacher assessment Why does assessment matter?

How can teachers use assessment to promote learning?

3

Background: Types of assessment and their problems

© 2010 Robert Coe, Durham University 4

Assessment types

   Functions of assessment  To certify achievement (summative)   To inform learning (formative) To control curriculum, teachers, students (accountability) Types of assessment  Formal / informal   Criterion / norm / self referenced Continuous / final / progressive Questions of quality  Validity: does it test what you want students to learn?

 Reliability: would it give the same result again?

5

Criterion-referenced assessment

  Define what you want learners to be able to do Assess whether they can 6 50 59 Approp riate referen cing and citation details.

Good in terms of structure, compre hensibility and quality of presen tation.

Good level of knowledge and understanding in terms of central ideas and concepts within the field of education theory, policy and context with appropriate supporting factual detail allied to an ability to make explicit the relationship of the ideas to the wider field of educational knowledge. A relevant line of argument is presented and the relationship of this to the wider field of educational knowledge is made explicit. Demonstrat es a critical understandi ng of the appropriaten ess of educational* research and its relationship to the topic. The theoretical justification for practice and the practical implications of theoretical ideas are discussed as are some of the dilemmas which arise from the relationship of educational theory to practice.

Evidence of sufficient reading focussing on a range of educational source material [e.g. government reports, research articles, books, personal experience etc.]  ‘Add numbers up to 20’   11+3 3+11  Context is everything!

Teaching to the test: the ‘Texas miracle’

7 Klein et al, 2000 © 2010 Robert Coe, Durham University

Teacher assessment

 Pros     Based on sustained performance Based on assessment of authentic learning tasks (validity) Incorporates a range of tasks Motivates students  Cons      Judgements often have low reliability Hard to translate into absolute standards Halo effects Teacher workload Problems if teachers are judged by results 8 © 2010 Robert Coe, Durham University

Rising standards

Performance of England in international surveys

560 550 540 Science (age 14, TIMSS) 530 520 Science (age 10, TIMSS) 510 Maths (age 14, TIMSS) 500 490 480 Maths (age 10, TIMSS) Reading (age 11, PIRLS) 25 point rise in PISA = +£4,000,000,000,000 GDP Scientific literacy (age 15, PISA) Reading literacy (age 15, PISA) Mathematical literacy (age 15, PISA)

Year

Assessment: Perhaps the single most important determinant of learning

11 © 2010 Robert Coe, Durham University

What factors influence achievement most?

12   John Hattie and his team have analysed data from all available studies on the effects of different strategies    From 50,000 studies Involving over 200,000,000 students Compiled in over 750 meta-analyses Which are most promising?

    Personalised learning?

Smaller classes?

Ability grouping?

Better headteacher?

0.20

0.21

0.25

0.30

© 2010 Robert Coe, Durham University

Assessment… Influence

Self-report grades Feedback Providing formative evaluation to teachers Frequent/ Effects of testing Teaching test taking skills

Effect Size

1.44

.72

.70

.46

.22

13 © 2010 Robert Coe, Durham University

www.suttontrust.com

‘Best buy’ strategies

10 Promising Feedback Meta-cognitive May be worth it Peer tutoring Homework 0 AfL £0 Learning styles Individualised learning Arts Ability grouping Summer schools Parental involvement Sports Performance pay Cost per pupil ICT After school Pre-school 1-1 tutoring Smaller classes No way £1000 Teaching assistants

“… we have each been asked several times by teachers, ‘What makes for good feedback?’—a question to which, at first, we had no good answer. Over the course of two or three years, we have evolved a simple answer—good feedback causes thinking.” (Black & Wiliam, 2003) © 2010 Robert Coe, Durham University 16

Focus on learning

 Does your ‘theory of learning’ explain why     ability grouping (setting) after-school clubs teaching assistants smaller classes do not work?

17 © 2011 Robert Coe, Durham University

Do we care about learning?

  Which of the following are evidence of learning?

   Students are busy: lots of work is done Students are engaged, interested, motivated Classroom is ordered, calm, under control What do school students value most?

    Social interactions & status with peers Keeping out of trouble Pleasing teachers: good marks, neat writing, polite Thinking hard about really challenging problems 18 © 2011 Robert Coe, Durham University

Assessment can help focus on learning

    How can you know what your students are learning?

How often do they need to think hard?

Learning is invisible, and is low on the agenda of many teachers and students If you want to evaluate what students are learning, you need appropriate tools to help you 19 © 2010 Robert Coe, Durham University

If you want your students to learn something difficult …

   You need to know how many of them have ‘got it’ They need to know whether they have ‘got it’ If they haven’t, you need to be able to do something about it 20 © 2010 Robert Coe, Durham University

Formative assessment

“Information used as feedback to modify teaching and learning” 21  Effect sizes between 0.4 and 0.7

 Advice to improve formative assessment:      Feedback should focus on work, with advice for improvement, not comparisons with others Pupils should be trained in self-assessment - to understand learning aims Teaching should allow pupils to express their understanding Create reflective dialogue between teacher and pupil Tests should be frequent and relevant; feedback and support to improve should be given Black and Wiliam, 1998 © 2010 Robert Coe, Durham University

Formative assessment in practice

     Sharing success criteria with learners Classroom questioning Feedback through marking, eg comment-only Peer-assessment and self-assessment Formative use of summative tests 22 Black et al, 2002 © 2010 Robert Coe, Durham University

Comments: instead or as well?

Comments group Comment and grade group Failed to recall Failed to recall Recalled comment (5%) 23 Recalled grade and comment (23%) Recalled grade (53%) Recalled comment (84%) © 2010 Robert Coe, Durham University Butler, 1988

“If you are going to grade or mark a piece of work, you are wasting your time writing careful diagnostic comments.”

24 Wiliam, 1999 © 2010 Robert Coe, Durham University

One evaluation of comment-only marking

“I would prefer to be in another form because we don’t get our test marks back” 25 © 2010 Robert Coe, Durham University Smith and Gorard, 2005

Which feedback is best?

a) Well done, that is very good work b) Well done, that is very good work (for you) c) Well done, you have shown a high level of ability in solving these problems d) Well done, that is one of the best essays in the class e) Well done, that is a big improvement on your previous work f) Well done, I can see you have worked hard at this g) Well done, I can see you have concentrated on reading the questions carefully and it has paid off © 2011 Robert Coe, Durham University 26

Feedback does not always enhance performance

Ideally, feedback should …      relate performance to clear, specific and challenging goals make people focus on the task, not themselves, nor compare them with others compare their performance with performance their own past be seen as informative, not controlling make people feel competent, but not complacent 27 Coe, 1998 © 2010 Robert Coe, Durham University

28

Feedback does not always enhance performance

Ideally, feedback should …      suggest reasons for performance that are alterable (eg effort, strategies) follow soon after performance be specific and task focused be given to individuals on their individual performance correct errors or inadequacies © 2010 Robert Coe, Durham University Coe, 1998

Further advice on assessment

29       Feedback should relate to individuals’ (mis)understandings Praise should be infrequent, credible, contingent, specific and genuine Praise should be related to factors within an individual’s control Don’t help too much: ‘minimal intervention’ Feedback should support a view of ability as incremental rather than fixed Students often don’t understand what they are meant to be doing and how it is assessed Wiliam, 1999, 2000 © 2010 Robert Coe, Durham University