Research on formative assessment practices

2013 AERA Classroom Assessment SIG Symposium: Advancing Research in Classroom Assessment Dylan Wiliam

Section 3: Formative assessment

2

 Four chapters  Formative and summative aspects of assessment: Theoretical and research foundations in the context of pedagogy (Black)  Gathering evidence of student understanding (Heritage)  Feedback and instructional correctives (Wiliam)  Examining formative feedback in the classroom context: New research perspectives (Ruiz-Primo & Li)

Formative and summative aspects of assessment: Theoretical and research foundations in the context of pedagogy

Paul Black

Formative and summative assessment

4

 A simple model of instruction  Clear aims  Planning activities  Interaction  Review of the learning  Summing up  Regulation of learning processes  Proactive  Interactive  Retroactive Tensions arise between summative and formative functions, especially in terms of instruments, methods, and agents

The role of teachers

5

 Teachers’ exclusion from summative functions of assessment weakens the kinds of inferences that can be made  But, inclusion of teachers brings its own issues  Poor construct definition makes assessment design more of a discretionary process than it should be  Teachers model their assessments on tests  “Summative drives out formative”  Alignment of curriculum, assessment and instruction  Case studies (Australia, England, Scotland, Sweden)

Gathering evidence of student understanding

Margaret Heritage

Eliciting evidence

7

 Two fundamental purposes of assessment  provide information on current achievement  inform future instruction

Sources and quality of evidence

8

  Sources of evidence  What learners say, write, make, or do (Griffin 2007)  Interactions (questions, discussions)  Tasks  Directed (model-eliciting activities)  Partially directed (activities)  Undirected (observations) Quality of evidence  Construct underrepresentation  Construct-irrelevant variance  systematic  random (i.e., unreliability)

Learning progressions

9

 “What gets better when someone gets better”  Two key aspects 1.

2.

progressions lay out in successive steps, increasingly more sophisticated understandings of core concepts and principles in a domain, and progressions describe typical development over an extended period of time.

 Two approaches to their development  Top-down (theoretical)  Bottom-up (empirical)

Feedback and instructional correctives

Dylan Wiliam

Reviews of research on feedback

11

        Fuchs & Fuchs (1986) Natriello (1987) Crooks (1988) Bangert-Drowns et al. (1991) Dempster (1991, 1992) Elshout-Mohr (1994) Kluger & DeNisi (1996) Black & Wiliam (1998)         Nyquist (2003) Brookhart (2004) Allal & Lopez (2005) Köller (2005) Brookhart (2007) Wiliam (2007) Hattie & Timperley (2007) Shute (2008)

12

Feedback: an evolving concept (Brookhart, 2007)

Conceptualization

 Information about the learning process…  … that teachers can use for instructional decisions…  …and students can use to improve performance…  …which motivates students

Source(s)

 Scriven (1967)  Bloom, Hastings and Madaus (1971)  Sadler (1983; 1989)  Natriello (1987); Crooks (1988); Black and Wiliam (1998); Brookhart (1997)

Back to the future

13

“These considerations of utility and alternative interventions suggest that even [a feedback intervention (FI)] with demonstrated positive effects on performance should not be administered whenever possible. Rather, additional development of [feedback intervention theory] is needed to establish the circumstance under which positive FI effects on performance are also lasting and efficient and when these effects are transient and have questionable utility. This research must focus on the processes induced by FIs and not on the general question of whether FIs improve performance—look at how little progress 90 years of attempts to answer the latter question have yielded.” (Kluger & DeNisi, 1996 p. 278)

Dual pathway theory (Boekaerts)

14

“It is assumed that students who are invited to participate in a learning activity use three sources of information to form a mental representation of the task-in-context and to appraise it: (1) current perceptions of the task and the physical, social, and instructional context within which it is embedded; (2) activated domain-specific knowledge and (meta)cognitive strategies related to the task; and (3) motivational beliefs, including domain-specific capacity, interest and effort beliefs.” (Boekaerts, 2006 p. 349)

15

 As a result of the appraisal, the student activates energy and attention along one of two pathways  the growth pathway (increasing competence)  the well- being pathway (prevent harm, threat or loss)  Integration of other theories  Mindset (Dweck, 2000)  Mastery and performance goals (Dweck, 2000)  Interest (Hidi & Harackiewicz, 2000)  Self-regulated learning (Deci & Ryan, 1994)

Examining formative feedback in the classroom context: New research perspectives

Maria Araceli Ruiz-Primo and Min Li

Formative assessment in the classroom

17

 Updated review of research on feedback in classroom learning  Vast majority of studies are poorly designed  No control groups  Feedback effects evaluated in the instructional session  No evidence of pedagogical orientation (e.g., orientations to the purpose of feedback, or how it could be used)

A new, expanded, notion of feedback

18

 Feedback should:   Be seen as a process guided by the learning goals towards which the teacher and students work Actively involve students in the process      Be considered as an instructional scaffold that goes beyond written or oral comments.

Be specifically intended to improve learning outcomes Ensure its usefulness by making feedback accessible and practical. Consider different sources of information Demonstrate, over time, alignment with a learning trajectory

Future priorities

19

 Defining feedback  Understanding variability in feedback practices  Understanding feedback in use  Reviewing feedback research from new theoretical understandings

Major unresolved questions

20

 Can teachers be involved in the summative assessment of their students without compromising the potential of formative assessment to improve achievement?

 How can learning progressions be developed?

 How can we better theorize the impact of feedback interventions?