Transcript Inside the black box: Raising standards through classroom

Quality in formative
assessment
Symposium at the 2008 meeting of AERA, New
York, NY
Discussant: Dylan Wiliam
www.dylanwiliam.net
Formative assessment
Formative assessment involves the creation of, and the
capitalization upon, moments of contingency in instruction
What gets formed?
Long-cycle
 Span: across units, terms
 Length: four weeks to one year
 Impact: Student monitoring; curriculum alignment
Medium-cycle
 Span: within and between teaching units
 Length: one to four weeks
 Impact: Improved, student-involved, assessment; teacher cognition about learning; before endof-unit adjustment to instruction.
Short-cycle
 Span: within and between lessons
 Length:
 day-by-day: 24 to 48 hours
 minute-by-minute: 5 seconds to 2 hours
 Impact: classroom practice; student engagement
For whom?
Individual students
Groups of students on whom data were collected
Other groups of students
Theory of action
Evidence of student learning is elicited (by teachers, peers, or by the
learners themsleves)
This evidence is intepreted in terms of a theory of learning
Decisions about next steps in instruction are taken that are either better,
or better founded, than the decisions they would have taken in the
absence of the evidence that was elicited
Competing definitions of quality
What is effective formative assessment?
Data is generated for the purpose of improving instruction (intention)
Data is generated that is likely to improve instruction (prediction)
Under any conditions
Under commonly encountered conditions
Under specified conditions
Data is generated that actually improves instruction (action)
Evidence collection
Kinds of assessment in support of learning
Monitoring
Whether the required learning is taking place
Diagnostic
When the required learning is not taking place, what is not being learned
Formative
When the required learning is not taking place, what needs to be done to
improve the situation
Evidence collection must be driven by a theory of learning
Learning progressions
“Unpacking” state standards
Horizontal unpacking—operationalization for testing
Vertical unpacking—operationalization for learning
Learning progressions
Are rarely independent of curriculum
Require both empirical and theoretical rationales
The role of content knowledge
Knowing what the correct answer is
Knowing what incorrect answers indicate
Knowing what instructional activities are likely to move learning on
Knowing what issues are worth bothering about
Specific comments
Herman & Choi
 An important existence proof
 A particularly well understood, and curriculum-independent, domain
 How much accuracy is needed?
Heritage, Kim & Vendlinski
 Importance of teacher x principle x task effects
 Planning next instruction is hard
 Requires an “anatomy of quality”
Wylie & Ciofalo
 Competing priorities
 Minimizing false positives
 Interpretability of true negatives