Transcript Presentation to ASCD Conference

Classroom assessment:
minute-by-minute and day-to-day
Session 1249T: ASCD Annual Conference 2007
17 March 2007; Anaheim, CA
Dylan Wiliam,
Institute of Education, University of London
www.dylanwiliam.net
11
Overview of presentation
Why raising achievement is
important
Why investing in teachers is the
answer
Why assessment for learning should
be the focus
Why teacher learning communities
should be the mechanism
How we can put this into practice
22
Raising achievement matters
For individuals
Increased lifetime salary
Improved health
For society
Lower criminal justice costs
Lower health-care costs
Increased economic growth
33
Where’s the solution?
Structure
Small high schools
K-8 schools
Alignment
Curriculum reform
Textbook replacement
Governance
Charter schools
Vouchers
Technology
44
It’s the classroom
Variability at the classroom level
is up to 4 times greater than at
school level
It’s not class size
It’s not the between-class
grouping strategy
It’s not the within-class grouping
strategy
It’s the teacher
55
Teacher quality
 A labor force issue with 2 solutions
 Replace existing teachers with better ones?
No evidence that more pay brings in better teachers
No evidence that there are better teachers out there deterred
by certification requirements
 Improve the effectiveness of existing teachers
The “love the one you’re with” strategy
It can be done
We know how to do it, but at scale? Quickly? Sustainably?
66
Why assessment for learning?
Several major reviews of the research
Natriello (1987): grades K-12
Crooks (1988): grades K-12
Kluger & DeNisi (1996): grades K-16, work
Black & Wiliam (1998): K-12
Nyquist (2003): grades 13-16
All find consistent, substantial effects
77
Cost/effect comparisons
Intervention
Class-size reduction by 30% (e.g.,
from 30 to 20)
Increase teacher content knowledge
from weak to strong (2 standard
deviations)
Formative assessment/
Assessment for learning
Extra months of
learning/yr
Cost/yr
3
$30k
1.5
?
6 to 9
$3k
88
Types of formative assessment
 Long-cycle
Span: across units, terms
Length: four weeks to one year
 Medium-cycle
Span: within and between teaching units
Length: one to four weeks
 Short-cycle
Span: within and between lessons
Length:
 day-by-day: 24 to 48 hours
 minute-by-minute: 5 seconds to 2 hours
99
Effects of formative assessment
 Long-cycle
Student monitoring
Curriculum alignment
 Medium-cycle
Improved, student-involved, assessment
Improved teacher cognition about learning
 Short-cycle
Improved classroom practice
Improved student engagement
10
10
Five Key Strategies …
Questioning
Engineering effective classroom discussions,
questions, and learning tasks
Feedback
Moving learners forward with feedback
Sharing Learning
Expectations
Clarifying and sharing learning intentions and
criteria for success
Self Assessment
Activating students as the owners of their own
learning
Peer Assessment
Activating students as instructional resources for
one another
11
11
…and one big idea
Use evidence about learning to adapt
instruction to meet student needs
12
12
Keeping Learning on Track (KLT)
 A pilot guides a plane or boat toward its
destination by taking constant readings
and making careful adjustments in
response to wind, currents, weather, etc.
 A KLT teacher does the same:
 Plans a carefully chosen route ahead of time
(in essence building the track)
 Takes readings along the way
 Changes course as conditions dictate
13
13
Eliciting evidence of student
achievement by engineering effective
classroom discussions, questions and
learning tasks
14
14
Kinds of questions: Israel
Which fraction is the smallest?
1
2
1
1
a) , b) , c) , d) .
6
3
3
2
Success rate 88%
Which fraction is the largest?
4
3
5
7
a) , b) , c) , d) .
5
4
8
10
Success rate 46%; 39% chose (b)
[Vinner, PME conference, Lahti, Finland, 1997]
15
15
Misconceptions
16
16
Misconceptions
3a = 24
a + b = 16
17
17
Molecular structure of water?
18
18
Moving learners forward with
feedback
19
19
Kinds of feedback: Israel
 264 low and high ability grade 6 students in 12 classes in 4
schools; analysis of 132 students at top and bottom of each class
 Same teaching, same aims, same teachers, same classwork
 Three kinds of feedback: scores, comments, scores+comments
Feedback
scores
Gain
none
Attitude
top
+ve
bottom -ve
comments
30%
all
[Butler(1988) Br. J. Educ. Psychol., 58 1-14]
+ve
20
20
Responses
Feedback
scores
Gain
none
Attitude
top
+ve
bottom -ve
comments
30%
all
+ve
What do you think happened for the students given
both scores and comments:
A: Gain: 30%; Attitude: all +ve
B: Gain: 30%; Attitude: top +ve, bottom -ve
C: Gain: 0%; Attitude: all +ve
D: Gain: 0%; Attitude: top +ve, bottom -ve
E: Something else
[Butler(1988) Br. J. Educ. Psychol., 58 1-14]
21
21
Kinds of feedback: Israel (2)
 200 grade 5 and 6 Israeli students
 Divergent thinking tasks
 4 matched groups
experimental group 1 (EG1); comments
experimental group 2 (EG2); grades
experimental group 3 (EG3); praise
control group (CG); no feedback
 Achievement
EG1>(EG2≈EG3≈CG)
 Ego-involvement
(EG2≈EG3)>(EG1≈CG)
[Butler (1987) J. Educ. Psychol. 79 474-482]
22
22
Effects of feedback
 Kluger & DeNisi (1996)
 Review of 3000 research reports
 Excluding those:
 without adequate controls
 with poor design
 with fewer than 10 participants
 where performance was not measured
 without details of effect sizes
 left 131 reports, 607 effect sizes, involving 12652 individuals
 Average effect of feedback substantial, but
 Effect sizes very variable
 40% of effect sizes were negative
23
23
Sharing learning intentions and
success criteria
Activating students as owners of
their own learning
Activating students as instructional
resources for one another
24
24
Student involvement in learning
 3 teachers each teaching 4 year 8 science classes
in two US schools
 14 week experiment
 7 two-week projects, scored 2-10
 All teaching the same, except:
 For a part of each week
Two of each teacher’s classes discusses their likes and
dislikes about the teaching (control)
The other two classes discusses how their work will be
assessed
[White & Frederiksen, Cognition & Instruction, 16(1), 1998].
25
25
Student involvement in learning (2)
Iowa Test of Basic Skills
Group
Low
Middle
High
Likes and dislikes
4.6
5.9
6.6
Reflective assessment
6.7
7.2
7.4
26
26
Techniques: questioning
 Key idea: questioning should
 cause thinking
 provide data that informs teaching
 Improving teacher questioning




generating questions with colleagues
closed v open
low-order v high-order
appropriate wait-time
 Getting away from I-R-E




basketball rather than serial table-tennis
‘No hands up’ (except to ask a question)
class polls to review current attitudes towards an issue
‘Hot Seat’ questioning
 All-student response systems
 ABCD cards, Mini white-boards, Exit passes
27
27
Questioning in math: discussion
Look at the following sequence:
3, 7, 11, 15, 19, ….
Which is the best rule to describe the
sequence?
A. n + 4
B. 3 + n
C. 4n - 1
D. 4n + 3
28
28
Questioning in math: diagnosis
In which of these right triangles is a2 + b2 = c2 ?
A
b
a
B
a
c
C
b
a
b
D
c
c
b
c
E
c
a
a
b
F
b
c
a
29
29
Questioning in science: discussion
Ice-cubes are added to a glass of water. What
happens to the level of the water as the ice-cubes
melt?
A.
B.
C.
D.
The level of the water drops
The level of the water stays the same
The level of the water increases
You need more information to be sure
31
31
Questioning in science: diagnosis
The ball sitting on the table is not moving. It is not moving because:
A. no forces are pushing or pulling on the ball.
B. gravity is pulling down, but the table is in the way.
C. the table pushes up with the same force that gravity pulls down
D. gravity is holding it onto the table.
E. there is a force inside the ball keeping it from rolling off the table
Wilson & Draney, 2004
32
32
Save the ozone layer
What can we do to preserve the ozone layer?
A. Reduce the amount of carbon dioxide produced
by cars and factories
B. Reduce the greenhouse effect
C. Stop cutting down the rainforests
D. Limit the numbers of cars that can be used
when the level of ozone is high
E. Properly dispose of air-conditioners and fridges
33
33
Questioning in English: discussion
Macbeth: mad or bad?
34
34
Questioning in English: diagnosis
Where is the verb in this sentence?
The dog ran across the road
A B
C
D
35
35
Questioning in English: diagnosis
Where does the subject end and the
predicate begin in this sentence?
The dog ran across the road.
A B C
D
36
36
Questioning in English: diagnosis
Which of these is a good thesis statement?
A.
B.
C.
D.
E.
F.
G.
H.
The typical TV show has 9 violent incidents
There is a lot of violence on TV
The amount of violence on TV should be reduced
Some programs are more violent than others
Violence is included in programs to boost ratings
Violence on TV is interesting
I don’t like the violence on TV
The essay I am going to write is about violence on TV
37
37
Questioning in history: discussion
In which year did World War II begin?
A.
B.
C.
D.
E.
1919
1937
1938
1939
1941
38
38
Questioning in History
Why are historians concerned with bias when
analyzing sources?
A. People can never be trusted to tell the truth
B. People deliberately leave out important details
C. People are only able to provide meaningful
information if they experienced an event
firsthand
D. People interpret the same event in different
ways, according to their experience
E. People are unaware of the motivations for their
actions
F. People get confused about sequences of events
39
39
What’s wrong with this item?
There are two flights per day from Newtown to Oldtown.
The first flight leaves Newtown each day at 9:05 and
arrives in Oldtown at 10:45. The second flight from
Newtown leaves at 2:15. At what time does the second
flight arrive in Oldtown? Show your work.
40
40
Figurative language
A.
B.
C.
D.
E.
F.
Alliteration
Hyperbole
Metaphor
Onomatopoeia
Personification
None of the above
1.
2.
3.
4.
5.
6.
7.
8.
He was a bull in a china shop.
May I have a drop of water?
This backpack weighs a ton.
The sweetly smiling sunshine…
He honked his horn at the cyclist.
I’ve told you a million times already.
The Redcoats are coming!
He was as tall as a house.
41
41
Triangle shirt waist factory fire, March 25th, 1911
Triangle factory fire
Which of the following sources is biased?
A.
B.
C.
D.
Photograph of the event
New York Times story on Mar 26, 1911
Description of the fire in the textbook
Transcript of talk by Frances Perkins, Sep 30 1964
43
43
Techniques: feedback
 Key idea: feedback should
 cause thinking
 provide guidance on how to improve




Comment-only grading
Focused grading
Explicit reference to mark-schemes and scoring guides
Suggestions on how to improve
 ‘Strategy cards’ ideas for improvement
 Not giving complete solutions
 Re-timing assessment
 (eg two-thirds-of-the-way-through-a-unit test)
44
44
Techniques: sharing learning intentions
 Explaining learning intentions at start of lesson/unit
 Learning intentions
 Success criteria
 Intentions/criteria in students’ language
 Posters of key words to talk about learning
 eg describe, explain, evaluate
 Planning/writing frames
 Annotated examples of different standards to ‘flesh out’
assessment rubrics (e.g. lab reports)
 Opportunities for students to design their own tests
45
45
Practical techniques:
peer and self-assessment
 Students assessing their own/peers’ work
with rubrics
with exemplars
“two stars and a wish”
 Training students to pose questions/identifying
group weaknesses
 Self-assessment of understanding
Traffic lights
Red/green discs
 End-of-lesson students’ review
46
46
Putting it into practice
47
47
Force-field analysis (Lewin, 1951)
 What kinds of forces present in your school/district will
promote or support the development of this kind of work?
 What kinds of forces present in your school/district will
oppose or constrain the development of this kind of work?
+
–
48
48
A model for teacher learning
Content (what we want teachers to
change)
Evidence
Ideas (strategies and techniques)
Process (how to go about change)
Choice
Flexibility
Small steps
Accountability
Support
49
49
Supporting Teachers and Schools to
Change through Teacher Learning
Communities
50
50
Implementing AfL requires changing
habits, not adding knowledge
Teachers “know” most of this already
So the problem is not a lack of knowledge
It’s a lack of understanding what it means to do AfL
That’s why telling teachers what to do doesn’t work
Experience alone is not enough—if it were, then the most
experienced teachers would be the best teachers—we know
that’s not true (Hanushek, 2005)
 People need to reflect on their experiences in systematic
ways that build their accessible knowledge base, learn from
mistakes, etc. (Bransford, Brown & Cocking, 1999)





51
51
That’s what TLCs are for:







TLCs contradict teacher isolation
TLCs reprofessionalize teaching by valuing teacher
expertise
TLCs deprivatize teaching so that teachers’ strengths and
struggles become known
TLCs offer a steady source of support for struggling
teachers
They grow expertise by providing a regular space, time,
and structure for that kind of systematic reflecting on
practice
They facilitate sharing of untapped expertise residing in
individual teachers
They build the collective knowledge base in a school
52
52
The synergy
Content: assessment for learning
Process: teacher learning communities
Components of a model
Initial workshops
Support for TLC leaders
Monthly TLC meetings
Peer observations
‘Drip-feed’ resources
Web-site
Writings
New ideas
53
53
Summary
 Raising achievement is important
 Raising achievement requires improving teacher
quality
 Improving teacher quality requires teacher
professional development
 To be effective, teacher professional development
must address
What teachers do in the classroom
How teachers change what they do in the classroom
 AfL + TLCs
A point of (uniquely?) high leverage
A “Trojan Horse” into wider issues of pedagogy,
psychology, and curriculum
54
54