Transcript Improving Education: A triumph of hope over experience

Improving Education:
A triumph of hope over experience
Robert Coe
Inaugural Lecture, Durham University, 18 June 2013
A triumph of hope over experience
 Experience
–
–
–
–
Have educational standards really risen?
School improvement: Isn’t it time there was some?
Can we identify effective schools and teachers?
∂
Is ‘evidence-based’ practice and policy the
answer?
 Hope
– So what should we do (that hasn’t failed yet)?
www.cem.org/publications
2
Have educational standards
really risen?
3
∂
4
Equivalent change in
GCSE grades
∂
5
∂
(Updated from Coe, 2007)
6
ICCAMS (Hodgen et al)
∂
7
School improvement:
Isn’t it time there was some?
8
Mistaking School Improvement (1)
(Coe, 2009)
1. Wait for a bad year or choose underperforming
schools to start with. Most things self-correct or revert
to expectations (you can claim the credit for this).
2. Take on any initiative, and ask everyone who put
effort into it whether they feel it worked. No-one wants
∂
to feel their effort was wasted.
3. Define ‘improvement’ in terms of perceptions and
ratings of teachers. DO NOT conduct any proper
assessments – they may disappoint.
4. Only study schools or teachers that recognise a
problem and are prepared to take on an initiative.
They’ll probably improve whatever you do.
Mistaking School Improvement (2)
(Coe, 2009)
5. Conduct some kind of evaluation, but don’t let the
design be too good – poor quality evaluations are
much more likely to show positive results.
6. If any improvement occurs in any aspect of
∂
performance, focus attention
on that rather than on
any areas or schools that have not improved or got
worse (don’t mention them!).
7. Put some effort into marketing and presentation of
the school. Once you start to recruit better students,
things will improve.
Can we identify effective
schools and teachers?
11
Problems with school effectiveness
research
 ‘Value-added’ is not effectiveness (Gorard, 2010;
Dumay, Coe & Anumendem, 2013)
 Characteristics of ‘effective schools’
– ‘strong leadership’, ‘high expectations’, ‘positive
∂ teaching and learning’
climate’ and a ‘focus on
– Too vague
– ‘Effects’ are tiny anyway (Scheerens, 2000, 2012)
 Correlations, not causes (Coe & Fitz-Gibbon, 1998)
– Can ‘effective’ strategies be implemented?
– If so, do they lead to improvement?
12
Is ‘evidence-based’ practice
and policy the answer?
13
Toolkit of Strategies to Improve Learning
∂
The Sutton Trust-EEF Teaching and Learning Toolkit
http://www.educationendowmentfoundation.org.uk/toolkit/
www.educationendowmentfoundation.org.uk/toolkit
Impact vs cost
Effect Size (months gain)
Promising
8
May be
worth it
Feedback
Meta-cognitive
Peer tutoring
Homework
(Secondary)
Collaborative
Early Years
1-1 tuition
∂
Behaviour
Small gp
Phonics
Parental
tuition
involvement
ICT
Social
Individualised Summer
schools
learning
Mentoring
Homework
(Primary)
Performance Aspirations
0
pay
£0 Ability grouping
Cost per pupil
Smaller
classes
After
school
Teaching
assistants
£1000
Not
worth it
Key messages
 Some things that are popular or widely
thought to be effective are probably not
worth doing
– Ability grouping (setting); After-school clubs;
∂
Teaching assistants; Smaller classes;
Performance pay; Raising aspirations
 Some things look ‘promising’
– Effective feedback; Meta-cognitive and self
regulation strategies; Peer tutoring/peer‐assisted
learning strategies; Homework
Clear, simple advice:
 Choose from the top left
 Go back to school and do it
∂
For every complex problem
there is an answer that is
clear, simple, and wrong
H.L. Mencken
17
Why not?
 We have been doing some of these things for a
long time, but have generally not seen
improvement
 Research evidence is problematic
– Sometimes the existing evidence is thin
∂ reflect real life
– Research studies may not
– Context and ‘support factors’ may matter (Cartwright and
Hardie, 2012)
 Implementation is problematic
– We may think we are doing it, but are we doing it right?
– We do not know how to get large groups of teachers and
schools to implement these interventions in ways that are
faithful, effective and sustainable
18
So what should we do
(that hasn’t failed yet)?
19
Four steps to improvement




Think hard about learning
Invest in effective professional development
Evaluate teaching quality
∂
Evaluate impact of changes
1. Think hard about
learning
www.educationendowmentfoundation.org.uk/toolkit
Impact vs cost
Effect Size (months gain)
Promising
8
May be
worth it
Feedback
Meta-cognitive
Peer tutoring
Homework
(Secondary)
Collaborative
Early Years
1-1 tuition
∂
Behaviour
Small gp
Phonics
Parental
tuition
involvement
ICT
Social
Individualised Summer
schools
learning
Mentoring
Homework
(Primary)
Performance Aspirations
0
pay
£0 Ability grouping
Cost per pupil
Smaller
classes
After
school
Teaching
assistants
£1000
Not
worth it
www.educationendowmentfoundation.org.uk/toolkit
Effect Size (months gain)
Impact vs cost
8
Feedback
Meta-cognitive
Peer tutoring
Homework
(Secondary)
These
work?
∂
Collaborative
Phonics
Does your
theory of
learning
explain why …
These
don’t?
Smaller
classes
After
school
0
£0
Ability grouping
Performance
pay
Aspirations
Cost per pupil
Teaching
assistants
£1000
Poor Proxies for Learning
 Students are busy: lots of work is done
(especially written work)
 Students are engaged, interested, motivated
 Students are getting attention: feedback,
explanations
∂
 Classroom is ordered, calm, under control
 Curriculum has been ‘covered’ (ie presented to
students in some form)
 (At least some) students have supplied correct
answers (whether or not they really understood
them or could reproduce them independently)
24
A simple theory of learning
Learning happens
when people have
to think hard
∂
Hard questions about your school
 How many minutes does an average pupil on
an average day spend really thinking hard?
 Do you really want pupils to be ‘stuck’ in your
∂
lessons?
 If they knew the right answer but didn’t know
why, how many pupils would care?
26
2. Invest in effective CPD
How do we get students to learn hard things?
Eg
 Place value
 Persuasive
writing
 Music
composition
 Balancing
chemical
equations
• Explain what they should do
• Demonstrate it
• Get them to do it (with
gradually
reducing support)
∂
• Provide feedback
• Get them to practise until it is
secure
• Assess their skill/
understanding
How do we get teachers to learn hard things?
Eg
 Using formative
assessment
 Assertive
discipline
 How to teach
algebra
• Explain what they should do
∂
What (probably) makes CPD
effective?
 Intense: at least 15 hours, preferably 50
 Sustained: over at least two terms
 Content focused: on teachers’ knowledge of
subject content & how ∂students learn it
 Active: opportunities to try it out & discuss
 Supported: external feedback and networks to
improve and sustain
 Evidence based: promotes strategies
supported by robust evaluation evidence
3. Evaluate teaching
quality
Every teacher needs to
improve, not because they
∂
are not good enough,
but
because they can be even
better.
Dylan Wiliam
Identifying the best teachers
Sources of evidence:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
Colleagues (peers, SMs) observing lessons
Trained outsiders observing lessons
Pupils’ test score gains
Progress in NC levels (from teacher assessment)
∂
Pupils’ ratings of teacher/lesson
quality
Teacher qualifications
Tests of teachers’ content knowledge
Parents’ ratings
Ofsted ratings
Colleagues’ (including senior managers) perceptions
Teachers’ self-evaluation
Next generation of CEM systems …


Assessments that are
– Comprehensive, across the full range of curriculum areas,
levels, ages, topics and educationally relevant abilities
– Diagnostic, with evidence-based follow-up
– Interpretable, calibrated against norms and criteria
– High psychometric quality ∂
Feedback that is
– Bespoke to individual teacher, for their students and classes
– Multi-component, incorporating learning gains, pupil ratings,
peer feedback, self-evaluation, …
– Diagnostic, with evidence-based follow-up

Constant experimenting
34
4. Evaluate impact of
changes
Bad reasons not to evaluate










We are sure this works
This is so important we need it to work
Everyone is working really hard and fully committed to this
Evaluating would be a lot of work
We don’t have the data to be able to evaluate
∂
We don’t know how to evaluate
We can’t do a really good evaluation, so what is the point of
doing it badly?
We do happy sheets and ask people what they thought of it;
isn’t that enough?
You can’t do randomised trials in education
What works is different in different schools or contexts
Key elements of good evaluation
∂
 Clear, well
defined
intervention
 Good
assessment of
appropriate
outcomes
 Well-matched
comparison
group
A triumph of hope over experience
 Experience
– So far, we haven’t cracked it: don’t keep doing the
same things
 Hope
–
–
–
–
∂
Think hard about learning
Invest in effective professional development
Evaluate teaching quality
Evaluate impact of changes
38