Practice and research in education: How can we make both better

Download Report

Transcript Practice and research in education: How can we make both better

Practice and research in education:

How can we make both better, and better aligned?

Robert Coe ResearchED 2013, Dulwich College, 7 Sept 2013 @ProfCoe

Improving practice and research

   Problems with research and evidence Can research tell us what works?

How can practice be improved?

– Invest in good CPD – Evaluate teaching quality – Evaluate impact of changes Improving Education: A triumph of hope over experience http://www.cem.org/attachments/publications/ImprovingEducation2013.pdf

Problems with research and evidence

Problems with evidence

  Evidence can be found to support any position in education Ofsted asks schools to produce evidence to  the gap (even though some of it may not have) DfE misunderstands/misuses evidence (relative gaps as percentage difference; small changes with small samples) 4

Problems with research

      Quality varies, but a lot is not very good Quality really matters How do you know who or what to trust?

∂ Academic papers are inaccessible Academic debates are (mostly) pointless Peer review doesn’t work 5

Small positives

    Impact agenda requires ‘public benefit’ EEF: funding for high quality evaluation Recurrent policy interest in Evidence-Based Education (see http://www.cem.org/evidence based-education/introduction ) Social media & internet gives instant critique, debate, interaction 6

Evidence about the effectiveness of different strategies

Toolkit of Strategies to Improve Learning ∂ The Sutton Trust-EEF Teaching and Learning Toolkit http://www.educationendowmentfoundation.org.uk/toolkit/

www.educationendowmentfoundation.org.uk/toolkit

Impact vs cost

Promising

8 Feedback Meta-cognitive

May be worth it

Peer tutoring 0 Homework (Secondary) ∂ Collaborative Behaviour Phonics Small gp tuition Social Homework ICT Individualised learning Parental involvement Summer schools Mentoring 1-1 tuition £0 (Primary) Ability grouping Performance pay Aspirations Cost per pupil Teaching assistants After school £1000 Early Years Smaller classes

Not worth it

Key messages

 Some things that are popular or widely thought to be effective probably can’t improve learning  Performance pay; Raising aspirations Some things look ‘promising’ – Effective feedback; Meta-cognitive and self regulation strategies; Peer tutoring/peer ‐ assisted learning strategies; Homework

Clear, simple advice:

  Choose from the top left Go back to school and do it ∂ For every complex problem there is an answer that is clear, simple, and wrong H.L. Mencken 11

 

Why not?

We have been doing some of these things for a long time, but have generally not seen improvement Research evidence is problematic – Sometimes the existing evidence is thin  – Context and ‘support factors’ may matter Implementation is problematic – We may think we are doing it, but are we doing it right?

– We do not know how to get large groups of teachers and schools to implement these interventions in ways that are faithful, effective and sustainable 12

So how might practice be improved?

Four steps to improvement

    Think hard about learning Invest in good professional development Evaluate impact of changes

1. Think hard about learning

www.educationendowmentfoundation.org.uk/toolkit

Impact vs cost

Promising

8 Feedback Meta-cognitive

May be worth it

Not worth it

Poor Proxies for Learning

      Students are busy: lots of work is done (especially written work) Students are engaged, interested, motivated Students are getting attention: feedback, explanations ∂ Classroom is ordered, calm, under control Curriculum has been ‘covered’ (ie presented to students in some form) (At least some) students have supplied correct answers (whether or not they really understood them or could reproduce them independently) 17

A simple theory of learning

Learning happens

∂

when people have to think hard

2. Invest in effective CPD

How do we get students to learn hard things?

Eg  Place value  Persuasive writing  Music composition  Balancing chemical equations • Explain what they should do • Demonstrate it • Get them to do it (with gradually reducing support) • Provide feedback • Get them to practise until it is secure • Assess their skill/ understanding

How do we get teachers to learn hard things?

Eg  Using formative assessment  Assertive discipline  How to teach algebra • Explain what they should do ∂

What CPD helps learners?

     

Intense

: at least 15 contact hours, preferably 50

Sustained

: over at least two terms

Content focused

: on teachers’ knowledge of subject content & how students learn it ∂

Active

: opportunities to try it out & discuss

Supported

: external feedback and networks to improve and sustain

Evidence based

: promotes strategies supported by robust evaluation evidence

3. Evaluate teaching quality

Classroom observation: The new Brain Gym?

  Validity evidence – Are observation ratings really a reflection of teaching quality?

∂ Impact evaluation – Does the process of observation and feedback lead to improvement?

– In what, how much and for what cost?

  

Validity evidence

Do observation ratings correspond with other indicators of teaching quality or effectiveness?

– Student learning gains – Student ratings – Peer (teacher) perceptions – Self ratings Are they consistent? – Across occasions – Across raters ∂ Are ratings influenced by spurious confounds – Charisma – Confidence – Subject matter – Students’ behaviour – Time of day 25

Does observation improve teaching?  Need studies with – Clearly defined intervention – High quality outcome measures (student learning) – Adequate sample – Measures of sustained impact  Just one would be nice … 26

4. Evaluate impact of changes

Mistaking School Improvement (1)

(Coe, 2009) 1. Wait for a bad year or choose underperforming schools to start with. Most things self-correct or revert to expectations (you can claim the credit for this). 2. Take on any initiative, and ask everyone who put effort into it whether they feel it worked. No-one wants to feel their effort was wasted.

Define ‘improvement’ in terms of perceptions and ratings of teachers. DO NOT conduct any proper assessments – they may disappoint.

4. Only study schools or teachers that recognise a problem and are prepared to take on an initiative. They’ll probably improve whatever you do.

Mistaking School Improvement (2) (Coe, 2009) 5.

Conduct some kind of evaluation, but don’t let the design be too good – poor quality evaluations are much more likely to show positive results.

6. If any improvement occurs in any aspect of performance, focus attention on that rather than on any areas or schools that have not improved or got worse (don’t mention them!).

7. Put some effort into marketing and presentation of the school. Once you start to recruit better students, things will improve.

Key elements of good evaluation

∂  Clear, well defined, replicable intervention  Good assessment of appropriate outcomes  Well-matched comparison group

 

Summary …

A lot of educational research is rubbish, but some is very good: relevant and rigorous Four steps to improve practice: – Think hard about learning – Invest in

good

CPD ∂ – Evaluate teaching quality (but not with dodgy observation) – Evaluate impact of changes [email protected]

Practice and research in education: How can we make both better

Transcript Practice and research in education: How can we make both better