Transcript Impact evaluation: making a difference Vivienne Porritt

Ascend and IOE
Partnership
Evaluating impact: how do you
know you are making a difference?
Sue Hellman, David Godfrey
and Sarah Seleznyov
London Centre for Leadership in Learning
Institute of Education www.ioe.ac.uk/lcll
• To deepen your understanding of impact framework and how to use
this to demonstrate impact in relation to the aims of the Ascend
project.
• To provide an opportunity for you to refine your impact frameworks
having compared your impact frameworks to a sample framework
• To hear about responses to the lesson study so far
• To explore explicit links between lesson study, subject audits and
impact framework
• Improve the quality of teachers’
subject specific knowledge
• Improve the quality of teaching
• Raise attainment and accelerate
progress for pupils with FSM,
underachieving and socially
disadvantaged pupils in Maths,
English and Science across Key
stages 1-3
Impact
Framework
Lesson Study
Pupil
Learning
Outcomes
Survey
Responses
Guskey’s
levels
pupil learning
outcomes
use of new
knowledge and skills
organization support and change
professional learning
initial reactions
What is impact
evaluation?
Input model:
Impact model:
Focus on what we do,
what we offer
Focus on the difference we
want to achieve
© Copyright LCLL, 2008
How will you
know?
From what ……….
Baseline picture
• Current practice
• Current data
• Current attainment
measure
to what?
Impact picture
• Changed practice
• Changed data
• Changed attainment
measure
© Copyright LCLL, 2007
‘…so that …’
Pupils’ learning and
outcomes improve…
Baseline – Current practice
Impact – Shift in practice
What will I
be doing?
What am I
doing?
What will I
be feeling /
thinking?
What am I
feeling /
thinking?
What will I
be saying?
What am I
saying?
Evidence – How do you know?
© Copyright LCLL 2008
Teacher
Practice
Baseline
Impact
I never sit with the lower ability children.
I will work with a different group each
day.
Lesson observations show that I never
sit with a group for more than 10
minutes at a time because I move
around the room responding to low
ability children who have raised their
hands for help.
Planning scrutinies show that I work
with a different ability group each day.
Lesson observations show that I sit
with a group for at least 20 minutes.
Fewer lower ability children raise their
hands and evidence that they use the
problem strategies independently.
I set problem solving extension tasks for There will be problem solving activities
the most able pupils.
for all children.
© Copyright LCLL, 2007
Baseline – Pupils’ learning
Impact – Shift in practice
What will
pupils be
doing?
What are
pupils
doing?
What will
pupils be
feeling /
thinking?
What are
pupils feeling /
thinking?
What will
pupils be
saying?
What are
pupils
saying?
Evidence – How do you know?
© Copyright LCLL 2008
Pupils’
Learning
Baseline
Impact
Most children struggle to stay engaged
and on-task during lessons.
Children will remain engaged and
focused throughout the lesson.
Lesson observations show that around
30% of the children drift into off-task
talk after about 10 minutes of work time
and around 4 or 5 children in any given
lesson will get up and wander around
the room. This means work outputs
are too low for around 70% of the
class.
Lesson observations will show that no
children wander around the room and
that 90% of the children remain on task
and focused for the full 20 minutes of
independent work time.
The target group write mainly simple
sentences.
The target group will add more
descriptive detail to their writing.
© Copyright LCLL, 2007
Working in pairs:
Looking at the sample Impact Frameworks
- How does the sample Framework compare with
your own?
- What will you need to do to further refine and
develop your own Impact Framework?
Working in pairs:
Take one person’s Impact Framework at a time.
You have 15 minutes to refine and develop the
Framework in line with our discussions.
Impact
Framework
Lesson Study
Pupil
Learning
Outcomes
Survey
Responses
How does this affect:
• the focus of the lesson study?
• the role of the observer?
Ascend and IOE
Partnership
Feedback from lesson study first
cycle and subject knowledge
audits
David Godfrey
London Centre for Leadership in Learning
Institute of Education www.ioe.ac.uk/lcll
Lesson Study
Pupil
learning
outcomes
Impact framework
Survey responses
• Surveys for Ascend project:
• Subject knowledge audits:
• All participants to complete for either English or Mathematics
• Post lesson-study survey:
• All participants to complete once as focus teacher and for each time
as observers (non-focus teacher)
• Subject knowledge audits:
• Subject knowledge audits:
•
•
•
•
More responses please!
Mathematics audit relies on NCETM categories and is about self-confidence
English audit tests participant knowledge but is also designed to highlight
areas where teacher may be more/less confident
Both are tools for self-evaluation and reflection on subject knowledge in
relation to areas you teach and key stage (particularly in relation to target
students)
• Lesson study evaluation:
• Two versions:
•
•
Focus teacher
Non-focus teacher (baseline and post-lesson evidence section omitted)
•
Sections:
•
•
Baseline data
Evidence of changed practice and pupil outcomes captured from lesson
study cycle
Key professional learning, challenges and future strategies
Participant reactions to the process
•
•
• Lesson study evaluation:
• Focus teacher reactions to the process:
• Lesson study evaluation:
• Non- focus teacher reactions to the process:
• Lesson study evaluation:
• Baseline data:
• Use impact frameworks to fill this out – can copy and paste from word
document into survey
• Student numbers: use teacher number plus student letter (e.g.
AL001a, AL001b)
• Add key baseline data – combination of impact framework and first
lesson of cycle
• Add observations from lesson study about target pupils especially
(soft data)
• Lesson study evaluation:
• Key professional learning, challenges and future strategies:
• Plenty of comments about insights into children’s learning and
effectiveness of approaches
• Need to make specific comments that relate to target students
• Some comments about the challenges of effective planning (making
time) and observation
•
•To sum up…
Relate lesson study evaluation to details of impact framework
Focus observations on target students (use identifier codes)
Focus on strategies that directly or indirectly (e.g. through your own
professional development needs) will lead to improved outcomes for target
students
Ascend and IOE
Partnership
Feedback from lesson study first
cycle and subject knowledge
audits
David Godfrey
London Centre for Leadership in Learning
Institute of Education www.ioe.ac.uk/lcll
Impact
Framework
Lesson Study
Pupil
Learning
Outcomes
Survey
Responses
Ascend and IOE
Partnership
Evaluating impact: how do you
know you are making a difference?
Sue Hellman
London Centre for Leadership in Learning
Institute of Education www.ioe.ac.uk/lcll
[email protected]