Transcript School Tools for Analysis and Research

How Data and Analysis techniques inform effective
Teaching and Learning
■
Powerful things that schools can now do with their data...
Within School Variation project website at: www.4matrix.org
Question for discussion
Should a school ever be surprised
by its Ofsted report?
Consensus view of group:
A school that knows itself well would not be surprised
by the findings of a school inspection
Are schools doing better or worse than they used to?
11% outstanding, 48% good, 34%
satisfactory, 8% inadequate
- Ofsted Annual Report
Nearly 60% of schools are good
or better
“Satisfactory was no longer good
enough” – HMCI Gilbert
“Half of schools are failing”
- Guardian October 06
Headline figures about ‘standards’ are not always helpful
The data used to judge schools
RAISEonline (ex-PANDA) data will form the basis of
the main judgements about a school’s standards
because:
• School inspections are too brief to add much to
these judgements
• A school’s own self-evaluation evidence provided in the
SEF tends to be descriptive rather than quantitative
e.g. “We do lots of lesson observations”
Issues to do with using National
Data to judge schools
• It can come too late in the year to be useful
rather than simply judgemental
• Ofsted global figures are good for Government
stats but less useful for school improvement
• Contextual Value Added adjustments are not a
precise way to compare one school with another
CVA
Contextual Value Added adjustment discounts one
or more variables (ethnicity, gender etc) so that
comparisons can be made on a fairer basis
but…
• It is a statistical technique that really only works with large
sample sizes.
• To make CVA accurate enough to make exact comparisons
between one school and another it needs to discount every
external factor known to affect a pupil’s performance, to
leave only those attributable to the influence of the school.
• It has no meaning at class level. A good teacher will aim to
do the best for every pupil - and not expect some individual
pupils to do less well because of their ethnicity or gender etc.
CVA
Where inspectors say that CVA data shows the
school should be doing better, what
independent, pupil-level source of measures
can a school use to put this judgement into
context?
A growing number of schools are able to provide
secure alternative evidence arising from their own
pupil-level analysis and research
Within School Variation (WSV)
• There is over 4 times the variation in
provision within schools than between
schools - source OECD
• Reducing negative variation would
raise standards by 10% - source DfES
"We have always known that there is a difference in performance
between schools. But what can make a bigger difference is the
experience that children have within one school. So a child can do
really well in one subject and not do well in another subject. And
that can make an even bigger difference to children's life chances
than differences between schools."
- Jane Creasy, Assistant Director of Research, NCSL
How would school self-analysis complement RAISEonline data?
RAISEonline
School self analysis
available late in the Autumn term
available the day the exam results
arrive
inspection could be based on
information that is out of date
inspection would be based on the
most up-to-date
performance information
ownership and control in the hands
of Government
ownership and control in the hands
of schools
offers an external evaluation of a
school’s performance
offers teachers an opportunity to
make their own evaluation
evaluates pupils’ performance with
national contextual data
evaluates pupils’ performance with
school contextual data
forms the basis of Ofsted judgments
on the school
provides measures to support the
school's own view of its performance
judges leadership on basis of CVA
comparisons with other schools
allows a school to show that good
leadership equates with successfully
tackling negative variation
What is the better basis on which school leadership should be judged?
High Standards
High Achievement and
Low Variation
5 A*-Cs
How well every pupil achieves
Some Children Matter
Every Child Matters
Headline figures can hide significant
pockets of underachievement
Headline figures should show how
well every child achieves
Good Leadership = high attainment
Good Leadership = evidence of doing
the best for every pupil
League table position
Measures of Within School Variation
Performance analysis is done by the
few and passed to the many
All teachers are involved with
analysing their pupils’ performance
Judged by official data
Secure school-level evidence of
effectiveness
School Improvement = more 5A-Cs
School Improvement = less negative
variation
“Schools that are proactive in showing
inspectors the evidence of their own
pupil-level analysis and research tend
to do better in their inspection.”
- Dr. Mike Treadaway, Fischer Family Trust,
Naace ‘Making Information Work’ Conference 27.04.07
Jack is a C level pupil with an F in this subject – should Jack or his teacher try harder?
An analysis of variation can shine a light on the achievement of different groups of pupils
http://www.4matrix.org/example
The ‘Data Confident School’ Toolkit
• The use of pupil-level data
analysis and research is
relatively underdeveloped at
this point in time
• Good schools will be those
that can show that they know
themselves well, are targeting
negative variation, and can
show evidence of improvement
“The Data Confidence Toolkit
is a good starting point for
schools”
- Dr Mike Treadaway
http://www.4matrix.org/toolkit/
Question:
Should every school be expected to
prove that for them ‘Every Child Matters’ rather than just those pupils who will get
5 A*-C grades?