Transcript School Subject Estimates

Using FFT Live
Secondary Schools
Mike Treadaway
Director of Research
Fischer Family Trust
Contents
Data Landscape
• Change and Continuity?
FFT Live
• Self-Evaluation
• Estimates - Particularly KS4
• Data Literacy – Applying Key Principles
Other Reports
• Student Explorer
• Exports
Data Landscape
FFT
DfE
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
Value-Added
Contextual
Value-Added
3 Year
Summaries
Estimates
Also:
• Floor Standards
• Expected Progress
Whether or not to Contextualise?
Type of Analysis
Advantages
Disadvantages
Raw Outcomes e.g. %L4+
Provides a view as to whether
pupils are attaining the
knowledge and skills needed
for their next stage of
education?
When used to compare
schools, depends strongly
upon attainment of the
school’s intake.
Value-Added
Enables a focus upon progress
irrespective of context e.g.
How well are our FSM pupils
doing compared to our nonFSM pupils?
Comparing value-added
outcomes for schools with
different contexts and makeup of intakes can be
misleading.
Contextual Value-Added
Aims to take account of
factors which are outside the
school’s control and which,
from analysing national data,
can be shown to influence
progress.
If used to look ahead
(estimates) then danger of
‘self-fulfilling prophecy’ for
pupils in groups who currently
under-perform nationally.
Differences – VA and CVA
• Compare PA (Value-added) and SX (Contextual Value-Added)
outcomes for KS2->4 in 2009/10.
– Use KS4 Capped Points Score as outcome measure.
– Look at how many change significant state.
– Analyses by grouping schools in different socio-economic contexts.
School
VA compared to CVA
School
Change in VA Score
Context
Better
Worse
Context
Avg
Min
Max
Q1
30%
2%
Q1
+7
-19
+25
Q2
9%
11%
Q2
-1
-53
+38
Q3
7%
17%
Q3
-3
-62
+32
Q4
14%
13%
Q4
0
-54
+34
To get some idea of the likely change for your
school, compare PA and SX value-added outcomes
on FFT Live or in FFT Self-Evaluation reports.
In Lambeth, most schools have VA
better than CVA – this is mainly
related to the progress of ethnic
minority pupils.
Recommendations
For Self-Evaluation:
• VA (FFT PA model) and CVA (FFT SX model) are both important
• Contextual Value-Added is probably still the best starting point
• For schools with high % of ethnic minority pupils (other than Black Caribbean) CVA is
likely to be LOWER than VA
• If CVA is good, particularly in a school serving a socio-economically disadvantaged area,
VA provide a view as to whether pupils are making good progress irrespective of context
For Estimates:
• Thinking about range is probably the most important principle
• PA model probably the best starting point, but need to take account of:
• What the school has achieved in previous years
• Impact of pupil characteristics and school context
• Other factors, including parental support, school interventions .....
And, in the end, what matters most to each pupil is the knowledge and
skills they take with them to the next stage of their education!
Expected Progress
• DfE Approach
– Whole Levels, Subject Specific Input
– So, for example, 68% L4 in Mathematics at KS2 attained grade C or
higher at KS4 in 2011
• FFT Approach
– Differentiate within levels (using Fine Grades)
– Take into account prior attainment in English and Mathematics
KS2 English
C+ KS4 Maths
KS2 Maths
KS4 C+ Maths
3
50%
4C
47%
4C
61%
4B
69%
4B
69%
4A
86%
4A
75%
5
80%
FFT Live
Includes:
• Latest
updates
• Future
releases
• Help Files
SETS
• Key Changes for 2011/12
–
–
–
–
Default changed to PA (Prior Attainment)
SX (Contextual VA) continues to be available
Options to SORT and FILTER (Pupil Groups) available on range of reports
Default layout for KS4 and KS5 Subject Summary VA Simplified
Self-Evaluation
Significant Areas Grid
• Range of pupil groups and indicators
• KS2->3, KS2->4, KS3->4
• 3 Year summary, Overall + Trends
Significant Areas Detail
• Shows actual attainment and VA/CVA scores for
each year
• Can be accessed by clicking on Significant Areas grid
Subject Summary Reports
• KS4 and KS5 Individual Subjects
• Summary Report (most users find this is sufficient)
• Detail Report
Pupil Value-Added
• Overall Indicators KS2->3, KS2->4, KS3->4
• Individual Subjects KS2->4, KS3->4, KS4->5
• VA or CVA (KS3 and KS4 outcomes)
Estimates
School Summary Estimates
• Range of Indicators and Pupil Groups
• A – PA 50th Percentile; B – SE 50th Percentile,
• D – SE 25th Percentile; S – School Previous
Pupil Summary Estimates
• Types A, B, D
• Range of indicators (e.g. 5A*C, 5A*CEM, Points)
School Subject Estimates
• Core Subjects including Expected Progress for English and
Mathematics
• Outcomes for all grades (A* to G)
• Some pupil groupings available
Pupil Subject Estimates
• Show likelihood of attaining all grades (A* to G or A* to E)
• Also provides option to show ‘most likely grade’ (GM)
Using Estimates
Two basic approaches
• Start with pupil data
• Take into account other
factors
• Add up and compare with
overall estimates
• Revise as necessary to
arrive at overall targets
• Use overall estimates and
evaluation of past performance
to draft overall targets
• Use ‘RANK’ option in FFT School
Subject Estimates to set overall
figure at your overall targets
• Click on link to pupil estimates
• Estimate -> Prediction -> Target
• Consider option of range - realistic and ambitious
• Don’t add up ‘estimated levels / grades’
Adding up estimated levels – why not?
• Consider a group of 50 ‘identical’ pupils
• They all have the same estimates:
G
F
E
D
C
B
A
A*
GM
HGM
GA
0%
0%
10%
20%
40%
20%
10%
0%
C
30%
B
If we add up the ‘GM’ grades
If we average the ‘chances’ for
each level and multiply by
number of pupils
E
D
C
B
A
0
0
50
0
0
E
D
C
B
A
10%
20%
40%
20%
10%
5
10
20
10
5
Note : Adding up predicted or target grades is fine –
assuming that they are not simply just using
unmoderated estimates!
KS4 Estimates
Default
Extended
• GM – Grade likely to be achieved by 50% of more ‘similar’ students
nationally
• HGM – likelihood of attaining above GM
• GA – one grade above GM
• EP – Expected Progress from KS2->KS4 (English and Mathematics only)
Other Reports
• Export Section
• Provides additional data (e.g. UPN,
FFT_ID) which is not available on
SETS reports
• Removes formatting and colours, so
easier to import into other systems
•
•
•
•
Student Explorer
Select students by range of characteristics
Look at individual pupil history
Developments:
• FSM as pre-set group (can currently
select using drop-down menus)
• Pilot estimate report which takes into
account factors such as EAL and when
arrived in UK
Questions
English Baccalaureate
• Currently, FFT Live includes:
• EXPORT report which shows school, LA and National data for a range of indicators (EBac
and our ‘best guess’ at what is likely to be the case for Basics indicators)
• DEVELOPMENT section has EBac estimates – based upon those who actually sat subjects
in all 5 EBac areas, so not consistent with KS4 Subject Estimates
• Should we do anything further?
• EBac value-added – or is this covered by looking at the relevant subjects in KS4 Subject
Value-Added?
• What about Estimates – are they needed?
Student Explorer
• Ability to select by FSM (current or ever) will soon be included
• Are there any other important pupil characteristics which would be useful to have easily
available on the main menu?
Any other suggestions for development?