Transcript Measuring Pupil Progress: getting the most out of value

Peter Hendry: CEM Consultant
Running a School on Data:
the role of Value-Added Data
Course: Using CEM data in Practice
Day 2 Session 4
Wed 30th May 2012
[email protected]
CASE STUDY : MidYIS Year 7 to GCSE Raw Residuals
0.0
Additional Science
Art & Design
-0.1
Business Studies
0.8
Design & Technology
0.8
1.5
Drama
English
-0.2
English Literature
-0.1
0.6
French
0.3
Geography
0.4
German
0.3
History
0.6
Home Economics
Mathematics
Music
-0.6
-0.2
0.6
Physical Education
Religious Studies
Science
Sociology
Welsh
-0.3
0.2
0.8
0.6
Comments?
-0.2
-0.1
-0.1 -0.1
-0.1
-0.5
0.1
0.2
0.2
0.2
0.0
0.5 0.4
0.4
0.4
0.3
0.4
0.5 0.5
0.5
0
-0.5
-1
-1.5
Average Standardised Residual
CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals
2
1.5
1.1
1
-2
Short Course Physical…
Welsh
Sociology
Science
Religious Studies
Physical Education
Music
Mathematics
Home Economics
History
German
Geography
French
English Literature
English
Drama
Design & Technology
Business Studies
Art & Design
Additional Science
CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals
Statistical Process Control Chart
Nationally Represesentative Standardisation
GCSE Overall
0.3
Average Standardised Residual
0.2
0.1
0
-0.1
-0.2
-0.3
2005
2006
2007
Exam Year
Examination
Year
2008
CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals
Distribution of Schools' Average Standardised Residuals (National Schools'
Standardisation) with 95% Confidence Intervals: GCSE School Level
Year 7 To GCSE
2.5
Average Standardised Residual
2
1.5
1
0.5
0
-0.5
-1
-1.5
0
50
100
150
200
250
Rank
300
350
400
450
CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals
Statistical Process Control Chart
Nationally Represesentative Standardisation
GCSE Mathematics
0.3
Average Standardised Residual
0.2
0.1
0
-0.1
-0.2
-0.3
-0.4
-0.5
-0.6
2005
2006
2007
Exam Year
Examination Year
2008
CASE STUDY : MidYIS Year 7 to GCSE Standardised Residuals
Distribution of Schools' Average Standardised Residuals (National Schools'
Standardisation) with 95% Confidence Intervals: GCSE Mathematics
Year 7 to GCSE
4
Average Standardised Residual
3
2
1
0
-1
-2
0
40
80
120
160
200
Rank
240
280
320
360
400
Yr 7 MidYIS to GCSE: all pupils
Year 7 MidYIS Score 2003/2004 Against GCSE Mathematics 2008
8.0
GCSE Points Score
7.0
6.0
5.0
4.0
3.0
2.0
1.0
0.0
50
60
70
80
90
100
110
120
130
MidYIS Test Score (Standardised on a Nationally Representative Sample)
140
150
By grade
1.5
Mean Standardised Residual
Case
Study:
Maths
All pupils
1
0.5
0
A*
A
B
C
D
E
F
G
-0.5
-1
-1.5
-2
-2.5
Grade
Grouped by GCSE Grade
Result
A*
A
B
C
D
E
F
G
8
7
6
5
4
3
2
1
Mean
Raw
Resid
Mean
Stand
Residual
1.17
-0.03
0.04
-0.61
-1.28
-1.88
-2.74
0.94
-0.02
0.03
-0.49
-1.03
-1.51
-2.21
Count
0
8
24
36
27
18
13
13
total
139
By quartile band
1
0.8
Mean Standardised Residual
Case
Study:
Maths
All pupils
0.6
0.4
0.2
0
D
-0.2
C
B
A
-0.4
-0.6
-0.8
-1
Band
Grouped by Quartile Band
Band
D
C
B
A
lower
bound
0
89.9
100
110.1
Mean
upper
Raw
bound Residual
89.9
-0.98
100
-0.80
110.1
-0.39
200
-0.69
Mean
Stand
Residual
-0.79
-0.65
-0.32
-0.55
Count
13
32
46
48
GIRLS
BOYS
By grade
2
2
1.5
1.5
Mean Standardised Residual
Mean Standardised Residual
By grade
1
0.5
0
A*
A
B
C
D
E
F
G
1
0.5
0
A*
A
B
-0.5
-1
D
E
F
G
-1
-1.5
-1.5
-2
-2
-2.5
-2.5
-3
Grade
Grade
Grouped by GCSE Grade
Grouped by GCSE Grade
Result
A*
A
B
C
D
E
F
G
C
-0.5
Mean
Raw
Resid
8
7
6
5
4
3
2
1
1.14
-0.08
-0.16
-0.50
-1.28
-1.66
-2.64
Mean
Stand
Residual
0.92
-0.06
-0.13
-0.41
-1.03
-1.34
-2.13
Count
0
4
11
16
14
12
6
8
total
71
Result
A*
A
B
C
D
E
F
G
8
7
6
5
4
3
2
1
Mean
Raw
Resid
Mean
Stand
Residual
1.19
0.02
0.19
-0.72
-1.29
-2.06
-2.88
0.96
0.01
0.16
-0.58
-1.04
-1.66
-2.32
Count
0
4
13
20
13
6
7
5
total
68
By quartile band
GIRLS
By quartile band
BOYS
1.5
Mean Standardised Residual
Mean Standardised Residual
1.5
1
0.5
0
D
C
B
A
-0.5
-1
-1.5
1
0.5
0
D
C
A
-0.5
-1
-1.5
Band
Band
Grouped by Quartile Band
Band
D
C
B
A
B
lower
bound
0
89.9
100
110.1
Mean
upper
Raw
bound Residual
89.9
-1.39
100
-1.13
110.1
-0.57
200
-0.42
Grouped by Quartile Band
Mean
Stand
Residual
-1.12
-0.91
-0.46
-0.34
Count
6
18
26
21
Band
D
C
B
A
lower
bound
0
89.9
100
110.1
Mean
upper
Raw
bound Residual
89.9
-0.63
100
-0.38
110.1
-0.16
200
-0.89
Mean
Stand
Residual
-0.51
-0.31
-0.13
-0.72
Count
7
14
20
27
AVERAGE SCORES
MidYIS Predicted
Test
GCSE
Score
Points
overall
girls
boys
105.8
105.2
106.5
4.8
4.8
4.9
Actual
GCSE
Points
4.2
4.0
4.3
Standard
Raw
ised
Residual Residual
-0.6
-0.7
-0.5
-0.5
-0.6
-0.4
Case study: Alis value-added data
Four sets of VAD are available!
From average GCSE baseline:
• all Alis cohort
• type of Institution
• Syllabus
From the baseline test
• all Alis cohort
SPC Chart with confidence limits: WHOLE SCHOOL
All Alis Cohort:
Syllabus
Institution
Using PARIS software: Baseline Test
Whole School
From your perspective, which set of VAD would you use for the
different user groups? (Governors, HoDs, Media, Parents, SMT/SLT...)