Transcript Predicting Achievement in the Early Years: How Influential

Reporting the attainments and progress
of pupils with Special Educational
needs (SEN) using the CEM P scales.
Dr. Francis Ndaji
[email protected]
Centre for Evaluation and Monitoring (CEM)
Durham University
www.cemcentre.org
Introduction (1)
• What are the P scales?
• Descriptions of attainment levels below level 1 of the
National Curriculum-published 1998.
• Levels P1(i) – P8 and the early levels of the NC
• Example:
• Level description for P4 in Number: Pupils show an
awareness of number activities and counting, for
example copying some actions during number rhymes,
songs and number games; following a sequence of
pictures or numbers as indicated by a known person
during number rhymes and songs.
Introduction (2)
The Durham P scales project
• Annual data collection and analysis of
P scales data at the CEM, Durham University
• How and when did it start?
• Project Aims to
– collect P scales data from schools (up to L4 of NC)
– analyse data
– produce feedback (pupil and whole school evaluation, and
target setting)
– prepare software
Schools that are expected to participate
• Special schools
• Mainstream schools
(a) with special units
(b) having pupils with special educational need in an
inclusion setting
Pupils expected to register for the project
• pupils with special educational needs in both special and
mainstream schools
• age groups of Year 1 to Year 14
• PMLD, SLD, MLD, SpLD, BESD, PD, VI, HI,MSI, ASD,
SLCN, Other.
• More than one categories of special need
Subjects (1)
• English
– Speaking
– Listening
– Reading
– Writing
• Mathematics
– Using & applying
– Number
– Shape, space & measure.
Subjects (2)
• Science
– Scientific enquiry
– Life processes and living things
– Materials & their properties
– Physical processes
• ICT
• PSHE & Citizenship
What do schools get?
The feedback to schools – in three parts
(1) Initial feedback – comparison of attainments
(2) Value added feedback – measures progress
(3) Software – additional analysis
Average attainment by learning difficulty
Average attainments by year group and learning difficulty
16
14
Average attainments
12
10
PMLD
8
SLD
MLD
6
4
2
0
-2
-1
0
1
2
3
4
5
6
Year group
7
8
9
10
11
12
Re-coding P levels into numbers
•
•
•
•
•
•
•
•
•
•
•
P1(i)=1
P1(ii)=1.5
P2(i)=2
P2(ii)=2.5
P3(ii)=3
P3(ii)=3.5
P4=4
P5=5
P6=6
P7=7
P8=8
L1c=9
L1b=10
L1a=11
L2c=12
L2b=13
L2a=14
L3=15
L4=16
Calculating the inter-quartile range for initial feedback
9
8
8
7
7
6
6
5
5
4
3.5
3.5
3
3
3
2.5
2.5
2
1.5
1
25%
25%
25%
25%
Initial feedback (1)
Graph
of Attainments in Speaking for pupils categories as SLD
Speaking
P levels
9
8
7
6
Sample 50% Band
5
4
Year Group Average
3
2
Scores by individual
students
1
0
-1
0
1
2
3
4
5
6
7
8
Year Group
9
10 11 12 13 14
Initial feedback (2)
First
Name
Last Name
Year
Group
Speaking
Reading
Writing
Jane
Kurts
3
2.5
2.5
4
John
Craig
3
4
5
4
Lynne
Adams
4
5
4
3
Dean
Fletcher
4
7
6
6
James
Homer
5
4
4
5
Sally
Luke
6
4
6
6
Initial Feedback (3)
Schools can use the initial feedback to compare attainments:
• Different subjects
• Example question: How does pupil A’s attainment in Speaking
compare with their attainment in Number or Reading or Shape,
space and measure?
First name
Last name
Yr. group
Speaking
Reading
Writing
Alex
Biggs
3
2.5
2.5
4
Julian
Craig
3
4
5
4
Initial Feedback (4)
Schools can use the initial feedback to compare attainments:
• Similar pupils in the school
• Example question: How does Pupil B’s attainment in Reading compare with
the average attainments in Reading of pupils of same age and principal
need in their school?
• Answer - 2 ways to do this (a) using the chart (b) using the averages table
Averages for school
Reading
Writing
Using and
applying
5.2
6
4.8
4.5
5.5
6.5
8.5
8
6
5.2
5.8
6
5.2
4.8
Year Group
Speaking
Listening
0
4.2
1
2
Speaking/
Listening
Initial Feedback (5)
Schools can use the initial feedback to compare attainments:
• Pupil with similar pupils in the whole sample
• Example question: How does pupil C’s attainment in Number compare
with the average attainments in Number for her year group and principal
need in the whole sample?
• Answer - 3 ways (a) using the chart (b) using the averages table (c) Table
of percentiles
Averages for whole sample
Year Group
Speaking
Listening
0
4.8
1
2
Speaking/
Listening
Reading
Writing
Number
5.1
6
4.7
4.5
5.1
5.8
6.7
6.1
5.7
5.4
6.1
7.2
6.3
6.5
Initial Feedback (6)
Schools can also use the percentiles table to see how well each pupil has done
compared to similar pupils in the whole sample e.g. table for MLD pupils.
Year
group
Reading
Percentile
score in
Reading
Number
Percentile
score in
Number
Forename
Surname
Amy
Gonns
7
P8
92
P8
94
Aisha
Tanni
7
P7
87
P7
85
Garry
Morgan
7
P7
87
P8
94
Adam
Rickwood
8
L2a
90
L3
92
Amy
Gordons
8
L2b
79
L2a
83
Ben
Levy
9
L2b
69
L3
90
Dobby
Jicks
9
L2b
69
L4
95
Jenny
Kelly
10
L1b
6
L1c
3
Harry
Tanney
10
L1a
10
L2c
28
Initial Feedback (7)
Schools can use the initial feedback to compare attainments of cohorts of pupils:
Cohorts of pupils
Example question: How do the attainments in Mathematics of MLD pupils in my
school compare with those of SLD pupils?
Answer – compare the average attainment for each year group of MLD and SLD
pupils (Tables are provided with feedback)
Average attainments for SLD pupils
Average attainments for MLD pupils
Year Group
Using &
applying
Number
5.1
0
4.2
4.8
5.1
5.8
1
4.3
5.2
5.4
6.1
2
4.7
5.6
Year Group
Using &
applying
Number
0
4.8
1
2
Initial Feedback (8)
hools can use the initial feedback to compare attainments of cohorts of pupils.
Cohorts of pupils:
Example question: How do the attainments of SLD pupils of my school compare to SLD pupils of
the whole sample.
Answer – Compare average attainments for each year group in School averages table with average
attainment for same year group in whole sample average table. (Tables are provided with feedback)
Average attainments of SLD pupils (School A)
Average attainments for SLD pupils (whole
sample)
Year Group
Reading
Writing
Year Group
Reading
Writing
0
2.5
2.1
0
2.2
2.7
1
2.8
3.5
1
2.5
3.7
2
3.1
3.7
2
3.4
3.5
Initial Feedback (9)
Attainment levels from one year to the other
P scales values
John Apples: Attainment levels from 2005 to 2009
7
6
6
5 5
5
4
5
4
3.5
2.5
2.5
3
2
5
4
6
5
4
5
4
4
2.5
5
4
4
2.5
2.52.52.5
2005
3.5
3.53.5
2.52.52.5
2006
2007
2008
1
0
2009
Speaking
Listening
SpkList
Reading
Writing
Using
Number
John Apples: Attainment levels from 2005 to 2009
4
3.53.5
3.53.5
P scales values
3.5
3
3.53.5
3
2.52.52.5
2.52.5
3.53.5
3
2.52.5
3.53.5
3
2.52.5
3
2.52.5
2005
2.5
2006
2
1.5
2007
1
2008
0.5
2009
0
Shape
SciEnq
Lif eProc
MatProp
PhysProc
ICT
PSHE
Initial feedback (10)
Average scores by year group and ethnic origin
Average scores in Maths for White - UK heritage ethnic group
14
Average P scales
12
10
8
Sample
6
School
4
2
0
-2
-1
0
1
2
3
4
5
6
7
Year group
8
9
10
11
12
13
14
Value added (1)
• Value added measures the progress pupils
make over a period of time.
• Should value added calculations take into
account the special needs of the pupils?
Expected scores and value added in Mathematics for pupils classified as PMLD,
SLD and MLD with prior and current attainments of P4 and P5 respectively
Year
group
PMLD
Expected
score
Value
added
SLD
Expected
score
Learning difficulty
ignored
MLD
Value
added
Expected
scores
Value
added
Expected
score
Value
added
1
4.2
100.8
4.5
100.5
5.2
99.8
4.7
100.3
2
4.3
100.7
4.6
100.4
5.1
99.9
4.7
100.3
3
4.1
100.9
4.4
100.6
5.0
100.0
4.5
100.5
4
4.1
100.9
4.5
100.5
5.1
99.9
4.5
100.5
5
4.1
100.9
4.4
100.6
4.9
100.1
4.4
100.6
6
4.2
100.8
4.4
100.6
5.2
99.8
4.4
100.6
7
3.9
101.1
4.5
100.5
5.2
99.8
4.3
100.7
11
4.0
101.0
4.2
100.8
5.8
99.2
4.3
100.7
12
4.0
101.0
4.3
100.7
4.8
100.2
4.2
100.8
13
4.0
101.0
4.5
100.5
5.0
100.0
4.3
100.7
14
4.0
101.0
4.6
100.4
4.4
100.6
4.3
100.7
Value added feedback (3)
• Value added calculation takes into
consideration:
– Special need category of pupil
– Year group of pupil
– Pupils’ prior attainment
Value added (5)
Value added for pupils classified as MLD
top
2.5%, top 16%, middle 68%, bottom 16%, bottom 2.5%
Name and Year group
First
name
Last
name
Harry
Dean
Jack
Orange
Zainab
Kash
Jenny
Butt
Nathan
Nans
Alexander Kridge
Sam
Dotton
Saddeq
Salman
Dick
Henns
Qadiri
Mansour
Literacy
Numeracy
Value
Average Average
Added
Year Score in Score in Predicted
for
Group 2008
2009
Score
Literacy
4
4
5
5
5
5
6
6
6
6
9.7
11.7
6.5
7.3
10.7
10.3
11
11.3
9.7
11.7
11
12.3
6.5
9
12.3
11.3
12
12
10.3
12
10.4
12.5
7
7.9
11.4
11.1
11.8
12.1
10.4
12.5
100.63
99.85
98.88
102.29
101.9
100.54
100.23
99.84
99.94
99.44
Average
Score in
2008
9.3
11.3
7.7
7.3
12
9.3
12.3
12.7
12.7
12.3
Average
Value
Score in Predicted Added for
2009
Score Numeracy
10
11.3
7.3
8.3
11.3
10.7
12
13.3
13.3
13
10
12.1
8.3
7.9
12.7
10
12.8
13.2
13.2
12.8
99.96
99.29
98.55
100.67
97.78
101.08
99.22
100.14
100.14
100.14
Value Added feedback (6)
The Value Added feedback can be used to compare:
• Progress in different subjects
• Example question: How does the progress of pupil A in English
compare with their progress in Mathematics or Science?
• Answer: Visual inspection of the value added table or using the
P scales software (to be demonstrated later).
Value added (7)
• Progress of similar pupils
• Example question: How does pupil A’s progress in Mathematics
compare with the progress in Mathematics of pupils of same age,
special need and prior attainment.
• Answer: The VA table is presented by special need. The calculation
also took into account age and prior attainment. Therefore direct
comparison with similar pupils is possible and straightforward.
• You can identify, say, a Year 3 MLD pupil with prior average
attainment of say 4 (P4) and compare their value added with that of
another Year 3 MLD pupil of same prior attainment.
Predicted scores for coming year for pupils classified as SLD.
Forename
Surname
Year
Group
Akash
Aisha
Ayo
Mary
Jenny
Kieron
Rilwanu
John
Kenny
Addo
Lisa
Jordan
Gini
Egere
Combes
Graham
James
Akara
Rivers
Kings
Mohammed
Scott
1
1
2
2
3
3
3
4
5
6
6
Predicted
Predicted
Score in score in Score in score in
Reading Reading in Writing Writing in
in 2008
2009
in 2008
2009
4
5
4
6
5
8
2
6
8
12
9
4.5
5.5
4.5
6.5
5.5
9
2.5
6.4
8.4
13
9.3
4
4
5
4
4
8
2
6
7
11
8
4.5
4.5
5.4
4.4
4.4
9
2.5
6.4
7.3
11.3
8.3
Setting and evaluating targets
Writing
End of KS1
attainment
Target
Challenging target
(Median)
(Upper Quartile)
P1(i)
P2(ii)
P1(ii)
P2(ii)
P2(ii)
P2(i)
P2(ii)
P3(i)
P2(ii)
P3(i)
P4
P3(i)
P3(ii)
P4
P3(ii)
P4
P5
P4
P6
P7
P5
P7
L1c
P6
L1c
L1a
P7
L1a
L2c
P8
L2c
L2b
L1c
L2b
L3
Compare attainments with targets
Higher attainment than predicted
Same attainment as predicted
Names and year group
Special
need
Speaking
Special
needs
Predicted for
2009
Attained in
2009
Predicted for
2009
Attained in
2009
Year group
Lower attainment than predicted
Listening
First name
Last name
Hanns
Bashir
9
SLD
P2(ii)
P2(ii)
P2(ii)
P2(i)
Hassan
Kenis
7
ASD
P2(i)
P2(ii)
P2(ii)
P3
Georgina
Battany
6
MLD
P6
P5
P7
P7
Georgina
Greena
6
ASD
P5
P5
P7
P7
Rachel
Betts
7
PMLD
P2(i)
P2(i)
P3(i)
P2(i)
Rachel
Gentt
7
MSI
P2(i)
P2(i)
P2(ii)
P2(i)
Priyadharshini
Arums
3
ASD
P3(i)
P2(ii)
P3(i)
P4
Khira
Harunah
3
ASD
P3(i)
P4
P3(i)
P2(ii)
Spike
Clement
Nursery
PMLD
P1(ii)
P1(ii)
P1(ii)
P1(i)
Brian
Bantu
SLD
P5
P6
P6
P5
11
LA level feedback (1)
Attainments by year group in Science by pupils classified as
MLD in School A compared to MLD pupils in their LA
Average P levels
12
10
8
School A
6
Whole LA
4
2
0
1
2
3
4
5
6
7
8
Ye ar group
9
10
11
12
13
14
LA level feedback (2)
Attainments in English
Attainments by year group in English by pupils classified as M LD
in Example LA compared with M LD pupils in the whole sample.
14
12
10
8
Example LA
6
Whole sample
4
2
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14
Year group
What does Ofsted want during school inspections?
• According to Ofsted, they want schools to be able to
make a judgment about what they consider as
satisfactory/good or better progress and be able to
show why this is so for individuals.
•
•
The value added calculation takes into account age, prior attainment and
principal need. Therefore the value added of a pupil measures their
progress relative to those of their age group and principal need that had
the same prior attainment as them. Using our system, good progress
would be when a child’s value added is average (100), better progress
would be when the pupil’s value added is above average (above 100).
This is a fair comparison because the calculation takes into account
pupils of similar circumstances.
Conclusion (1)
Is the P scales project successful?
• Only users of the feedback can say whether the P scales project is
successful.
A
teacher said, ‘The use of the P scales and CEM’s analysis has helped
to bring the schools’ assessment, recording and reporting procedures into
sharper focus and as a result enabled us to ensure that we set achievable
but challenging targets for pupils and the school, and has supported us in
our dialogues with our School Improvement Partner. The data and its
analysis also supported our school self assessment process and helped us
to be judged an ‘outstanding’ school by Ofsted in 2005 and again in 2008’
Conclusion (3)
• Our OfSted Lead Inspector said that:
• In his opinion the Durham data – particularly the
predicted scores and the Value Added calculations were
the most useful analysis that he had seen for looking at
and comparing Special School Data Nationally. Having
seen Durham data in the past, he commented that this
year’s (2006) analysis is the most useful to date.
Conclusions (4)
‘The feedback has assisted the school in setting
objective-led targets and assessing progress for children
in the Early Years, Reception class and Year 1/Year 2
class.’
The school has been using the levels extensively since
their initial publication and CEM data (the feedback)
greatly helps in the continuing evaluation of their use and
the school’s progress.’