Transcript No Slide Title

Introduction to
CEM Secondary Pre-16
Information Systems
Neil Defty
Secondary Systems Programme Manager
Ensuring Fairness
Principles of Fair Analysis :
1. Use an Appropriate Baseline
2. Compare ‘Like’ with ‘Like’
3. Reflect Statistical Uncertainty
CEM Systems
Principle of Fair Analysis No1 : Use an Appropriate Baseline
The Assessments
MidYIS - Year 7, 8 or 9
1 session – Computer
adaptive or paper-based
Developed ability assessments:
Vocabulary
Maths
Non-verbal (Cross-sections, Block Counting and Pictures)
Skills (Proof Reading and Perceptual Speed & Accuracy)
SOSCA (“INSIGHT Core”) – Year 9
3 sessions - Computer
adaptive
Curriculum-based assessments:
Reading (Passage Comprehension, Text Comprehension and Speed Reading)
Maths (Number & Algebra, Handling Data and Space, Shapes & Measures)
Science (Biology, Chemistry and Physics)
INSIGHT – Year 8* or 9
(* depending on numbers)
4 sessions - Computer
adaptive
Developed ability assessments (‘MidYIS’) plus Curriculum-based assessments (‘SOSCA’)
Yellis - Year 10 or 11
Developed ability assessments:
Vocabulary
Maths
Patterns (Matching and Reflections & Rotations)
1 session - Computer
adaptive or paper-based
Computer-adaptive vs Paper-based testing
Computer Adaptive
Number of
Students per
Test Session
Limited by number of
computers available – multiple
testing sessions
Cost
Roughly 30% cheaper than
paper-based test
Processing of Baseline feedback available
within a couple of hours of
Baseline
testing
Feedback
Preparation
Student
Experience
Must be able to install the
software or access the internet
version of the test
“Tailored” assessment
Paper-based
Can test all students in a
single session (in hall or in
form groups) or in more than
the one session
Standard cost
Takes around 2-4 weeks for
papers to be marked
No pre-test set up
All students see all questions,
irrespective of suitability
The Analysis
.
Linear Least Squares Regression
Outcome
Subject
SubjectXX
8
7
6
5
4
3
2
1
0
-ve VA
+ve VA
50
70
80
90
100
100
110
120
150
130
Baseline
Residuals
Regression Line (…Trend Line, Line of Best Fit)
Outcome = gradient x baseline + intercept
Correlation Coefficient (~ 0.7)
Some Subjects are More Equal than Others….
Trend segments for 2009 GCSEs: drawn over 68% of Yellis intake
(mean +/-1 standard deviation)
GCSE point score (A*=8, A=7, B=6, …)
8
7
6
English
5
Maths
Art & Design
Geography
4
Biology
French
3
Business Studies
Core Science
2
Additional Science
Add Appl Science
1
20
30
40
50
60
70
80
Yellis Test Score (%)
Principle of Fair Analysis No2 : Compare ‘Like’ with ‘Like’
The Assessments
Maths
Vocabulary
Non-verbal
Skills
Maths
Science
Reading
Baseline Assessment and
Predictive Feedback
Individual Pupil Recordsheets (IPRs)
Band Profile Graphs
Predictions
Chances Graphs
English Language - band D
English Language - band C
50
40
31
30
24
23
20
10
Percent
Percent
50
10
9
2
1
20
20
0
0
29
30
10
0
35
40
1
2
U
G
7
6
U
G
F
E
D
C
B
A
A*
F
E
Grade
0
50
Percent
21
20
7
10
0
0
5
2
0
0
U
G
F
E
D
C
Grade
B
A
A*
50
40
24
C
English Language - band A
40
30
D
Grade
English Language - band B
Percent
1
0
B
A
A*
35
40
26
30
23
20
10
8
0
0
0
1
U
G
F
E
7
0
D
C
Grade
B
A
A*
Value Added Feedback
Burning Question :
What is my Value-Added Score ?
Better Question :
Is it Important ?
Principle of Fair Analysis No3 : Reflect Statistical Uncertainty
Grade Points Equivalent
The Scatter Plot
Look for Patterns…
General under- or over-achievement ?
Baseline Score
Do any groups of students stand out ?
– high ability vs low ability ?
– male vs female ?
Value Added Feedback
Year 7 Pupil Level Residuals to GCSE
-0.2
-0.5
Double Science
Single Science
0.1
Religious Studies
-0.7
Physics
-0.5
Music
Media Studies
Maths
Information Technology
-0.4 -0.4
Home Economics
-0.7 -0.7
History
-0.2
German
Geography
-0.6
French
-0.2 -0.2 -0.1
English Literature
0.1
English Language
-1.0
Drama
Design and Technology
Chemistry
Biology
Art
Average Standardised Residual
Standardised Residuals Graph
4.0
3.0
2.0
2.0
1.0
0.1
0.4
0.0
-0.4
-2.0
-2.2
-3.0
-4.0
Statistical Process Control (SPC) Chart
Attitudinal Surveys
Attitudinal Feedback
Your data is above the Yellis average
Your data is below the Yellis average
Your data is about the same as the Yellis average
Your data is about the same as the Yellis average
Attitudinal Feedback
Secondary Pre-16 Contact Details
Tel:
0191 334 4255
Email:
[email protected]
[email protected]
[email protected]
[email protected]
Web:
www.cemcentre.org