Transcript VCAA Study Design

Statistical Moderation
and Study Score
What and Why
Stat Mod and Study Score
• A process that Moderates the Schoolassessed Coursework scores
• Uses the exam scores because it is the only
task that is exactly the same for each student
in the study
What is Statistical
Moderation?
Stat Mod and Study Score
Process to ensure comparability of results
Aims to eliminate marking differences between
schools, not within
Aligns external and internal assessment scales
Maintains rank order given by the school
The Common Scale
Stat Mod and Study Score
New Zealand $300
Hong Kong $1300
USA $250
Which is the Cheapest ?
The Camera from Hong Kong
Why is it Needed?
Stat Mod and Study Score
Assessment marking varies between schools
Measurements are only comparable when they are
expressed on the same scale
School-based assessment can only be included in the
ATAR (ENTER) if it is moderated
How is it Done?
Stat Mod and Study Score
Identify the moderation groups in each study
Calculate an external score for each student
Align the distributions of coursework and external
scores
Calculate moderation formula and then apply to the
coursework score for each student
Using the Common Scale
Raw School
Assessment
Stat Mod and Study Score
External Score =
Moderated School
Assessment
Highest Score in School
Highest Score in external
Upper Quartile in School
Upper Quartile in external
Median in School
Lower Quartile in School
Zero score in School
Median in external
Lower Quartile in external
Zero score in external
Typical Group
Stat Mod and Study Score
Course Raw Score (0-100)
Examination (0-120)
External Score (0-100)
Moderated Score
100
95
89
89
99
92
87
89
92
86
84
85
86
83
82
79
85
77
78
78
83
72
75
75
83
69
74
75
81
69
74
73
68
59
68
67
65
56
66
66
62
52
63
61
61
49
61
60
60
43
58
58
59
35
53
56
57
32
51
51
47
31
51
51
40
30
50
50
35
29
49
49
MAXIMUM
UPPER QT.
MEDIAN
LOWER QT
Capable Student in
a Lower Performing Class
Stat Mod and Study Score
CRS
CRS
CRS
(0-100)
(0-100)
(0-100)
EXAM
EXAM
EXAM
(0-120)
(0-120)
(0-120)
EXT
EXT
EXT
(0-100)
(0-100)
(0-100)
MOD
MOD
MOD
(0-100)
(0-100)
(0-100)
93
93
111
111
98
98
78
78
88
88
84
80
72
72
62
62
72
69
69
72
72
60
60
70
68
68
70
70
47
47
68
66
66
61
61
38
38
64
55
55
59
59
37
37
63
54
54
56
56
34
34
62
52
51
53
53
30
30
60
50
50
MAXIMUM
MAXIMUM
MAXIMUM
MEDIAN
MEDIAN
MEDIAN
Tie at the Top
Stat Mod and Study Score
CRS
CRS
EXAM
EXAM
EXT
EXT
MOD
MOD
100
100
120
120
100
100
100
100
100
100
119
119
100
100
100
100
100
100
119
119
100
100
100
100
CRS
CRS
EXAM
EXAM
EXT
EXT
MOD
MOD
98
98
120
120
100
100
100
100
98
98
119
119
100
100
100
100
98
98
119
119
100
100
100
100
CRS
CRS
EXAM
EXAM
EXT
EXT
MOD
MOD
100
100
119
119
100
100
88
100
100
81
81
82
82
88
100
100
81
81
82
82
88
Partnerships and
Statistical Moderation
Stat Mod and Study Score
• Statistical Moderation works most effectively with groups
larger than 5
• The VCAA requires that schools form partnerships
• Schools who find it difficult to find a partner for any
reason; lack of time / lack of access / lack of willingness
can be reassured that the statistical moderation process
is run as usual with the addition of precautionary checks
VET and
Statistical Moderation
Stat Mod and Study Score
• The moderation group for VCE VET coursework scores
is the RTO that has delivered the program for a group of
students.
• The group may be all the schools auspiced by the RTO
Statistical Moderation Report on VASS
Statistical Moderation Report on VASS
Modifications to the Report
Modification to the Report
Important Points to
Remember about Stat Mod
Stat Mod and Study Score
• Required so that school assessment can be used fairly as part of
VCE assessment
• Any adjustment to a students’ coursework scores is made solely on
the basis of the relationship between the groups coursework scores
and external scores for the group.
• Does not change the rank order of students as determined by the
school
• Not influenced by students with negative anomalous results as they
are excluded from the moderation process before moderation
formulas are calculated
• Students whose scores are excluded have their scores moderated
using the formulas calculated for the rest of the group
Study Score Calculation
Stat Mod and Study Score
• To be eligible for a study score a student must achieve ‘S’
results for units 3&4 in the one year, unless granted Interrupted
Studies status.
• Achieve two or more GA results in the study
• A study score is calculated using the students’ final numeric
score in each GA with each GA contributing a specified
percentage
• Number between 0-50 with mean 30 and standard deviation 7
• Study scores are distributed according to the properties of a
normal distribution curve
Study Score Calculation
Stat Mod and Study Score
Study Score Calculation
• Calculate
• Subtract
• Divide
• Multiply
• Add
Stat Mod and Study Score
Study Score
Example 1
Stat Mod and Study Score
MAX. SCORE
WEIGHTING
MEAN
STANDARD
DEVIATION
A+ GRADE
SCALE
GA1
100
25%
69.21
15.52
84 – 100
GA2
100
25%
70.78
14.95
85 – 100
GA3
120
50%
66.32
18.61
94 – 120
STUDENT A
SCORE
GRADE
STANDARDISED
SCORE
WEIGHTED
SCORE
GA1
100
A+
1.984716
0.496179
GA2
100
A+
1.953861
0.488465
GA3
106
A+
2.132657
1.066328
STUDENT B
SCORE
GRADE
STANDARDISED
SCORE
WEIGHTED
SCORE
GA1
84
A+
0.953468
0.238367
GA2
90
A+
1.285180
0.321295
GA3
95
A+
1.541442
0.770721
2.050972 = 47
1.330383 = 40
Study Score
Example 2
Stat Mod and Study Score
MAX. SCORE
WEIGHTING
MEAN
STANDARD
DEVIATION
A+ GRADE
SCALE
GA1
100
25%
69.21
15.52
84 - 100
GA2
100
25%
70.78
14.95
85 – 100
GA3
120
50%
66.32
18.61
94 - 120
STUDENT C
SCORE
GRADE
STANDARDISED
SCORE
WEIGHTED
SCORE
GA1
96
A+
1.726904
0.431726
GA2
98
A+
1.820124
0.455031
GA3
92
A
1.380202
0.690101
STUDENT B
SCORE
GRADE
STANDARDISED
SCORE
WEIGHTED
SCORE
GA1
84
A+
0.953468
0.238367
GA2
90
A+
1.285180
0.321295
GA3
95
A+
1.541442
0.770721
1.576858 = 42
1.330383 = 40
Things to Remember
Stat Mod and Study Score
• Indicates how the student performed against all other students who
did the study in that year.
• Calculated using the numeric results for the GA not the grade.
• Moderated coursework scores are used when calculating study
scores to ensure fairness.
• A strong performance on one GA and a weak performance in
another GA will not necessarily result in a high study score
• Studies where there are less than 250 students are reviewed each
year by the Study Score Review Panel, consisting of Executive
members of the VCAA and VTAC and adjustments made based on
the available data from the previous five years.
What is Used in the
Calculation of a DES
• School assessment is the primary data
• In addition, other data is used where appropriate
• If the student has the DES application approved, and the best
predictor is better than the achieved score then the predictor
becomes the student’s final score.
• For all examinations, analysis indicates that the two other GA
scores provide the greatest contribution to all the predictors.
Indicative Grades
Indicative Grades:
• primary purpose is to be used as part of the quality
assurance procedures for marking examinations, however
they are also used in the calculation of the DES.
• are the prediction of a student’s actual level of
achievement on the examination
• the rank order and level of spread of the indicative grades
for the school cohort in the study is of prime importance
The published previous years grade range can be used to
inform this process.
Contact Details
Student Records and Results
[email protected]