Transcript Severe Grading in MFL - the people • Helen Myers • David Blow

Severe Grading in MFL
draft Sat 18th
- the people
•
Helen Myers (Past President, ALL & Asst
Head, The Ashcombe School)
ALL = Assoc. for
Language Learning
•
David Blow (Head, The Ashcombe School)
•
Duncan Byrne (Chair ISMLA)
•
Geoffrey Plow (Exams, ISMLA)
•
John Dunford (Gen. Secretary ASCL)
•
Peter Downes (former Chief ML examiner,
Head, President ALL & SHA)
ISMLA = Independent
Schools ML Assoc
Severe Grading in MFL
- brief history
• Long-standing concern - research by Nuttall in
1974; Peter Downes raised over time, leading to
recommendation in the Nuffield Report
• Series of presentations by Helen Myers and
David Blow using DfES and other data
• Continuing concern in ML community, esp as
Languages is no longer compulsory and numbers
go into freefall
• Series of meetings by ALL, ISMLA etc with QCA
and exam boards (‘06)
GCSE MFL grading
• Our outline principle is that students
studying a particular GCSE subject should
have a reasonable expectation that they
will get comparable grades across a range
of subjects.
• The PANDA Relative Performance
Indicator measures the performance of
students in their other subjects compared
with the particular one. By the principle
above, these should be negligible.
They are not.
“Severe grading”
thanks to Peter
Tymms for this
phrase
Terminology is important. Words like “hard”,
“standards”, “demand” carry multiple connotations.
Restate principle that pupils of comparable prior
attainment should attain comparable grades across
a range of subjects whether Business Studies
which will be only started at GCSE, Maths, Drama,
etc all of which have very varying situations for
different pupils
Depressing exams
(NB separate issue from grading)
As professionals, we know (or should know) that
a “difficult” paper with low scores does not need
to lead to low grades as the grade boundaries
/conversion to UMS will be adjusted to
compensate...
BUT pupils leaving that exam will perceive that
they have performed badly, which compounds the
“severe grading” problem
Issues for pupils
• Perceive that they are performing worse
in languages in other subjects
• Reduces the take-up at KS4 and A-level
• In each case, pupils are doing relative
comparisons on grades (note key issue
at AS in French as compare with other
subjs - at A2 Univs can make subj. by
subj. adjustments)
Issues for staff &
managers
• Heads and Governors may think that
languages are “under-performing”
• Mixed messages re school targets and
DCFS targets
• Ritual debate each year about whether
standards are slipping / pupils are
choosing “soft” subjects........
Dearing Review
• National outcry at drop in numbers GCSE Aug 06
- Alan Johnson sets up Dearing Review
• “Mr Johnson wants to see what more can be done to
encourage14-16 year olds to study GCSE or other
language courses leading to a recognised qualification.”
DfES Press Notice 2006/0144 Oct 06
• Meeting Nov 06 with Lord Dearing
–Idea of broad comparability with Maths grading
–Awareness of public and political sensitivity
Numbers - GCSE French
‘02
‘06
German - similar
pattern but 40% as
many, e.g 120,000
(French 300,000)
Grades
‘02
• drop of 30,000 in
number gaining
A*-C from ‘02 to
‘06
• drop of 40,000 in
number gaining
A*-C from ‘02 to
‘08
German - similar pattern
but 10,000 (‘02 to ‘06) and
18,000 (‘02 to ‘08)
‘08
Joint Proposal to Dearing (1)
The mandate of QCA and the examination boards is to ensure that
"standards are maintained over time". This is an enormously
challenging task and one which has a high public profile.
Comparability of grading
It is now accepted by QCA and the exam boards that pupils of
similar prior/concurrent attainment will gain a lower grade in
languages than most other subjects.
The presented documentation is available at http://www.alllondon.org.uk/severe_grading.htm
Because this difference is embedded historically, and their
mandate is to maintain standards over time, a decision from the
DfES is needed to change this.
Joint Proposal (2)
Opportunity for change
In the light of the crisis at KS4, there is now the opportunity
for a change.
The very strong experience of modern language teachers,
advisers and senior managers is that the discrepancy in
grading is one significant negative factor in the decline.
Making changes here will not in itself solve the overall
problem, but, to use a metaphor, will remove having a
hand tied behind one's back.
Joint Proposal
Proposal
That QCA, and thus the exam boards are mandated to
ensure that the grades awarded in Modern Languages
move to become comparable, on average, to those
awarded in Mathematics. Different methodologies exist to
measure the comparability (described in the papers), and
these should all be used, so that the pupils of similar
prior/concurrent attainment have a similar average grade
in Modern Languages to that in Mathematics.
This proposal should be treated independently of the
many other ideas and proposals to deal with the
issue of KS4 M.L.
Dearing Review
• Led to recommendation in Consultation Report
(Jan 07) confirmed in Final Report (Mar 07) to
have definitive study published on “perception” of
severe grading (with implicit corollary that there
should be action...) - the consultation had “found
strong confirmation of the view that award of
grades is more demanding than for most other
subjects”
Dearing Review - press
• Dr Coe, of Durham's curriculum, evaluation and
management centre, believed such trends were repeated
in other years, but insisted "the question of difficulty is not
about content of the subject ... It is purely about the
examination and grading process."
• David Willetts, Conservative education spokesman, said:
"If there is evidence modern languages is tougher than
other GCSEs, then that is something that has to be
corrected. They should be the same level of challenge as
traditional academic GCSEs."
http://education.guardian.co.uk/gcses/story/0,,2031769,00.html
James Meikle, education correspondent
Monday March 12, 2007
Political context
• “ I would like to wait until I have this advice
before making any decisions about
changing grade boundaries” Alan
Johnson, Secretary of State, May 08
referring to the QCA study set up in
response to Dearing recommendation when
replying to joint letter from ASCL, ALL,
ISMLA
• Meetings with Jim Knights, David Laws
(LibDem) and Nick Gibb (Con) to ensure
cross-party awareness
Sensitivity
• We do not underestimate the sensitivity of
this topic
• Remember Feb 08
–“get GCSE ML without speaking”
–“ML Orals abolished!”
• Through all of our involvement, have sought
to make politically realistic proposals - e.g
Lord Dearing liking the idea of comparability
with Maths “no-one will say that Maths is
easy”
QCA report - Feb 08
• Introduced two definitions of comparability:
1. “Attainment–based comparability” - § 8
• “each boundary must be set using
professional judgement. The judgement
must reflect the quality of candidates’ work,
informed by relevant technical and
statistical evidence.' - § 8
• “Fundamentally, awarding meetings are
required to ensure that the quality of work
describing a particular grade this year is the
same as that of the previous year.” - § 9
QCA report - Feb 08
2. Aptitude–based comparability
The ‘aptitude-based’ conception of comparability
holds that two examinations may be seen as
comparable if students of a certain ‘calibre’ have an
equal chance of achieving a particular grade in any
examination (either within subject areas or between
them).
One examination is harder than another if the
results of a (representative) group of students
taking both examinations are worse in it. This is the
approach to comparability on which the ALL
proposal is based.
QCA report - Feb 08
statistical case of “severe
grading” now accepted - see
joint press release
• What if French grades based on candidates’ key stage 3
test scores and their results in GCSE mathematics? The
analyses indicated that the changes would be marked at
the higher grades.
• About half the candidates presently awarded a grade B
would gain a grade A as the threshold mark or
performance standard for a grade A would have to move
down by about half a grade width. There would be a
similar effect at grade C.
evidence suggests that for D grade in French 70% gain higher grade in Maths
QCA report had said that only 123 schools (excluding grammar schools) had
maintained a high rate of GCSE languages take-up (over 90 per cent entered
in each year from 2003 to2006).
QCA report - Feb 08
•
Conclusion: “The present examination system is not based on an
aptitude-based conception of comparability and its adoption would
create a major threat to public confidence in students’ results. We
have similar concerns about applying a new performance standard to
languages but not to any other subjects, some of which might also
want to claim special status.
We do not have evidence that there have been significant changes to
grade standards in GCSE languages in recent years. In conclusion,
we do not therefore, recommend any adjustment to national grade
standards in GCSE languages.”
•
unfortunate implication in conclusion of report that T&L special
problem with MFL :
“Instead we should focus on improving levels of teaching and learning
in modern languages in order to gain students' commitment and raise
performance.”
Brigshaw School & Language College
• 2007 GCSE results:
– 53% 5A*-C; 40% 5A*-C inc En + Ma
–24% A*-C in MFL i.e. 59 out of 243
presented at
Language College
Conference May 08
and Oct 08
• comparing MFL results with Maths:
– 70% lower grade than Maths
– 23% same grade in Maths
– 7% higher grade than Maths
• so in Year 11 those not in line for C switched to
Asset Languages (138 of 240) of which expect 46
to get equiv grade C or higher. So (46 +102) / 240
= 60% A*-C in MFL for Jun ‘08
but is it T&L issue?
Brigshaw School & Language College
“What is most striking about the school is the pervasive sense
that it is a place where learning languages matters. The school
has extensive provision for languages, with good links across
subjects so that pupils can see how and why they can apply their
language skills and knowledge. Pupils appreciate the very good
opportunities to get involved in visits abroad. New courses are
ensuring that, whilst everyone has to study a language
throughout their time at school, there are courses to suit different
needs. Extensive use is made of the expertise in the language
college to support other schools and the community.”
Ofsted Nov 06
The school had also called in the examiners to look at the
papers, and they agreed that the school had properly prepared
the candidates
Way forward
• We believe it is unrealistic to move to
comparable grading for all subjects
• Currently 4 main bands
– ML
– Maths, Sci, His, Geo
– English
– Art, Drama, PE,
• We are proposing simply that ML moves
into the 3rd band - tiny changes in grade
boundaries involved, similar to those which
have taken place - “one-off step change”
data from
CEM
SCORE
paper
subject
1
2
ave
ave score in
score in
subject
OTHER
G.S.
73.0
Biology
79.5
Physics
82.7
Chemistry
85.8
German
86.4
French
89.0
Music
79.5
History
85.2
Mathematics
89.7
Economics
89.9
Other Soc St
81.8
Eng Lit
85.6
Geography
85.3
R.S.
87.5
DT
78.0
Sport St/PE
75.5
Business St
81.8
Drama
85.1
Sociology
84.7
Comm St (Media)83.5
3
subjects
difference
87.8
90.8
92.2
95.0
95.3
96.2
84.9
90.0
93.5
92.4
84.0
86.5
83.7
85.6
75.6
72.8
78.2
80.2
79.4
75.8
-14.8
-11.3
-9.5
-9.2
-8.9
-7.2
-5.4
-4.8
-3.8
-2.5
-2.2
-0.9
1.6
1.9
2.4
2.7
3.6
4.9
5.3
7.7
Situation of
MFL - A/L
To asses the relative performance of subjects in the
context of uneven grading at a national level, there is
a calculation in the PANDA of
nearly
1 grade
2) the average points score of those candidates in
the OTHER subjects they are taken.
on
average
The difference between the two figures
shows how
1) the average pts score of candidates in the subject
the grading for the subject compares to other
subjects. Because the second figure is calculated
ONLY for the candidates doing the first subject, it
automatically takes account of the ability profile of
the candidates doing the subject
This establishes the national context, and if the
same calculation is done for the school, it removes
the "severe grading" issue from the consideration of
performance
from post-16 PANDA
national data for Relative
Performance Indicator (RPI)
20 points
= 1 grade
French - blue
German - black
Situation of MFL - A/L
relative grading of subjects - A/L 2004
nearly
1 grade
on
average
0
-5
-10
Art
Comm St (Media)
Sociology
Drama
Business St
Sport St/PE
DT
R.S.
Geography
Eng Lit
Other Soc St (e.g. Psych)
Economics
Mathematics
History
Music
French
German
G.S.
-20
Chemistry
-15
Physics
"difference"
as in
previous
graph
5
Biology
Col. 3
difference in UCAS pts
10
subject
from post-16 PANDA national data for
Relative Performance Indicator (RPI)
20 points
= 1 grade
Situation of MFL - A/L
French - blue
German - black
average pt v "severe grading" A/L 2004
10
Comm St (Media)
Art
Col. 3
Sport St/PE
DT
relative grading of subject
"difference"
as in
previous
graph
Sociology
Drama
Business St
5
0
0
70
75
-5
R.S.
Geography
C
89.
0 90
85
Economics
Other Soc St
-7.2 Mathematics
Eng Lit
80
B
95
History
Music
French
Physics
-10
German
Chemistry
Biology
-15
G.S.
-20
ave pts per subject
20 points =
1 grade
Col. 1 - average score in subject
100
French - blue
German - black
Situation of MFL - A/L
average pt in OTHER subjects v "severe grading" A/L 2004
10
Comm St (Media)
Art
Sociology
5
Col. 3
relative grading of subject
"difference"
as in previous
graph
Sport St/PE
DT
0
0
-5
70
75
Drama
Business St
C
80
7.2
R.S.
Geography
96. B
95 2
100
Economics
-7.2
Mathematics
Eng Lit
85
90
Other Soc St (e.g.
Psych)
History
Music
French
-10
Chemistry
Physics
German
Biology
-15
G.S.
-20
ave pts per subject in OTHER subjects = "ability profile"
20 points =
1 grade
Col. 2 - average score in OTHER
subjects (related to ability profile)
Issues for MFL - A/L
• Note that French & German have high “severe
grading factor” at A/L.
• French & German have the highest average grades
in OTHER subjects
• So
– will suffer in comparison with other subjects,
– are only left with the most able
• Compounded by situation at AS:
for these 2004 A/L grades,
in 2003 at AS French had: subject ave 22.9
OTHER subject ave 29.9 (diff = - 7.7)
A virtuous circle???
• Language grading is fair, so...
• Pupils choose MFL at KS4, feel good
after their exams and get grades in line
with other subjects so...
• More choose at AS and get grades in
line with other subjects so...
• More continue to A2 and so...
• Everyone is happy!