Transcript (CEFR). - ASER Pakistan

CBSE-CAER International Conference 2014
Delhi-NCR, 28th and 29th April 2014
From Norm- to Standards-based assessment
What role can the CEFR play?
Dr Claudia Harsch
Centre for Applied Linguistics
University of Warwick
Centre for Applied Linguistics
Overview
• Norm- and Criterion-Oriented Assessment
• The Common European Framework of Reference
(CEFR) and its use in assessment
– Classroom-based assessment
– Educational Monitoring
– International Certificates
• Transition from norm- to criterion-orientation:
Chances and Challenges
– Preparing students and teachers
– Aligning exams to the criterion (CEFR)
– The multi-lingual context
Centre for Applied Linguistics
Norm-Orientation
Criterion-Orientation
Centre for Applied Linguistics
Comparison (e.g. Bachman & Palmer 2010, Douglas 2010)
Norm-Orientation
Criterion-Orientation
•
•
•
•
•
• Qualitative description of ...
- what skills / competencies
learners have acquired
- how well learners perform in
relation to learning outcomes
• Transparent expectations, goals,
outcomes expressed in criteria
• Comparability via common goal
posts
Quantitative score
Ranking learners
Comparing learners
Placement decisions
Grading decisions
Centre for Applied Linguistics
The Common European Framework
of Reference (CEFR)
in criterion-oriented assessment
1.
2.
3.
Classroom-based assessment
Educational Monitoring
International certification
Centre for Applied Linguistics
The CEFR
• Instrument to enhance communication and mobility
(language policy of the Council of Europe)
• At core: Proficiency model with levels of proficiency
• Description of learners’ abilities as positive ‘Can do’
statements
• Framework allows qualitative description of
learning outcomes and assessment results (rather
than only numerical scores) –> reference criteria
Centre for Applied Linguistics
The CEFR levels
A1
A2
B1
B2
C1
C2
Centre for Applied Linguistics
The CEFR scale system
Communicative
Competence
Communicative
Language Activities
Reception
Communicative
Communicative
Language Competences
Strategies
Interaction
Production
Grammar
Vocabulary
Orthography
Pronunciation
Centre for Applied Linguistics
CEF Proficiency Levels
Student with
high proficiency
Student with
low proficiency
C2: Can understand a wide range
of long and complex texts,
appreciating subtle distinctions of
style and implicit as well as
explicit meaning.
A1: Can understand very short,
simple texts a single phrase at a
time, picking up familiar names,
words and basic phrases and
rereading as required.
Centre for Applied Linguistics
CEFR in Classroom-based assessment
• Defining learning outcomes, develop curriculum using
CEFR descriptors
• Self-assessment: via the self-assessment grid or the
European Language Portfolio
• Peer-assessment via common criteria, to be derived and
specified on basis of e.g. grid or CEFR descriptors
• Teacher assessment via the CEFR scales, need to specify
for particular performances
• Advantage of common and transparent goals, criteria and
levels, shared expectations
• Possibility of comparing different views on learners
Centre for Applied Linguistics
CEFR in Educational Monitoring
• CEFR as external standard / criterion,
e.g. National Educational Standards (NES) Germany:
• Defining learning outcomes /competencies expected
at the end of secondary schooling in relation to CEFR
levels and descriptors
• Learning outcomes operationalised in standardsbased tests
• Tests formally aligned to the CEFR
• Test results reported on competency scale (like PISA),
accompanied by qualitative descriptions (CEFR, NES)
• Advantage of linking external monitoring to internal
teacher-/student-assessment
Centre for Applied Linguistics
CEFR in International Certification
•
•
•
•
•
•
•
CEFR as common proficiency framework
International tests aligned to CEFR levels
Test results/reports: scores and CEFR levels
Comparability: CEFR as common reference point
Challenge: “Is my B1 your B1?”
High-stakes decisions, e.g. university admission
Political agenda, e.g. visa and immigration
decisions (ethical?)
Centre for Applied Linguistics
Transition from norm- to
criterion-based assessment
1. Preparing teachers and students
2. Aligning exams to the CEFR
3. The Multi-lingual Context
Centre for Applied Linguistics
Preparing teachers and students
• Traditional focus on ranking, competition,
errors, correct answers
• Now: shift towards criteria, outcomes, skills
• Focus on what learners can do with their
language, whether they can perform certain
communicative tasks
• Requires willingness and readiness for change
• Requires training resources
Centre for Applied Linguistics
Preparing teachers and students
• Teacher Training: Familiarity with CEFR
– Document available online: www.coe.int/lang-CEFR
– Online training: www.helsinki.fi/project/ceftrain
• Teacher Training: Familiarity with rating approaches,
self- and peer-assessment, Portfolio
http://www.coe.int/t/dg4/education/elp/Default_EN.asp
• Students: Introduction to self-assessment and
Portfolio by their teachers, familiarity over time
• Classroom: Developing a culture of peer-assessment,
supportive and honest but respectful feedback
Centre for Applied Linguistics
Aligning Exams to the CEFR
• Manual to support such endeavours
http://ww.coe.int/t/dg4/linguistic/manuel1_en.asp
• Several steps to follow – resources needed
• Range of standard setting methods available –
expertise needed (e.g. Cizek et al 2007)
• Human judgement – uncertainties and biases
• Research projects ongoing to
– report exam alignments in a transparent way
– examine most feasible ways of alignment
(e.g. Eckes 2012, Figueras et al. 2005, Harsch et al. 2010)
Centre for Applied Linguistics
The Multi-Lingual Context
• Specific multi-lingual skills, learning outcomes,
expectations
• Not covered in the CEFR
• Possibility to define such a set of multi-lingual
skills for the Indian context
– Relevance
– Transparency
– Added value: setting an example for other multilingual contexts
Centre for Applied Linguistics
Thank You for Your Attention
Claudia Harsch
[email protected]
Centre for Applied Linguistics
Selected References
Bachman, L. & Palmer, A. (2010). Language Assessment in Practice. OUP.
Cizek, G.J., & Bunch, M.B. (2007). Standard-setting. A guide to establishing and
evaluating performance standards on tests. California: Sage Publications Inc.
Council of Europe (2001). The Common European Framework of Reference for Languages. Learning Teaching
and Assessment. CUP. Online www.coe.int/lang-CEFR
Council of Europe (2009). Manual for Relating Language Examinations to the Common European Framework
of Reference for Languages (CEFR). Retrieved from www.coe.int/t/dg4/linguistic/Manuel1_EN.asp
Concil of Europe (2009). Relating Language Examinations to the Common European Framework of Reference
for Languages: Learning, Teaching, Assessment (CEFR). Further Material on Maintaining Standards
across Languages, Contexts and Administrations by exploiting Teacher Judgment and IRT Scaling.
Retrieved from www.coe.int/t/dg4/linguistic/Manuel1_EN.asp
Douglas, D. (2010). Understanding Language Testing. London: Hodder.
Eckes, T. (2012). Examinee-centered standard setting for large-scale assessments: The prototype group
method. Psychological Test and Assessment Modeling, 54(3), 257-283.
Figueras, N, North, B, Takala, S, Verhelst, N and Van Avermaet, P (2005). Relating Examinations to the
Common European Framework: a Manual. Language Testing 22 (3), 1–19.
Harsch, C., Pant, H.A. & Köller, O.(Eds.) (2010): Calibrating Standards-based Assessment Tasks for English as a
First Foreign Language. Standard-setting Procedures in Germany. Münster: Waxmann.
Centre for Applied Linguistics