The CEF Levels & Descriptor Scales Brian North

Eurocentres Foundation

Developing the CEF Descriptor Scales

• • •

Full account:

North, B. (2000).

The development of a common framework scale of language proficiency.

New York, Peter Lang.

More Technical:

North, B. and Schneider, G. (1998). Scaling descriptors for language proficiency scales.

Language Testing

15, 2, 217 –262.

Less Technical:

North, B. (2002a). Developing descriptor scales of language proficiency for the CEF common reference levels. In Alderson, J.C.A. (ed.)

Case Studies in applying the Common European Framework

, Strasbourg, Council of Europe, 87-105.

Common Framework of Reference • Relevant areas of concern

–

Descriptive scheme

•

based on descriptive theory

• Stages of attainment in those areas

–

Common reference levels

•

based on measurement theory

Descriptive Scheme

• “….. actions performed by a social agent who, as an individual, has at his or her disposal and develops a range of

general competences

and in particular

communicative language competence

. He or she draws on these competences in different kinds of

language activities

in order to process

text

(receptively or productively) in relation to specific

domains

, activating those

strategies

which seem most appropriate for carrying out the

tasks

to be accomplished”.

Scales in Descriptive Scheme

•

Communicative language competence

– Linguistic, Pragmatic, Socio-linguistic •

Communicative language activities

– Reception, Interaction, Production, (

Mediation)

•

Use of Strategies

– Reception, Interaction, Production

CEF Scales: C.L. Competences

• • • • • •

Linguistic Competence

General Linguistic Range Vocabulary Range Grammatical Accuracy Vocabulary Control Phonological Control Orthographic Control •

Sociolinguistic Competence

Sociolinguistic Appropriateness • • • • • •

Pragmatic Competence

Flexibility Turntaking Thematic Development Cohesion and Coherence Propositional Precision Spoken Fluency

CEF Scales: C.L. Activities.

RECEPTION Overall Listening Comprehension

• • • • • Understanding conversation between native-speakers Listening as a member of an audience Listening to announcements and instructions Listening to audio media and recordings Watching TV and film • • • •

Overall Reading Comprehension

Reading correspondence Reading for orientation Reading for information and argument Reading instructions

CEF Scales: C.L.Activities

INTERACTION Overall Spoken Interaction

• • • • • • • • Understanding a native-speaker interlocutor Conversation Informal discussion Formal discussion and meetings Goal-oriented co-operation Transactions to obtain goods and services Information exchange Interviewing and being interviewed • •

Overall Written Interaction

Correspondence Notes, messages and forms

CEF Scales: C.L. Activities

PRODUCTION

• •

Overall Spoken Production

• • • • Sustained monologue: Describing experience Sustained monologue: Putting a case (e.g. in debate) Public announcements Addressing Audiences

Overall Written Production

Creative Writing Reports and Essays

CEF Scales: C.L. Activities

• •

HANDLING TEXT

Note-taking Processing Text

CEF Scales: C.L. Strategies

•

Reception Strategies

Identifying cues/ inferring • • •

Interaction Strategies

Turntaking Cooperating Asking for clarification • • •

Production Strategies

Planning Compensating Monitoring and Repair

Common Reference Levels • •

A1 A2

• •

B1 B2

• •

C1 C2

Breakthrough Waystage Threshold Vantage Effective Operational Proficiency Mastery

Common Reference Levels

• 

Table 1 Global Scale Table 2 Self-assessment Grid

(Listening, Reading, Spoken Interaction,. Spoken Production, Writing) 

Table 3 Assessor Grid

(Range, Accuracy, Fluency, Interaction, Coherence)  50 Individual Scales for

PROFILING

Common Reference Levels

Proficient User B2 Independ ent User B1 Basic User C2 C1 A2 A1

Can understand with ease virtually everything heard or read. Can summarise information from different spoken and written sources, reconstructing arguments and accounts in a coherent presentation. Can express him/herself spontaneously, very fluently and precisely, differentiating finer shades of meaning even in more complex situations.

Can understand a wide range of demanding, longer texts, and recognise implicit meaning. Can express him/herself fluently and spontaneously without much obvious searching for expressions. Can use language flexibly and effectively for social, academic and professional purposes. Can produce clear, well-structured, detailed text on complex subjects, showing controlled use of organisational patterns, connectors and cohesive devices.

Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialisation. Can interact with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without strain for either party. Can produce clear, detailed text on a wide range of subjects and explain a viewpoint on a topical issue giving the advantages and disadvantages of various options.

Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc. Can deal with most situations likely to arise whilst travelling in an area where the language is spoken. Can produce simple connected text on topics which are familiar or of personal interest. Can describe experiences and events, dreams, hopes & ambitions and briefly give reasons and explanations for opinions and plans.

Can understand sentences and frequently used expressions related to areas of most immediate relevance (e.g.

very basic personal and family information, shopping, local geography, employment). Can communicate in simple and routine tasks requiring a simple and direct exchange of information on familiar and routine matters.

Can describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need.

Can understand and use familiar everyday expressions and very basic phrases aimed at the satisfaction of needs of a concrete type. Can introduce him/herself and others and can ask and answer questions about personal details such as where he/she lives, people he/she knows and things he/she has. Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.

Common Reference Levels • Where do the CEF Levels come from?

• How were the descriptors developed and scaled?

• How were the “cut-points” between levels set?

• How coherent is the scaling of content?

• How stable are the scale values?

Common Reference Levels

•

1913 Cambridge Proficiency C2

•

1990-91 Cambridge Advanced; DALF C1

•

1938 Cambridge First Certificate B2

•

1975 The Threshold Level B1

•

1978? Waystage A2

•

A1

Common Reference Levels

• • • • • • • Wilkins 1978 Ambilingual Proficiency Comprehensive Operational Proficiency Adequate Operational Proficiency Limited Operational Proficiency • • • Basic Operational Proficiency (Threshold Level) • Survival Proficiency Formulaic Proficiency • UCLES 1992 Proficiency CAE FCE Vantage PET Threshold KET Waystage CoE 1992-6 • Mastery • • Effective Operational Proficiency Vantage • Threshold • • Waystage Breakthrough

Developing the CEF Levels

Swiss Research Project 1993-7 to develop:

• scaled

descriptor bank

for the CEF levels • overview of language learning achievement in Swiss educational sectors • prototype European Language Portfolio.

Proficiency Scales before CEF

 Wording tended to be relative. The descriptors were seldom stand-alone criteria one could rate “Yes” or “No”  Situation of descriptors at a particular level was arbitrary - following convention/cliché  Wording often created semantic appearance of a scale, without actually describing anything  Lower levels tended to be worded negatively

CEF scaled criterion statements

• • •

Can take an active part in informal discussion in familiar contexts, commenting, putting points of view clearly, evaluating alternative proposals and making and responding to hypotheses.

Can with some effort catch much or what is said around him/her in discussion, but may find it difficult to participate effectively in discussion with several native speakers who do not modify their language in any way.” Can account for and sustain his/her opinions in discussion by providing relevant explanations, arguments and comments.

B2 Informal Discussion

Swiss Research Project 1993-7

A learner model:

"… a simplified description of selected aspects of the infinite varieties of skills and knowledge that characterise real students" (Mislevy 1995:343) "A learner’s state of competence at a given point in time is a complex constellation of facts and concepts, and the net works that interconnect them; of automatized procedures and conscious heuristics, ….; of perspectives and strategies, and the management capabilities by which the learner focuses his efforts. There is no hope of providing a description of such a state. Neither is there any need to." (Mislevy 1993: 28)

Swiss Research Project 1993-7

A measurement model:

“…the scale values of the statements should not be affected by the opinions of the people who helped to construct it. This may turn out to be a severe test in practice, but the scaling method must stand such a test before it can be accepted as being more than a description of the people who construct the scale. At any rate, to the extent that the present method of scale construction is affected by the opinions of the readers who help sort out the original statements into a scale, to that extent the validity of the scale may be challenged. ” (Thurstone 1928: 547–8)

Swiss Research Project 1993-7

• Intuitive Phase: – Creating a pool of classified, edited descriptors • Qualitative Phase: – Analysis of teachers discussing proficiency – 32 teacher workshops sorting descriptors • Quantitative Phase: – Teacher assessment of learners on questionnaires – Assessment (by all) of videos of some learners • Interpretation Phase: – Setting “cut-points” for common reference levels

Swiss Research Project 1993-7 • 1994: English • 1995: French, German, English • 1996: Portfolio 2,800 learners, 500 classes, 300 teachers Lower & upper secondary, vocational, adult

Swiss Research Project 1993-7

Data Collection:

Quest. C Quest. B Quest. A

Swiss Research Project 1993-7

Setting the cut-points between levels

1. Marking out equal intervals on the scale 2. Identifying „jumps“ in content described, gaps between clusters of descriptors 3. Comparing to original scale author intention 4. Comparing to Waystage, Threshold 5. Fine-tuning for equal intervals 6. Checking for consistency, coherence

Common Reference Levels

Finer Levels (Swiss)

Mastery Effectiveness Vantage + Vantage Threshold + Threshold Waystage + s Waystage Breakthrough Tourist M E V+ V T+ T W+ W B Tour

Council of Europe Cut-off on logit scale Range on logit scale Mastery Effectiveness Vantage Threshold Waystage Breakthrough

-- 3.90

2.80

1.74

0.72

-0.26

-1.23

-2.21

-3.23

-4.29

-5.39

1.10

1.06

1.02

0.98

0.97

0.98

1.02

1.06

1.10

Content coherence

Level A1

 is the point at which the learner can:

interact in a simple way, ask and answer simple questions about themselves, where they live, people they know, and things they have, initiate and respond to simple statements in areas of immediate need or on very familiar topics

, rather than relying purely on a rehearsed repertoire of phrases.

Content coherence

Level A2

reflects

Waystage

• majority of descriptors stating social functions :

greet people, ask how they are and react to news; handle very short social exchanges

;

ask and answer questions about what they do at work and in free time; make and respond to invitations; discuss what to do, where to go and make arrangements to meet

;

make and accept offers.

• descriptors on getting out and about:

make simple transactions in shops, post offices or banks; get simple information about travel; ask for and provide everyday goods and services

.

Content coherence

Level B1

reflects

Threshold Level

• maintain interaction and get across what you want to:

give or seek personal views and opinions in an informal discussion with friends; express the main point he/she wants to make comprehensibly; keep going comprehensibly, even though pausing for grammatical and lexical planning and repair is very evident, especially in longer stretches of free production.

• cope flexibly with problems in everyday life:

deal with most situations likely to arise when making travel arrangements through an agent or when actually travelling; enter unprepared into conversations on familiar topics; make a complaint.

Content coherence

Level B2

reflects three new emphases:  effective argument:

account for and sustain opinions in discussion by providing relevant explanations, arguments and comments; explain a viewpoint on a topical issue giving the advantages and disadvantages of various options.

 holding your own in social discourse: i

nteract with a degree of fluency and spontaneity that makes regular interaction with native speakers quite possible without imposing strain on either party; adjust to the changes of direction, style and emphasis normally found in conversation.

 a new degree of language awareness: c

orrect mistakes if they have led to misunderstandings; make a note of "favourite mistakes" and consciously monitor speech for them.

Content coherence

Level C1

is characterised by access to a broad range of language: fluent, spontaneous communication: 

express him/herself fluently and spontaneously, almost effortlessly; Has a good command of a broad lexical repertoire allowing gaps to be readily overcome with circumlocutions. There is little obvious searching for expressions or avoidance strategies; only a conceptually difficult subject can hinder a natural, smooth flow of language.



produce clear, smoothly-flowing, well-structured speech, showing controlled use of organisational patterns, connectors and cohesive devices

Content coherence

• •

Level C2

represents the degree of precision and ease with the language of highly successful learners:

convey finer shades of meaning precisely by using, with reasonable accuracy, a wide range of modification devices; has a good command of idiomatic expressions and colloquialisms with awareness of connotative level of meaning;

Stability of scale values

Validation Studies:

– Swiss Replication Year 2: 0.99

– DIALANG (List; Read; Write): 0.90

– Basle: University entrance: 0.90

– ALTE / UCLES: (anchors): 0.97

Developing the CEF Descriptor Scales

• • •

Full account:

North, B. (2000).

The development of a common framework scale of language proficiency.

New York, Peter Lang.

More Technical:

North, B. and Schneider, G. (1998). Scaling descriptors for language proficiency scales.

Language Testing

15, 2, 217 –262.

Less Technical:

North, B. (2002a). Developing descriptor scales of language proficiency for the CEF common reference levels. In Alderson, J.C.A. (ed.)

Case Studies in applying the Common European Framework

, Strasbourg, Council of Europe, 87-105.