Transcript THE DUTCH “PAMPHLET” AND THE USE OF THE CEFR

THE DUTCH “PAMPHLET” AND THE USE OF
THE CEFR
UNIVERSITY OF LEUVEN
Piet Van Avermaet & Veerle Depauw
Kransjka Gora, May 2004
OVERVIEW
•
•
•
•
•
The Pamphlet of the Dutch Language Union
Misuse and impact of the CEFR?
The CEFR as a point of reference or as a prescriptive tool?
Example of how VDAB uses the CEFR
The CEFR providing a meta language for communication
The Pamphlet of the Dutch Language Union
1 Objectives for testing have to be set socially and educationally.
2 These objectives for testing can subsequently be related to regional (Flemish),
national (Dutch) or international (e.g. CEFR) frameworks or collections of
objectives. Regional or national frameworks are preferably related to the Common
European Framework of Reference.
3 The Common European Framework of Reference should as such be used as a
frame of reference and not as a prescriptive instrument.
4 Linguistic competence should be tested functionally in a user context.
5 When describing and assessing linguistic competence, differentiation with regards
to content is needed; differences as to social, educational and professional roles
(contexts of language use) will have to be visible in tests and in language products
to be assessed.
Why this pamphlet?
Concern that CEFR would become more and more a prescriptive instrument?
Based observation, communication
Is concern legitimate/justifiable?
A few examples
Misuse of the CEFR?
•
•
CEFR
a non prescriptive instrument
misuse
Do we all see the CEFR as a point of reference instead of a normative
instrument?
•
Conflicting discourses or mismatch between discourse and behaviour
•
discourse
behaviour
point of reference
not prescriptive
compendium
…a basis for…
item bank
templates
What is a template? A special shape or pattern used to make identical ...
Mismatch between discourse and behaviour?
How comes that there is this mismatch?
• We want to remain faithful to basic principles and goals of CEFR
• But we also assume/claim that everybody is familiar with the CEFR, uses it as
a basis for test development, syllabus design, etc… I.e. use as a rather
normative instrument
• However, who are the people who use it as such? The same who state that the
CEFR is a point of reference?
The CEFR and rationalisations
What about the others?
The not us, policy makers, teachers, curriculum
designers, language planners, etc…
Why do they use it? Why have they decided to use the scales?
Rationalisations?
The attraction of the dominant discourse, of power
subversive mechanism
IMPACT of CEFR
We don’t now!
What is the actual impact (political, educational, societal)?
What is current behaviour in Europe in relation to CEFR?
What are people’s perceptions of the CEFR?
How do these perceptions relate to their behaviour?
What made people decide to use it?
How do people use it?
How do they interpret it?
How would they use if they knew it in depth?
CEFR as prescriptive instrument
To what extent is it acceptable, desirable to impose a norm?
Do we want to change the CEFR from a point of reference to a more prescriptive
instrument?
But we don’t want to!
We want to remain faithful to its basic principles
Wait, let’s have a look at a few things that are going on now in Europe:
idea of European items (decontextualised?, across languages?)
specifications across languages
one construct underlying the CEFR
templates for development of test items
CEFR scales as basis for and an item bank for Barcelona indicators project
How do I know that my B1 is your B1?
•
How do we have to interpret this question in relation to what I just said?
•
How do I know that my mojito is your mojito?
–
–
–
–
–
•
Different rums
how much rum
lemon or lime
how many drops of lime/lemon
syrup of sugar cane, real sugar cane, white sugar, vanilla sugar
different mints
how many leaves
what is interaction effect of these different ingredients on a mojito?
Do all ingredients of a mojito have to be exactly similar to call it a mojito? Can
they differ according to the ingredients available, to the local context?
How do I know that my B1 is your B1?
Suppose you claim your exam is at B1
fair question
What is rationale behind having an exam at B1?
impact of CEFR?
Who wants to know what people have to do at B1?
Who wants to know whether a person is at B1?
Or do we want to know what people have to do with language in a given situation?
Suppose you start from a needs analysis and develop real life tasks: different
features are at different levels. What claim should one make? How can one
answer the question how do I know that my B1 is your B1?
Do all items have to be at B1?
Do all testtasks have to be at B1?
Do all components of an examination have to be at B1?
How do I know that my B1 is your B1?
•
Can we answer the B1 question for examination/items/testtasks from a content
and consequential perspective, from a functional perspective?
–
–
–
–
context validity features (Weir 2004)
for each feature specifications along the CEFR scales. Is this possible?
What is effect of interaction of features on level?
What if an item/test task consists of different levels for different goals and/or
features?
B1
A2
1
A2
A1
PROFILING
B1
B1
2
B2
A2
How do I know that my B1 is your B1?
Answering this question is even more problematic if CEFR is basis for test
development.
In that case danger is that we end up with items/test tasks that are perfect at a
level, but have low real life validity.
A kind of artificial test tasks based on an approach of development through
segmentation.
In real life situations people often have to perform in a more integrated way often
with features at different levels.
How far can a test be from real life?
Who are we?
• VDAB: one of the main providers of
vocational training for adults in
Flanders
• An important challenge: we meet a
lot of non-native adults (DSL) who
want to attend a vocational training
course
• The development of a (language)
curriculum with (language)
assessment procedures at important
stages in the learning process
(vocational & language)
The language curriculum in Flanders
BASIC LEVEL (richtniveau 1.1) Provided by another supplier
CEFR
NATIONAL
FRAMEWORK
C2
SHORT DESCRIPTION
RECOMMENDED
LANGUAGE
TRAINING
DURATION
In total: 1200
hours
15 hours a
week = one and
a half year!
C1
Richtniveau 4
240 h
B2
Richtniveau 3
240 h
B1
Richtniveau 2
480 h
A2
Richtniveau 1.2.
1.1. + 1.2. : 240 h
A1
Richtniveau 1.1.
(=basic
proficiency)
A few important characteristics of our
language learners
• They are aiming at a job, as soon as possible
• They have a financial need to find a suitable job
• Language learning is not a goal as such, but
rather a means
No mastery level before entering the
vocational courses
• Due to the learners’ profile
• Due to our view on language learning:
performing tasks in a natural environment
instead of exclusively learning the language
in the classroom
NEEDS ANALYSIS
A SET OF SPECIFIC GOALS
LANGUAGE CURRICULUM & ASSESSMENT
The language curriculum in Flanders
(a professional perspective )
TRAINING FLOOR
VDAB-trainee
Example: assessment ‘The Springboard’
SPECIFIC LANGUAGE
VDAB-trainee
Example: the Elevator
BASIC LEVEL
Provided by another supplier
A sample of the global scale
B2
Can understand the main ideas of complex text on both concrete
and abstract topics, including technical discussions in his/her field of
specialisation …
B1
Can understand the main points of clear standard input on familiar
matters regularly encountered in work, school, leisure, etc. Can deal
with most situations likely to arise whilst travelling in an area where
the language is spoken.
The CEFR is a very useful instrument
A meta language for communication
A point of reference for comparing and profiling
But above all a compendium for more than only testing and assessment
Some, however, have doubts and say that the emperor has no clothes.
However, in the light of the above (ie the danger of becoming more and more
prescriptive) does the emperor really need clothes?
Or maybe he needs different sets of clothes for different occasions.
The same question for the need of an underlying construct. Does the CEFR need
one underlying construct? What if people start from another construct?
What we need in the first place is a clear rationale and clear test specifications for
different examination contexts (for each test) which can be compared to each
other using the CEFR as a point of reference and can be used for profiling tests.
The CEFR is a very useful instrument
•
We much ask ourselves how far we want to go in using the CEFR as a
prescriptive instrument and in the development of instruments along this line.
•
How far does Europe want to go?
•
How far do national education boards want to go?
•
How far do schools, teachers, employers, students, candidates want to go?
The CEFR is a very useful instrument
Tests and frameworks are powerful tools and have enormous impact.
Instruments like templates, itembanks, specifications etc, increase their power.
A European item bank looks great. And might be ok (although: decontextualised
items across languages) as long as it functions as a point of reference like the
CEFR. But we know that the CEFR is becoming more and more used as a
prescription.
We don’t have to make our own mojito anymore. We buy it (mixed, shaken and
stirred). The best. Quality ascertained. And reliable: the same for everybody and
everywhere
HOWEVER
Languages and people
=
coca cola