Transcript Testing 09 - Xiamen University

Testing 09
Problems and Directions
• Past experience with language help us come
to the conclusion that we should keep a
balance between linguistic and
psychometric considerations.
• Shift from psychometric-structures to
psychometric-comunicative
Authenticity: communicative
approach in testing
• Two approaches in testing: discrete-point vs.
integrative
• Discrete-point: emphasize on structure,
lexicon
• Integrative: emphasize on communicative
effect
Authenticity
• Two definitions of authenticity
• 1. Real-life performance (RL): the extent to which
test performance replicates some specified nontest language performance. Its main concerns are
(1) face validity -- appearance or perception of the
test and its effect on test performance and test use,
(2) predictive utility -- accuracy with which test
performance predict future non-test performance.
This approach does not distinguish between
language ability and the context in which this
ability is observed.
Authenticity
• 2. Functionality or illocutionary purpose or
interactional/ability (IA).Ability: theories of
verbal abilities. Iinteractional:
communicative language use.
Real life language use and
authenticity
• Proficiency and authenticity are effectively
synonymous.
• Direct proficiency test: the testing format and
procedure attempts to duplicate as closely as
possible the setting and operation of the real-life
situations in which the proficiency is normally
demonstrated.
• Indirect test: they may in many cases bear little
formal resemblance to linguistic situations that the
student would encounter in real life.
Real life language use and
authenticity
• Because of the difficulty of replicating nontest performance in tests, authenticity is
described as a continuum between direct
and indirect.
• Validity: The primary validities of RL
approach are predictive utility, face validity
and content relevance and coverage.
Criticisms of the RL approach
• Failure to distinguish ability from behavior:
competence can not be tested in any direct way.
We can measure it through manifestations of it in
performance.
• Inadequate basis for validation: Face validity is
the mere appearance of validity and that face
validity judgments are naïve because appearances
in testing are treacherous, and well-established
deceivers.
Authenticity as interactive
language use
• If we could develop a means of classifying
test tasks on the basis of dimensions, or
factors that we abstract from authentic
language use, we should be able to
characterize the relative authenticity of a
given test task in terms of its potential for
generating an authentic interaction with the
abilities of a given groups of test takers.
Case study: RL and IA in oral
proficiency testing
• The RL and IA have distinct implications
for both the design and interpretation of a
given type of language test.
• RL: ILR Language Skill Level Descriptions
• ACTFL Proficiency Guidelines
• OPI (Oral Proficiency Interview)
• IA: Oral Interview Test of Communicative
Proficiency in English
ILR Level Two
• Can handle with confidence but not with
facility most social situations including
introductions and casual conversations
about current events, as well as work,
family, and autobiographical information;
can handle limited work requirements,
needing help in handling complications or
difficulties. Can give directions from one
place to another.
ACTFL Advanced Level
• Able to satisfy the requirements of everyday
situations and routine school and work
requirements. Can handle with confidence but not
with facility complicated tasks and social
situations, such as elaborating, complaining, and
apologizing. Can narrate and describe with some
details, linking sentences together smoothly. Can
communicate facts and talk casually about topics
of current public and personal interest, using
general vocabulary.
IA: Grammatical competence
• pp. 326-327
Difference between RL and AI
• Different views of language proficiency.
• RL: language proficiency is viewed as a
unitary ability.
• AI: three main scales—grammatical
competence, pragmatic competence and
sociolinguistic competence, corresponding
to the different components of language
ability.
Difference between RL and AI
• Relationship between the contextual
features of the elicitation procedure and the
ability to be measures.
• RL: there is no distinction between the
ability to be measured and the features of
the context in which language performance
takes place.
• AI: scales are defined in terms of levels on
the various component abilities.