Transcript Language, Dialect, and Register

Language, Dialect, and Register:
Sociolinguistics and the Estimation of
Measurement Error in the Testing of
English-Language Learners*
Guillermo Solano-Flores
American Institutes for Research
NCCRESt Conference,
Arizona State University in Tempe, AZ
November 18, 2004
*Research funded by the National Science Foundation, REC-0336744
A warm-up exercise
What is the animal that eats shoots and
leaves?
What is the animal that eats, shoots, and
leaves?
Overview
• The sociolinguistic dimension of ELL
testing
• Key concepts: dialect, register, linguistic
misalignment
• Heterogeneity of populations, score
variation, and measurement error
• Dialect and register: as important as
language in ELL testing
ELL testing: Levels of analysis
language
dialect
register
The concept of dialect
Dialects used by
different speech
communities
Language
Dialect in testing: An example
How much money does Laura need to pay
for two candies that cost 45 cents each?
How much money does Laura need to pay
for two sweets that cost 45 cents each?
How much money does Laura need to pay
for two 45-cent candies?
Dialect intersection
Dialect A:
Speech
Community A
A
AB
B
Dialect B:
Speech
Community B
The concept of register
Registers: Uses
of language in
different contexts
Language
Register in testing: An example
Reproduced from TIMSS Population 1
Item Pool. Copyright © 1994 by IEA, The Hague
The interaction of
dialect and register
Standard English:
The length of a dinosaur was reported to have
been 80 feet (rounded to the nearest 10 feet).
What length other than 80 feet could have been
the actual length of this dinosaur?
The interaction of
dialect and register
Standard Spanish:
Se reportó que la longitud de un dinosaurio era 80
pies (redondeada a los 10 pies más cercanos).
¿Cuál longitud además de 80 pies podría haber
sido la longitud real de este dinosaurio?
Southern California:
Se reportó que un dinosaurio midió 80 pies de
largo (redondeado a la decena más próxima).
Aparte de 80 pies, ¿qué pudo haber sido la
medida exacta del dinosaurio?
Dialect and register
Miami:
Brooklyn:
Yo di yon dinozò te
mezire 80 pye de longè
(Rapwoche pi pre 10
pye). Ki lòt longè pase
80 pye ki ta kapab vrè
longè dinozò a?
Repons:___________
Yo rapòte longè yon
dinozò te 80 pye (longè
a awondi nan dizèn ki
pi pre a). Apa 80 pye,
ki lòt vrè longè dinozò
a ta ka genyen?
Repons:___________
Linguistic misalignment:
Frequency and severity
How much money does Laura need to pay
for two candies that cost 45 cents each?
How much money does Laura need to
purchase two sweets at 45 cents a piece?
How much money does Laura need to pay
for two 45-cent candies?
Linguistic misalignment
Test:
• In standard
dialect
• Assumes full
command of
testing register
• Content area
register based on
standards
documents
T
T S
S
Student:
• Non-standard
dialect
• Partially
unfamiliar with
testing register
• Content area
register shaped
by instructional
context
Find instances of misalignment:
Sam can purchase his lunch at school.
Each day he wants to have juice that costs
50¢, a sandwich that costs 90¢, and fruit
that costs 35¢. His mother has only $1.00
bills. What is the least number of $1.00
bills that his mother should give him so he
will have enough money to buy lunch for 5
days?
NAEP: Mathematics Grade 4,
1996 Public Release
Students’ interpretations* of
His mother has only $1.00 bills
1. His mother has only bills of one dollar
2. His mother has only one dollar
3. His mother has only dollar bills
*As reflected by students’ read-alouds
Frequency
of misalignment
Linguistic misalignment
probabilistic space
Linguistically
challenging items
Linguistically
sound items
Severity of misalignment
Generalizability (G) theory in one slide
Object of
measurement
Facets
Sources of
measurement error
Error due to
unknown sources
student (s)
item (i)
Main effect
dialect (d)
si
sd
Interaction
id
effect
sid,e
Miami students tested in Standard English
and Standard Haitian-Creole
Source
student (s)
rater (r)
item (i)
language (l)
sr
si
sl
ri
rl
il
sri
srl
sil
ril
sril,e
Percentage
of score variation
20
0
6
5
0
11
7
0
0
1
0
0
39
0
11
Miami students tested in Miami HaitianCreole and Standard Haitian-Creole
Source
student (s)
rater (r)
item (i)
dialect (d)
sr
si
sd
ri
rd
id
sri
srd
sid
rid
srid,e
Percentage
of score variation
22
0
5
0
0
23
2
0
0
0
0
0
33
0
14
Approaches to ELL testing
•
•
•
•
•
Item response
theory
Scaling
Differential item
functioning
Between-group
designs
Reference groups
Population
homogeneity
Generalizability
theory
• Measurement error
• Score dependability
• Within-group
designs
• No reference groups
• Population
heterogeneity
Conclusions
• Dialect and register are as important as
language in the testing of ELLs
• Testing and psychometric approaches for
ELLs should and can be well grounded on
reasonings from sociolinguistics