Transcript Testing Considerations for English as a Second Language

CLEAR 2008 Annual Conference
Testing Considerations for
English as a Second
Language (ESL)
Candidates
Nisha Mittal
Anthony R Zara
Paul D Naylor
Anchorage, Alaska
Topics for this Session…
• Why is the ESL issue more important now than
ever before?
• What challenges are faced by ESL candidates?
• What are the common requests made by ESL
candidates?
• What are the general strategies for assisting
ESL candidates?
• What accommodations are currently used for
ESL candidates in the United States?
• How do you decide on the level of
accommodation for your exam?
CLEAR 2008 Annual Conference
Anchorage, Alaska
Why is the ESL issue more
important now than ever before?
• Changes in the immigrant population in the United
States
• Increase in the need for professional/nonprofessional
workers from developing countries
• Enforcement of strict laws by employing agencies to
provide authentic proof of knowledge, skills, and
abilities required for jobs (certification/licensure tests)
• Strict legal/governmental regulations in assessment
while being fair to all
– test security
– costs
CLEAR 2008 Annual Conference
Anchorage, Alaska
What challenges are faced by ESL
candidates?
• Knowledge of common English vocabulary is limited
• Knowledge of “idiomatic phrases that occasionally occur in some test
items” is limited
• Knowledge of specialized vocabulary or jargon used in professional
settings is limited
• Grasp of moderately complex English grammar used in test questions is
limited
• They are challenged to simultaneously think in a second language while
applying cognitive strategies and prior knowledge to determine correct
answers
• They are challenged to select the “best” answer to a test item when the
candidate’s sociolinguistic background suggests that other answers may
be appropriate
• They may experience increased test anxiety resulting from any or all of the
reasons cited above.
CLEAR 2008 Annual Conference
Anchorage, Alaska
Challenges continued…
Like other candidates, ESL candidates may also score
low on an exam because of the following:
• they are unfamiliar with the test blueprint and what is
required to prepare for the test;
• they may need additional practice to feel comfortable
taking a multiple-choice test, especially if they have
been out of school for some time;
• they may need additional practice to feel comfortable
taking a computer-based test; or
• they may lack the minimal competence.
CLEAR 2008 Annual Conference
Anchorage, Alaska
What are the common requests
made by ESL candidates?
•
•
•
•
•
Translated exams
Oral interpretation of test questions
Waivers
Provisional licenses
Special accommodations
– Dictionaries
– Extended time
CLEAR 2008 Annual Conference
Anchorage, Alaska
What are the general strategies for
assisting ESL candidates?
•
•
•
•
Test translation
Oral interpretation of test questions
Printed language aids
Modifications to the English used in the
test questions
CLEAR 2008 Annual Conference
Anchorage, Alaska
Test Translation
• Accuracy of translation
• Equivalence of the translated exam to the
original exam
• International Test Commission (ITC)
provided 22 test adaptation guidelines
• Costs (large versus low-volume exams)
CLEAR 2008 Annual Conference
Anchorage, Alaska
Further Considerations for Test Translation
• How many potential languages and dialects
would require translation?
• If one language is to be translated, do all others
need to be translated?
• Would the cost of translation be passed on to all
candidates or just ESL candidates?
• If ESL candidates bear the cost, should
individual candidates from small linguistic
groups bear a greater cost than candidates from
larger linguistic groups?
CLEAR 2008 Annual Conference
Anchorage, Alaska
Differences in Approaches to Test
Translation
• National Center for Construction
Education and Research (NCCER)
• Nursing (NCLEX)
CLEAR 2008 Annual Conference
Anchorage, Alaska
Oral Interpretation of Test Questions
What? Professional bilingual interpreters recruited to
translate any item text that the candidate feels unsure of,
from individual words to entire sentences.
Advantages
• Reduced expenses by limiting use of interpreters to just
those candidates who demonstrate a need
• Could translate any text the candidate feels unsure of
• Provides instantaneous assistance, as requested,
allowing the candidate more time to focus on answering
the questions instead of looking up the words in the
dictionary
CLEAR 2008 Annual Conference
Anchorage, Alaska
Continued…
Disadvantages
• May threaten the security of the examination
• May increase the number of examinations
administered under nonstandard conditions
• Who should determine the credentials of
interpreters?
• Who should pay the oral interpreters?
CLEAR 2008 Annual Conference
Anchorage, Alaska
Printed Language Aids
What are these?
• English and/or bilingual dictionaries
• Quick reference dictionary of English vocabulary
for each exam
• Bilingual versions of quick-reference dictionaries
• Standardized English language glossary of
professional terminology which could be
provided as examination study guides
• Bi-lingual versions of the standardized
professional glossary
CLEAR 2008 Annual Conference
Anchorage, Alaska
Continued…
Advantages
• Inexpensive
• Standardized format
Disadvantages
• Maybe insufficient aid to help examinees
with very limited English proficiency
CLEAR 2008 Annual Conference
Anchorage, Alaska
Modifications to the English in Exams
What? Phrase the test questions so that ESL
candidates understand them more easily.
Readability of test questions
• Assess readability by using formulas devised for
measuring the complexity of written text
• Simplify vocabulary, sentence length, and
grammatical complexity
CLEAR 2008 Annual Conference
Anchorage, Alaska
Continued…
The sociolinguistic biases in test questions
• Assess test items for idiomatic phrases or professional
jargon unfamiliar to ESL examinees
• Avoid test item content that reflects judgments about
appropriate actions or decisions that reflect the majority
culture
• Avoid test item content that contains any implicit beliefs
that reflect only the majority culture
Presumption
Candidates have some 8th to 10th grade reading level
English proficiency
CLEAR 2008 Annual Conference
Anchorage, Alaska
What accommodations are provided in
the United States?
•
•
•
•
•
•
Undergraduate admission tests
Professional school admissions
ETS higher education tests
Certification programs
Licensure programs
Others
CLEAR 2008 Annual Conference
Anchorage, Alaska
In conclusion, testing accommodation for ESL
candidates in the United States
• is ambiguous (there are no clear guidelines for qualifying
for an accommodation).
• differs from state to state.
• is better established for candidates with disabilities.
More common accommodations are:
– use of up to two dictionaries
– extended time
Less common accommodations are:
– oral interpretation
– translated exams
CLEAR 2008 Annual Conference
Anchorage, Alaska
Clearly there is a need to:
• Establish clear policies for ESL candidates
• Provide clear and well-documented
guidelines to determine ESL eligibility for
accommodations
• Make these policies consistent across all
states
• Evaluate ESL accommodations made in
other countries
CLEAR 2008 Annual Conference
Anchorage, Alaska
How can you determine the level of
accommodation for your exam?
• Determine the knowledge, skills, and abilities
required in a profession
• Evaluation of the need for English language skills
in the profession should be based on the skills
needed to function at minimum competency
within the profession
• Consider costs associated with accommodations
• There can be no ONE consistent policy across all
professions
CLEAR 2008 Annual Conference
Anchorage, Alaska