Transcript No Slide Title

ELL-Language-based Accommodations for
Content Area Assessments
The University of Central Florida
Cocoa Campus
Jamal Abedi
University of California, Davis
July 7, 2011
1
ELL Language-Based Accommodations
 English dictionary
 English glossary
 Bilingual dictionary/glossary
 Customized Dictionary
 Native language testing
 Read-aloud test items or directions
 Linguistically modified test
 Computer testing with pop-up glossaries
2
English Dictionary





Providing an English Dictionary is another commonly
used accommodation for ELL students (Abedi, Courtney,
& Leon, 2003; Abedi, Lord, Boscardin & Miyoshi, 2000).
The use of a dictionary and extra time affect the
performance of all students (Abedi, Lord, Hofstetter &
Baker, 2000; Hafner, 2001; Maihoff, 2002; Thurlow, 2001;
Thurlow & Liu, 2001).
By gaining access to definition of content-related terms,
recipients of a dictionary may be advantaged over those
who did not have access to the dictionaries. This may
compromise the validity of assessment (Abedi, Courtney,
Mirocha, Leon & Goldberg, 2005).
The dictionary as a form of accommodation suffers from
yet another major limitation, the feasibility issue (Abedi,
Courtney, Mirocha, Leon & Goldberg, 2001).
Consequently, the results of accommodated and nonaccommodated assessment may not be aggregated.
3
English Glossary





English glossary with extra time raised performance of both
ELL and non-ELL students (Abedi, Hofstetter & Lord,
2000).
ELL students’ performance increased by 13% when they
were tested under the glossary with extra time
accommodation.
While this looks promising, it does not present the entire
picture.
Non-ELL students also benefited from this accommodation,
with an increase of 16% (Abedi, Hofstetter & Lord1998,
2000).
Thus, the results of the accommodated outcome cannot be
aggregated with the non-accommodated outcome.
4
Customized Dictionary
 Customized dictionary was introduced as a more valid
alternative to English/bilingual dictionaries (Abedi,
Courtney, Mirocha, Leon & Goldberg, 2001).
 It is a cut-and-paste of the actual dictionaries.
 In only includes terms that are: (1) in the test and (2) noncontent related.
 Results of studies suggest that it is highly effective and
valid accommodation for ELL students.
5
Linguistically Modified Test
There are, however, some accommodations that help ELL
students with their English language needs without
compromising the validity of assessment.
Studies suggested that the linguistically modified version of
the tests items is an effective and valid accommodation for
ELL students (Abedi, Hofstetter, Lord & Baker, 2000;
Maihoff, 2002; Rivera & Stansfield, 2001).
This accommodation also helped students with learning
disabilities.
Thus, an accommodation may have the potential to be
effective and help provide valid assessment outcomes for
ELL students.
6
Computer Testing
 Research findings suggest computer testing as an
effective and valid accommodation for ELL students
(Abedi, Courtney, Mirocha, Leon, & Goldberg, 2001).
 ELL students show higher levels of motivation on the
assessments administered by computer.
 Different types of accommodations that have been
shown to be useful for ELL students may be
incorporated into the computer testing system (Abedi,
et al, 2011).
7
Native Language Testing




Translating tests into students’ native language is an
accommodation used by many states across the country
(Abedi, Lord, Hofstetter & Baker, 2000; Rivera,
Stansfield, Scialdone & Sharkey, 2000).
Issues concerning translation and content coverage
across the forms must be seriously considered.
Students’ background variables, particularly their level of
proficiency in L1 and L2, must be studied before
considering this accommodation.
Using native language assessment may not produce
desirable results if the language of instruction and
assessment are not aligned (Abedi, Hofstetter & Lord,
2004).
8
Examining Complex Linguistic Features
in Content-Based Test Items
Unnecessary complex linguistic features slow down the
reader, make misinterpretation more likely, and add to the
reader’s cognitive load; thus interfering with concurrent
tasks. These features include:
Concrete vs. abstract or impersonal presentations
Item length
Unfamiliar Vocabulary
Nominal heaviness
Relative clause
Conditional clause
Passive voice
Long noun phrases
Subordinate clauses
9
Impact of language factors on content
assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
1
Items with no linguistic complexity:
 Familiar or frequently used words; word length
generally shorter
 Short sentences and limited prepositional phrases
 Concrete item(s) and a narrative structure
 No complex conditional or adverbial clauses
 No passive voice or abstract or impersonal
presentations
10
Impact of language factors on content
assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
2
Items with a minimal level of linguistic complexity:
 Familiar or frequently used words; short word length
 Moderate sentence length with a few prepositional
phrases
 Concrete item(s)
 No subordinate, conditional, or adverbial clauses
 No passive voice or abstract or impersonal
presentations
11
Impact of language factors on content
assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
3
Items with a moderate level of linguistic complexity:
 Unfamiliar or seldom used words
 Long sentence (s)
 Abstract concept (s)
 Complex sentence/conditional tense/adverbial
clause(s)
 A few passive voice or abstract or impersonal
presentations
12
Impact of language factors on content
assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
4
Items with a high level of linguistic complexity:
 Relatively unfamiliar or seldom used words
 Long or complex sentence(s)
 Abstract concept(s)
 Difficult subordinate, conditional, or adverbial
clause(s)
 Passive voice/ abstract or impersonal presentations
13
Impact of language factors on content
assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
5
Items with a maximum level of linguistic complexity:
 Highly unfamiliar or seldom used words
 Very Long or complex sentence(s)
 Abstract concept(s)
 Very difficult subordinate, conditional, or adverbial
clause(s)
 Many passive voice/ abstract or impersonal
presentations
14
Sample Original and Revised Item
• Below is an example of a test items that deemed to be
linguistically complex and a linguistically modified version of
the items.
• Original: If Y represents the number of newspapers that Lee
delivers each day, which of the following represents the total
number of newspapers that Lee delivers in 5 days?
– A) 5 + Y
– B) 5 x Y
– C) Y + 5
– D) (Y + Y) x 5
• Modified: Lee delivers Y newspapers each day. How many
newspapers does he deliver in 5 days?
(Adopted from Abedi, Lord & Plummer, 1997, p. 21)
15
Linguistic Modifications
made on the item
•
•
•
•
•
•
•
•
Conditional clause changed to separate sentence
Two relative clauses removed and recast
Long nominals shortened
Question phrase changed from “which of the following
represents“ to “how many”
Item length changed from 26 to 13 words
Average sentence length changed from 26 to 6.5
words
Number of clauses changed from 4 to 2
Average number of clauses per sentence changed
from 4 to 1
16
Conclusions and Recommendation
Assessments and instructions for ELLs :
Must be based on a sound psychometric principles
Must be controlled for all sources of nuisance or
confounding variables
Must be free of unnecessary linguistic complexities
Must include sufficient number of ELLs and SWDs in its
development process (field testing, standard setting,
etc.)
Must be free of biases, such as cultural biases
Must be sensitive to students’ linguistics and cultural
needs
17
Conclusions and Recommendations
Accommodations:





Must be relevant in addressing assessment issues for
ELL students
Must be effective in making assessments more
accessible to ELL students
Should not alter the construct being measured
Must provide results that can be aggregated with the
assessment outcomes under standard conditions
Must be feasible in national and state assessments
18
Conclusions and Recommendations
Examples of research-supported accommodations:

Providing a customized dictionary is a viable
alternative to providing traditional dictionaries.

The linguistic modification of test items that reduce
unnecessary linguistic burdens on students is among
the accommodations that help ELL students without
affecting the validity of assessments.

Computer testing with added extra time and glossary
was shown to be a very effective, yet valid
accommodation (Abedi, Courtney, Leon and
Goldberg, 2003)
19