Transcript Slide 1

Five Key Considerations in
Translating Standardized Tests
for Braille Readers
Presentation for AER International
July 23, 2010
Lynne Jaffe, Ph.D.
Learning Disabilities Consulting & Evaluation
Tucson, AZ
Woodcock-Johnson III
Richard Woodcock, Kevin McGrew, Nancy Mather (2001)
Tests of
Cognitive
Abilities
,,brl
Tests of
Achievement

Oral language

Academic skills &
knowledge
Objectives in Developing a Braille
Adaptation of the WJ III ACH
1.
Provide a standardized means of assessing the
academic achievement and language abilities of
individuals who are blind;
2.
Provide a single, standard braille version of the
test.
3.
Produce this adaptation with permission and
approval of Riverside Publishing Company and the
authors of the WJ III.
Important Considerations in Adapting
Standardized Tests into Braille
1.
Maintain item/test integrity: Retain assessment intent
and task demands of each item and test.
2.
Replicate standardized administration procedures
3.
Adhere to BANA guidelines as closely as possible.
4.
Decide on braille format - uncontracted & contracted.
5.
Provide for consistency in scoring across all test
subjects.
Consideration 1: Maintain item integrity –
assessment intent & task demands
1.
What specific skill is this item intended to assess?
2.
How is the task administered?
3.
What does the individual have to do to respond?
4.
Would needed adaptations change task demands and,
thereby, draw on a different skill/knowledge than
intended?
5.
Does the item contain extraneous information that is:
6.

intended to be identified and ignored?

unrelated to the task & would disadvantage a blind
person?
What foundational skills, knowledge or experience does
the performance of this task assume?
Little/no adaptations needed
Test 11: Story Recall
Martha went to the store to buy groceries. When
she got there, she discovered that she had
forgotten her shopping list. She bought milk, eggs,
and flour. When she got home she discovered
that she had remembered to buy everything
except the butter.
Copyright © 2002. Mather, N & Jaffe, L. WJ III: Reports, Recommendations & Strategies. NY: Wiley
Little/no adaptations needed
Letter-Word Identification
another
Uncontracted
another
Contracted
Ano!r
large
large
l>ge
different
Different
di6}5t
play
play
Play
learn
learn
Le>n
world
world
_w
again
again
ag
Problems
1.
Items incorporating concepts unlikely to be within
a blind individual’s experience/knowledge (e.g.,
Reading Fluency - colors of objects)
2.
Item sequences that differ substantially from
instructional sequence of letters/sight words taught
in popular braille programs (e.g., Letter-Word
Identification)
3.
Pictorial items (e.g., Passage Comprehension)
Problem: How to represent pictures that are
integral to the meaning of the test item?
Raised outlines and miniatures don’t work.
Solution 1: Substitute Form B items for
Form A items of equal difficulty
 Passage Comprehension – Pictorial items
Form A: There are many tall buildings in most ___.
Form B: A bird has two
.
 Reading Fluency – Visually biased items
Form A: The color of grass is red.
Form B: The letter B is a number.
Solution 2: Use tangible objects
Passage Comprehension
Original item 24
Something is in the wagon. It is a ____.
Adapted
Say: Please feel these objects. Guide the
subject to explore the ball and basket. Say:
Read this to yourself and tell me one
word that goes in the blank space.
,"s?+ is 9 ! basket4 ,x is a ----4
Solution 3: Use tactile graphics
Applied Problems
How many dogs
are there?
How many triangles
are there?
No solution: Adaptations not possible
Tests omitted

Too dependent on pictures





Understanding Directions
Writing Samples
Picture Vocabulary
Academic Knowledge
Task demand considerably more
complicated
 Writing Fluency
Writing Fluency
books
likes
read
Copyright © 2002. Mather, N & Jaffe, L. WJ III: Reports, Recommendations & Strategies. NY: Wiley
Tests Added:
Oral Vocabulary & Sound Blending
New Tests
New Clusters
Oral Vocabulary
Oral Comprehension
Oral Language Comprehension
Oral Vocabulary
Oral Comprehension
Story Recall
Total Oral Language – Braille
Sound Blending
Sound Awareness
Phonemic Awareness – Braille
Lynne Jaffe, University of Arizona
Tests
Standard Battery
1 Letter-Word Ident.
2 Calculation
3 Passage Comprehension
Shape Recognition
4 Applied Problems
5 Oral Comprehension
6 Word Attack
7 Quant Concepts
8 Spelling of Sounds
9 Oral Vocabulary
10 Spelling
Extended Battery
11 Story Recall
12 Reading Fluency
13 Sound Blending
14 Math Fluency
15 Reading Vocabulary
16 Editing
17 Sound Awareness
18 Punctuation & Capital.
19 Story Recall – Delayed
Lynne Jaffe, University of Arizona
Clusters: Standard & Extended Batteries
Oral Language
Oral Language Compre.
Total Oral Language – Braille
Phonemic Awareness - Braille
Math
Math Calculation
Math Reasoning
Brief Math
Broad Math
Writing
Basic Writing Skills
Reading
Phon-Graph. Knowl
Basic Reading Skills
Reading Compre.
Brief Reading
Broad Reading
Total Reading
Intra-Reading
Discrepancies
General Academics
Academic Skills
Brief Achievement
Lynne Jaffe, University of Arizona
Variation and Discrepancy Procedures
Intra-Reading Variations
• Letter-Word Identification
• Reading Fluency
• Passage Comprehension
• Word Attack
• Reading Vocabulary
• Spelling of Sounds
Intra-Academic Variations
• Brief Reading
• Brief Math
• Basic Writing Skills
Total Oral Language – Braille
Discrepancies
• Broad Reading
• Brief Reading
• Basic Reading Skills
• Reading Comprehension
• Broad Math
• Brief Math
• Math Calculation Skills
• Math Reasoning
• Basic Writing Skills
Lynne Jaffe, University of Arizona
Consideration 2: Replicate standardized
administration
Passage Comprehension
Starting With Item 27
Guide subject to first item on subject’s page and say: Read this to yourself
and tell me one word that goes in the blank space. Do not read items or
tell subject any words during this test.
28. The book is one of a series of over eighty volumes. Each volume is
designed to provide convenient _____ to a wide range of carefully
selected articles.
Correct: access
Incorrect: information, availability
Copyright © 2002. Mather, N & Jaffe, L. WJ III: Reports, Recommendations & Strategies. NY: Wiley
Consideration 3: Adhere to BANA Guidelines
Exceptions
 Letter-Word Identification: Test items were laid out
horizontally rather than in a left-justified column.
Which word says “cat”?
run cat hill the

3
+3
run cat hill !
Math Fluency: Spatial layout was changed
2
+1
6
+3
3+3=
2+1=
6+3=
Consideration 4: Uncontracted or contracted
braille?
Problems:
1.
Educational institutions vary regarding how and
when contractions are taught.
2.
Subjects will vary re: achievement in knowledge
of contractions.
3.
Sequence of WJ III ACH items is not aligned with
progression of any braille instructional program.
Solution:
Provide both –
but entire battery must be given in same format.
Word Attack – uncontracted only
flib
bungic
dotlention
flib
bungic
dotlention
Spelling of Sounds – uncontracted only
1.
(im)
2.
(trop)
3.
(smuff)
4.
(barches)
Lynne Jaffe, University of Arizona
Consideration 5: Provide for consistency
in scoring across all test subjects.
 Subject Test Book - braille
 Examiner Test Books - print
 Supplementary Examiner’s Manual
 Test Record
 WJ III ACH – Braille Compuscore®
and Profiles Program CD
 Norms: WJ III Normative Update Tests of
Achievement
What about the norms?

WJ III Normative Update Tests of Achievement

Norms obtained on people without significant visual
impairments

Reasons for using current norms

Time limits on fluency tests
Examiner Teams
Primary examiner (PE): A person
who meets all of the qualifications
for administering the WJ III ACH
in print and who has received
training in administering the WJ III
ACH - Braille
Ancillary examiner (AE): An education professional who
reads braille and Nemeth Code fluently and who teams with a
primary examiner (PE) who does. The AE should also have
knowledge of the effects of blindness on the various aspects
of an individual’s development and information regarding the
student’s vision history.
Lynne Jaffe, University of Arizona
Primary Examiner Qualifications

Administration requires
 thorough knowledge of exact administration and
scoring procedures
 understanding of the importance of adhering to these
standardized procedures
 proficiency in administering the WJ III ACH - Braille

Interpretation requires
 graduate-level training in educational assessment
 background in diagnostic decision-making
 sensitivity to conditions that may compromise or
invalidate standardized test results
Typical educational training: completion of applicable
graduate-level program of study that includes (at a minimum)
 Practicum-course covering administration &
interpretation of tests of academic achievement.
 state, provincial, or professional certification, registration,
or licensure in a field/profession that includes
responsibility for providing educational assessment and
interpretation services.
Adapted from: Mather., & Woodcock, R. W. (2001). Examiner’s Manual. WoodcockJohnson III Tests of Achievement. Rolling Meadows, IL: Riverside Publishing.
Ancillary Examiner Qualifications
Education professional who:
• reads braille and Nemeth Code fluently
• has knowledge of the effects of blindness on the
various aspects of an individual’s development
• (preferably) has worked with the student being
assessed and is familiar with his/her background
• teams with a primary examiner (PE) who does not
read braille and Nemeth fluently.
What is the role of the ancillary examiner?
For test administration
• Deals with equipment that primary examiner may not be
familiar with.
• Monitors when student has reached basal and ceiling.
• Ensures that student understands and is following
instructions (e.g., using uncontracted braille for Spelling).
• Ensures that student is reading in correct place and not
skipping items.
• Transcribes student’s brailled responses immediately after
test session.
Lynne Jaffe, University of Arizona
For qualitative analysis & interpretation
• Helps to ascertain patterns of errors in reading,
spelling, & calculation that are specific to braille and
Nemeth.
• Helps to make distinctions between a problem in
acquiring specific skills vs. gaps in knowledge typical of
blind individuals.
Recap: Key Issues
1.
Item integrity: Retain assessment intent and task
demands of each item and test.
2.
Replicate standardized administration procedures
3.
Adhere to BANA guidelines as closely as possible.
4.
Decide on braille format - uncontracted &
contracted.
5.
Provide for consistency in scoring across all test
subjects.
Lynne Jaffe, University of Arizona
Assessment Purposes
WJ III ACH-Braille Adaptation may be used to
• provide a sampling of skills and abilities
• help describe the present functioning or
achievement in a particular academic or language
area in relation to age or grade-peers
• help to identify the nature of specific academic
weaknesses or strengths
• provide a baseline against which to measure
progress
Acknowledgements
For development and contribution of the
WJ III ACH Braille Compuscore® and Profiles Program
and for alternative items:
Woodcock-Munoz Foundation
Dr. Richard Woodcock
Dr. Fredrick Schrank
Mary Ruef
For permission to develop a braille adaptation of the WJ III
ACH
Riverside Publishing Company