Transcript Introduction to the WIAT-III

Introduction to the WIAT-III
Presented by
George McCloskey, Ph.D.
Philadelphia College of Osteopathic Medicine
[email protected]
[email protected]
WIAT-III: What’s New?
• New Age Range
(temporary): 4-0 to 19-11
• Adult Norms planned for
2010
WIAT-III: What’s New? Materials
• Record Form
–Longer – 52 pages, but space is
well-used to improve ease of
administration
– All administration and item
directions are in the Record Form;
no need to refer to the manual
during testing
Quick Administration Guide for Each
Subtest on Record Form
General Directions for Each
Subtest on Record Form
Correct Answer Key for Each
Subtest Item on Record Form
WIAT-III: What’s New? Materials
• Stimulus Booklet
– Only one booklet, not two
– Contains only items; no subtest
directions in easel
– Items on both sides of easel pages;
work through one side then flip
over to continue
– More durable construction
WIAT-III: What’s New? Materials
• Oral Reading Fluency Booklet
– Durable, reusable, washable
booklet; student reads from this
booklet
• Word Reading and Pseudoword
Decoding Cards
– New items but no design changes
WIAT-III: What’s New? Materials
• Response Booklet
– Longer - 32 pages
– Used for 9 different Subtests
– Improved Item layout for
Numerical Operations
WIAT-III: What’s New? Materials
• Examiner’s Manual
– Administration and scoring;
analysis and interpretation
– Not used for test administration
– Appendices contain expanded
scoring guidelines for specific
subtests
WIAT-III: What’s New? Materials
• Technical Manual on CD
only
– Contains Norms Tables
– Information about
psychometric properties
WIAT-III: What’s New? Materials
• Scoring Assistant
– Provided as part of basic kit
– Expanded capabilities
• Item Scoring (interactive scoring
guide for Essay Composition)
• Item skill analyses
• Goal & Objective statements
Derived Scores for
Composites and Subtests
•
•
•
•
Standard Score
Percentile Rank
Normal Curve Equivalent
Stanines
• Grade Equivalent
• Age Equivalent
• Growth Scale Values
Status
Scores
Growth
Scores
Assessment in a Three Tier Model
Educators need to be
aware of psychometric
pitfalls in attempting to
assess response to
intervention using standard
scores from normreferenced tests.
15
Attempting to Assess Response to
Intervention Using Standard Scores
100
100
?
78
Time 1
Time 2
16
Attempting to Assess Response to
Intervention Using Standard Scores
100
100
78
78
Time 1
Time 2
17
Attempting to Assess Response to
Intervention Using Standard Scores
100
100
78
Time 1
78
?
Time 2
18
Attempting to Assess Response to
Intervention Using Standard Scores
100
100
78
Time 1
78
70
Time 2
19
Attempting to Assess Response to
Intervention Using Standard Scores
The score graphs demonstrate the
fact that the use of age- or gradereferenced standard scores to
monitor performance across time
will not effectively characterize
progress in situations where the
amount of progress made is less than
the average amount of progress
made by same-age or same-grade
peers.
20
Constructing Grade Equivalents
30
25
20
15
Interpolate
10
0
K.6
1.6
2.6 3.6
2.9
4.6
5.6
6.6
Word Reading Subtest
109 (6.3)
100
82
100
102 (7.1)
100
(1.6)
Time 1
11/03
Grade 4.3
Time 2
6/04
Grade 5.10
Time 3
3/06
Grade 7.7
22
WIAT-III: What’s New?
Growth Scale Values
• Growth Scale Values can be used
to monitor progress instead of
Standard Scores or Grade
Equivalent
• Grow Scale Values are
psychometrically sound and
reflect change in a more accurate
manner than Grade Equivalents
Assessing Response to Intervention
Using Growth Scale Values
503
453
445
360
Time 1
Time 2
24
WIAT-III: What’s New?
Growth Scale Values
• When interpreting student
performance:
• Use Standard Scores to report
how the student is performing
compared to same age or same
grade peers
• Use Growth Scale Values to
reflect intra-individual growth
Subtest Administration Order
• Whenever possible, subtests should be
administered according to the order of
subtests in the Record Form
– Simply skip any unwanted subtests
• Alterations to the order should be based
on clinical need (not examiner
preference)
• Every effort should be made to
administer subtests in one session.
Start and Stop Points
• Generally acceptable to choose an earlier
or later start point if the gradeappropriate start point is too
easy/difficult
• HOWEVER – If you choose an earlier start
point and then child gets credit on gradeappropriate start item and next 2 items
– THEN full credit is rewarded on all
items preceding grade-appropriate start
point
ERASERS - WHY NOT???
• “The student must write his or her
responses using a pencil without an
eraser. Students who participated in the
standardization sample were required to
use pencils without erasers because
erasure marks can make the student’s
response difficult to read and score. FOR
THIS REASON, STUDENTS MUST CORRECT
ERRORS BY MARKING OUT AND
REWRITING.”
WIAT-III: What’s New?
Subtests and Scores
• Reading
– 5 Subtests; 9 Scores
• Oral Language
– 2 Subtests; 7 Scores
• Written Expression
– 4 Subtests; 9 Scores
• Mathematics
– 5 Subtests; 5 Scores
Reading Subtests
Reading Subtests
Early Reading Skills
Word Reading
Word Reading
Pseudoword
Decoding
Pseudoword
Decoding
Reading
Comprehension
Reading
Comprehension
Oral Reading Fluency
Reading Scores
Early Reading Skills
Word Reading
Pseudoword
Decoding
Reading
Comprehension
Oral Reading Fluency
Word Reading Rate
Pseudoword
Decoding Rate
OR Accuracy
OR Rate
Written Expression
Subtests
Written Expression Subtests
Spelling
Spelling
Alphabet Writing Fluency
Written
Expression
Sentence Composition
Essay Composition
Written Expression Scores
Spelling
Alphabet Writing
Fluency
Sentence
Composition
Essay
Composition
Sentence Combining
Sentence Building
Word Count
Theme Dev & Text Org
Grammar & Mechanics
Oral Language
Subtests
Oral Language Subtests
Listening
Comprehension
Oral
Expression
Listening
Comprehension
Oral
Expression
Oral Expression Scores
Listening
Comprehension
Oral
Expression
Receptive Vocabulary
Oral Discourse
Comprehension
Expressive Vocabulary
Oral Word Fluency
Sentence Repetition
Mathematics
Subtests
Math Subtests
Numerical
Operations
Numerical
Operations
Math
Reasoning
Math
Problem
Solving
Math Fluency - Addition
Math Fluency - Subtraction
Math Fluency - Multiplication
Math Scores
Numerical
Operations
Math
Problem
Solving
Math Fluency - Addition
Math Fluency - Subtraction
Math Fluency - Multiplication
Achievement Testing vs
Diagnostic Skill Assessment
• Achievement testing indicates
overall level of performance in
a very global manner
• Diagnostic skills assessment
identifies specific component
skill strengths and weaknesses
Copyright 2007 George McCloskey, Ph.D.
42
Achievement Testing vs
Diagnostic Skill Assessment
• Achievement testing focuses
on composite scores
• Diagnostic skills assessment
focuses on subtest and
process scores
Copyright 2007 George McCloskey, Ph.D.
43
Interpretive Levels Framework
Global Composite
Level
Specific Composite Indexes /
Clinical Clusters Level
Subtest Level
Item Level
Task Cognitive
Capacities
Level
Copyright 2007 George McCloskey,
Ph.D.
44
Reading Subtests
Early Reading Skills
• Item types originally included on WIAT-II
Word Reading Subtest (Items 1-47):
– Letter Names (Items 1-11)
– Letter-Sound Relationship (Items 12,
13, 27-29)
– Rhyming (Items 14-17)
– Sound Awareness (Items 18-24)
– Sound Blending (Items 25, 26)
– Orthographic Awareness (Items 30-34)
Early Reading Skills
• Grades PK-3
• 34 Items (47 Items on WIAT-II)
• All students start with Sample A and
Item 1
• Administer ALL subtest items
• Use routinely for PK-1, for low
functioning in grades 2 & 3
• Total Raw Score to Standard Score
Word Reading
Grades 1-12
75 Items (84 Items on WIAT-II)
All students start with Item 1
Discontinue after 4 items (7 on
WIAT-II)
• Time and Record number of items
completed in first 30 seconds
•
•
•
•
Word Reading
• Student are told to read words aloud but
not to read as quickly as possible
• Students can use their fingers but do not
provide materials for place marking
• Credit should be given for regional or
dialectal pronunciations of words
• Pronunciations must be fluent to receive
credit. For dysfluent attempt, say “Say it
all together.” If 2nd attempt not fluent,
score 0.
Word Reading
• Total Raw Score to
Standard Score
• Words Read in 30 seconds
to Word Reading Speed
Cumulative Percentage
Pseudoword Decoding
• Grades 1-12
• 52 Items (55 Items on WIAT-II)
• Items revised to measure a broader range
of symbol-sound relationships
• All students start with Item 1
• Discontinue after 4 items (7 on WIAT-II)
• Time and Record number of items
completed in first 30 seconds
Pseudoword Decoding
• Student are told to decode words aloud
but not to decode as quickly as possible
• Students can use their fingers but do not
provide materials for place marking
• Credit should be given for regional or
dialectal pronunciations of words
• Pronunciations must be fluent to receive
credit. For dysfluent attempt, say “Say it
all together.” If 2nd attempt not fluent,
score 0.
Pseudoword Decoding
• Total Raw Score to Standard
Score
• Nonsense Words Decoded in
30 seconds to Pseudoword
Decoding Speed Cumulative
Percentage
Reading Comprehension
• Grades 1-12
• 84 Items (140 Items on WIAT-II)
• Pre-passage items replaced with a new
lower level reading passage
• All other passages retained from WIAT-II
• Oral reading sentence items deleted
• Supplemental scores dropped
• Administered in item sets like WIAT-II
Reading Comprehension
• Start administration with grade level
item set; administer all items in the set
• Can start one set lower if student is
known to be a very poor reader;
otherwise, use reverse rule
• Can reverse only a maximum of 3 levels
below grade level
• Reverse Rule: less than 2 points earned
on items from 1st passage
Reading Comprehension
• Total Raw Score to Weighted
Raw Score to Standard Score
• Note: The Weighted Raw
Score Table is reversed from
WIAT-II; Weighted Scores in
outside column; Raw scores
in body of table
Reading Comprehension
Reading Comprehension
• If the student performs poorly on
lower item set
– Probably means they have a weakness
in comprehension
• OR If the student performs well on
lower item set
– Probably means they have a weakness
in word identification and/or decoding
and/or vocabulary
Reading Comprehension
• While Reading Comprehension Subtest Scores, in
general, correlate highly between measures such as
the WIAT-III, KTEA-II, and WJ-III, scores for
individual students can vary greatly due to:
• Differences in Output Demands (free response
[WIAT-II, KTEA-II] vs cloze [WJ-III])
• Differences in Processing Demands (e.g.,
literal vs inferential item types and passage
length variations resulting in different demands
for reasoning, reading speed, working memory,
and executive functions)
Oral Reading Fluency
Grades 1-12
Student reads two passages
Administer grade level passages
Can start one grade level lower if student
is known to be a very poor reader;
otherwise, use reverse rule
• Reverse rule: If reading of 1st passage
takes longer than the reverse rule time
limit go back one level
•
•
•
•
Oral Reading Fluency
• Provide unknown words after 5 seconds
• A comprehension question is asked after
the reading; these questions are not
scored formally; the items are used to
cue the student to read at a level that
engages comprehension
• Student reads entire passage; record the
total reading time per passage
• Keep track of and record word addition
errors and other word reading errors
Oral Reading Fluency
• Oral Reading Fluency Administration Steps:
1. Begin timing as student begins reading
2. Record errors as student reads
• Addition Errors: Any word added
• Other Errors: Words supplied after 5”
of attempted reading of word (mark
with G for “given”); mispronunciations,
substitutions, omissions and
transpositions
Oral Reading Fluency
Oral Reading Fluency
• Oral Reading Fluency Administration Steps:
3. End timing as student finishes
passage and record stop time
4. Repeat administration steps for 2nd
passage
5. Transfer Completion Time, Addition
Errors, Other Errors, and Word Count
Totals to the ORF Passage Subtotal
Raw Scores Table on Page 46
Oral Reading Fluency
• Oral Reading Fluency Administration Steps:
6. Calculate Oral Reading Fluency Total
Raw Score, Oral Reading Accuracy Total
Raw Score, and Oral Reading Rate Total
Raw Score
7. Use Total Raw Scores to find
Weighted Raw Scores in Appendices
A.2, A.3 and A.4
8. Convert Weighted Raw Scores to
Standard Scores using Table B.1 or C.1
Written Expression
Subtests
Alphabet Writing Fluency
• Grades PK-3 Only
• Student writes letters of the
alphabet for 30”
• Raw Score is number of legible
letters produced in 30”
• If all letters are made before 30”
record completion time
Alphabet Writing Fluency
• Scoring Criteria are in Appendix B.1
• Scoring Criteria are extremely
flexible in order to reduce scoring
time and difficulty
• Typically used for Grades PK-2, can
be used for lower functioning 3rd
graders
Spelling
• Grades K-12
• First 5 items are pre-spelling skills
(12 items on WIAT-II)
• 63 Items (53 Items on WIAT-II)
• Start at Grade Level Start Point
• Can start 1 level below but give full
credit if basal at Grade Level Start
Point is met
Spelling
• Basal: score of 1 on all of the first
three items in Start Point Set
• Reverse Rule: Incorrect on any of first 3
items administered; give items in
reverse order until basal is met
• Discontinue after 4 items (7 on WIAT-II)
• Raw Score to Standard Score
• Intra-item error analysis optional
Sentence Composition
• Grades 1-12
• 5 Sentence Combining Items (Most
from WIAT-II) and 7 Sentence Building
Items (“Write a sentence using the
word ___”)
• All students start with Sentence
Combining Sample A and Items 1 and 2
• Discontinue Sentence Combining if
score of 0 on first 2 items
Sentence Composition
• If at least 1 point is earned on first 2
items, administer ALL remaining
Sentence Combining items
• After completing Sentence
Combining, ALL students take
Sentence Building Sample A and
Items 1 and 2
• Discontinue Sentence Building if
score of 0 on first 2 items
Sentence Composition
• If at least 1 point is earned on first 2
item, administer ALL remaining
Sentence Building items
• Separate Raw Scores and Standard
Scores for Sentence Combining and
Sentence Building
• Add SC and SB standard scores and
use as Raw Score to obtain Sentence
Composition Standard Score
Sentence Composition
• Sentence Composition Scoring: Appendix
B.2 provides Basic Rules of Written
Grammar and Mechanics
• Sentence Combining Scoring (Appendix
B.3): Prerequisite Scoring Criteria
– Does response include essential information?
– Does response simply restate original
sentences?
– Does response have more than one sentence,
run-on sentence or sentence fragment?
Sentence Composition
• Sentence Combining: If a sentence does
not meet all 3 Prerequisite Criteria, score
the sentence 0
• Sentence Combining: If a sentence does
meet all 3 Prerequisite Criteria, use the
additional scoring criteria to determine the
raw score for each scoring criterion:
– Semantics & Grammar
– Mechanics
– Extra Credit
Sentence Composition
• Sentence Building Scoring (Appendix B.4):
Prerequisite Scoring Criteria
• Does response include the target word?
• Is the response a fragmented sentence?
• Is the target word use in a title or as the
subject or object of the sentence?
• Sentence Building: If a sentence does not
meet all 3 Prerequisite Criteria, score the
sentence 0
Sentence Composition
• Sentence Building: If a sentence does
meet all 3 Prerequisite Criteria, use
the additional scoring criteria to
determine the raw score for each
scoring criterion:
– Semantics & Grammar
– Mechanics
• The Scoring Workbook provides
unscored and scored samples
Essay Composition
• Grades 3-12
• Writing from a prompt; identical to
the WIAT-II Paragraph sub-item (My
favorite game is…)
• All students write using same prompt
• 10 minutes maximum time
• Scoring Criteria in Appendix B.5 and
B.6
Essay Composition
• Essays are scored for:
– Word Count
– Theme Development and Organization
• Introduction (0-2)
• Conclusion (0-2)
• Paragraphs (0-5)
• Transitions (0-5)
• Reasons Why (0-3)
• Elaborations (0-3)
– Grammar and Mechanics
Essay Composition
• The Scoring Workbook
provides unscored and scored
samples
• The Scoring Assistant Includes
an interactive scoring guide
that can be used to complete
the scoring process
Essay Composition
• Raw Scores for Word Count, Theme
Development & Text Organization
(TDTO) and Grammar & Mechanics are
converted into Standard Scores using
B.1 or C.1
• Word Count and TDTO Standard Scores
are summed to derive the Essay
Composition Raw Score to obtain the
Essay Composition Standard Score in
B.1 of C.1
Oral Language
Subtests
Listening Comprehension
• Grades PK-12
• Receptive Vocabulary retained –
19 items (16 items on WIAT-II)
• Oral Discourse Comprehension (27
items) replaces Sentence
Comprehension format (10 items)
• Expressive Vocabulary moved to
Oral Expression Subtest
Listening Comprehension
• ALL students start with Receptive
Vocabulary (RV) Item 1
• Discontinue RV after 4 consecutive
scores of 0
• All students go on to Oral Discourse
Comprehension (ODC)
• ODC administered best using the
audio CD
Listening Comprehension
• Grades PK-3 start with Item 1;
Grades 4-12 start with Item 6
• All items scored objectively 0 or 1
• Apply Reverse Rule for Grades 4-12,
move backward until 3 consecutive
scores of 1 are obtained
• Discontinue ODC after 4 consecutive
scores of 0
Listening Comprehension
• Raw Scores for RV and ODC are
converted into Standard Scores for
each using B.1 or C.1
• RV and ODC Standard Scores are
summed to derive the Listening
Comprehension Raw Score to obtain
the Listening Comprehension
Standard Score in B.1 of C.1
Oral Expression
• Grades PK-12
• Oral Word Fluency Expanded into
a sub-area (2 items instead of 1)
• Sentence Repetition expanded to
15 items (9 items on WIAT-II)
• Expressive Vocabulary expanded
to 17 items and moved to OE
from LC
Oral Expression
• ALL students start with Expressive
Vocabulary (EV) Item 1
• Discontinue EV after 4 consecutive
scores of 0
• ALL students go on to Oral Word
Fluency (OWF) and take both items
• ALL students go on to Sentence
Repetition (SR) and start with Sample
A and Item 1
Oral Expression
• Discontinue SR after 4 consecutive
scores of 0
• Raw Scores for EV and OWF and SR are
converted into Standard Scores for each
using B.1 or C.1
• EV and OWF and SR Standard Scores are
summed to derive the Oral Expression
Raw Score to obtain the Oral Expression
Standard Score in B.1 of C.1
Mathematics
Subtests
Math Problem Solving
• Grades PK-12
• Name changed from Math
Reasoning
• Items increased to 72 (67 items
on the WIAT-II)
• Pencil and paper are permitted,
but calculators are not
Math Problem Solving
• Start points vary based on Grade Level
• Basal: score of 1 on first three items in
Start Point Set
• Reverse Rule: Incorrect on any of first 3
items administered; give items in reverse
order until basal is met.
• Discontinue after 4 items (6 on WIAT-II)
• Raw Score to Standard Score
Numerical Operations
• Grades K-12
• Items increased to 61 (54 items on
the WIAT-II)
• Starts with 7 Pre-calculation concept
items for Grades K-1
• Start points vary based on Grade
Level
• Items completed in Response Booklet
Numerical Operations
• Basal: score of 1 on first three
items in Start Point Set
• Reverse Rule: Incorrect on any of
first 3 items administered; give
items in reverse order until basal is
met.
• Discontinue after 4 items (6 on
WIAT-II)
• Raw Score to Standard Score
Math Fluency
• Addition and Subtraction Grades
1-12; Multiplication Grades 3-12
• Items completed in Response Booklet
• Administer only 1 item set of each
Fluency Subtest; 60 second time
limit for all item sets
Referenced Resources
• Berninger, V. & Richards, T. (2003).
Brain Literacy for Educators and
Psychologists. Academic Press
• Marzano, Pickering, & Pollock
(2001). Classroom Instruction that
Works: Research-based Strategies
for Improving Student Achievement.
ASCD