Transcript How do we make educational decisions with DIBELS?

Advances in the Dynamic Indicators of
Basic Early Literacy Skills (DIBELS™)
NASP Workshop
Dallas, TX, March 31, 2004
http://dibels.uoregon.edu
Roland H. Good III
University of Oregon
Ruth A. Kaminski
Pacific Institutes for Research
Advances in DIBELS™ Overview


March 31, 2004
Introduction
 Core Components of Beginning Reading
 Review of DIBELS ™ Measures
Use of DIBELS™ within an Outcomes Driven Model
 Assessing individual students and making
instructional recommendations
 Providing individual consultation to teachers
 Providing systems-wide consultation to schools
and districts
Dallas, TX
2
Beginning Reading Core Components
#1. Phonemic Awareness: The ability to hear and
manipulate sound in words.
#2. Phonics: The ability to associate sounds with letters and
use these sounds to read words.
#3. Fluency : The effortless, automatic ability to read words
in isolation (orthographic reading) and connected text.
#4. Vocabulary Development: The ability to understand
(receptive) and use (expressive) words to acquire and
convey meaning.
#5. Reading Comprehension: The complex cognitive
process involving the intentional interaction between
reader and text to extract meaning.
National Reading Panel. (2000). Teaching children to read: An evidence-based
assessment of the scientific research literature on reading and its implications for reading
instruction: Reports of the subgroups. Bethesda, MD: National Institute of Child Health and Human
Development. Available: http://www.nationalreadingpanel.org/.
March 31, 2004
Dallas, TX
3
Reading First:
Four Kinds/Purposes of Reading Assessment
An effective, comprehensive, reading program
includes reading assessments to accomplish four
purposes:
 Outcome - Assessments that provide a bottom-line
evaluation of the effectiveness of the reading program.
 Screening - Assessments that are administered to
determine which children are at risk for reading
difficulty and who will need additional intervention.
 Diagnosis - Assessments that help teachers plan
instruction by providing in-depth information about
students’ skills and instructional needs.
 Progress Monitoring - Assessments that determine if
students are making adequate progress or need more
intervention to achieve grade level reading outcomes.
March 31, 2004
Source: Reading First Initiative: Secretary’s Leadership Academy
Dallas, TX
4
Using an Outcomes Driven Model to
Provide Decision Rules for Progress
Monitoring
Outcomes Driven model: Decision making steps
1. Identifying Need for Support
2. Validating Need for Instructional Support
3. Planning and Implementing Instructional Support
4. Evaluating and Modifying Instructional Support
5. Reviewing Outcomes for Individuals and Systems
Good, R. H., Gruba, J., & Kaminski, R. A. (2002). Best Practices in Using Dynamic
Indicators of Basic Early Literacy Skills (DIBELS) in an Outcomes-Driven Model. In A. Thomas
& J. Grimes (Eds.), Best Practices in School Psychology IV (pp. 679-700). Washington, DC:
National Association of School Psychologists.
March 31, 2004
Dallas, TX
5
Identify Need
for Support
Validate Need
for Support
Plan
Instructional
Support
Using the Outcomes
Driven Model
Implement
Instructional
Support
Provide Instructional Support
Based on Integrated
Assessment - Intervention
Feedback Loop
Evaluate
Support
3 times per year progress monitoring
- Low Risk
Frequent progress monitoring
- At Risk
Review
Outcomes
March 31, 2004
Dallas, TX
6
Progress Monitoring Model for
Beginning Reading Core Areas
Big Ideas in
Beginning
Reading
Phonological
Awareness
Dynamic
Indicators of
Big Ideas in
Beginning
Reading
Benchmark Goal
Fall
Timeline for
Assessing Big
Ideas K-3
OnRF
ISF
PSF
Winter
Spring
Kindergarten
Accuracy &
Fluency with
Connected Text
Alphabetic
Principle
NWF
Fall
Winter
First Grade
ORF
Spring
ORF
Fall
Winter Spring
Second Grade
High-Stakes
Reading
Outcome
ORF
Fall
OSA
Winter Spring
Third Grade
3 times per year progress monitoring - Low Risk
Frequent progress monitoring - At Risk
Good, R. H., Simmons, D. C., & Kame'enui, E. J. (2001). The importance and decisionmaking utility of a continuum of fluency-based indicators of foundational reading skills for thirdgrade high-stakes outcomes. Scientific Studies of Reading, 5, 257-288.
March 31, 2004
Dallas, TX
7
Progress Monitoring


March 31, 2004
Repeated, formative assessment to evaluate progress
toward important goals for the purpose of modifying
instruction or intervention.
Frequency of Progress Monitoring
 3 times per year for students at low risk (All
Students)
 Benchmark
 1 per month for students with some risk
 Strategic
 1 per week for students at risk
 Intensive
Dallas, TX
8
Research on Progress Monitoring



March 31, 2004
Progress monitoring has been extensively researched
in Special Education
For example:
Fuchs, L. S., & Fuchs, D. (1986). Effects of
systematic formative evaluation: A meta-analysis.
Exceptional Children, 53, 199-208.
With Reading First, progress monitoring is not just for
special education any more.
Dallas, TX
9
Effects of Progress Monitoring



March 31, 2004
Fuchs and Fuchs (1986) found the average effect size
associated with progress monitoring was:
 +0.70 for monitoring progress
 +0.80 when graphing of progress was added
 +0.90 when decision rules were added
A student at the 50th percentile would be expected to
move to the 82nd percentile
(i.e., a score of 100 would move to a score of 114)
Perhaps more important, a student at the 6th
percentile would be expected to move to the average
range (25th percentile)
(i.e., a score of 76 would move to a score of 90)
Dallas, TX
10
Progress Monitoring Tools


March 31, 2004
Meaningful and important goals, waypoints, or
benchmarks representing reading health or wellness.
 Meaningful and Important
 Public and Measurable
 Ambitious
Brief, repeatable, formative assessment of progress
toward benchmark goals that is sensitive to
intervention.
 Brief and Efficient
 Repeatable - weekly or monthly
 Reliable and Valid indication of risk and growth
Dallas, TX
11
Secretary’s Leadership Academy
Assessment Committee
Team Leader Edward J. Kame’enui, University of Oregon

David Francis, University of Houston

Lynn Fuchs, Vanderbilt University

Roland Good, University of Oregon

Rollanda O’Connor, University of Pittsburgh

Deborah Simmons, University of Oregon

Gerald Tindal, University of Oregon

Joseph Torgesen, Florida State University
Kameenui, E. J., Francis, D., Fuchs, L. Good, R. O’Connor, R. Simmons, D., Tindal, G.,
Torgesen, J. (2002). Secretary’s Leadership Academy, Reading First Initiative,
Assessment Committee Presentation. US Dept. of Education: Washington, DC.
March 31, 2004
Dallas, TX
12
idea.uoregon.edu/assessment
March 31, 2004
Dallas, TX
13
March 31, 2004
Dallas, TX
14
March 31, 2004
Dallas, TX
15
March 31, 2004
Dallas, TX
16
March 31, 2004
Dallas, TX
17
Progress Monitoring Model for
Beginning Reading Core Areas
Big Ideas in
Beginning
Reading
Phonological
Awareness
Dynamic
Indicators of
Big Ideas in
Beginning
Reading
Benchmark Goal
Fall
Timeline for
Assessing Big
Ideas K-3
OnRF
ISF
PSF
Winter
Spring
Kindergarten
Accuracy &
Fluency with
Connected Text
Alphabetic
Principle
NWF
Fall
Winter
First Grade
ORF
Spring
ORF
Fall
Winter Spring
Second Grade
High-Stakes
Reading
Outcome
ORF
Fall
OSA
Winter Spring
Third Grade
3 times per year progress monitoring - Low Risk
Frequent progress monitoring - At Risk
Good, R. H., Simmons, D. C., & Kame'enui, E. J. (2001). The importance and decisionmaking utility of a continuum of fluency-based indicators of foundational reading skills for thirdgrade high-stakes outcomes. Scientific Studies of Reading, 5, 257-288.
March 31, 2004
Dallas, TX
18
DIBELS™ Assess the Big Ideas
Big Idea of Literacy
DIBELS/CBM Measure
Phonological Awareness
Initial Sound Fluency
Phoneme Segmentation Fluency
Alphabetic Principle
Nonsense Word Fluency
Accuracy and Fluency with
Connected Text
Oral Reading Fluency
Comprehension
At least through grade 3:
A combination of Oral
Reading Fluency & Retell
Fluency
Vocabulary – Oral Language
Word Use Fluency
Indicator of Risk
Letter Naming Fluency
DIBELS™ Initial Sound Fluency
Dynamic Indicators of Basic Early Literacy Skills™ 6th Ed.
Directions for Administration and Scoring*
P hone m e S egm entation Fluency
Letter N am in g F luen cy
Initial S ound Fluency
B eg
M id
E nd
P reschool


B eg
C B M R ea ding G 1
N onsen se-w ord Flue ncy
M id
E nd
K inde rg arte n
B eg
M id
E nd
First G rade
B eg
M id
E nd
S econd G rad e
Initial Sound Fluency is intended for most children from the last year of preschool
through the middle of kindergarten. It may be appropriate for monitoring the
progress of older children with very low skills in phonological awareness.
The benchmark goal is 25 to 35 in the middle of kindergarten. Below 10 in the
middle of kindergarten is indicates need for intensive instructional support.
BIG IDEA of Early Reading: Phonemic Awareness
D y n a m ic In d ic a to rs o f B a sic E a rly L ite ra cy S kills
U n iv e rs ity o f O re g o n
In itia l S o u n d F lu e n c y -S a m p le
DIBELS Initial
Sound Fluency
This is a mouse, flowers,
pillow, letters (point to each
picture while saying its
name).
Mouse begins with the
sound /m/ (point to the
mouse). Listen: /m/,
mouse. Which one begins
with the sounds /fl/?
March 31, 2004
Dallas, TX
21
Phoneme Segmentation Fluency
Dynamic Indicators of Basic Early Literacy Skills™ 6th Ed.
Directions for Administration and Scoring*
P h o n em e S e g m e n ta tio n F lu e n cy
B eg
M id
End
P re s c h o o l


B eg
M id
End
K in d e rg a rte n
B eg
M id
End
F irs t G ra d e
B eg
M id
End
S e c o n d G ra d e
B eg
M id
End
T h ird G ra d e
Phoneme Segmentation Fluency works well for most children from winter of
kindergarten through spring of first grade. It may be appropriate for monitoring
the progress of older children with low phonological awareness skills.
The benchmark goal is 35 to 45 correct phonemes per minute in the spring of
kindergarten and fall of first grade. Students scoring below 10 in the spring of
kindergarten and fall of first grade may need intensive instructional support to
achieve benchmark goals.
BIG IDEA of Early Reading: Phonemic Awareness
DIBELS Phoneme Segmentation Fluency
bad
/b/ /a/ /d/
lock
/l/ /o/ /k/
______/6
that
/TH/ /a/ /t/
pick
/p/ /i/ /k/
______/6
mine
/m/ /ie/ /n/
noise /n/ /oi/ /z/
______/6
coat
/k/ /oa/ /t/
spin
/s/ /p/ /i/ /n/
______/7
meet
/m/ /ea/ /t/
ran
/r/ /a/ /n/
______/6
wild
/w/ /ie/ /l/ /d/
dawn /d/ /o/ /n/
______/7
woke
/w/ /oa/ /k/
sign
/s/ /ie/ /n/
______/6
fat
/f/ /a/ /t/
wait
/w/ /ai/ /t/
______/6
side
/s/ /ie/ /d/
yell
/y/ /e/ /l/
______/6
jet
/j/ /e/ /t/
of
/o/ /v/
______/5
land
/l/ /a/ /n/ /d/
wheel /w/ /ea/ /l/
______/7
beach
/b/ /ea/ /ch/
globe /g/ /l/ /oa/ /b/
______/7
I am going to say a
word. After I say it, you
tell me all the sounds in
the word. So, if I say,
“sam,” you would say
/s/ /a/ /m/. Let’s try one.
(one second pause).
Tell me the sounds in
“mop”
Ok. Here is your first
word.
Total ______/75
March 31, 2004
Dallas, TX
23
DIBELS™ Nonsense Word Fluency
Dynamic Indicators of Basic Early Literacy Skills™ 6th Ed.
Directions for Administration and Scoring*
N o n s e n s e W o rd F lu e n c y
B eg
M id
End
P re s c h o o l


B eg
M id
End
K in d e rg a rte n
B eg
M id
End
B eg
F irs t G ra d e
M id
End
S e c o n d G ra d e
B eg
M id
End
T h ird G ra d e
Nonsense Word Fluency is intended for most children from spring of kindergarten
through spring of first grade. It may be appropriate for monitoring the progress of
older children with low skills in alphabetic principle.
The benchmark goal for Nonsense Word Fluency is 50 correct letter sounds per
minute by mid first grade. Students scoring below 30 in mid first grade may need
intensive instructional support to achieve first grade reading goals.
BIG IDEA of Early Reading: Alphabetic Principle
DIBELS Nonsense Word Fluency
k ik w o j
sig
fa j
y is
kaj
fek
av
zin
zez
la n
n u l zem
og nom
yuf pos vok
v iv
feg
bub
vus
to s
d ij
w u v n ij
sij
p ik n o k m o t
n if
v ec
al
b o j n en
su v
y ig
d it tu m
jo j
yaj
zo f
u m v im
v el
tig m a k so g w o t sa v
March 31, 2004
Here are some more makebelieve words (point to the student
probe). Start here (point to the first
word) and go across the page
(point across the page). When I
say, “begin”, read the words
the best you can. Point to
each letter and tell me the
sound or read the whole
word. Read the words the
best you can. Put your finger
on the first word. Ready,
begin.
Dallas, TX
25
DIBELS™ Oral Reading Fluency
Dynamic Indicators of Basic Early Literacy Skills™ 6th Ed.
Directions for Administration and Scoring*
D IB E L S O ral R e a d in g F lu e n c y
B eg
M id
P re s c h o o l

End
B eg
M id
End
K in d e rg a rte n
B eg
M id
End
F irs t G ra d e
B eg
M id
End
S e c o n d G ra d e
B eg
M id
End
T h ird G ra d e
DIBELS™ Oral Reading Fluency is intended for most children from mid first
grade through third grade. The benchmark goals are 40 in spring of kindergarten,
90 in spring of second grade, and 110 in the spring of third grade. Students may
need intensive instructional support if they score below 10 in spring of first grade,
50 in spring of second grade, and below 70 in spring of third grade.
BIG IDEA of Early Reading: Accuracy and Fluency with Connected Text
T h e R ob in ’s N est
T h ere w as a rob in ’s n est ou tsid e ou r k itch en w in d ow . T h e
n est w as in a tall b u sh . T h e m oth er rob in sat in th e n est all d ay
DIBELS Oral
Reading Fluency
lon g. O n e d ay w h en I w as w atch in g, th e m oth er b ird flew
aw ay. I saw th e eggs sh e w as sittin g on . T h ere w ere fou r b lu e
eggs.
I w atch ed an d w atch ed . T h e eggs m oved . I w atch ed som e
m ore. T h e eggs started to crack . F in ally, th e eggs h atch ed . I
saw fou r b ab y b ird s. T h e b ab y b ird s op en ed th eir b eak s w id e.
I h eard th em p eep in g. S oon th e m oth er b ird cam e b ack . T h en
th e m oth er rob in p u t w orm s in th eir m ou th s.
E very d ay I w atch ed th e b ab y b ird s an d th eir m oth er.
P retty soon th e b ab ies w ere so fat th ere w as n o room for th e
m oth er. T h en on e m orn in g th e n est w as gon e from th e b u sh .
Please read this (point)
out loud. If you get
stuck, I will tell you the
word so you can keep
reading. When I say,
“stop” I may ask you to
tell me about what you
read, so do your best
reading. Start here (point
to the first word of the passage).
D IB E L S O ral R eading F luency
© 2001 D ynam ic M easu rem ent G roup
March 31, 2004
F irst G rade B enchm ark 2
R evised: 03/28/02
Dallas, TX
Begin.
27
DIBELS™ Retell Fluency
Dynamic Indicators of Basic Early Literacy Skills™ 6th Ed.
Directions for Administration and Scoring*
D IB E L S O ral R e a d in g F lu e n c y
B eg
M id
P re s c h o o l

End
B eg
M id
End
K in d e rg a rte n
B eg
M id
End
F irs t G ra d e
B eg
M id
End
S e c o n d G ra d e
B eg
M id
T h ird G ra d e
DIBELS™ Retell Fluency is intended for most children from mid first
grade through third grade who are reading at least 40 words per minute.
It has been developed to provide a comprehension check for the DORF
Assessment.
BIG IDEA of Early Reading: Comprehension
End
DIBELS Retell Fluency
Please tell me all about what you just read. Try
to tell me everything you can. Begin. Start your
stopwatch after you say “begin”.
0 1 2
26 27
49 50
72 73
March 31, 2004
3 4 5
28 29
51 52
74 75
6 7 8
30 31
53 54
76 77
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94
Dallas, TX
29
Validity of ORF with RTF for Reading
Comprehension




March 31, 2004
Desirable standards: r = .60 to .80
First grade: ORF with consistent retell correlates with
Woodcock Johnson Broad Reading Cluster r = .81
(average of 2 probes)
 But, ORF with inconsistent retell correlates r = .42
Third grade: A single probe ORF and RTF correlates
with Oregon State Assessment -- Reading and
Literature Subtest: r = .73
RTF by itself generally correlates in the .20s, .30s, .40s
and .50s with a variety of measures of comprehension.
Dallas, TX
30
Inconsistent Retell in First Grade
60
Retell Fluency
50
40
30
20
10
0
0
20
40
60
80
Oral Reading Fluency
100
120
Students reading more than 40 words correct per minute, typical
retell is about 50% of ORF score.
Consistent retell is greater than or equal to 25% of ORF score.
An inconsistent retell is less than 25% of the ORF score.
March 31, 2004
Dallas, TX
31
ORF with inconsistent Retell has lower
validity with Reading Outcomes
150
WJ Broad Reading Cluster SS
140
130
120
110
100
90
80
70
60
50
10
20
30
40 50 60 70 80 90
VIP Oral Reading Fluency
100 110 120 130
ORF with consistent retell r = .81
ORF with inconsistent retell r = .42
March 31, 2004
Dallas, TX
32
Inconsistent Retell in Third Grade
Retell Fluency
100
90
80
70
60
50
40
30
20
10
0
0
20
40
60
80 100 120 140 160 180 200 220 240
Oral Reading Fluency
Students reading more than 40 words correct per minute, typical
retell is about 50% of ORF score.
Consistent retell is greater than or equal to 25% of ORF score.
An inconsistent retell is less than 25% of the ORF score.
March 31, 2004
Dallas, TX
33
DIBELS™ Word Use Fluency
Dynamic Indicators of Basic Early Literacy Skills™ 6th Ed.
Directions for Administration and Scoring*
L e tte r N a m in g F lu e n c y
B eg
M id
P re s c h o o l
End
B eg
M id
End
K in d e rg a rte n
B eg
Word Use Fluency
M id
End
F irs t G ra d e
B eg
M id
End
S e c o n d G ra d e
B eg
M id
T h ird G ra d e
Word Use Fluency (WUF) is intended for most children from fall of
kindergarten through third grade. A benchmark goal is not provided
for WUF because additional research is needed to establish its linkage
to other big ideas of early literacy. Tentatively, students in the lowest
20 percent of a school district using local norms should be considered
at risk for poor language and reading outcomes and those between the
20th and 40th percentile should be considered at some risk.
BIG IDEA of Early Reading: Vocabulary and Oral Language
End
DIBELS™ Word Use Fluency

Format: Examiner orally presents word and asks child to tell a
sentence using the word.
 “Listen to me use this word in a sentence. Jump. I like to
jump rope. Your turn to use a word in a sentence. Pool.”
P robe 1
pool
1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 17 18 19 20 21 22
_______ 
tr ie d
1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 17 18 19 20 21 22
_______ 
w orry
1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 17 18 19 20 21 22
_______ 
happened
1 2 3 4 5 6 7 8 9 10 11 12 13 14 16 17 18 19 20 21 22
_______ 
Word Use
Fluency
Street: Don’t go in the street (5)
Bottom: Sit on your bottom (4)
Today: Today have a fun day (5)
Anyone: Anyone can go to my party (6)
Against: You’re against me (3)
Dress: Dress yourself. I’m not gonna
dress you.(7)
Snow: I like to play in the snow
(7)
Bats: Bats are scary (3)
Middle: Careful that’s middle (0)
Total = 40
WUF Distributions
100
80
60
40
20
WUF Fall Benchmark
0
WUF Winter Benchmark
3rd
WUF Spring Benchmark
e
ad
gr
e
ad
gr
2nd
ird
Th
nd
co
Se
1st
de
ra
tg
n
rte
ga
K
rs
Fi
er
nd
Ki
-20
Grade Level
March 31, 2004
Dallas, TX
37
Statewide WUF Distributions
Kindergarten 02-03/03-04
= 2002-2003
= 2003 -2004
March 31, 2004
Dallas, TX
38
Statewide WUF Distributions
First Grade 02-03/03-04
= 2002-2003
= 2003 -2004
March 31, 2004
Dallas, TX
39
Statewide WUF Distributions
Second Grade 02-03/03-04
= 2002-2003
= 2003 -2004
March 31, 2004
Dallas, TX
40
Statewide WUF Distributions
Third Grade 02-03/03-04
= 2002-2003
= 2003 -2004
March 31, 2004
Dallas, TX
41
Word Use Fluency
Summary

1-month Alternate forms reliability = .59 - .65
1-week Alternate forms reliability = .65 -.71 (4-5 probes for r = .90)
Criterion-Related Validity
 PPVT = .31 - .55
 TOLD = .44 - .55
 EVT = .22 - .57
 WJ-LC = .36 - .47
 WRMT Reading Comprehension = .28 - .41
 Language Sample DWR = .44 - .72
Sensitive to growth over time in K-1 (mean slope of 3.12 words per
minute per month)
Easy and practical to administer

No Benchmark goals established -- Use local norms




March 31, 2004
Dallas, TX
42
DIBELS™ Letter Naming Fluency
Dynamic Indicators of Basic Early Literacy Skills™ 6th Ed.
Directions for Administration and Scoring*
L e tte r N a m in g F lu e n c y
B eg
M id
P re s c h o o l


End
B eg
M id
End
K in d e rg a rte n
B eg
M id
End
F irs t G ra d e
B eg
M id
End
S e c o n d G ra d e
B eg
M id
T h ird G ra d e
Letter Naming Fluency works well for most children from fall of kindergarten through fall of
first grade.
Students are considered at risk for difficulty achieving early literacy benchmark goals if they
perform in the lowest 20% of students in their district. That is, below the 20th percentile using
local district norms. Students are considered at some risk if they perform between the 20th and
40th percentile using local norms. Students are considered at low risk if they perform above the
40th percentile using local norms.
BIG IDEA of Early Reading: NONE; indicator of risk
End
P ro b e 1
c
c
N
u
Q
M
u
h
S
i
n
b
e
N
F
f
o
a
K
k
g
p
k
p
a
H
C
e
G
D
b
w
F
i
h
O
x
j
I
K
x
t
Y
q
L
d
f
T
g
v
T
V
Q
o
w
P
J
t
B
X
Z
v
U
P
R
l
V
C
l
W
R
J
m
O
z
D
G
y
U
Y
Z
y
A
m
X
z
H
S
M
E
q
n
j
s
W
r
d
s
B
I
r
A
E
L
c
c
N
u
Q
M
LNF Probes



T o ta l: __ __ /1 10
Each probe is a random
sort of 2 lower case and 2
upper case alphabets.
Lines help students to
keep their place.
Serial naming and fluency
aspects of the task are
important.
Using DIBELS™ Within an Outcomes
Driven Model to Provide Decision
Rules for Progress Monitoring
Outcomes Driven model: Decision making steps
1. Identifying Need for Support
2. Validating Need for Instructional Support
3. Planning and Implementing Instructional Support
4. Evaluating and Modifying Instructional Support
5. Reviewing Outcomes for Individuals and Systems
Good, R. H., Gruba, J., & Kaminski, R. A. (2002). Best Practices in Using Dynamic
Indicators of Basic Early Literacy Skills (DIBELS) in an Outcomes-Driven Model. In A. Thomas
& J. Grimes (Eds.), Best Practices in School Psychology IV (pp. 679-700). Washington, DC:
National Association of School Psychologists.
March 31, 2004
Dallas, TX
45
Three Levels of Assessment

Benchmark Assessment


Strategic Monitoring


Assess all children 3 - 4 times/year (e.g., Fall, Winter, Spring)
 How is the program (e.g.,classroom, school, curriculum, instruction)
doing overall?
 Are there children who may need additional support to achieve
outcomes?
 Which children may need additional support to achieve outcomes?
Assess at risk children more frequently (e.g., monthly)
 Is current program sufficient to keep progress on track or are additional
supports/intervention needed?
Continuous or Intensive Care Monitoring

March 31, 2004
Assess students needing more intensive, effective intervention weekly
 Are instructional supports/strategies effective or is a change in
intervention needed?
Dallas, TX
46
Benchmark Assessment - First Grade
Dynamic Indicators of Basic Early Literacy SkillsTM 6th Ed.
University of Oregon
First Grade Benchmark Assessment
Name:
Teacher:
School:
District:
Benchmark 1
Beginning/Fall
Benchmark 2
Middle/Winter
Benchmark 3
End/Spring
(middle score)
(middle score)
(middle score)
(middle score)
(Optional)
(Optional)

Date
Letter Naming
Fluency
Phoneme
Segmentation
Fluency
Nonsense
Word Fluency
DIBELS Oral
Reading
Fluency2

Retell Fluency
(Optional)
Word Use
Fluency
(Optional)
(Optional)


© 2002 Dynamic Measurement Group, Inc.
March 31, 2004
Revised: 07/02/02
Page 1
Dallas, TX
Benchmark assessment –
screening all children to
identify need for support to
achieve goals in Core
Components of literacy:
phonemic awareness,
alphabetic principle, accuracy
and fluency with connected
text,
Vocabulary, and
Reading Comprehension for
all children.
Beginning: September,
October, or November
Middle: December, January, or
February
End: March, April, May, or
June
47
1. Identifying Need for Support
Key Decision for Screening Assessment:

Which children may need additional instructional support to
attain important reading outcomes?
Data used to inform the decision:

Compare individual student’s performance to benchmark goals
or local normative context to evaluate need for additional
instructional support.

Benchmark Goals: A deficit in a foundation skill is a
strong indicator that instructional support will be needed to
attain later benchmark goals.

Local normative context: First, choose a percentile
cutoff. 20th percentile seems a good place to start, but a
district could choose 15th percentile or 25th percentile or
other cutoff depending on resources.
March 31, 2004
Dallas, TX
48
http://DIBELS.uoregon.edu
March 31, 2004
Dallas, TX
49
Beginning of
Kindergarten
March 31, 2004
Dallas, TX
50
Identify Students who Need Support
to Reach NEXT Benchmark Goal


March 31, 2004
In September of Kindergarten, Melissa has a deficit on
initial sounds. She may need additional instructional
support to achieve kindergarten benchmark goals in
Phonemic Awareness.
Tevin is on track with to achieve Phonemic Awareness
goals with effective core curriculum and instruction.
Dallas, TX
51
Longitudinal Outcomes for DIBELS
Benchmark Assessment
Odds of achieving subsequent early literacy goals for
DIBELS Benchmark Assessments at the beginning,
middle, and end of kindergarten, first, second, and third
grades (12 screening points across K - 3) are available at
dibels.uoregon.edu/techreports/decision_rule_summary.pdf
 Students are at risk if the odds are against achieving
subsequent early literacy goals.
 The purpose of screening is to provide additional
instructional support -- strategic or intensive -sufficient to thwart the prediction of difficulty achieving
reading outcomes.

March 31, 2004
Dallas, TX
52
Sample Odds of Achieving Early Literacy Goals
for Different Patterns of DIBELS Performance
T able 4
Instructional R ecom m en dations for Individual P atterns of P erform ance on M iddle of K indergarten D IB E L S B enchm ark A ssessm ent
P ercent M eeting L ater G o als
Initial S o und
F lue nc y
L etter
N a m ing
F lue nc y
P ho ne m e
S eg m en tatio n
F lue nc y
P ctile
E nd K
PSF
M id 1
NW F
E nd 1
ORF
A v g.
Incid ence
Instruc tio nal S up p o rt R eco m m end atio n
D eficit
A t R isk
A t R isk
3
18
14
19
17
M o re C o m m o n
Inten sive - N eed s S ub sta ntial Interve ntio n
D eficit
A t R isk
S o m e R isk
7
34
13
21
23
U n usual
Inten sive - N eed s S ub sta ntial Inter ve ntio n
E m ergin g
A t R isk
A t R isk
9
28
20
28
25
M o re C o m m o n
Inten sive - N eed s S ub sta ntial Interve ntio n
E m ergin g
A t R isk
S o m e R isk
11
41
17
22
27
M o re C o m m o n
Inten sive - N eed s S ub sta ntial Interve ntio n
D eficit
S o m e R isk
A t R isk
13
24
28
48
33
M o re C o m m o n
Inten sive - N eed s S ub sta ntial Interve ntio n
D eficit
A t R isk
L o w R isk
15
60
21
25
35
U n usual
Inten sive - N eed s S ub sta ntial Interve ntio n
D eficit
S o m e R isk
S o m e R isk
16
37
30
40
36
U n usual
S trategic - A d d itio nal In terve n tio n
E stab lished
A t R isk
A t R isk
17
45
32
31
36
E xtre m ely R are
S trategic - A d d itio nal In terve n tio n
E m ergin g
S o m e R isk
A t R isk
18
37
30
49
38
U n usual
S trategic - A d d itio nal In terve n tio n
D eficit
L o w R isk
A t R isk
20
30
37
58
42
U n usual
S trategic - A d d itio nal In terve n tio n
E stab lished
S o m e R isk
A t R isk
21
42
38
49
43
E xtre m ely R are
S trategic - A d d itio nal In terve n tio n
E m ergin g
S o m e R isk
S o m e R isk
22
47
36
51
45
M o re C o m m o n
S trategic - A d d itio nal In terve n tio n
E stab lished
A t R isk
S o m e R isk
24
52
38
47
45
E xtre m ely R are
S trategic - A d d itio nal In terve n tio n
E m ergin g
A t R isk
L o w R isk
26
75
29
36
47
M o re C o m m o n
S trategic - A d d itio nal In terve n tio n
D eficit
L o w R isk
S o m e R isk
28
43
42
68
51
U n usual
S trategic - A d d itio nal In terve n tio n
D eficit
S o m e R isk
L o w R isk
29
66
41
55
54
E xtre m ely R are
S trategic - A d d itio nal In terve n tio n
E m ergin g
L o w R isk
A t R isk
31
42
50
70
54
M o re C o m m o n
S trategic - A d d itio nal In terve n tio n
E stab lished
S o m e R isk
S o m e R isk
33
55
44
64
54
U n usual
S trategic - A d d itio nal In terve n tio n
E stab lished
A t R isk
L o w R isk
34
82
34
47
54
U n usual
S trategic - A d d itio nal In terve n tio n
E m ergin g
L o w R isk
S o m e R isk
38
53
53
80
62
M o re C o m m o n
S trategic - A d d itio nal In terve n tio n
E m ergin g
S o m e R isk
L o w R isk
44
82
47
59
63
M o re C o m m o n
S trategic - A d d itio nal In terve n tio n
E stab lished
L o w R isk
A t R isk
47
51
58
89
66
E xtre m ely R are
B ench m ark - A t grad e leve l
E stab lished
L o w R isk
S o m e R isk
49
58
62
87
69
M o re C o m m o n
B ench m ark - A t grad e leve l
D eficit
L o w R isk
L o w R isk
52
74
60
75
70
U n usual
B ench m ark - A t grad e leve l
E stab lished
S o m e R isk
L o w R isk
54
88
56
69
71
M o re C o m m o n
B ench m ark - A t grad e leve l
E m ergin g
L o w R isk
L o w R isk
64
88
68
83
80
M o re C o m m o n
B ench m ark - A t grad e leve l
E stab lished
L o w R isk
L o w R isk
86
93
80
93
89
M o re C o m m o n
B ench m ark - A t grad e leve l
N ote. P ercent m eetin g go al is the conditio nal percent of children w ho m eet the end of first grade go al of 40 or m ore on D IB E LS O R F .
B ased on n o f approx im ately 32000 students, 638 schools, and 255 school districts.
dibels.uoregon.edu/techreports/decision_rule_summary.pdf
March 31, 2004
Dallas, TX
53
Letter
Initial Sound
Naming
Fluency
Fluency
Deficit
At Risk
Deficit
At Risk
Emerging
At Risk
Emerging
At Risk
Deficit
Some Risk
Phoneme
Segmentation
Fluency
At Risk
Some Risk
At Risk
Some Risk
At Risk
Pctile
3
7
9
11
13
[Table Continues]
Established
Emerging
Established
Some Risk
Low Risk
Low Risk
Low Risk
Low Risk
Low Risk
54
64
86
Pattern of performance based on the
DIBELS Benchmark Assessment
[Table Continues]
Instructional Recommendations for Individual Patterns of
Performance on Middle of Kindergarten DIBELS
Benchmark Assessment
Percentile Rank for
the pattern of
performance. For
example, a child with
established ISF, some
risk on LNF, and low
risk on PSF is at the
54th percentile
compared to other
children in the middle
of kindergarten. He
or she achieved as well
or better than 54% of
children in
participating schools
on DIBELS.
dibels.uoregon.edu/techreports/decision_rule_summary.pdf
March 31, 2004
Dallas, TX
54
Instructional Recommendations for Individual Patterns of
Performance on Middle of Kindergarten DIBELS
Benchmark Assessment (continued)
Phoneme
Segmentation
Fluency
[Table Continues]
Average Percent
Initial Sound
End K Mid 1 End 1
achieving
Fluency
PSF NWF ORF Avg. subsequent early
Deficit
At Risk
At Risk
18
14
19
17
literacy goals.
Deficit
At Risk
Some Risk
34
13
21
23
For example, a
Emerging
At Risk
At Risk
28
20
28
25
Emerging
At Risk
Some Risk
41
17
22
27
student with a
Deficit
Some Risk At Risk
24
28
48
33
Deficit, Some
[Table Continues]
Risk, At Risk
Established Some Risk Low Risk
88
56
69
71
pattern on
Emerging
Low Risk
Low Risk
88
68
83
80
DIBELS has
Established Low Risk
Low Risk
93
80
93
89
33% odds of
Odds of achieving specific early literacy goal. For example, 69% achieving later
of students with Established, Some Risk, Low Risk pattern in the literacy goals on
middle of kindergarten achieved the end of first grade DIBELS average.
Oral Reading Fluency goal of 40 or more words read correct per
minute.
Letter
Naming
Fluency
Percent Meeting Later Goals
dibels.uoregon.edu/techreports/decision_rule_summary.pdf
March 31, 2004
Dallas, TX
55
Letter
Initial Sound
Naming
Fluency
Fluency
Deficit
At Risk
Deficit
At Risk
Emerging
At Risk
Emerging
At Risk
Deficit
Some Risk
Phoneme
Segmentation
Fluency
At Risk
Some Risk
At Risk
Some Risk
At Risk
[Table Continues]
Instructional Recommendations for Individual Patterns of
Performance on Middle of Kindergarten DIBELS
Benchmark Assessment (continued)
Incidence
More Common
Unusual
More Common
More Common
More Common
[Table Continues]
Established
Emerging
Established
Some Risk Low Risk
Low Risk Low Risk
Low Risk Low Risk
More Common
More Common
More Common
Extremely rare patterns may indicate a need to retest.
For example, it would be extremely rare for a student
to have Established ISF, Low Risk on LNF, and At
Risk status on PSF. Their PSF score may not be
accurately estimating their phonemic awareness skill.
Incidence or how often a
pattern of performance
occurs. For example,
among students with a
Deficit on ISF and Some
Risk on LNF, achieving
in the At Risk range on
PSF would be a more
common pattern, but
achieving in the Some
Risk range would be an
unusual pattern.
dibels.uoregon.edu/techreports/decision_rule_summary.pdf
March 31, 2004
Dallas, TX
56
Initial Sound
Fluency
Deficit
Deficit
Emerging
Letter
Naming
Fluency
At Risk
At Risk
At Risk
Phoneme
Segmentation
Fluency
At Risk
Some Risk
At Risk
[Table Continues]
Emerging
Established
Emerging
Some Risk Some Risk
At Risk
Some Risk
At Risk
Low Risk
[Table Continues]
Instructional Recommendations for Individual Patterns of
Performance on Middle of Kindergarten DIBELS
Benchmark Assessment (continued)
Instructional Support Recommendation
Intensive - Needs Substantial Intervention
Intensive - Needs Substantial Intervention
Intensive - Needs Substantial Intervention
Strategic - Additional Intervention
Strategic - Additional Intervention
Strategic - Additional Intervention
[Table Continues]
Established
Emerging
Established
Some Risk Low Risk
Low Risk Low Risk
Low Risk Low Risk
Benchmark - At grade level
Benchmark - At grade level
Benchmark - At grade level
Instructional Support Recommendation. For students with odds in favor of
achieving subsequent literacy goals, benchmark instruction is recommended. For
students with odds against achieving subsequent literacy goals, intensive support
is recommended. For about 50 – 50 odds, strategic support is recommended.
dibels.uoregon.edu/techreports/decision_rule_summary.pdf
March 31, 2004
Dallas, TX
57
Decision Utility of DIBELS

Pattern of performance on DIBELS measures determines
overall risk status and instructional recommendation. In
fall of first grade, for example,

LNF >= 37, DIBELS PSF >= 35, DIBELS NWF >= 24
Instructional Recommendation: Benchmark - At grade level. Effective
core curriculum and instruction recommended,
 Odds of reading 40 or more words correct per minute at the end of
first grade: 84%
 Odds of reading less than 20 words correct per minute at the end of
first grade: 2%
LNF < 25, DIBELS PSF < 10, DIBELS NWF < 13
Instructional Rec: Intensive - Needs substantial intervention:
 Odds of reading 40 or more words correct per minute at the end of
first grade: 18% (unless given intensive intervention)
 Odds of reading less than 20 words correct per minute at the end of
first grade: 48% (unless given intensive intervention)
Value of knowing the instructional recommendation and the goal early
enough to change the outcome: Priceless.


March 31, 2004
Dallas, TX
58
Sample Cutoffs for Low Risk, Some Risk,
At Risk for Kinder DIBELS Performance
DIBELS 3 Benchmark Goals and Indicators of Risk
Kindergarten
DIBELS
Measure
Beginning of Year
Month 1 - 3
Scores
Status
Middle of Year
Month 4 - 6
Scores
Status
End of Year
Month 7 - 10
Scores
Status
DIBELS Initial
Sound Fluency
ISF < 4
4 <= ISF < 8
ISF >= 8
At risk
Some risk
Low risk
ISF < 10
10 <= ISF < 25
ISF >= 25
Deficit
Emerging
Established
DIBELS Letter
Naming Fluency
LNF < 2
2 <= LNF < 8
LNF >= 8
At risk
Some risk
Low risk
LNF < 15
15 <= LNF < 27
LNF >= 27
At risk
Some risk
Low risk
LNF < 29
At risk
29 <= LNF < 40 Some risk
LNF >= 40
Low risk
DIBELS Phoneme
Segmentation
Fluency
PSF < 7
7 <= PSF < 18
PSF >= 18
At risk
Some risk
Low risk
PSF < 10
10 <= PSF < 35
PSF >= 35
Deficit
Emerging
Established
DIBELS Nonsense
Word Fluency
NWF < 5
5 <= NWF < 13
NWF >= 13
At risk
Some risk
Low risk
NWF < 15
15 <= NWF <
25
NWF >= 25
At risk
Some risk
Low risk
March 31, 2004
Dallas, TX
59
Middle of
Kindergarten
March 31, 2004
Dallas, TX
60
End of
Kindergarten
March 31, 2004
Dallas, TX
61
Identify Need for Support: Using Local
Norms
X
X
X
March 31, 2004
Dallas, TX
62
2. Validate Need for Support
Key Decision:
 Are we reasonably confident the student needs
instructional support?
 Rule out easy reasons for poor performance:
Bad day, confused on directions or task, ill, shy, or
similar.
 More reliable information is needed to validate
need for support than for screening decisions.
Data used to inform the decision:
 Repeated assessments on different days under
different conditions
 Compare individual student’s performance to local
normative context or expected performance to
evaluate discrepancy.
March 31, 2004
Dallas, TX
63
Identify Need:
Which children may need additional support?
Teacher’s list of children in class and DIBELS™ scores: January of K
P ercen tile
In itial
S ou n d S k ill
S tatu s
S core
P ercen tile
T ., S an d ra
9
4
D eficit
1
7
R ., M atrix
7
2
D eficit
1
7
W ., H alley
14
12
E m ergin g
2
9
M ., L atish a
19
22
E m ergin g
3
11
A ., B ran d on
9
4
D eficit
3
11
R ., T iffan y
42
86
E stab lish ed
13
31
M ., D an ielle
5
1
D eficit
14
33
M ., Josep h
38
75
E stab lish ed
15
35

A ll S ou n d s
S k ill S tatu s
At
D eficit
risk
At risk
D eficit
Some risk
D eficit
Some risk
D eficit
Some risk
D eficit
Lowgrisk
E m ergin
Lowgrisk
E m ergin
Lowgrisk
E m ergin
L etter N am in g F lu en cy
P ercen tile
S tu d en t
S core
Fluency
P h on em e S egm en tation
F lu en cy
S core
O n set R ecogn ition
Initial Sound
F lu en cy
R isk S tatu s
8
13
A t risk
In ten sive su p p ort in d icated .
11
19
A t risk
In ten sive su p p ort in d icated .
29
46
L ow risk
S trategic su p p ort.
35
59
L ow risk
S trategic su p p ort.
24
35
S om e risk
In ten sive su p p ort in d icated .
48
85
L ow risk
B en ch m ark .
21
28
S om e risk
S trategic su p p ort.
37
66
L ow risk
B en ch m ark .
In stru ction al R ecom m en d ation s B ased
P rim arily on P S F
In January of Kindergarten:

Sandra, Matrix, Brandon, and Danielle have a deficit on Initial Sound Fluency. They may
need additional instructional support to attain kindergarten benchmarks.

Joseph and Tiffany are on track with established skills on ISF.
Halley and Latisha have emerging skills and should be monitored strategically

Aggregating multiple, brief
assessments increases reliability
Reliability of Aggregate
1.00
0.90
0.80
0.70
0.60
1
2
3
4
5
6
7
8
9
10
Number of Probes

March 31, 2004
When brief, 1-minute probes are used, it is important
to consider error as one possible cause of poor
performance. A pattern of low performance across 3 - 4
probes is much more reliable.
Dallas, TX
65
Validating Need for Support
Phoneme Segmentation Fluency

Verify need for instructional support by retesting with
alternate forms until we are reasonably confident.
60
50
40
30
20
Danielle
Matrix
Mid-year cutoff at risk
10
Sandra
Brandon
Dec.
S co re s
March 31, 2004
Ja n .
S c o re s
Feb.
S co res
M arc h
S co re s
Dallas, TX
A p ril
S c o re s
M ay
S co re s
June
S co res
66
3. Planning and Implementing
Instructional Support
Key Decisions for Diagnostic Assessment:
 What are the Goals of instruction?
 Where are we? Where do we need to be? By
when? What course do we need to follow to get
there?
 What skills should we teach to get there?
 Focus on the beginning reading core areas:
Phonological Awareness, Alphabetic Principle,
Accuracy and Fluency with Connected Text
 Specific skills based on error analysis or
additional diagnostic assessment (e.g., CTOPP).
 What kind of instructional support is needed?
 Intensive Instructional Support
 Strategic Instructional Support
 Benchmark Instruction
March 31, 2004
Dallas, TX
67
Exploring Support - Aimline for Brandon
Phoneme Segmentation Fluency
The aimline connects where we are to where we need to be
by when, and shows the course to follow to get there.
60
50
40
End-year Benchmark Goal
Aimline
30
20
10
End-year cutoff at risk
Dec.
S co re s
March 31, 2004
Ja n .
S c o re s
Feb.
S co res
M arc h
S co re s
Dallas, TX
A p ril
S c o re s
M ay
S co re s
June
S co res
68
Phoneme Segmentation Fluency
Planning Support - Aimline for Sandra
The aimline connects where we are to where we need to be
by when, and shows the course to follow to get there.
60
50
End-year Benchmark Goal
40
Aimline
30
20
10
End-year cutoff at risk
Dec.
S co re s
March 31, 2004
Ja n .
S c o re s
Feb.
S co res
M arc h
S co re s
Dallas, TX
A p ril
S c o re s
M ay
S co re s
June
S co res
69
Instructional Goals for Core
Components of Beginning Reading
Benchmark Goals to be On Grade Level
 Step 1: Phonological Awareness with 25 - 35 on DIBELS
Initial Sound Fluency by mid kindergarten (and 18 on PSF)
 Step 2: Phonemic Awareness with 35 - 45 on DIBELS
Phoneme Segmentation Fluency by end of kindergarten
(and 25 on NWF)
 Step 3: Alphabetic principle 50 - 60 on DIBELS Nonsense
Word Fluency by mid first grade (and 20 on DORF)
 Step 4: Fluency with 40 - 50 on DIBELS Oral reading
fluency by end of first grade.
 Step 5: Fluency with 90 + on DIBELS Oral reading fluency
by end of second grade
 Step 6: Fluency with 110 + on DIBELS Oral reading
fluency by end of third grade
March 31, 2004
Dallas, TX
70
Instructional Steps from Kindergarten to
Successful Reading Outcomes
The outcome of each step depends on (a) students beginning skills, (b)
effectiveness of core curriculum and instruction, and (c) effectiveness
of system of additional instructional support.
March 31, 2004
Dallas, TX
71
Planning Support:
What skills should we teach?

Focus on the Big Ideas:
 Initial Sounds Fluency and Phoneme Segmentation
Fluency - Phonemic Awareness
 Nonsense Word Fluency - Alphabetic Principle
 Oral Reading Fluency - Accuracy and Fluency with
Connected Text
 Retell Fluency - Comprehension
 Word Use Fluency - Vocabulary
March 31, 2004
Dallas, TX
72
What specific skills to teach?

For specific skill level use:





March 31, 2004
Error analysis of DIBELS performance
Knowledge of child performance in class
Curriculum-linked assessment, e.g., mastery
measures
Use supplementary assessment as needed
What can the child do/not do?
Dallas, TX
73
Phonemic Awareness
Is the child accurate but not fluent? Build
fluency.
 Does the child have errors? What is the error
rate?
 Few (5-10%), some (10-33%), many (3390%), all?
 What is the pattern of errors?

March 31, 2004
Dallas, TX
74
ISF Response Patterns











Produces multiple random errors
Substitutes name of letter for initial sound
Repeats word when prompted for sound
Recognizes but does not produce initial sounds consistently
Recognizes and produces initial sounds confidently
Difficulty with consonant sounds
Difficulty with vowel sounds
Pronunciation differences due to dialect or second language
Frequent articulation difficulties, consider referral to language
specialist
Difficulty remembering picture names
Frequent self corrections
March 31, 2004
Dallas, TX
75
PSF Response Patterns













March 31, 2004
Stage 1: repeats entire word
Stage 2: produces initial sound or sounds only
Stage 3: produces onset and rhyme
Stage 4: produces initial and final sounds correctly; errors on
middle sounds
Stage 5: produces initial, middle and final sounds correctly; does
not segment blends
Stage 6: correctly segments all phonemes including phonemes in
blends
Produces consonant sounds correctly; misses vowel sounds
Omits final sounds
Pronunciation differences due to dialect or second language
Frequent articulation difficulties, consider referral to language
specialist
Frequent phoneme additions
Frequent phoneme omissions
Frequent self corrections
Dallas, TX
76
Sequence of Phonological Awareness Skills - K
1. Sound and Word Discrimination
* Tells whether words or sounds are the same or different (cat/cat = same;
cat/car=different).
* Identifies which word is different (e.g., sun, fun, sun).
* Tells the difference between single speech sounds (e.g., Which one is different? s, s, k).
2. Rhyming
* Identifies whether words rhyme (e.g., cat/mat; ring/sing).
* Produces a word that rhymes with another (e.g., "A word that rhymes with rose is nose.
Tell me another word that rhymes with rose.)
3. Blending
* Orally blends syllables (mon-key) or onset-rimes (m-ilk) into a whole word.
* Orally blends 2-3 separately spoken phonemes into one-syllable words (e.g., m-e: me; up: up; f-u-n: fun).
4. Segmentation
* Claps or counts the words in a 3-5 word sentence (e.g., Sue can jump far).
* Claps or counts the syllables in 1-, 2-, and 3-syllable words.
* Says each syllable in 2- and 3-syllable words (di-no-saur).
* Identifies the first sound in a one-syllable word (e.g., /m/ in man).
* Segments individual sounds in 2- and 3-phoneme, one-syllable words (e.g., run: /r/ /u/ /n/;
feet: /f/ /ee/ /t/).
March 31, 2004
Dallas, TX
77
Alphabetic Principle

Nonsense Word Fluency
 Is the child accurate but not fluent? Build
fluency.
 Does the child have errors? What is the
error rate?
 Does the child have errors? What is the
pattern of errors?
March 31, 2004
Dallas, TX
78
NWF Response Patterns













Stage 1: Has isolated letter-sound correspondences but lacks a systematic
strategy for attacking unknown words.
Stage 2: Produces correct consonant sounds; incorrect vowel sounds.
Stage 3: Produces most sounds correctly sound-by-sound, but does not
recode into complete word.
Stage 4: Produces sounds correctly sound-by-sound and then recodes into
complete word (e.g., /m/ /o/ /t/ “mot”)
Stage 5: Fluently applies systematic trategy for attacking unknown words
(i.e., reads mot as “mot”)
Substitutes real words for nonsense words
Produces sounds correctly sound-by-sound; recodes sounds out of order
(e.g.,/b…i…s…/ “sib”)
Consistent error for a specific consonant/vowel sound requiring review
Pronunciation differences due to dialect or second language
Frequent articulation difficulties, consider referral to language specialist
Frequent sound additions
Frequent sound omissions
Frequent self corrections
March 31, 2004
Dallas, TX
79
Critical Alphabetic Principle Skills

Letter-Sound Correspondences



Sounding Out Words



Example:
(Teacher points to letter m on board). "The sound of this
letter is /mmmmm/. Tell me the sound of this letter.”
Example: (Teacher points to the word map on the board,
touches under each sound as the students sound it out,
and slashes finger under the word as students say it
fast.) "Sound it out." (/mmmmmmmmaaaaaaap/) "Say
it fast." (map)
Reading Words
Reading Words in connected text
March 31, 2004
Dallas, TX
80
Accuracy and Fluency with
Connected Text

Oral Reading Fluency


Is the child accurate but not fluent? Build fluency.
Does the child have errors? What is the pattern of errors?



March 31, 2004
Correctly decodes easy, phonetically correct words,
misses long and/or irregular words
Consistently makes errors on words with specific blends,
digraphs, etc.
Only reads simple and common words correctly
consistently (e.g., “the” “and”)
Dallas, TX
81
Vocabulary

WUF Response Patterns
 Stereotypical
response pattern, e.g., “I like to
____”
 Word use is sparse and employs minimum
utterances
 Word use is fluent and confident employing
elaborated sentences
 Response often unrelated to target word
 Student appears shy and reticent to talk
 Student uses similar sounding word, may have
difficulty hearing target word
 Student frequently asks for the word to be
repeated, may have difficulty hearing target
word
March 31, 2004
Dallas, TX
82
What Curriculum and/or program?
Good News - Bad News - Good News



Good News: All but small number of children
can learn to read.
Bad News: No Magical Curriculum or
program that is effective for all students.
Good News: The Magic is in the system of
support that matches each child with the
support that is effective for her/him.
March 31, 2004
Dallas, TX
83
Planning Support:
What curriculum/program to use?

Three levels of instructional support



March 31, 2004
Benchmark -- Core Curriculum
Strategic -- Supplemental Curriculum
Intensive -- Intervention Curriculum
Dallas, TX
84
Benchmark Instruction Core Comprehensive Reading Programs

Purpose: to provide complete instruction in the core
components of reading

Examples:
 Open Court Reading, SRA/McGraw Hill
 Houghton Mifflin
 Reading Mastery
March 31, 2004
Dallas, TX
85
Strategic Support Supplemental Reading Programs

Purpose: to provide additional instruction in one or more
areas of reading for students who require strategic
instructional support to reach benchmark goals.

Examples:

phonemic awareness programs
Phonemic Awareness in Young Children: A Classroom
Curriculum, Paul H. Brookes Publishing Co., Inc.

fluency building programs
Read Naturally, Read Naturally, Inc.
Read Well, Sopris West

comprehension strategy programs
Soar to Success, Houghton Mifflin Co.
Collaborative Strategic Reading
March 31, 2004
Dallas, TX
86
Intensive Support Intervention Reading Programs

Purpose: to provide additional instruction to
students who have skill deficits and need intensive
support to reach benchmark goals.

Examples:
Corrective Reading, SRA/McGraw-Hill
Scott Foresman Early Reading Intervention
(Optimize), Scott Foresman
 Phonological Awareness Training for Reading,
AGS Publishing


March 31, 2004
Dallas, TX
87
Instructional Strategies


Grouping
 Small group instruction
 Flexible instructional grouping
Effective Instruction
 Focused and systematic
 Explicit





March 31, 2004

Direct explanation
Modeling
High student engagement
 Lots of opportunities to respond
 Lots of Guided Practice with immediate
feedback
Scaffolding to support learning
Integration of skills
Review
Dallas, TX
88
4. Evaluating and Modifying
Instructional Support
Key Decision for Progress Monitoring Assessment:

Is the intervention effective in improving the child’s early
literacy skills?
How much instructional support is needed?

Enough to get the child on trajectory for Benchmark
Goal.
When is increased support needed?

March 31, 2004
Monitor child’s progress during intervention by
comparing their performance and progress to past
performance and their aimline. Three consecutive
assessments below the aimline indicates a need to increase
instructional support.
Dallas, TX
89
Evaluating Support
Phoneme Segmentation Fluency
Sandra: Is the intervention working?
60
50
40
Aimline
30
20
10
Dec.
S co re s
March 31, 2004
Ja n .
S c o re s
Feb.
S co res
M arc h
S co re s
Dallas, TX
A p ril
S c o re s
M ay
S co re s
June
S co res
90
Evaluating Support
Brandon: Is the intervention working?
Phoneme Segmentation Fluency
Whoops! Time to make a change!
60
50
40
Aimline
30
20
10
Dec.
S co re s
March 31, 2004
Ja n .
S c o re s
Feb.
S co res
M arc h
S co re s
Dallas, TX
A p ril
S c o re s
M ay
S co re s
June
S co res
91
Phoneme Segmentation Fluency
Evaluating Support Brandon:Is Instructional Support Sufficient Now?
60
50
40
Aimline
30
20
10
Dec.
S co re s
March 31, 2004
Ja n .
S c o re s
Feb.
S co res
M arc h
S co re s
Dallas, TX
A p ril
S c o re s
M ay
S co re s
June
S co res
92
Where are we?
What is our goal?
What course should we follow?
How are we doing?
Our Goal
Desired
Course
We are
Here
March 31, 2004
Actual
Course
Dallas, TX
93
Progress Monitoring: The Teacher’s Map
The GPS for Educators
A change in intervention
60
50
40
Aimline
30
20
10
Dec.
S co re s
March 31, 2004
Ja n .
S c o re s
Feb.
S co res
M arc h
S co re s
Dallas, TX
A p ril
S c o re s
M ay
S co re s
June
S co res
94
Dynamic Interventions Build in an
Assessment  Intervention
Feedback Loop



March 31, 2004
Good interventions are identified by their outcomes not our philosophy, or beliefs, or the quality of their
packaging.
Good interventions are individual – an effective
intervention for one child may not be effective for
another.
Integrating assessment and intervention driven by
outcomes is a key aspect of an effective intervention.
Dallas, TX
95
Step 1: Initial Sound Fluency in First Half
of Kindergarten


40
30
20

10
Sept.
Scores
Oct.
Scores
Nov.
Scores
Dec.
Scores
Jan.
Scores
Week 1
Week 2
Feb.
Scores

Mid year goal:
25 on ISF
Beginning K
 Low risk: >= 8
 At risk: < 4
Middle K
 Low risk: >= 25
 At risk: < 10
Additional Goal
PSF >= 18
Week 3
Week 4
March 31, 2004
Dallas, TX
96
Step 2: Phoneme Segmentation Fluency in
Second Half of Kinder
60

50

40
30

20
10
Dec.
Scores
Jan.
Scores
Feb.
Scores
March
Scores
April
Scores
May
Scores
Week 1
Week 2
Week 3
June
Scores

End K goal:
35 on PSF
Middle K
 Low risk: >= 18
 At risk: < 7
End K PSF
 Established:
PSF >= 35
 Deficit: < 10
Additional Goal NWF
>= 25
Week 4
March 31, 2004
Dallas, TX
97
Step 3: Nonsense Word Fluency in First
Half of First Grade

80
70

60
50
40

30
20
10
Sept.
Scores
Oct.
Scores
Nov.
Scores
Dec.
Scores
Jan.
Scores
Week 1
Feb.
Scores

Middle first goal:
50 on NWF
Beginning first
 Low risk: >= 24
 At risk: < 13
Mid first NWF:
 Established:
NWF >= 50
 Deficit: < 30
Additional Goal:
ORF >= 20
Week 2
Week 3
Week 4
March 31, 2004
Dallas, TX
98
Step 4: Oral Reading Fluency in Second
Half of First Grade

70

60
50
40

30
20
10

Dec.
Scores
Jan.
Scores
Feb.
Scores
March
Scores
April
Scores
May
Scores
Week 1
June
Scores
End first goal:
40 on ORF
Middle first ORF:
 Low risk: >= 20
 At risk: < 8
End first ORF:
 Low risk: >= 40
 At risk: < 20
Additional Goal:
Retell > ORF/4
Week 2
Week 3
Week 4
March 31, 2004
Dallas, TX
99
Step 5: Oral Reading Fluency in Second
Grade

120
110
100

90
80
70
60
50

40
30
20
10
Sep.
Scores
Oct.
Scores
Nov.
Scores
Dec.
Scores
Jan.
Scores
Feb.
Scores
March
Scores
April
Scores
May
Scores
Wk 1

Wk 2
Wk 3
Wk 4
March 31, 2004
June
Scores
Dallas, TX
End second goal:
90 on ORF
Beg second ORF:
 Low risk: >= 44
 At risk: < 26
End second ORF:
 Low Risk:
>= 90
 At Risk: < 70
Additional Goal:
Retell > ORF/4
100
Step 6: Oral Reading Fluency in Third
Grade

140
120

100
80
60

40
20
Sept.
Scores
Oct.
Scores
Nov.
Scores
Dec.
Scores
Jan.
Scores
Feb.
Scores
March
Scores
April
Scores
May
Scores
Wk 1
Wk 2
Wk 3
June
Scores

End third goal:
110 on ORF
Beg third ORF:
 Low risk: >= 77
 At risk: < 53
End third ORF:
 Low Risk:
>= 110
 At Risk: < 80
Additional Goal:
Retell > ORF/4
Wk 4
*Each tick is 4 points.
March 31, 2004
Dallas, TX
101
Is Progress is Related to Outcomes?
The logic of the Evaluating and Modifying Support step relies on
evidence that amount of progress toward goals is related to
important reading outcomes.

Is slope of progress on NWF in the Fall of first grade related to
first grade reading outcomes? This questions was recently
examined by
Fuchs, L. S., Fuchs, D., and Compton, D. L. (in press). Monitoring
early reading development in first grade: Word Identification
Fluency versus Nonsense Word Fluency. Exceptional Children.

Fuchs et al. also examined the validity of spring slope, whole
year slope, and the validity of fall level, all of which will not
be addressed here.

March 31, 2004
Dallas, TX
102
Validity of Slope on NWF in Fall of
First Grade for Oral Reading Fluency
Outcomes

Based on 151 “at risk” children, Fuchs et al. correlated slope of
progress in fall of first grade with spring of first grade reading
outcomes:
Fall WIF
Slope
Fall NWF
Slope
WRMT-R Word Identification
.43
.05
WRMT-R Word Attack
.27
-.03
CRAB Fluency
.54
.16
CRAB Comprehension
.49
-.04
Spring Outcome Measure
Note. WIF is Word Identification Fluency
March 31, 2004
Dallas, TX
103
Conclusions: Validity of NWF Slope


March 31, 2004
“coefficients for the nonsense word fluency measure slopes
were disappointingly low, ranging from -.04 to .16. Because
nonsense word fluency is recommended for progress
monitoring in the fall of first grade within the DIBELS
system (Good et al., 2001), these findings raise serious
concern. An increasing pattern of scores through the first
semester of first grade on DIBELS nonsense word fluency
appears to bear little relationship to students’ end-of-year
reading status.” (p. 21)
“practitioners can have confidence that increases in word
identification fluency over time reflect improved
performance on important end-of-year reading outcomes.
As our results suggest, the same is not true for DIBELS
nonsense word fluency, and findings are particularly
compelling because data were collected on the same group
of children using the same methods.” (p. 23)
Dallas, TX
104
Concerns and Questions
Before accepting these conclusions, some concerns
should be addressed.
1. The simple correlation between slope and reading
outcome addresses the wrong question.
 There is no rational or logical reason why slope by
itself should be related to reading outcomes
without considering the students initial skills.
 The crucial question is, Given the student’s initial
skills, does slope of progress add to the variance
explained in reading outcomes?
 Nick has NWF slope of +0.70 while Nora has NWF
slope of +1.50. Who would you expect to have
higher reading outcomes in the spring?

March 31, 2004
Dallas, TX
105
Answer: It depends on initial skills.

Nora has a slope twice that of Nick, but substantially
lower reading outcome because her initial skills are so
much lower.
NWF Correct Letter Sounds
80
70
60
Nick: slope = +0.70
Spring DORF = 51
50
40
30
Nora: slope = +1.50
20
Spring DORF = 27
10
0
0
4
8
12
16
20
24
28
32
36
Week
Slope, by itself without considering initial skills is not
enough to predict outcomes.
March 31, 2004
Dallas, TX
106
Now Consider Nora and Nell

Nora and Nell have similar initial skills – Nell’s
higher slope predicts higher skills in middle of first
grade and higher reading outcomes.
NWF
NWF Correct
Correct Letter
Letter Sounds
Sounds
80
80
70
70
60
60
Nick: slope = +0.70
50
50
Nell: slope = +2.56
40
40
Spring DORF = 37
30
30
+1.50
= +1.50
slope =
Nora: slope
Nora:
20
20
Spring DORF = 27
10
10
00
00
March 31, 2004
44
88
12
12
20
16
20
16
Week
Week
Dallas, TX
24
24
28
28
32
32
36
36
107
Given Initial Skills, Does Slope Add to
Predictions of Outcomes?

Students with complete data from 2002-2003 in the
DIBELS Data System were examined for level of risk,
slope of progress, and reading outcomes.
NWF Slope
Beginning NWF
Group
N
Mean
Std Dev
N
Mean
Std Dev
At Risk
20739
5.46
4.23
20739
1.54
1.02
Some Risk
20606
18.08
3.13
20606
1.47
0.97
Low Risk
38082
34.62
7.09
38082
1.23
1.16
Hi AP
12288
70.32
22.55
12288
1.24
1.73
Total
91715
29.09
22.12
91715
1.36
1.19
March 31, 2004
Dallas, TX
108
Fuchs et al. “At Risk” Sample
Mean DIBELS NWF score is in the low risk range.
An estimated 70% of the sample would be above the
NWF cutoff of 23 for low risk.


Initial Skills
Variable
N
Mean
CBM WIF
151
10.11
DIBELS NWF
151
31.29
March 31, 2004
Slope
Std Dev
N
Mean
9.26
151
0.90
0.90
14.47
151
1.92
2.04
Dallas, TX
Std Dev
109
Utility of Initial NWF Risk Categories

Initial skills on NWF are a very strong predictor of
reading outcomes.
Ending ORF
Group
0 -12
March 31, 2004
N
Mean
Std Dev
Odds of Achieving
Benchmark Goal
At Risk
20739
26.52
21.13
22%
Some Risk
20606
42.81
24.47
47%
Low Risk
38082
62.07
28.74
76%
Hi AP
12288
102.19
34.44
97%
Total
91715
55.08
35.68
60%
Dallas, TX
110
Does Slope Add to the Prediction of
Reading Outcomes After Risk Level and
Initial Skills?

March 31, 2004
Rules for evaluating effects:
1. Significance. With N > 20,000 everything is
significant.
2. Percent of variance explained. More than 10% of
variance explained is a good indication of a strong
effect. Greater percent is stronger.
3. Educationally meaningful effects. Analysis of
outcomes to see if the predicted differences would
be educationally important to teachers, students,
parents.
Dallas, TX
111
Does Slope Add to the Prediction of
Reading Outcomes After Risk Level and
Initial Skills?

Sequential model predicting first grade DORF reading
outcomes from (1) risk category, (2) initial NWF skill
given risk, and (3) slope given risk and initial skill.
Source
DF
R2 change
NWF Risk Category
3
0.40
Initial NWF Skill Given Risk
1
0.08
Slope Given Risk, Initial Skill
1
0.11
Total
91714
1.00
Risk category, initial skills, and slope combined explain
59% of reading outcomes.
March 31, 2004
Dallas, TX
112
Variance Explained by Slope for Each
Risk Category

A separate analysis was conducted for each risk
category.
Percent of Risk Category
Variance in Reading Outcomes
Explained
Group
NWF Initial
Skills
NWF Slope Given
Initial Skills
At Risk
8%
26%
Some Risk
2%
21%
Low Risk
8%
21%
25%
11%
Hi AP
But, is the variance
explained by slope
(given risk and
initial skills)
educationally
important?
Rate of progress is especially important for students
who are at risk for low reading outcomes.
March 31, 2004
Dallas, TX
113
Variability in Slope for At Risk
Students

About 68% of student’s trajectories are between the
low slope and the high slope.
NWF Correct Letter-Sounds
80
Mean NWF - 1sd slope
Mean NWF Mean slope
70
Mean NWF + 1sd slope
60
50
Hi Slope
40
30
20
Lo Slope
10
0
0
March 31, 2004
4
8
12
16
20
Week
Dallas, TX
24
28
32
36
114
Are Differences in Slope Educationally
Meaningful for At Risk Students?

Yes. Predicted reading outcomes are substantially
different.
45
40
35
Hi Slope
30
25
20
15
10
5
Lo Slope
Predicted End First DORF
50
0
0.52
0.77
1.03
1.28
1.54
1.79
2.05
2.30
2.56
MSlope - 1sd to MSlope + 1sd
March 31, 2004
Dallas, TX
115
Conclusions:
Validity of DIBELS NWF Slope



March 31, 2004
Initial risk status and initial skills on DIBELS Nonsense
Word Fluency are very important in predicting reading
outcomes in first grade, explaining 48% of variance in
outcomes.
An increasing pattern of scores through the first semester
of first grade on DIBELS Nonsense Word Fluency appears
to be a very important predictor of reading outcomes for
students who are at risk and for each risk category.
We can be confident that increases in DIBELS Nonsense
Word Fluency reflect improved performance on essential
skills that contribute to important end-of-year reading
outcomes.
Dallas, TX
116
5. Reviewing Outcomes
Key Decisions for Outcome/Accountability Assessment:
 Does the child have the early literacy skills predictive
of successful reading outcomes?
 Does the school have core curriculum and instruction
as well as a system of effective instructional support so
their students achieve literacy outcomes?
Data used to inform the decision:
 Evaluate individual student’s performance with
respect to benchmark goals that with the odds in favor
of achieving subsequent literacy goals.
 Compare school/district outcomes to goals and
outcomes from previous year.
 Evaluate core curriculum and system of additional
support for each step to identify strengths and areas for
improvement.
March 31, 2004
Dallas, TX
117
Reviewing Outcomes - School Level
1998 – 99 First Grade Reading
CBM Reading
60
50
Frequency
40
30
20
10
+
75
-7
4
70
-6
9
65
-6
4
60
-5
9
55
-5
4
50
-4
9
45
-4
4
40
-3
9
35
-3
4
30
-2
9
25
-2
4
20
-1
9
15
-1
4
10
-9
5
0
-4
0
Correct Words
March 31, 2004
28% Established Readers
57% Emerging Readers
15% Non-Readers
Dallas, TX
118
Reviewing Outcomes - School Level
1999 – 00 First Grade Reading
CBM Reading
140
120
Frequency
100
80
60
40
20
+
75
-7
4
70
-6
9
65
-6
4
60
-5
9
55
-5
4
50
-4
9
45
-4
4
40
-3
9
35
-3
4
30
-2
9
25
-2
4
20
-1
9
15
-1
4
10
-9
5
0
-4
0
Correct Words
March 31, 2004
57% Established Readers
36% Emerging Readers
6% Non-Readers
Dallas, TX
119
Heartland Early Literacy Project Across Year
First Grade Oral Reading Fluency Outcomes



March 31, 2004
2001-2002 Beginning: Middle: 4229 End: 4414
2000-2001 Beginning: Middle: 4037 End: 4152
1999-2000 Beginning: Middle: 1595 End: 1879
Dallas, TX
120
Reviewing Outcomes: Effectiveness of
Benchmark Instruction (Core Curriculum)


March 31, 2004
For each step toward literacy outcomes, a school with
an effective core curriculum and instruction supports
students who are on track (i.e., low risk or
benchmark) to achieve the goal.
For students with the odds in favor of achieving
literacy goals, it is the job of the core to teach the core
components so that all students (100%) achieve the
goals.
Dallas, TX
121
Reviewing Outcomes: Effectiveness of
Strategic and Intensive Intervention


March 31, 2004
For each step toward literacy outcomes, a school with
an effective system of effective interventions supports
students who are not on track (i.e., at some risk or at
risk of difficulty achieving literacy goals) to achieve
the goal.
For students with the odds against achieving literacy
goals unless we provide an effective intervention, it is
the job of the system of additional support to augment
the core curriculum so that all students (100%) achieve
the same benchmark goals.
Dallas, TX
122
Instructional Steps from Kindergarten to
Successful Reading Outcomes
Big Ideas in
Beginning
Reading
Phonological
Awareness
Dynamic
Indicators of
Big Ideas in
Beginning
Reading
ISF
Benchmark Goal
Timeline for
Assessing Big
Fall
Ideas K-3
Step
1
Instructional
Step
PSF
Winter Spring
Step
2
Kindergarten
Accuracy &
Fluency with
Connected Text
Alphabetic
Principle
NWF
Fall
ORF
High-Stakes
Reading
Outcome
ORF
ORF
HSA
Winter Spring Fall Winter Spring Fall Winter Spring
Step
3
Step
4
First Grade
Step
5
Second Grade
Step
6
Third Grade
The outcome of each step depends on (a) students beginning skills, (b)
effectiveness of core curriculum and instruction, and (c) effectiveness
of system of additional instructional support.
March 31, 2004
Dallas, TX
123
Step by Step, Core and Intervention
Effectiveness
of Core
Step
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Step 1: Phonemic Awareness
Step 2: Phonemic Awareness and Phonics
Step 3: Phonics and Fluency
Step 4: Fluency and Comprehension
Step 5: Fluency and Comprehension
Step 6: Fluency and Comprehension
A
B
Effectiveness of Benchmark (core) for School A
A
S
Effectiveness of Strategic support for School A
A
I
March 31, 2004
Effectiveness of Intensive support for School A
Dallas, TX
124
Evaluating the Effectiveness of the
Core Curriculum and Instruction
1. Is the core curriculum and instruction getting at least
95% of Benchmark students to the next early literacy
goal?
 If children are on track, the core should keep
them on track.
 What would it take to achieve 100%?
2. Is the core curriculum and instruction as effective as
other schools in getting Benchmark students to the
goal?
 If typical schools are not getting 100% of
Benchmark students to the goal, then
supplementing the core in this area can improve
reading outcomes.
March 31, 2004
Dallas, TX
125
March 31, 2004
Dallas, TX
126
Step 1: Beginning K to Middle K
Middle kindergarten outcomes for students with benchmark,
strategic, and intensive instructional recommendations at the
beginning of kindergarten
Number of Schools
300
Intensive
Median
School
250
Intensive
Strategic
Note. Graph based
on all schools
participating in the
DIBELS Data
System in the
2001 – 2002
academic year.
Benchmark
200
Benchmark
Median
School
150
A
S
A
I
100
A
B
50
0
0
-1
0
11
-2
0
21
-3
0
31
-4
0
41
-5
0
51
-6
0
61
-7
0
71
-8
0
81
-9
0
91
00
1
-
Conditional Percent Reaching ISF Goal
A typical (middle) school had 62% of children with a beginning kindergarten
benchmark recommendation achieve the middle of kindergarten goal, and 2%
of children with intensive support recommendation.
March 31, 2004
Dallas, TX
127
March 31, 2004
Dallas, TX
128
Step 2: Middle K to End K
End of kindergarten outcomes for students with benchmark,
strategic, and intensive instructional recommendations in the
middle of kindergarten
250
A
B
Number of Schools
Intensive
150
A
S
Strategic
200
Benchmark
A
I
Benchmark
Median
School
Intensive
Median
School
100
50
-1
00
0
91
81
-9
0
71
-8
0
-7
0
61
-6
0
51
41
-5
0
31
-4
0
21
-3
0
-2
11
0
-1
0
0
Conditional Percent Reaching PSF Goal
A typical (middle) school had 90% of children with a middle kindergarten
benchmark recommendation achieve the end of kindergarten goal, and
26% of children with intensive support recommendation.
March 31, 2004
Dallas, TX
129
March 31, 2004
Dallas, TX
130
Step 3: Beginning First to Middle First
Middle of first grade outcomes for students with benchmark,
strategic, and intensive instructional recommendations in the
beginning of first grade
Number of Schools
300
250
A
Intensive
I
Median
School
Intensive
Strategic
Benchmark
200
150
Benchmark
A Median
B School
A
S
100
50
-1
00
0
91
81
-9
0
-8
0
71
-7
0
61
51
-6
0
41
-5
0
31
-4
0
21
-3
0
-2
11
0
-1
0
0
Conditional Percent Reaching NWF Goal
A typical (middle) school had 68% of children with a beginning first grade
benchmark recommendation achieve the middle of first grade goal, and
0% of children with intensive support recommendation.
March 31, 2004
Dallas, TX
131
March 31, 2004
Dallas, TX
132
Step 4: Middle First to End First
End of first grade outcomes for students with benchmark,
strategic, and intensive instructional recommendations in the
middle of first grade
Number of Schools
350
A Intensive
I Median
School
300
Benchmark A
B
Median
School
Intensive
Strategic
250
Benchmark
200
150
A
S
100
50
-1
00
0
91
81
-9
0
71
-8
0
-7
0
61
-6
0
51
41
-5
0
31
-4
0
21
-3
0
-2
11
0
-1
0
0
Conditional Percent Reaching ORF Goal
A typical (middle) school had 96% of children with a middle first grade
benchmark recommendation achieve the end of first grade goal, and 0% of
children with intensive support recommendation.
March 31, 2004
Dallas, TX
133
March 31, 2004
Dallas, TX
134
Step 5a: Beginning Second to Middle Second
Middle of second grade outcomes for students with benchmark,
strategic, and intensive instructional recommendations at the
beginning of second grade
Number of Schools
140
120
A
I Intensive
Median
School
Intensive
Strategic
100
BenchmarkBenchmark
Median
A
School
B
80
60
A
S
40
20
-1
00
0
91
-9
0
81
71
-8
0
61
-7
0
51
-6
0
41
-5
0
31
-4
0
21
-3
0
-2
11
0
-1
0
0
Conditional Percent Reaching ORF Goal
A typical (middle) school had 90% of children with a beginning second
grade benchmark recommendation achieve the middle of second grade
goal, and 0% of children with intensive support recommendation.
March 31, 2004
Dallas, TX
135
March 31, 2004
Dallas, TX
136
Step 5b: Middle Second to End Second
End of second grade outcomes for students with benchmark,
strategic, and intensive instructional recommendations at the
middle of second grade
Intensive
Median
School
Number of Schools
140
120
Benchmark
Median
School
Intensive
Strategic
100
Benchmark
80
A
B
A
I
60
A
S
40
20
-1
00
0
91
81
-9
0
71
-8
0
61
-7
0
51
-6
0
41
-5
0
31
-4
0
21
-3
0
-2
11
0
-1
0
0
Conditional Percent Reaching ORF Goal
A typical (middle) school had 92% of children with a middle second grade
benchmark recommendation achieve the end of second grade goal, and 4%
of children with intensive support recommendation.
March 31, 2004
Dallas, TX
137
March 31, 2004
Dallas, TX
138
Step 6a: Beginning Third to Middle Third
Middle of third grade outcomes for students with benchmark,
strategic, and intensive instructional recommendations at the
beginning of third grade
Number of Schools
120
Intensive
Median
School
100
Intensive
Strategic
Benchmark
80
Benchmark
Median
School
60
40
A
B
A
S
A
I
20
-1
00
0
91
81
-9
0
71
-8
0
61
-7
0
51
-6
0
41
-5
0
31
-4
0
-3
0
21
-2
11
0
-1
0
0
Conditional Percent Reaching ORF Goal
A typical (middle) school had 90% of children with a beginning third grade
benchmark recommendation achieve the middle of third grade goal, and
0% of children with intensive support recommendation.
March 31, 2004
Dallas, TX
139
March 31, 2004
Dallas, TX
140
Step 6b: Middle Third to End Third
End of third grade outcomes for students with benchmark,
strategic, and intensive instructional recommendations at the
middle of third grade
Number of Schools
140
120
A
I Intensive
Median
School
Intensive
Strategic
100
Benchmark
Median
School
Benchmark
80
60
A
S
40
A
B
20
-1
00
0
91
81
-9
0
71
-8
0
-7
0
61
51
-6
0
41
-5
0
31
-4
0
21
-3
0
-2
11
0
-1
0
0
Conditional Percent Reaching ORF Goal
A typical (middle) school had 91% of children with a middle third grade
benchmark recommendation achieve the end of third grade goal, and 0%
of children with intensive support recommendation.
March 31, 2004
Dallas, TX
141
Step by Step, Core and Intervention
Effectiveness
of Core
Step
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Step 1: Phonemic Awareness
Step 2: Phonemic Awareness and Phonics
Step 3: Phonics and Fluency
Step 4: Fluency and Comprehension
Step 5a: Fluency and Comprehension
Step 5b: Fluency and Comprehension
Step 6a: Fluency and Comprehension
Step 6b: Fluency and Comprehension
Support – Effectiveness is less than a typical school and less than goal.
Typical – Effectiveness is less than goal, but typical of other schools.
Strength – Effectiveness is at goal or greater than typical of other schools
March 31, 2004
Dallas, TX
142
Identify Need
for Support
Validate Need
for Support
Plan
Instructional
Support
Outcomes Driven Model
Repeat for Each Step
Implement
Instructional
Support
Provide Instructional Support
Based on Integrated
Assessment - Intervention
Feedback Loop
Evaluate
Support
3 time per year progress monitoring
- Low Risk
Frequent progress monitoring
- At Risk
Review
Outcomes
March 31, 2004
Dallas, TX
143
Instructional Steps from Kindergarten to
Successful Reading Outcomes
Big Ideas in
Beginning
Reading
Phonological
Awareness
Dynamic
Indicators of
Big Ideas in
Beginning
Reading
ISF
Benchmark Goal
Timeline for
Assessing Big
Fall
Ideas K-3
Step
1
Instructional
Step
PSF
Winter Spring
Step
2
Kindergarten
Accuracy &
Fluency with
Connected Text
Alphabetic
Principle
NWF
Fall
ORF
High-Stakes
Reading
Outcome
ORF
ORF
HSA
Winter Spring Fall Winter Spring Fall Winter Spring
Step
3
Step
4
First Grade
Step
5
Second Grade
Step
6
Third Grade
Step by step to important reading goals and outcomes. Implicit in
this logic is a linkage to High Stakes Reading Outcomes.
March 31, 2004
Dallas, TX
144
Third Grade Oral Reading Fluency to
Oregon Statewide Assessment Test
250
r = .73
53% of
Variance
OSAT Total Score
240
230
Exceeds
220
210
Meets
200
Does not meet
Expectations
190
180
170
160
0


March 31, 2004
20
40
60
80 100 120 140 160 180 200 220 240
TORF May Grade 3
Odds of “meets expectation” on OSAT given 3rd grade
TORF of 110 : 90 of 91 or 99%.
Odds of “meets expectation” on OSAT given 3rd grade
TORF below 70: 4 of 23 or 17%.
Dallas, TX
145
Linkage of Third-Grade TORF to
Illinois State Assessment Test (ISAT)
r = .79
63% of
Variance
ISAT, Spring Grade 3
200
190
180
Exceeds Standards
170
160
Meets Standards
Below Standards
150
140
0


20
40
60
80
100 120 140
TORF, Spring Grade 3
160
180
Odds of “meets standards” on ISAT given Third-Grade TORF of
110 or above: 73 of 74 or 99%.
Odds of “meets standards” on ISAT given Third-Grade TORF of 70
or below: 1 of 8 or 12%.
Sibley, D., Biwer, D., & Hesch, A. (2001). Unpublished Data. Arlington Heights, IL: Arlington Heights
School District 25.
March 31, 2004
Dallas, TX
146
525
Advanced
inE Reading
Alaska State Benchmark
BM 1 -R -SC OR
500
475
450
425
400
375
350
Proficient
325
Proficient
Above 110, the
odds are strong
the student will
rank “proficient”
on the AK State
Benchmark.
300
Below Proficient
275
250
Below 70, the
odds are low the
student will rank
“proficient” on
the AK State
Benchmark.
Not Proficient
225
200
175
150
125
100
25
50
75
100
125
150
175
200
225
250
3rd Grade Benchmark in Reading - CBM
Linner,Words
S. (2001,
January).
Curriculum
Based Assessment in reading used as a predictor
Per Minute:
TORF
3.3
for the Alaska Benchmark Test. Paper presented at the Alaska Special Education Conference,
Anchorage, AK.
147
March 31, 2004
Dallas, TX
Linkage of Oral Reading Fluency to
State Reading Outcome Assessments
Above 110, the
odds are 91% the
student will rank
“adequate” on the
FL State
Assessment.
550
500
450
400
350
300
250
200
150
100
50
0
20
40
60
80
100
120
140
160
180
200
220
240
Below 80, the
odds are 19% the
student will rank
“adequate” on the
FL State
Assessment.
Oral Reading Fluency
Buck, J., & Torgesen, J. (2003). The relationship between performance on a
measure of oral reading fluency and performance on the Florida Comprehensive
Assessment Test (Technical Report 1). Tallahassee, FL: Florida Center for Reading
Research,.
March 31, 2004
Dallas, TX
148
Themes






March 31, 2004
Don’t loose track of the bottom line. Are we getting
closer to important and meaningful outcomes?
Monitor Progress on -- and teach -- what is important:
Phonemic Awareness, Alphabetic Principle, Accuracy
and Fluency with Connected Text
Oral Reading Fluency is an important instructional
goal and target of progress monitoring.
Use progress monitoring to make decisions that change
outcomes for children.
Progress monitoring should be efficient and
purposeful.
Start early! Trajectories of reading progress are very
difficult to change.
Dallas, TX
149
Step by Step, Core and Intervention
Step
Step 1: Phonemic Awareness
Effectiveness
of Core
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Support
Strength
Strength
Step 2: Phonemic Awareness and Phonics
Step 3: Phonics and Fluency
Step 4: Fluency and Comprehension
Step 5a: Fluency and Comprehension
Step 5b: Fluency and Comprehension
Step 6a: Fluency and Comprehension
Step 6b: Fluency and Comprehension
Support – Effectiveness is less than a typical school and less than goal.
Typical – Effectiveness is less than goal, but typical of other schools.
Strength – Effectiveness is at goal or greater than typical of other schools
March 31, 2004
Dallas, TX
151
Step by Step, Core and Intervention
Effectiveness
of Core
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Step 1: Phonemic Awareness
Support
Strength
Strength
Step 2: Phonemic Awareness and Phonics
Strength
Strength
Support
Step
Step 3: Phonics and Fluency
Step 4: Fluency and Comprehension
Step 5a: Fluency and Comprehension
Step 5b: Fluency and Comprehension
Step 6a: Fluency and Comprehension
Step 6b: Fluency and Comprehension
Support – Effectiveness is less than a typical school and less than goal.
Typical – Effectiveness is less than goal, but typical of other schools.
Strength – Effectiveness is at goal or greater than typical of other schools
March 31, 2004
Dallas, TX
152
Step by Step, Core and Intervention
Effectiveness
of Core
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Step 1: Phonemic Awareness
Support
Strength
Strength
Step 2: Phonemic Awareness and Phonics
Strength
Strength
Support
Typical
Strength
Typical
Step
Step 3: Phonics and Fluency
Step 4: Fluency and Comprehension
Step 5a: Fluency and Comprehension
Step 5b: Fluency and Comprehension
Step 6a: Fluency and Comprehension
Step 6b: Fluency and Comprehension
Support – Effectiveness is less than a typical school and less than goal.
Typical – Effectiveness is less than goal, but typical of other schools.
Strength – Effectiveness is at goal or greater than typical of other schools
March 31, 2004
Dallas, TX
153
Step by Step, Core and Intervention
Effectiveness
of Core
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Step 1: Phonemic Awareness
Support
Strength
Strength
Step 2: Phonemic Awareness and Phonics
Strength
Strength
Support
Step 3: Phonics and Fluency
Typical
Strength
Typical
Step 4: Fluency and Comprehension
Typical
Support
Typical
Step
Step 5a: Fluency and Comprehension
Step 5b: Fluency and Comprehension
Step 6a: Fluency and Comprehension
Step 6b: Fluency and Comprehension
Support – Effectiveness is less than a typical school and less than goal.
Typical – Effectiveness is less than goal, but typical of other schools.
Strength – Effectiveness is at goal or greater than typical of other schools
March 31, 2004
Dallas, TX
154
Step by Step, Core and Intervention
Effectiveness
of Core
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Step 1: Phonemic Awareness
Support
Strength
Strength
Step 2: Phonemic Awareness and Phonics
Strength
Strength
Support
Step 3: Phonics and Fluency
Typical
Strength
Typical
Step 4: Fluency and Comprehension
Typical
Support
Typical
Step 5a: Fluency and Comprehension
Typical
Typical
Typical
Step
Step 5b: Fluency and Comprehension
Step 6a: Fluency and Comprehension
Step 6b: Fluency and Comprehension
Support – Effectiveness is less than a typical school and less than goal.
Typical – Effectiveness is less than goal, but typical of other schools.
Strength – Effectiveness is at goal or greater than typical of other schools
March 31, 2004
Dallas, TX
155
Step by Step, Core and Intervention
Effectiveness
of Core
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Step 1: Phonemic Awareness
Support
Strength
Strength
Step 2: Phonemic Awareness and Phonics
Strength
Strength
Support
Step 3: Phonics and Fluency
Typical
Strength
Typical
Step 4: Fluency and Comprehension
Typical
Support
Typical
Step 5a: Fluency and Comprehension
Typical
Typical
Typical
Step 5b: Fluency and Comprehension
Support
Strength
Strength
Step
Step 6a: Fluency and Comprehension
Step 6b: Fluency and Comprehension
Support – Effectiveness is less than a typical school and less than goal.
Typical – Effectiveness is less than goal, but typical of other schools.
Strength – Effectiveness is at goal or greater than typical of other schools
March 31, 2004
Dallas, TX
156
Step by Step, Core and Intervention
Effectiveness
of Core
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Step 1: Phonemic Awareness
Support
Strength
Strength
Step 2: Phonemic Awareness and Phonics
Strength
Strength
Support
Step 3: Phonics and Fluency
Typical
Strength
Typical
Step 4: Fluency and Comprehension
Typical
Support
Typical
Step 5a: Fluency and Comprehension
Typical
Typical
Typical
Step 5b: Fluency and Comprehension
Support
Strength
Strength
Step 6a: Fluency and Comprehension
Strength
Typical
Strength
Step
Step 6b: Fluency and Comprehension
Support – Effectiveness is less than a typical school and less than goal.
Typical – Effectiveness is less than goal, but typical of other schools.
Strength – Effectiveness is at goal or greater than typical of other schools
March 31, 2004
Dallas, TX
157
Step by Step, Core and Intervention
Effectiveness
of Core
Effectiveness
of Strategic
Support
Effectiveness
of Intensive
Support
Step 1: Phonemic Awareness
Support
Strength
Strength
Step 2: Phonemic Awareness and Phonics
Strength
Strength
Support
Step 3: Phonics and Fluency
Typical
Strength
Typical
Step 4: Fluency and Comprehension
Typical
Support
Typical
Step 5a: Fluency and Comprehension
Typical
Typical
Typical
Step 5b: Fluency and Comprehension
Support
Strength
Strength
Step 6a: Fluency and Comprehension
Strength
Typical
Strength
Step 6b: Fluency and Comprehension
Support
Typical
Typical
Step
Support – Effectiveness is less than a typical school and less than goal.
Typical – Effectiveness is less than goal, but typical of other schools.
Strength – Effectiveness is at goal or greater than typical of other schools
March 31, 2004
Dallas, TX
158