Transcript Assessing Rate of Improvement of Individual Students

Calculating and Interpreting
Slope: Implications for
School Psychologists
Caitlin S. Flinn, Ed.S. &
Andrew E. McCrea, M.S., NCSP
ASPP/PSU Conference
October 8, 2009
Translations…

ROI = Rate of Improvement, Slope

RTI = Response to Intervention

SLD = Specific Learning Disability

DD = Dually Discrepant
Overview





Conceptualization
Importance
Definitions
Functions
Graphing in Excel

Individual Graphs
Calculating ROI





Decision Making
Grounding the Data
Interpreting Growth


Individual Student
Student Groups
Considerations
What is Rate of Improvement?
Graphs in Education

Core instruction efficacy


Data-driven instruction


Batsche, Castillo, Dixon, & Forde, 2008
Setting goals


Colvin, Sugai, Good, & Young-Yon, 1997
Intervention effectiveness


NCLB, 2001; IDEA, 2004
Incremental skill growth


Batsche, et al., 2005
Shapiro, 2008
Progress monitoring

Wright, 2008; Fuchs & Fuchs, 1997
Graphs in Education

Assessing level and rate


Self monitoring


Sulzer-Asaroff & Mayer, 1991
Treatment integrity


Shinn, 2008
Professional accountability


Fuchs et al., 2006
IEP goals


Batsche, Castillo, Dixon, & Forde, 2008
Mortenson & Witt, 1998
Teacher performance feedback

Noell et al., 2005
Why do we like graphs so much?



Seeing is believing!
A picture is worth a thousand words!
People remember



Speeches that included visuals, especially in color,
improved immediate recall by 8.5% and delayed
recall (3 days) by 10.1% (Vogel, Dickson, & Lehman,
1990)
Visual aids are more effective for communicating
large amounts of information quickly
Transcend language barriers (Karwowski, 2006)
Is seeing enough?

If there are only two data points, basic
slope calculation is accurate.
Chart Title
y=x+1
12
10
10
8
Data1
6
Trendline1
4
2
2
0
1
2
3
4
5
6
7
8
9
Why do I need to know ROI?

This is also how to calculate an aimline.
Chart Title
y=x+1
12
10
10
8
Data1
6
Trendline1
4
2
2
0
1
2
3
4
5
6
7
8
9
Why Bother with all the Data?

If there are multiple data points, should you calculate the
slope between the first and last data points only?
Chart Title
y=x+1
12
10
10
8
Data1
6
Trendline1
4
2
2
0
1
2
3
4
5
6
7
8
9
ROI in Educational Decisions
Data
(slope)
Interpretation
(progress?)
Decision
Instructional Need
Quick Stretch of the Mind
Diego's
Progress
 Data

Words Per Minute
120

100

80
Interpretation

60

40
1
4
7
10
Benchmark
Diego
???
Will Diego catch up?
you28 have
13 16 
19 Do
22 25
31 34 enough
info?
School Week

0
NOT GOOD!
Decision

20
Benchmark: 77
Diego: 41
Quick Stretch of the Mind
Diego's Progress
 Data

Words Per Minute
120

100

80
Interpretation

60

40
1
4
7
10 13 16 19
Benchmark
Diego
???
Will Diego catch up?
 Do you have enough
22 25 28 31 34
info?

0
Slightly better
Decision

20
Benchmark: 77, 92
Diego: 41, 62, 63
School Week
Quick Stretch of the Mind
Diego's Progress
 Data

Words Per Minute
120

100

80
Interpretation

60

40

0
1
4
7
10 13 16 19 22
School Week
Good work
Decision

20
Benchmark: 77, 92
Diego: 41 … 104
Benchmark
Diego
Cured!!!
Will Diego maintain
25
28 progress?
31 34
that
Quick Stretch of the Mind
Diego's Progress
 Data

Words Per Minute
120

100

80
Interpretation

60

40

0
1
4
7
10 13 16 19 22
School Week
Ummm
Decision

20
Benchmark: 77, 92
Diego: 41 … 74
Benchmark
Diego
Uh oh…
What EXACTLY IS
25 28 31 34
Diego’s Progress??
Quick Stretch of the Mind
Diego's Progress
Words Per Minute
120
100
Benchmark
80
Diego
60
Diego's ROI
40
Goal Slope
20
0
1
4
7 10 13 16 19 22 25 28 31 34
School Week

Now can you make a decision?
Rationale for Importance of ROI
Shift from discrepancy to RtI for SLD
eligibility determination
 Use of ROI to determine lack of RtI
 Questions to be Empirically Answered

What parameters of ROI indicate lack of RtI?
 How does ROI present between SLD and
non-SLD?
 What are reasonable goals using ROI?

Rationale for Importance of ROI
School
psychological
practices including
more CBM as a
result of IDEA 2004
and NCLB 2001
 CBM is great
because:



















Comparisons
Dual discrepancy
Efficient
Federal regulations
Graph
GOM
Goal setting
Instructional planning
Multi-tiered assessment
Match instructional level
Norms available
Parents
Progress monitoring
Survey level assessments
Sensitive to change
Screening
Systems-level
Technically adequate
Rate of Improvement

Algebraic terms:


slope is the change in the output because of the
input, over time
a way of looking at growth

Aimline: expected performance
Trendline: actual performance

Definitions:



Fuchs and Fuchs (1998)
Batsche, Castillo, Dixon, and Forde (2008)
Rate of Improvement

Shapiro (2008)


Criteria for setting reasonable, achievable, and
ambitious goals
Fuchs, Fuchs, Hamlett, Walz, and Germann
(1993)


weekly rates of growth in curriculum-based
measurements
slopes between 1.5 and 2.0 times that of their peers
were most likely to remediate skill deficits with current
instruction
Functions of ROI

Comparisons

peers/class

local norms/district

national norms
Functions of ROI

A decision tool

Shinn’s estimate for decision-making in RTI
trainings in Pennsylvania (Kovaleski, 2008)
 Slope
of 2.0x or greater (than expected slope, to
close the gap)
Rate of Improvement and RTI

Fuchs and Fuchs (1998)

hallmark components of RTI
 ongoing
formative assessment
 identifying non-responsive students
 treatment fidelity of core / supplemental instruction
 “dual discrepancy”

a student performing at or greater than one standard
deviation from typically performing peers in
 (a) level (i.e., grade) and
 (b) rate (slope)
Eligibility and ROI

School psychologists
Adopt use of additional information (CBM!)
 “Snapshot” data

 Good

Data continuously collected over time
 Data

day, bad day data
Versus
that accounts for good days and bad days!
Eyeball the graph vs. decision guidelines
Determining SLD in an RTI Model

Gresham, 2001


RTI is viable alternative for identifying learning
disabilities
Stuebing, Fletcher, LeDoux, Lyon,
Shaywitz, & Shaywitz, 2002

Poor reliability and validity of discrepancy
model
Determining SLD in an RTI Model

Gresham, 2001

3 models of RTI
 predictor-criterion
 dual
discrepancy
 applied behavior analytic
Determining SLD in an RTI Model

Gresham, 2001

All three models involved:
 multiple
tiers of intervention
 progress monitoring
 effective instructional strategies
 support
for applying the RTI model to
identifying students for learning
disabilities
Eligibility and ROI

Deno, Fuchs, Marston, and Shin, 2001
slopes of children identified as frequently nonresponsive to robust interventions
approximated the slopes of children already
identified as having a specific learning
disability
 supportive evidence for dual-discrepancy
model

Summary of ROI Research
Through an RTI model, students receive
instruction based on data. Rate of
improvement, or slope, has surfaced in the
literature as an indicator for making
instructional decisions.
Although research is expanding in terms of
reliability and validity for both RTI and
ROI, extant studies allude to strong utility
for educational planning.
Skills Typically Graphed

Reading



Oral Reading Fluency (ORF)
Word Use Fluency (WUF)
Reading Comprehension








Math Computation
Math Facts
Early Numeracy
Early Literacy Skills






Math
MAZE
Retell Fluency



Initial Sound Fluency (ISF)
Letter Naming Fluency (LNF)
Letter Sound Fluency (LSF)
Phoneme Segmentation Fluency
(PSF)
Nonsense Word Fluency (NWF)
Spelling
Written Expression
Behavior


Oral Counting
Missing Number
Number
Identification
Quantity
Discrimination
Get out your laptops!
Open Microsoft Excel
I love
ROI
Setting Up Your Spreadsheet



Open Microsoft Excel
In cell B2, type School
Week to represent the
weeks of school
In cell C2, type
Benchmark to
represent the
benchmarks or goals
of the skill you are
graphing

In cell D2, type WPM
to represent Words
Per Minute (or an
abbreviation the basic
skill you are graphing)
Labeling School Weeks




In cell B3, type 1 to
represent the first
week of school
Continue numbering
to 36 in column 2
Why 36? There’s
typically 36 school
weeks per school
year.
Finish at cell B38
Entering Benchmarks/Goals

In cell C3, type the
number that represents
the fall benchmark
(months 1-3) of the skill
you are graphing (e.g.,
77). This score should be
next to school week 1.

In cell C20, type the
number that represents
the winter benchmark
(months 4-6) of the skill
you are graphing (e.g.,
92). This score should be
next to school week 18.

In cell C38, type the
number that represents
the spring benchmark
(months 7-10) of the skill
you are graphing (e.g.,
110). This score should be
next to school week 36.
Entering Student Scores


In cells D3 through D38
type the number that
represents the score the
student achieved during
that week of the school
year.
If a student was not
assessed during a certain
week of the school year,
leave that cell blank*.
Entering Student Scores
 *Do
NOT enter zero (0) or a
score of zero will be calculated
into the trend line and interpreted
as the student having read zero
words correct per minute that
week.
Creating a Graph





Highlight the data in
columns C and D.
Left click with mouse in
cell C2.
While holding down
mouse, highlight
columns C and D from
row 2 through row 38.
These will be your data
points contributing to
your graph.
The selection should
include the blank cells
in column C.
Creating a Graph


Left click “Insert” in
the tool bar (typically
in the top row) while
columns C and D are
still highlighted.
Left click “Chart” in
the drop-down menu.
A “Chart Wizard”
window will appear.
Creating a Graph


In the Chart Wizard
are two tabs. Make
sure you are in the
left tab, “Standard
Types.”
Next, you want to left
click on the “Line”
graph on the left side
of your Chart Wizard.
Creating a Graph


On the right side is
“Chart sub-type.”
Choose the graph with
the description “Line
with markers displayed
at each data value.”
This option is typically
the first graph icon in
the second row.
Left click “Next” at the
bottom of your Chart
Wizard.
Creating a Graph

The upper left
tab says “Data
Range.” The
bottom half of
that tab has
your data range
and series type.
You want to
select
“Columns.”
Creating a Graph



The top right tab is
labeled “Series.” Left
click on “Series”
The top half of the
“Series” tab has an icon
of your graph. The
bottom half shows you
the label your data for
the legend.
Left click “Next” at the
bottom of your Chart
Wizard.
Creating a Graph





The next options include tabs
for Titles, Axes, Gridlines, etc.
If you would like to title your
chart, left click on the “Titles”
tab. Enter a title (i.e., Diego’s
Rate of Improvement) in the
first box.
Enter a description in the
Category (X) Axis, which is
your number of school weeks.
Suggestion: School Week
Enter a description in your
Value (Y) Axis, which is the
number of the skill you are
graphing. Suggestion: Words
Per Minute.
Left click “Next” at the bottom
of your Chart Wizard.
Numerical Label to Data Points




Click on “Data Labels”
Click to put a check
mark next to “Value”
Click on the “Next”
Button
Note: This can clutter
your graph but
provide useful info.
Creating a Graph


Left click the bottom circle to choose “As
object in: …” This will put the graph in the
worksheet you have open.
Left click “Finish” in your Chart Wizard. This
will place your graph in your Excel Document.
- OR -
Creating a Graph




You can choose to have the graph created in a new
worksheet within your Excel document.
Click next to “As new sheet…”
Label the worksheet by clicking on the tab at the bottom of the
sheet and typing a name (i.e., Diego’s Graph)
Left click “Finish” in your Chart Wizard. This will place your
graph in a new sheet in your Excel Document.
Creating a Graph
Resizing the Graph
You may resize the graph by clicking in
any white space within your graph
which will bring up squares at the
corners of your graph.
 You can put your mouse over these
squares and drag the graph to a size
you prefer.
 Keep in mind that a large graph may not
print easily.

Coloring the Graph


Right click in the gray
area and a drop down
menu appears
Click on “Format Plot
Area”
Coloring the Graph




The Format Plot Area
menu appears.
To eliminate a color
border, click next to
“None”
To eliminate a color
plot area, click next to
“None” or choose a
color.
Click “OK”
Coloring Data Points



Right click on the data
points for which you
would like to change
the color
Click on “Format Data
Series”
Choose the
foreground,
background, and line
color
Adding Trendlines


RIGHT click on any of the student’s scores/data
points within your graph.
Left click “Add Trendline” from the menu that
appears.
Adding Trendlines

Under the “Type” tab,
click on “Linear.”



Under the “Options” tab, click
next to “Custom” and type
“Diego’s ROI”
Click by “Display Equation…”
Click “OK”
Adding Trendlines

Left clicking on the equation will highlight a
“box” around it, and clicking on that box
will allow you to move the equation above
the chart legend to see it better.
Adding Trendlines




Repeat this process for the other set of data – your
benchmarks. Begin by right clicking on any of your three
goal data points.
For your benchmark data, label the trendline title
“Expected Slope”
Click next to “Display equation on chart”
Click “OK”
Adding Trendlines

Move your equation under the first one
Lines, Lines, Lines
Diego's Rate of Improvement
y = 1.6317x + 50.928
Words Per Minute
120
y = 0.9434x + 75.704
100
Benchmark (3rd)
80
Diego's Scores (3rd)
60
Diego's ROI
40
Goal Slope
20
0
1
4 7 10 13 16 19 22 25 28 31 34
School Week
Cautions in Interpreting Slope
(my disclaimer!)




The graph depicts the actual scores as data points on
the graph.
You can change the size of the font for the numeric
values by right clicking on any of them and choosing a
smaller font size. This may reduce some of the clutter on
your graph.
The rate of improvement, or trendline, is calculated using
a linear regression, a simple equation of least squares.
This line takes into account each score entered to
portray an average rate of improvement across school
weeks.
The equation indicates the slope, or rate of
improvement. The number, or coefficient, before "x" is
the average improvement, which in this case is the
average number of words per minute per week gained
by the student.
Cautions in Interpreting Slope
(my disclaimer continued!)


The slope can change depending on which week
(where) you put the benchmark scores on your chart.
Enter benchmark scores based on when your school
administers their benchmark assessments for the most
accurate depiction of expected student progress.
To add additional progress monitoring/benchmark scores
once you’ve already created a graph, enter additional
scores in Column D in the corresponding school week.
Remember to leave cells blank for the weeks that no
score was obtained. The graph will incorporate that
score into the set of data points and into the trendline.
Program Excel to Calculate ROI

Type “ROI” in cell B39
(below your last week
of school)
Program Excel to Calculate ROI




To calculate the
expected slope as per
the benchmarks:
Click on cell C39
Put your cursor at the
top next to the fx
Type:
=SLOPE(C3:C38,B3:B38)

Hit Enter/Return
Program Excel to Calculate ROI




To calculate the
student’s slope:
Click on cell D39
Put your cursor at the
top next to the fx
Type:
=SLOPE(D3:D38,B3:B38)

Hit Enter/Return
ROI as a Decision Tool
within a Problem-Solving Model
Steps
1.
2.
3.
4.
Gather the data
Ground the data
Interpret the data
Figure out how to fit Best Practice into
Public Education
Step 1: Gather Data
Universal Screening
Progress Monitoring
Common Screenings in PA
DIBELS
 AIMSweb
 MBSP
 4Sight
 PSSA

Validated Progress
Monitoring Tools
DIBELS
 AIMSweb
 MBSP


www.studentprogress.org
Step 2: Ground the Data
To what will we compare our
student growth data?
Multiple Ways to
Look at Growth
Needed Growth
 Expected Growth & Percent of Expected
Growth
 Fuchs et. al. (1993) Table of Realistic and
Ambitious Growth
 Growth Toward Individual Goal*

*Best Practices in Setting Progress Monitoring Goals for Academic Skill Improvement (Shapiro, 2008)
Looking at Percent of
Expected Growth
Tier I
Tier II
Tier III
Greater than
150%
Between
110% & 150%
Possible LD
Between 95%
& 110%
Likely LD
Between 80%
& 95%
May Need
More
May Need
More
Likely LD
Below 80%
Needs More
Needs More
Likely LD
Tigard-Tualatin School District
(www.ttsd.k12.or.us)
Oral Reading Fluency Adequate
Response Table
Realistic
Growth
Ambitious
Growth
1st
2.0
3.0
2nd
1.5
2.0
3rd
1.0
1.5
4th
0.9
1.1
5th
0.5
0.8
Fuchs, Fuchs, Hamlett, Walz, & Germann
(1993)
Digit Fluency Adequate
Response Table
1st
Realistic
Growth
0.3
Ambitious
Growth
0.5
2nd
0.3
0.5
3rd
0.3
0.5
4th
0.75
1.2
5th
0.75
1.2
Fuchs, Fuchs, Hamlett, Walz, & Germann
(1993)
Oral Reading Fluency
01/15/09 01/22/09 01/29/09 02/05/09 02/12/09 02/19/09 02/26/09 03/05/09 03/12/09 03/19/09 03/26/09 04/02/09 04/09/09 04/16/09 04/23/09 04/30/09 05/07/09 05/14/09
1
Benchmark
Aiden
Ava
Noah
Olivia
Liam
Hannah
Gavin
Grace
Oliver
Peyton
Josh
Riley
Mason
Zoe
Ian
Faith
David
Alexa
Hunter
Caroline
2
3
4
5
6
7
8
9
10
11
12
13
14
68
40
49
43
49
48
65
17
18
Needed RoI* Actual RoI** % of Expected
RoI
49
45
60
71
95
1.61
2.17
167%
77
57
54
87
92
2.28
2.76
213%
69
61
54
84
2.28
2.01
156%
57
70
79
83
1.39
1.50
116%
36
54
70
83
1.94
1.58
122%
52
60
82
1.72
1.20
93%
67
68
84
79
1.44
1.66
129%
46
60
74
79
2.06
1.76
136%
51
51
57
78
2.22
1.45
112%
53
54
64
64
69
40
53
48
44
63
46
68
50
49
38
42
49
53
1.29
52
49
55
50
16
90
61
59
15
47
58
75
77
1.50
1.12
87%
55
48
36
67
77
2.28
1.62
125%
54
69
67
50
76
2.67
1.76
136%
49
50
64
74
2.06
1.17
91%
34
38
42
68
55
51
58
3.11
1.44
111%
41
31
45
49
47
30
46
2.72
0.24
19%
29
36
35
36
36
29
45
44
3.39
0.75
58%
30
23
44
52
43
19
63
38
3.33
0.79
61%
18
19
25
33
33
23
28
37
4.00
0.94
73%
23
23
48
38
32
34
3.72
0.75
58%
28
20
40
37
19
30
3.44
0.02
2%
* Needed RoI based on difference betw een w eek 1 score and
Benchmark score for w eek 18 divided by 18 w eeks
53
24
28
Expected RoI at Benchmark Level
25
Oral Reading Fluency Adequate Response Table
** Actual RoI based on linear regression of all data points
Benchmarks based on DIBELS Goals
60
Realistic Grow thAmbitious Grow th
1st Grade
2.0
3.0
2nd Grade
1.5
2.0
3rd Grade
1.0
1.5
4th Grade
0.9
1.1
5th Grade
0.5
0.8
(Fuchs, Fuchs, Hamlett, Walz, & Germann 1993)
Step 3: Interpreting Growth
What do we do when we do not
get the growth we want?
When to make a change in instruction and
intervention?
 When to considered SLD?

When to make a change in
instruction and intervention?
Enough data points (6 to 10)?
 Less than 100% of expected growth.
 Not on track to make benchmark (needed
growth).
 Not on track to reach individual goal.

When to consider SLD?
Continued inadequate response despite:
 Fidelity with Tier I instruction and Tier II
intervention.
 Multiple attempts at intervention.
 Individualized Problem-Solving
approach.
Whole Class Example
Computation
01/15/10 01/22/10 01/29/10 02/05/10 02/12/10 02/19/10 02/26/10 03/05/10 03/12/10 03/19/10 03/26/10 04/02/10 04/09/10 04/16/10 04/23/10 04/30/10 05/07/10 05/14/10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Needed RoI* Actual RoI** % of Expected
RoI
0.35
50th Percentile
25
31
25th Percentile
19
23
Student
6.5
9
8
Student
6
7.5
8.5
Student
4.5
Student
13
Student
8.5
0.24
5.5
11
13
1.72
0.61
5
11
11.5
1.72
0.57
161%
5.5
6.5
9.5
10.5
1.72
1.06
300%
173%
8
9.3
8
5.6
9.6
9.6
1.72
-0.23
-66%
8
10.5
10.5
5.6
9.3
9
1.72
-0.03
-7%
9
8
4
8
9
1.72
0.07
21%
6
10.5
9
1.72
0.43
122%
6
8
1.72
0.07
20%
7
1.72
-0.25
-71%
-119%
Student
8.5
5.5
Student
6.5
5.5
Student
6.5
9
4.5
Student
8
10.5
4.5
6.5
4
Student
9
10
5.6
6.6
5
4.6
6.6
1.72
-0.42
8
8
8.5
4
8
6.6
1.72
-0.18
-51%
3.5
6.5
1.72
-0.24
-67%
26%
Student
Student
9
4.5
4.5
4
3.5
Student
6.5
5
6.5
9
7.5
6.5
1.72
0.09
Student
5.5
3
8
4
6.5
6.3
1.72
0.19
55%
Student
7.5
10
6.6
3.3
3
6.3
1.72
-0.46
-130%
Student
5
5.5
6.5
6
5
6
1.72
0.04
11%
Student
5
4
8
8.5
10
8
6
1.72
0.25
71%
Student
4.5
3.5
5.5
1.72
-0.03
-8%
5
5.3
1.72
-0.14
-40%
Student
6
5
2.5
5.5
4.5
10.5
* Needed RoI based on difference betw een w eek 1 score and Benchmark score for w eek 18 divided by 18 w eeks
11
Digit Fluency Adequate Response Table
** Actual RoI based on linear regression of all data points
Percentiles based on AIMSw eb Grow th Tables
Expected RoI at 50th Percentile
Expected RoI at 25th Percentile
Realistic Grow thAmbitious Grow th
1st Grade
0.3
0.5
2nd Grade
0.3
0.5
3rd Grade
0.3
0.5
4th Grade
0.75
1.2
5th Grade
0.75
1.2
(Fuchs, Fuchs, Hamlett, Walz, & Germann 1993)
3rd Grade Math Whole Class
Who’s responding?
 Effective math
instruction?
 Who needs more?

N=19
 4 > 100% growth
 15 < 100% growth
 9 w/ negative
growth

Small Group Example
Oral Reading Fluency
09/11/09 09/18/09 09/25/09 10/02/09 10/09/09 10/16/09 10/23/09 10/30/09 11/06/09 11/13/09 11/20/09 11/27/09 12/04/09 12/11/09 12/18/09 01/01/10 01/08/10 01/15/10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Needed RoI* Actual RoI** % of Expected
RoI
68
1.41
Benchmark
44
Student
35
39
41
45
42
45
52
57
62
1.83
1.49
106%
Student
28
38
42
40
50
55
64
72
74
2.22
2.77
196%
Student
26
28
32
31
27
29
35
34
38
2.33
0.57
41%
Student
31
35
39
45
42
47
53
58
65
2.06
1.90
135%
Student
40
44
38
48
52
64
72
74
78
1.56
2.62
186%
* Needed RoI based on dif ference between week 1 score
and Benchmark score for week 18 divided by 18 weeks
Oral Reading Fluency Adequte Response Table
** Actual RoI based on linear regression of all data points
Benchmarks based on DIBELS Goals
Expected RoI at Benchmark Level
Realistic GrowthAmbitious Growth
1st Grade
2.0
3.0
2nd Grade
1.5
2.0
3rd Grade
1.0
1.5
4th Grade
0.9
1.1
5th Grade
0.5
0.8
(Fuchs, Fuchs, Hamlett, Walz, & Germann 1993)
Intervention Group
Intervention working for how many?
 Can we assume fidelity of intervention
based on results?
 Who needs more?

Individual Kid Example
2nd Grade Reading Progress
100
y = 1.5333x + 42.8
90
90
80
79
Words Read Correct Per Minute
74
70
68
60
60
56
53
y = 0.9903x + 36.873
53
50
48
46
45
44
40
31
30
20
10
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
09/12/08 09/19/0809/26/0810/03/08 10/10/08 10/17/08 10/24/08 10/31/08 11/07/08 11/14/08 11/21/08 11/28/08 12/05/08 12/12/08 12/19/08 01/16/09 01/23/09 01/30/09 02/06/0902/13/09 02/20/0902/27/0903/06/09 03/13/0903/20/0903/27/0904/03/09 04/10/09 04/17/0904/24/09 05/01/09
Benchmark
Linear (Benchmark)
Linear
Individual Kid
Making growth?
 How much (65% of expected growth).
 Atypical growth across the year (last 3
data points).
 Continue? Make a change? Need more
data?

Step 4: Figure out how to fit
Best Practice into Public
Education
Things to Consider
Who cares about Rate of Improvement?
 Who is At-Risk and needs progress
monitoring?
 Who will collect, score, enter the data?
 Who will monitor student growth, when,
and how often?
 What changes should be made to
instruction & intervention?

Who cares about Rate of
Improvement?
Explaining the concept of RoI.
 Creating buy-in with ORF, M-CBM, etc.
 Defending frequent data collection (the
“we spend too much time testing and not
enough time teaching” argument).

Who is At-Risk and needs
progress monitoring?

Below level on universal screening
Entering 4th Grade Example
DORF
(110)
Student A
115
ISIP
TRWM
(55)
58
4Sight
(1235)
PSSA
(1235)
1255
1232
Student B
85
48
1216
1126
Student C
72
35
1056
1048
Who will collect, score, and
enter the data?
Using MBSP for math, teachers can
administer probes to whole class.
 DORF probes must be administered oneon-one, and creativity pays off (train and
use art, music, library, etc. specialists).
 Schedule for progress monitoring math
and reading every-other week.

Week 1
Reading
1st
Reading
X
X
X
X
X
Math
X
X
4th
5th
Math
X
2nd
3rd
Week 2
X
X
Who will monitor student
growth, when, and how often?
Best Practices in Data-Analysis Teaming
(Kovaleski & Pedersen, 2008)
 Chambersburg Area School District
Elementary Response to Intervention
Manual (McCrea et. al., 2008)
 Derry Township School District Response
to Intervention Model

(http://www.hershey.k12.pa.us/56039310111408/lib/56039310111408/_files/Microsoft
_Word_-_Response_to_Intervention_Overview_of_Hershey_Elementary_Model.pdf)
What changes should be made
to instruction & intervention?
Ensure treatment fidelity!!!!!!!!
 Increase instructional time (active and
engaged)
 Decrease group size
 Gather additional, diagnostic, information
 Change the intervention

RoI and Behavior?
Percent of Time Engaged in Appropriate Behavior
100
90
y = 7.2143x - 1.5
80
70
y = 3.9x + 19.8
Percent
60
50
40
y = 2x + 22
30
20
10
0
1
2
Baseline
3
4
Condition 1
5
6
Condition 2
7
8
9
Linear (Baseline)
10
11
12
Linear (Condition 1)
13
14
Linear (Condition 2)
15
16
17
Linear (Condition 2)
18
Check These Out
 www.interventioncentral.com
 www.aimsweb.com
 http://dibels.uoregon.edu
 www.nasponline.org
Check These Out




www.fcrr.org
Florida Center for Reading Research
http://ies.ed.gov/ncee/wwc//
What Works Clearinghouse
http://sites.google.com/site/rateofimprovement/
Rate of Improvement
http://www.rti4success.org
National Center on RtI
References
Batsche, G. M., Castillo, J. M., Dixon, D. N., & Forde, S. (2008). Best
practices in linking assessment to intervention. In A. Thomas and J.
Grimes (Eds.), Best practices in school psychology V. (pp. 177-193).
Bethesda, MD: National Association of School Psychologists.
Batsche, G. M., Elliot, J., Graden, J. L., Kovaleski, J. F., Prasse, D., et al.
(2005). Response to intervention: Policy considerations and
implementation. Alexandria, VA: National Association of School
Psychologists.
Colvin, G., Sugai, G., Good, R. H., III, & Young-Yon, L. (1997). Using active
supervision and precorrection to improve transition behaviors in an
elementary school. School Psychology Quarterly, 12, 344-363.
Deno, S. L. (1985). Curriculum-based measurement: The emerging
alternative. Exceptional Children, 52, 219-232.
Deno, S. L., & Fuchs, L. S. (1987). Developing curriculum-based
measurement systems for databased special education problem
solving. Focus on Exceptional Children, 19(8), 1-15.
Deno, S. L., Fuchs, L.S., Marston, D., & Shin, J. (2001). Using curriculumbased measurement to establish growth standards for students with
learning disabilities. School Psychology Review, 30, 507-524.
References
Flinn, C. S. (2008). Graphing rate of improvement for individual students.
InSight, 28(3), 10-12.
Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Germann, G. (1993).
Formative evaluation of academic progress: How much growth can we
expect? School Psychology Review, 22, 27-48.
Fuchs, L. S., & Fuchs, D. (1998). Treatment validity: A unifying concept for
reconceptualizing the identification of learning disabilities. Learning
Disabilities Research and Practice, 13, 204-219.
Fuchs, L. S., & Fuchs, D. (1997) Use of curriculum-based measurement in
identifying students with learning disabilities. Focus on Exceptional
Children, 30, 1–16.
Fuchs, L. S., Fuchs, D., Hamlett, C. L., Hope, S. K., Hollenback, K. N. Capizzi,
A. M., Craddock, C. F., & Brothers, R. L. (2006). Extending responsivenessto-intervention to math problem-solving at third grade. Teaching Exceptional
Children, 38, 59-63.
Gresham, F. (2001, August). Responsiveness to intervention: An alternative
approach to the identification of learning disabilities. In R. Bradley, L.
Danielson, & D. P. Hallahan (Eds.). Identification of learning disabilities:
Research in practice (pp. 467–519). Mahwah, NJ: Erlbaum.
Individuals with Disabilities Education Improvement Act (IDEIA) of 2004, Pub.
L. No. 108,446.
References
Kovaleski, J. F. (2008, September 19). Pennsylvania promotes best
practices in school psychology. Paper presented at the Pennsylvania
Training and Technical Assistance Network, King of Prussia, PA.
Mortenson, B. P., & Witt, J. C. (1998). The use of weekly performance
feedback to increase teacher implementations of a prereferral academic
intervention. School Psychology Review, 27, 613-628.
No Child Left Behind Act of 2001. Pub. L. No. 107-110, 115 Stat. 1425
(2002).
Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Giatti, S. L., Williams,
K. L., et al. (2005). Treatment implementation following behavior
consultation in schools: A comparison of three follow-up strategies.
School Psychology Review, 34(1), 87-106.
Shapiro, E. S. (2008). Best practices in setting progress monitoring goals
for academic skill improvement. In A. Thomas and J. Grimes (Eds.),
Best practices in school psychology V (Vol. 2, pp. 141-157). Bethesda,
MD: National Association of School Psychologists.
Shinn, M. R. (Ed.). (1998). Advanced applications of curriculum-based
measurement. New York: Guilford Press.
References
Shinn, M. R. (2008). Best practices in using curriculum-based
measurement in a problem-solving model. In A. Thomas and J.
Grimes (Eds.), Best practices in school psychology V (Vol. 2, pp.
243-261). Bethesda, MD: National Association of School
Psychologists.
Sulzer-Azaroff, B., & Mayer, G. R. (1991). Behavior analysis for lasting
change. Fort Worth: Holt, Rinehart, & Winston.
Stuebing, K., Fletcher, J., LeDoux, J., Lyon, G. R., Shaywitz, S., &
Shaywitz, B. (2002). Validity of IQ-discrepancy classifications of
reading disabilities: A meta-analysis. American Educational
Research Journal, 39, 469–518.
Upah, K. R. F. (2008). Best practices in designing, implementing, and
evaluating quality interventions. In A. Thomas and J. Grimes (Eds.),
Best practices in school psychology V. (pp. 209-221). Bethesda,
MD: National Association of School Psychologists.
Wright, J. (2008). Internet resources for ‘response to intervention’.
Retrieved June 24, 2008, from
http://www.jimwrightonline.com/php/rti/rti_wire.php