Transcript Slide 1

Response to Intervention:
Using Data to Enhance
Outcomes for all Students
Amanda VanDerHeyden
Education Research and Consulting, Inc.
Objectives Today
• Overview of RTI, RTI decision making, and
expected outcomes
• Specific How-To for RTI
– Interpreting Assessment Data to determine need for
Tiers 1, 2, and 3 Interventions
– Selection and implementation of Tier 2 Interventions
– Selection and implementation of Tier 3 Interventions
• Implementing intervention for sustenance and
system change
16 x 3 = 48 hours
What is RTI?
• A science of decision making and way of
thinking about how educational resources
can be allocated (or reallocated) to best
help all children learn
• Major premium on child outcomes
RTI is Not
• A program, a curriculum, an intervention, a
particular model
Data allow us to
• Provide faster, more effective services for ALL
children
• Work “smarter” not harder, better utilize the
talents of the school psychologist and schoolbased assessment and intervention teams.
• Make implementation SIMPLE and EASY for
teachers (low cost, few errors)
• Prevent diagnosis
Early Screening Identifies Children At Risk
of Reading Difficulty
J
5.2
Reading grade level
5
4
3
Low Risk on
Early
Screening
2.5
2
1
At Risk on Early Screening
1
2
3
4
Grade level corresponding to age
This Slide from Reading First
Experts From Reading First
Early Intervention Changes Reading
Outcomes
5.2
5
Reading grade level
J
4.9
With substantial
instructional
intervention
4
3
With research3.2 based core but
without extra
2.5 instructional
intervention
Low Risk
on Early
Screening
2
1
At Risk on Early Screening
1
2
3
4
Grade level corresponding to age
This Slide from Reading First
From Reading First
Experts
Evolution
• “Wait to Fail”
 Let’s provide services early!
• Costly sp ed programs not improving learning
 Let’s shift resources to provide services in less restrictive setting!
• Increasing numbers of children struggling in general ed
 Let’s provide help in general education!
• Traditional measures are de-contextualized and the
constructs are problematic
 Let’s help children who struggle academically by measuring
performance in response to certain intervention strategies and
then deliver what works!
Rationale for System Change
200-300% increase in SLD
•
•
•
•
Level and Rate of Performance
Return to General Education
Lack of Certified Teachers
No demonstrated instructional techniques that
differentially benefit SLD
• Drop-out
• Disproportionate Representation by Ethnicity
History of RTI
• Effective Instruction Lit, CBA/M Lit
• Lab-Quality Intervention Programs
• Progress Monitoring Data and ProblemSolving Models
• Reading First data
Consistent with
• NCLB
• Reauthorized IDEA
• Recommendations of panel reports on:
Minority students in special education,
National Reading Panel, Science and
Math Initiative
Impetus
• Faster, more effective services for ALL children
• Work “smarter” not harder, better utilize the talents of the
school psychologist and school-based assessment and
intervention teams.
• Make implementation SIMPLE and EASY for teachers
(low cost, few errors)
Why RTI?
• Viable alternative to traditional diagnosis of highincidence disabilities, particularly Learning Disability (LD)
• Reauthorized IDEA guidelines for identifying LD state
that:
a) A severe discrepancy between achievement and
intellectual ability shall not be required
b) A response to intervention (RTI) may be considered
Why RTI?
• RTI can address the problem of disproportionate
identification of children with LD by race and
gender
• The utility of curriculum-based measures (CBM)
for:
– Identifying children not likely to benefit from the
general education curriculum without assistance,
– Predicting important long-term outcomes
– Tracking individual student growth and informing
instructional programming changes has been
established
Considerations
• There has been some consensus concerning
the need for change; however, there has not
been consensus on how this change can best
be achieved.
• Whereas RTI has considerable promise as a tool
within special and general education, it is a
vulnerable construct if misapplied.
Improved Treatment Validity
• Direct link to treatment or consequential validity
• Efforts are focused to:
– Properly articulate a concern
– Develop targeted intervention to resolve the concern
– Collect information to determine whether or not the
concern has been adequately addressed or whether
different solution efforts need to be implemented
• This approach changes the goal of assessment
from what some have described as “admiration”
of the problem to problem-solving.
Contextualized Decision Making
• RTI emphasizes the pre-referral conditions (child
and environment) and this context becomes part
of the decision-making equation.
• Allows practitioners to quantify:
– The state of instructional affairs in the child’s regular
education environment
– Potential learning given optimal instructional
conditions
• RTI may enable improvement in general
education programming leading to children
receiving assistance in a more efficient manner.
Improved Identification
Accuracy for LD
• Under RTI models intervention becomes a
specified, operationalized variable, thus false
positive identification errors should be reduced
dramatically.
• Removing the current reliance on teacher
identification and requiring direct measures of
child performance in context will enhance
identification accuracy.
More Effective Intervention
• RTI is likely to facilitate less restrictive
interventions and placements for children.
• RTI allows school psychologists to bring their
expertise to bear on assessment strategies at
the classroom level and assist teachers to use
data formatively to enhance their instructional
programming.
Possible Challenges
of RTI
Decision-making Criteria
• Must be operationalized and validated
through research
• The purpose of RTI will be critical to
determining how implementation should
proceed
New Challenges for Teams
• Effective intervention delivery will depend
on relevant intervention variables
• To be effective the intervention must have
been:
– Properly identified
– Implemented with integrity and with sufficient
frequency, intensity, and duration
STEEP Model
Screening to Enhance Educational Progress
Tier 1: Math Screening
• Math Probe:
– Group administered.
– Materials: Worksheet consisting of a series of
problems sampling the target skill(s) (e.g.,
sums to 5, double digit multiplication with
regrouping).
– Timing: 2 minutes
– Information obtained: digits correct in two
minutes.
Math Probe Example
• Total Digits: 38
•Errors: 5
•Digits Correct: 33
Tier 1: Writing Screening
• Writing Probe:
– Group administered.
– Materials: story starter (e.g., If I had a million
dollars…) printed at the top of a blank page.
– Timing: 1 minute to think, 3 minutes to write.
– Scoring: words written or correct word sequences in
three minutes.
Writing Example
Tier 1: Reading Screening
• Reading Probe:
– Individually administered
– Materials: A content-controlled reading passage.
– Procedure: The student reads aloud as the teacher
listens and records errors.
– Timing: 1 minute
– Information obtained: words read correctly in one
minute.
CBM Reading: Sample Scoring



TRW=63
Errors=6
CRW=58
Class-wide Screening
QuickTime™ and a
Cinepak decompressor
are needed to see this picture.
Feedback to Teachers
Tier 2: Class-wide Intervention
Digits Correct in Two Minutes
Intervention
Baseline
120
100
m
a
s
t
e
r
y
80
60
40
instructional range
20
0
1
2
3
Sessions
4
5
Mary
Chiquita
Randy
Sandy
Brandy
Colvin
Jolisha
Daleesha
Kiera
Bradley
Jared
Alfred
Sienna
Jarian
Trey
Robert
Andrea
Ashley
Jaren
No Class-wide Problem Detected
Tier 2: Can’t Do/Won’t Do
Assessment
• “Can’t Do/Won’t Do”
• Individually-administered
• Materials
3-7 minutes per
child
– Academic material that student performed poorly
during class assessment.
– Treasure chest: plastic box filled with tangible items.
Can’t Do/Won’t Do Assessment
QuickTime™ and a
Cinepak decompressor
are needed to see this picture.
Decision Rule Following Can’t
Do/Won’t Do Assessment
Tier 3: Individual Intervention
QuickTime™ and a
Cinepak decompressor
are needed to see this picture.
#Correct
Response to Intervention
Before
Intervention
During Intervention
Avg. for his Class
Each Dot is one
Day of Intervention
Intervention Sessions
Intervention in Reading
#Correct
Response to Intervention
Before
Intervention
During Intervention
Avg. for his Class
0
Breaux Aisha S
Chapdelain Lily M
Robb Sydney M
Mangione Nikole M
White Bailey V
Ramey Sara B
Littsen Lucas R
Foxhoven Shane A
Gonzales Audryana
Foley Thomas J
Hermes Jay M
Thueson Lila D
Svob Seth H
Salsbury Mariah A
Wills Donna D
Blakeley Brandee E
Kellogg Anthony S
Negrete Sara M
Ashton Julia E
Dailey Brandon L
Showers Phillip J
Counes James G
Hilkemeyer Austin R
Howe Ashley B
Strider Katie N
Santa cruz Daniel R
Gallego Angela M
Lewandowski
Sisk Cody A
Forsyth Ian E
Blake Nicholas K
Hatch Vanessa L
Machain Anthony F
Peterson Tyler L
White Alexa L
Nutbrown Jordan C
Bluemke Megan J
Casamasa Gregory L
Roche Alyssa R
Elias Elizabeth L
Beeston Kristine D
Lopez Theresa A
Pierce Shannon M
Lanier Matthew W
Nanna Caitlin N
Smith Shelby N
Iturralde Jacqueline R
Mcharg Jordan E
Brechbiel Shari L
Cota Alexia K
Jackson Damion M
Lamadrid Leonardo
Oliver Riley W
Layton Marissa M
Mueller Lane E
Turner Alana K
Rowlan Paige E
Dumes Scott M
Riordan Timothy D
Hicks Coltin C
Kenton Chelsee M
Cornwell Kimberly M
Crater Shelbie M
Rytting Ryan C
French Joshua M
Davila Ariel N
Thompson Tasha N
Ryckman Shelby L
Ayers Megan L
Dunham Clayton J
Mausert James R
Morales Eric A
Bain Bryce G
Gryczkowski samluk
Cuff Matthew J
Hackman Lindsey S
Whitlock John C
Benson Brad J
Stanfield Benjamin C
Martinez Nathaniel P
Carrizosa Robert A
Webb Brianna J
Evans Joseph B
Laye Lestot D
Bazzanella Stephen L
Crowl Robert S
Thompson Krysta E
Tipton Emily N
Meyer Sean M
Couture Anne S
Gibbons Cody D
Fuhrman Autum C
Nolen Jayd L
Drake Justin T
Neale Shaine R
Purcell John E
Rugotska Colton J
Bong Samantha L
Johnson Amanda L
Rodriguez Ryan T
Bergstrom Matthew A
Pannell Marina S
Digits Correct Two Minutes
Vehicle for System Change:
System-wide Math Problem
4th Grade Math
Multiplication 0-9
120
100
80
60
Instructional range
40
20
Frustrational range
Each bar is a student’s performance
0
Neale
Frost Joshua
Franklin
Ryckman
Turner Alana
Ayers
Smith
Montano
Kenton
Banken
Mausert
White Alexa
Brechbiel
Hatch
Meyer Sean
Santa cruz
Oliver Riley
Cornwell
Wills Donna
Bluemke
Williams
Cuff Matthew
Nanna
Hilkemeyer
Robb
Gryczkowski
Gonzales
Mangione
Foxhoven
Layton
Negrete
Tipton Emily
Gavino
Lamb Nicole
Chapdelain
Roche
Digits Correct Two Minutes
Re-screening Indicates No
Systemic Problem
Fourth Grade
Fourth Grade Multiplication 0-9
140
120
100
80
60
40
20
Rest of Grade at Standard
Classroom A
B
C
D
E
F
Spring 2003– Classroom F
F
Teacher moved to lower grade in
Fall 2003
Digits Correct Two Minutes
0
11/18/2003
11/14/2003
11/7/2003
10/31/2003
10/24/2003
Class-wide Intervention
Teacher F Mult 0-12
120
100
80
60
40
20
Weeks
Increased Difficulty- Intervention
Continues
140
120
100
80
60
40
20
1/23/2004
1/15/2004
1/8/2004
12/18/2003
12/5/2003
0
11/21/2003
Digits Correct Two Minutes
Teacher F Div 0-12
Weeks
Mixed Mult/Div/Fractions Probe
Classroom F
May
April
March
February
aimline
January
Dec
Nov
90
80
70
60
50
40
30
20
10
0
Oct
Sept
Digits Correct Two Minutes
Growth Obtained
Rush
actual growth
Effect on High-Stakes Scores
Percent of Students Passing HighStakes Mathematics
3rd Grade Math Performance
100
90
80
70
60
50
40
30
20
10
0
School 1
School 2
School 3
School 4
School 5
VanDerHeyden, in prep
Effect on High-Stakes Scores
Percent of Students Passing HighStakes Mathematics
5th Grade Math Performance
100
90
80
70
60
50
40
30
20
10
0
School 1
School 2
School 3
School 4
School 5
VanDerHeyden, in prep
District-wide Implementation Data
• Vail Unified School District
– www.vail.k12.az.us
• Three years, system-wide implementation
of STEEP grades 1-8
System Outcomes
• Referrals reduced greater than half
• % who qualify from 50% stable baseline
over three years to nearly 100%
• SLD down from 6% of children in district in
2001-2002 (with baseline upward trend) to
3.5% in 2003-2004 school year
• Corresponding gains on high-stakes tests
(VanDerHeyden & Burns, 2005)
• Intervention successful for about 95 to 98%
of children screened
VanDerHeyden, Witt, & Gilbertson, 2007
Cost Reduction
160,000
Cost in Dollars
140,000
120,000
100,000
Baseline
STEEP
80,000
60,000
40,000
20,000
0
Assessment
Placement
VanDerHeyden, Witt, & Gilbertson, 2007
Findings
• Number of Evaluations dramatically
reduced– 70% at highest referral school
• Diverse settings, psychologists of diverse
backgrounds and no prior experience with
CBM or functional academic assessment
• Percentage qualify increased at 4 of 5
schools
• Disproportionate representation of males
positively affected
• Number of children placed dramatically
reduced
VanDerHeyden, Witt, & Gilbertson, 2007
Team Decision-Making Agreement
RTI + and
Evaluated
RTI- and Did
Not Evaluate
2003-2004
(3 schools)
100%
41%
2004-2005
(5 schools)
100%
87%
VanDerHeyden, Witt, & Gilbertson, 2007
Team Decision-Making
Baseline
STEEP + and Team
STEEP- and Team
Decided to Evaluate
Decided to Evaluate
2003-2004 Cases, Schools 1-3 55%
89%1
50%2
2004-2005 Cases, Schools 1-5 52%
88%3
29%4
VanDerHeyden, Witt, & Gilbertson, 2007
Fall to Spring Reading Growth
WC/Min Growth Per Week
16
14
12
10
Minority
Caucasian
Expected
8
6
4
2
0
1
10
Weeks
VanDerHeyden & Witt, 2005
Percent of Minority and Caucasian
Students in Risk Category
What Proportion of Ethnicity Represented
Before and After Intervention in Risk
Category?
100
90
80
70
60
Minority
Caucasian
50
40
30
20
10
0
Before
Intervention
After Intervention
Expected
VanDerHeyden & Witt, 2005
Criterion +
Sens- .76
Spec- .89
PPP- .59
NPP- .95
+ LR- sens/(1-spec)
- LR- (1-sens)/spec
Criterion -
PVS +
13
9
PVS -
4
75
.76/.11=6.9: children with learning problem
about 7 times more likely to have a failed RTI
than children without a learning problem
.24/.89=.27: children without a learning
problem are 3.7 times more likely to have a
successful RTI than those children with a
learning problem
Criterion +
Sens- .94
Spec- .54
PPP- .29
NPP- .98
Criterion -
Probe +
16
39
Probe -
1
45
.94/.46=2.0: children with learning problem
about 2 times more likely to perform below
screening benchmark than children without a
learning problem
.06/.54=.11: children without a learning
problem are 9 times more likely to have a
above criterion screening score than those
children with a learning problem
Identification Accuracy
• High-achieving classrooms (<20%)
– Procedures paired with RTI criterion were
more accurate than other commonly used
screening devices
• Low-achieving classrooms (>50%)
– Procedures paired with RTI criterion were
more accurate than other commonly used
screening devices
VanDerHeyden & Witt, 2005
Questions
• Is there a classwide problem?
• Is there a gradewide problem?
• What’s the most efficient way to deliver
intervention?
Screening tells you
• How is the core instruction working?
• What problems might exist that could be
addressed?
• Most bang-for-the-buck activity
• Next most high-yield activity is classwide
intervention at Tier 2.
Screening Guidelines
• Efforts at Tier 1 pay off with fewer children
needing individual intervention
• 3 times per year, single probe
• Use small team of trained coaches
• Prepare all needed materials in a packet for
each teacher
• Score and return within 1 week on graph
• Use data to generate aimlines, can be used to
set benchmarks
Here is 3rd grade at Cottonwood. Each circle corresponds to a students score on a reading CBM
probe in March and the AIMS reading score the same month. So the circle near the blue lines is a
child who read 158 wc/min and scored 486 on the AIMS Reading. The diagonal line represents
the “best fit” line or the line closest to all the circles. This shows there is a strong positive
correlation between CBM and AIMS reading scores with this group.
How to Set a Benchmark
431 = “pass” for AIMS.
This line does not move
This is words read
correctly per
minute. You move
this line up and
down to “catch” as
many of those who
will not pass as
possible.
Setting 95 wc/min as the “pass”
standard for CBM
These are the
children
predicted to pass
AIMS who
actually failed.
“False Negative
Errors.” The
worst kind of
error.
These are the
children who were
predicted to fail
AIMS based on
CBM who did fail.
“Hits”
These are
children who
were predicted
to pass AIMS
based on CBM
and did pass.
“Hits”
These are the
children
predicted to fail
AIMS who
actually passed.
“False Positive
Errors.”
Moving the horizontal line up will “catch” two more cases who failed the AIMS,
but will result in many more “false positive errors.”
Thus, 95 wc/min at 3rd grade at Cottonwood is a good standard that will
tell you which children are likely to fail the AIMS reading section.
ROC printout
Coordinates of the Curve
Test Result Variable(s): CBM Reading score closest to testing (4/21/05 )
Positive if
Greater Than
or Equal To(a)
Sensitivity
1 - Specificity
30.000 0
1.000
1.000
44.5000
.989
1.000
60.5000
.978
1.000
63.5000
.978
.875
74.0000
.978
.750
86.5000
.966
.750
89.5000
.966
.625
90.5000
.955
.625
91.5000
.944
.625
92.5000
.933
.250
94.0000
.921
.250
95.5000
.921
.125
96.5000
.910
.125
99.0000
.899
.125
101.5000
.888
.125
102.5000
.876
.000
Mastery Model Measurement
(CBA)
Letter naming
fluency
Isolated sound
fluency
Beginning
sound fluency
Ending sound fluency
General Outcome
Measurement (CBM)
Words read correctly per minute
Letter naming
fluency
Isolated sound
fluency
Beginning
sound fluency
Ending sound fluency
Tracking Year-Long Growth
mastery
Digits Correct Two Minutes
45
40
aimline
35
30
25
20
15
10
instructional
5
0
1
3
5
7
9
11 13 15 17 19 21 23 25 27 29 31
Number of Instructional Weeks
Pass the
AIMS
1
Weeks
12
Any
Curriculum
Area
Academic Systems
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•High Intensity
•Of longer duration
1-5%
80-90%
5-10%
Students
Universal Interventions
•All students
•Preventive, proactive
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•Intense, durable procedures
1-5%
5-10%
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
Behavioral Systems
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
80-90%
Universal Interventions
•All settings, all students
•Preventive, proactive
Dave Tilly, 2005
“Weighing a cow doesn’t
make it fatter.”
Any
Curriculum
Area
Academic Systems
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•High Intensity
•Of longer duration
1-5%
80-90%
5-10%
Students
Universal Interventions
•All students
•Preventive, proactive
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•Intense, durable procedures
1-5%
5-10%
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
Behavioral Systems
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
80-90%
Universal Interventions
•All settings, all students
•Preventive, proactive
Dave Tilly, 2005
Class-wide Intervention
• Use pair-peered practice (classwide peer
tutoring, PALS)
• Model, Guided Practice, Independent
timed practice with delayed error
correction, reward contingency
Instructional Hierarchy
Finally, problem-solving/ application
practice should occur here with a
mastery level skill– Core InstructionNot Manipulated but could be
Generalization
Fluency
Acquisition
But fluency building should
happen here with an
instructional level skill–
Intervention Focus was here
To gain the steepest growth,
introduction of new skills
should happen here– Core
Instruction- Not manipulated
The Nuclear Readi ng Interve ntion for 1st and 2nd Grade S tudents
This intervention is designed t o build fluency in reading and increase accuracy.
Requires approximately 7 minutes each day.
Materi als Ne e de d:2 copies of a reading passage that are controlled t o reflect
instructional level words, digital timer, and pencil.
Te ach er C oachC ard (complete these steps every day):
Ta ke out the stu de ntÕ scopy of th e re adi ng passage .
MO DEL: Re ad the 60 words of th e passagealou d to the stu de nt. Re ad
slower th an you n ormal ly would an d poi nt to the words as you re ad.
GUIDE PRAC TIC E: Have th e stu dent re ad the first 60 words of th e passage
alou d to you. Instruct the student t o use his/her pointer finger to follow the words as he
or she reads. If the student gets stuck on a word for 3 seconds, tell the student the word.
P rompt the student to pronounce words correctly and immediately correct mistakes.
INDEPENDENT PRACTIC E: Ta ke ou t you rcopy of th e re adin g passage
for scori n g.
S e t th e timer for 1 mi n u te.
Have the stu de nt re ad alou d in de pe n de nlyt for 1 mi nute, while you follow
along on a separate copy of the same passage.
Mark errors (see below) as the stu de nt re ads. W hen the timer rings, draw a
verti cal li n e after th e last word re ad ( | ).
C ou nt n u m ber of words re ad corre ctly. Write this number at the top of the
page.
What is a missed word? skipped words, mispronounced words, and words
told to the student after the 3-second hesitation.
Reading classwide intervention
Select a Few Good Interventions to
Keep it Simple
Classwide
Individual
Math
Flash card
Practice
Cover copy
compare
Cue Cards
Highlighted errors
Reading
Listening Preview
Repeated
Readings
Error Correction
Key Words
Teacher:
Grade:
Date:
Classwide Intervention: Teaching Math Facts (Use with Flashcards)
This intervention is designed to build math fact fluency and increase accuracy and can be used for
addition, subtraction, multiplication, or division facts.
Teacher Coach Card (conduct these steps every day):
Instruct students to find their math partner and get out flashcards quickly and quietly.
GUIDED PEER PRACTICE
Set timer for 3 minutes and tell students, “Begin practicing.”
When timer rings, tell students, “Stop. Switch flashcards.”
Set timer for 3 minutes and tell students, “Begin practicing.”
When timer rings, tell students, “Stop practicing.”
TIMED INDEPENDENT PRACTICE
Pass out worksheets face-down on students’ desks. Tell students, “Write your name on the back of
your paper. Don’t turn them over until I tell you to.”
Set timer for 2 minutes. Say, “On your mark, get set.” Begin the timer, and say, “Go.”
When the timer rings, tell students, “Hold your papers up in the air so that I can see that you are no
longer working.”
Tell students, “Trade papers with your math partner for scoring. When I call out the answers,
mark the answers ‘right’ or ‘wrong’.”
ERROR CORRECTION
Call out the correct answers. Review answers that several students miss.
Tell students, “Give papers back to their owners now. If you missed problems, write the correct
answer under the problem where your partner wrote it.”
Tell students, “Write your score on your progress chart and pass your papers to the front so I can
pick them up.”
REWARD/MOTIVATION
Shuffle the papers. Randomly draw a paper from the stack. If the score on this randomly selected
paper is higher than the randomly selected score from the day before (or the class median if you have calculated
it), then deliver a classwide reward (e.g., 5 minutes free time).
Teachers: Every Friday, record each student’s score on the Daily Intervention datasheet in the
“intervention” column.
Teacher:
Grade:
Date:
Classwide Math Intervention: Moving Beyond Basic Facts (Use with Practice Sets)
Distribute the correct worksheet to students and tell students to get into their working pairs.
Instruct students to write their names and the date on math sheet.
GUIDED PEER PRACTICE
Students should complete the first row (or as many as possible in 3 minutes) of the worksheet with help
from their math buddy.
Tell students to switch roles. Now, the other student should complete the second row of problems (or as
many as possible in 3 minutes) with help from their math buddy.
*** The goal is for students to work as quickly as possible completing as many problems as possible in the
short amount of time with 100% accuracy. If one student is stronger than another, then you will have to monitor
to make sure that the stronger student does not simply supply the answer but explains how to get the answer
when that student is acting as the “coach” or “tutor.”
INDEPENDENT TIMED PRACTICE
Set timer for 2 minutes.
Work problems below the practice line for 2 minutes.
When timer rings, tell students to stop working.
ERROR CORRECTION
Have students trade papers and score.
Provide a mini-lesson/review when the same kind of error is made by many students.
Have students count the number of digits correct at the top of the page.
Write the correct answer for the problems you missed.
REWARD/MOTIVATION
Shuffle the papers. Randomly draw a paper from the stack. If the score on this randomly selected
paper is higher than the randomly selected score from the day before (or the class median if you have calculated
it), then deliver a classwide reward (e.g., 5 minutes free time).
Teachers: Each Friday, record student’s scores on the Daily Intervention Datasheet in the “Intervention
Column.”
Intervention Plan- 15 Min per Day
• Protocol-based classwide peer tutoring,
randomized integrity checks by direct
observation
• Model, Guide Practice, Independent Timed
Practice with delayed error correction
• Group performance contingency
• Teachers encouraged to
– Scan papers for high error rates
– Do 5-min re-teach for those with high-error rates
– Provide applied practice using mastery-level
computational skill
Sample Sequence
3RD GRADE
1. addition and subtraction facts 0-20
2. fact families addition and subtraction 0-20
3. 3 digit addition without and with regrouping
4. 3 digit subtraction without and with regrouping
5. 2 and 3 digit addition and subtraction
6.
7.
8.
9.
with and without regrouping
multiplication facts 0-9
division facts 0-9
fact families multiplication and division 0-9
add/subtract fractions with like denominators
flash cards
practice set – same as skill
practice set – same as skill
practice set – same as skill
practice set – same as skill
flash cards
flash cards
practice set – same as skill
practice set – same as skill
(3rds, 4ths, 8ths, 10ths, no regrouping)
10. single digit multiplied by double/triple digit
practice set – same as skill
without regrouping
11. single digit multiplied by double/triple digit
practice set – same as skill
with regrouping
12. single digit divided into double/triple digit
without remainders
13. add and subtract decimals to the hundredths
practice set – same as skill
practice set – same as skill
Intervention Plan
• Class Median reaches mastery range for
skill, next skill is introduced
• Following promising results at one site in
2002-2003, lead to implementation districtwide grades 1-8 for all children by 20042005.
Class-wide Math Intervention
With teacher support
• Consider time, resources, materials
• Remove skill barriers with
– classroom training for students
– classroom coaching for teachers
• Remove implementation barriers after use new steps
– follow-up supportive meetings to problem solve.
– frequent acknowledgment of a teacher’s efforts
Training Package
Tell
Rational
Step by step protocol
Show
Model
Do
Train students
Implement with guided practice
Implement independently with support
88% of interventions are not used
without support
Decisions do not always correspond to data
(someone must check)
Tier 2 Intervention
Effects
Class 1 at Screening
Class 1: Following 10 Days
Intervention
Class 1: Following 15 Days
Intervention
Class 2 at Screening
Class 2: Following 5 Days
Intervention
Class 2: Following 10 days
Intervention
Class 3 at Screening
Class 3: Following 5 days
Intervention
Following 10 Days Intervention
Tier 1 Screening Indicates Classwide Problem
Digits Correct Two Minutes
0
11/18/2003
11/14/2003
11/7/2003
10/31/2003
10/24/2003
Tier 2: Class-wide Intervention
Teacher F Mult 0-12
120
100
80
60
40
20
Weeks
Increased Difficulty- Intervention
Continues
140
120
100
80
60
40
20
1/23/2004
1/15/2004
1/8/2004
12/18/2003
12/5/2003
0
11/21/2003
Digits Correct Two Minutes
Teacher F Div 0-12
Weeks
Contextually-Relevant Comparisons
and Use of Trend Data
Mary
120
Chiquita
Classwide Intervention
Baseline
Randy
Correct Initial Sounds per Minute
Sandy
100
Brandy
Colvin
Jolisha
80
Daleesha
Kiera
60
Bradley
Alfred
40
Sienna
Jarian
20
Trey
Robert
0
Andrea
1
2
3
Weeks
4
5
Ashley
Jaren
5th Grade Math Intervention
Mr. Ezell CW Math Fact Families
79
5/31/2006
Median
5/30/2006
61
66.5 65.5
5/25/2006
49.5
5/24/2006
100
80
60
40
20
0
5/23/2006
Digits Correct in 2
Minutes
Mastery = >80 dc2m
Session Date
0
Breaux Aisha S
Chapdelain Lily M
Robb Sydney M
Mangione Nikole M
White Bailey V
Ramey Sara B
Littsen Lucas R
Foxhoven Shane A
Gonzales Audryana
Foley Thomas J
Hermes Jay M
Thueson Lila D
Svob Seth H
Salsbury Mariah A
Wills Donna D
Blakeley Brandee E
Kellogg Anthony S
Negrete Sara M
Ashton Julia E
Dailey Brandon L
Showers Phillip J
Counes James G
Hilkemeyer Austin R
Howe Ashley B
Strider Katie N
Santa cruz Daniel R
Gallego Angela M
Lewandowski
Sisk Cody A
Forsyth Ian E
Blake Nicholas K
Hatch Vanessa L
Machain Anthony F
Peterson Tyler L
White Alexa L
Nutbrown Jordan C
Bluemke Megan J
Casamasa Gregory L
Roche Alyssa R
Elias Elizabeth L
Beeston Kristine D
Lopez Theresa A
Pierce Shannon M
Lanier Matthew W
Nanna Caitlin N
Smith Shelby N
Iturralde Jacqueline R
Mcharg Jordan E
Brechbiel Shari L
Cota Alexia K
Jackson Damion M
Lamadrid Leonardo
Oliver Riley W
Layton Marissa M
Mueller Lane E
Turner Alana K
Rowlan Paige E
Dumes Scott M
Riordan Timothy D
Hicks Coltin C
Kenton Chelsee M
Cornwell Kimberly M
Crater Shelbie M
Rytting Ryan C
French Joshua M
Davila Ariel N
Thompson Tasha N
Ryckman Shelby L
Ayers Megan L
Dunham Clayton J
Mausert James R
Morales Eric A
Bain Bryce G
Gryczkowski samluk
Cuff Matthew J
Hackman Lindsey S
Whitlock John C
Benson Brad J
Stanfield Benjamin C
Martinez Nathaniel P
Carrizosa Robert A
Webb Brianna J
Evans Joseph B
Laye Lestot D
Bazzanella Stephen L
Crowl Robert S
Thompson Krysta E
Tipton Emily N
Meyer Sean M
Couture Anne S
Gibbons Cody D
Fuhrman Autum C
Nolen Jayd L
Drake Justin T
Neale Shaine R
Purcell John E
Rugotska Colton J
Bong Samantha L
Johnson Amanda L
Rodriguez Ryan T
Bergstrom Matthew A
Pannell Marina S
Digits Correct Two Minutes
Pre-post changes to performance
detected by CBM
4th Grade Math
Multiplication 0-9
120
100
80
60
Instructional range
40
20
Frustrational range
Each bar is a student’s performance
0
Neale
Frost Joshua
Franklin
Ryckman
Turner Alana
Ayers
Smith
Montano
Kenton
Banken
Mausert
White Alexa
Brechbiel
Hatch
Meyer Sean
Santa cruz
Oliver Riley
Cornwell
Wills Donna
Bluemke
Williams
Cuff Matthew
Nanna
Hilkemeyer
Robb
Gryczkowski
Gonzales
Mangione
Foxhoven
Layton
Negrete
Tipton Emily
Gavino
Lamb Nicole
Chapdelain
Roche
Digits Correct Two Minutes
Fourth Grade
Fourth Grade Multiplication 0-9
140
120
100
80
60
40
20
Effect on SAT-9 Performance
SAT-9 Standard Scores and t-test Results for Pre- and Post-Implementation Years by Grade
2001-2002
Grade
n
Third
85
Fourth
2002-2003
M
t
SD
n
M
SD
562.06
143.80
129
602.54
35.20
3.07**
116
611.09
120.61
117
638.22
33.39
2.35*
Fifth
113
636.73
109.86
107
659.17
35.77
2.01*
Total
314
607.04
126.83
353
631.53
41.93
3.42**
* p < .05
** p < .01
Cohen’s d (effect size between years)
Third .45
Fourth .35
Fifth
.31
Total .29
Effect on CBM Scores
Mean Digits Correct/2 Minutes Scores for Monthly Mathematics Probes
January
February
March
Grade
M
M
SD
M
SD
M
SD
F
Third
27.9
9.9
37.9
13.4
35.3
13.0
38.8
12.3
13.45*
Fourth
39.7
17.6
50.1
22.0
53.4
25.3
58.9
27.0
35.02*
Fifth
47.6
22.9
50.8
22.7
52.0
24.8
59.3
25.1
25.52*
Total
41.1
20.5
48.0
21.5
49.2
24.0
55.1
25.1
64.29*
SD
* p < .001
Cohen’s d (effect size between January and April scores)
Third .97
Fourth .86
Fifth
.49
April
Computation Gains Generalized to
High Stakes Test
Improvements
(Gains within Multiple Baseline
shown as pre-post data)
Percent of Students Passing HighStakes Mathematics
3rd Grade Math Performance
100
90
80
70
60
50
40
30
20
10
0
School 1
School 2
School 3
School 4
School 5
Gains within Multiple Baseline
(shown as pre-post data)
Percent of Students Passing HighStakes Mathematics
5th Grade Math Performance
100
90
80
70
60
50
40
30
20
10
0
School 1
School 2
School 3
School 4
School 5
Identification Accuracy
CBA + RTI Criterion
STEEP
WJ-R
Sensitivity
.76
1
.58
Specificity
.89
.99
.77
Positive Predictive Power
.59
.67
.44
.95
1
.86
Sensitivity
.46
.33
.42
Specificity
.69
.94
.85
Positive Predictive Power
.19
.17
.45
.89
.97
.83
Negative Predictive Power
Teacher Referral
ITBS
Negative Predictive Power
VanDerHeyden, et al., 2003
Percent Identified at each Tier
Identified
CBM (Classwide Assessment)
55 (15%)
CBM + Reward (Performance/skill Deficit Assessment)
40 (11%)
CBM + Reward + Instruction
(STEEP +)
22 (6%)
Teacher Referral
32 (19%)
CIBS-R
64 (18%)
DRA
17 (9%)
RTI Criterion Assessment
17 (5%)
WJ-R
ITBS deficit
12
3 (4%)
VanDerHeyden, et al., 2003
Any
Curriculum
Area
Academic Systems
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•High Intensity
•Of longer duration
1-5%
80-90%
5-10%
Students
Universal Interventions
•All students
•Preventive, proactive
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•Intense, durable procedures
1-5%
5-10%
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
Behavioral Systems
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
80-90%
Universal Interventions
•All settings, all students
•Preventive, proactive
Dave Tilly, 2005
Tier 3
• Assessment Data
– Instructional level performance
– Error analysis (high errors, low errors, pattern)
– Effect of incentives, practice, easier task
– Verify intervention effect
• Same implementation support as Tier 2
• Instructional-level materials; Criterion-level
materials
Digits Correct Per Minute
Assessment Data
Baseline
Baseline
Reward
50
Reward
40
30
20
10
0
1
2
3
4
Sessions
Digits Correct Per Minute
Assessment Data
Baseline
Baseline
Reward
25
Instruction with Reward
Instruction
20
15
10
5
0
1
2
3
4
5
Sessions
Tier 3
• Implement for 5-15 consecutive sessions
with 100% integrity
• Link to referral decision
• Weekly graphs to teacher and weekly
generalization probes outside of
classroom, supply new materials
• Troubleshoot implementation weekly
Lessons Learned
• Most individual interventions for reading
• Standard protocols with slight modifications are
best
• Most interventions are successful (should be
successful)
• Generalization must be attended to
• Team will not follow data without support and
training to do so
• Coordinate intervention start times with principal
and stagger start dates (10-15 at a time plus Tier
2’s).
• Organize master schedule for data collection
and Tier times
Tier 3 Intervention
• >5% of children screened (total
population) IF solid Tier 1
• Possibly as low as 2% IF solid Tier 1 and
Tier 2
• About 1-2% failed RTI; 10% of most at-risk
Successful RTI
WCPM
Lorena C.'s Reading Progress
100
90
80
70
60
50
40
30
20
10
0
Baseline
Intervention
Generalization
Can't/Won't
In-Class Probe
2
5
7
10
Session
12
15
2004- 3rd Grade
WCPM
Amadi's Reading Progress
90
80
70
60
50
40
30
20
10
0
-10Baseline
Intervention
Generalization
Can't/Won't
In-Class Probe
2
5
7
10
Session
12
15
Successful RTI
Vincent's Reading Progress
100
WCPM
80
60
Intervention
Generalization
Can't/Won't
In-Class Probe
40
20
0
Baseline
-20
2
5
7
10
Session
12
15
Successful Writing
Words/Word Seq.
Taylar's Writing Progress
20
18
16
14
12
10
8
6
4
2
0
Baseline
Spelling
Passage
Can't Do/Won't Do
In-Class Probe
2
5
7
10
Session
12
15
Successful RTI
WCPM
Aaron Archuleta's Reading Interv ention Progress
60
50
40
30
20
10
0
instruction
daily gen
nov el gen.
1
2
3
4
5
6
Session
Measurement Error
50
40
30
3
2
20
10
0
1
Digits Correct in
2 Min.
Amanda DuRain's Math Progress
Sessions
Successful Math
DCP2M
Destiny Knott's Math Interv ention Progress
daily practice
inst. lev el gen.
criterion gen.
7
6
5
4
3
2
1
50
40
30
20
10
0
-10
Session
Unsuccessful Math
Digits Correct Two Minutes
Math Intervention
BL
45
interv ention 3 lev els below criterion
interv ention 2 lev els below criterion
40
aimline 60 day s
35
30
25
20
15
10
5
generalization
0
1
2
4
6
9
11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59
Days
Tips for Effective
Implementation
Our Recipe for Intervention
Success
PREPARE
• Identify and Use standard protocols for
intervention
• Develop all needed materials
• Develop packets or put on a central web site
• Determine graphing program
Our Recipe for Intervention
Success
TRAIN
• Explain
• Watch the teacher do it with the actual child
before you leave
• Call or meet teacher after first day to problem
solve
Our Recipe for Intervention
Success
COLLECT DATA AND SUPPORT
• Each week, graph intervention performance and do
a generalization check with the child.
• Graphed feedback to teachers with generalization
checks for individual intervention once per week
• Response-dependent performance feedback to
sustain implementation accuracy
• Monthly CBM to track growth and enhance existing
Tier 1 Programs or advise new Tier 1
• Data to principal weekly. Summarize effects and
integrity of procedures.
Tracking Title Progress
80%
70%
60%
50%
40%
School 1
30%
20%
10%
0%
Exceeding Slope
Not Exceeding
Slope
Our Recipe for Intervention
Success
USE DATA TO MAKE DECISIONS
• RTI successful if child performs criterion-level probe
(from screening) in the instructional range. RTI
unsuccessful if 15 consecutive intervention sessions and
criterion probe is not in the instructional range.
• Increase task difficulty for intervention if child scores at
mastery on task during intervention sessions
Infrastructure for Implementation
•
•
•
•
Grade-level planning periods can be utilized
Special education “team” at school can be utilized
School Psych must be on-site 1 day/week
Developing master schedule for Tier 1, 2, and 3
intervention times is useful
• Integrate efforts with evaluation referral team efforts
(consider major reduction in meeting time and shift to
intervention efforts!)
Materials needed
• Computer and software to organize data
• Student data imported. Clerical person to enter data onsite for tier 1 screen only.
• Color printer to print graphs + extra color cartridges
• Probe materials, digital count-down timers
• Intervention protocols, intervention materials (e.g.,
flashcard sets, reading materials)
• Access to copier and some assistance with copying
• Reinforcers for treasure chest (no more than $500 per
school)
Guidelines for Implementers
• Use single trial scores for screening
• Following screening, grade-wide graphs to
principal
• Return data to teachers within 48 hours with
personal interpretation at grade-level team
meeting
• Include principal in critical meetings
• Involve teachers at all stages
Guidelines for Implementers
• Learn about curriculum and instruction.
• Integrate RTI with ongoing school and system reform
efforts
• Thoughtfully merge to subtract duplicate activities and to
enhance more comprehensive supplemental and core
instructional support activities that may be in place
• Use RTI data to evaluate the value of ALL instructional
programs or resource allocation decisions. Quantify
bang for the buck using student performance data.
Lessons Learned from Vail USD
• Infrastructure for education being a resultsbased enterprise
• Accountability
• Principal is the Instructional Leader of a school
• Principal as change agent
• School psychologist as change agent
• Replace resources/substitute don’t add
• Minimize meetings
• Track outcomes that matter
QuickTime™
and a and a
QuickTime™
decompressor
decompressor
are needed
to see to
this
picture.
are needed
see
this picture.
QuickTime™ and a
decompressor
are needed toQuickTim
see this picture.
e™ and a
decompressor
are needed to see this picture.
Upset parent
Tantruming child
Bus 11 is late
Police on
site for child
abuse
report
Principal
Visit from health
department,
bathroom out of
paper towels
again
Teacher out sick
for rest of year
Meeting at district office/items due
Principal
FILTER-- How much time allocated to instruction? Children actively
engaged? Standards introduced? Effective instruction occurring?
Upset parent
DATA on Learning
Check on health dept
Check on police interview
Goal Setting
Etc.
Teacher Evaluation
Allocation of Instructional Resources
Great Instructional Leaders
• Have a filter
• Allocate time and resources according to
their filter
• Use an AIMLINE
• Have a framework for making data-driven
decisions (know how to access the data
they need to reach timely decisions)
• Hold teachers, staff, students accountable
• Research findings on effective schooling
Great School Psychologists
• Hand the principal the data the principal did not
know to ask for but can’t live without
• Follow the aimline and attend to implementation
integrity
• Understand the variables of effective instruction
and engage in contextualized assessment that is
technically valid for the purposes needed AND
has treatment utility
• Minimizes meeting time and “avoids the science
of strange behavior…”
Great Districts
• Minimize time away from school, but use time together to
review school improvement implementation efforts and
ongoing results
• Have the will to proactively chart the course of a district
• Provide adequate resources and space for principals to be
effective instructional leaders and hold them accountable for
results
• Respect the role of parents and actively engage them
• Have a framework for evaluating results (know how to
access data for decision making)
• Evaluate quality of all programs locally and make decisions
about continued use based on DATA.
Great Teachers
• Use data to identify where
more/different/less instruction is needed
• Have as a goal to accelerate all learning of
all children
• Proactively address barriers to learning
• Take responsibility for learning that occurs
in the classroom
• Are confident and ready to collaborate in
the classroom
• Appreciate childhood and children (a little
humor, lots of patience, enthusiasm)
For More Information
• [email protected]
• www.isteep.com
• Thank you to the US Dept of Education for
providing all film clips shown in this
presentation