Transcript Slide 1

Response to Intervention:
Using Data to Enhance
Outcomes for all Students
Amanda VanDerHeyden
Education Research and Consulting, Inc.
16 x 3 = 48 hours
Data allow us to
• Provide faster, more effective services for ALL
children
• Work “smarter” not harder, better utilize the
talents of the school psychologist and schoolbased assessment and intervention teams.
• Make implementation SIMPLE and EASY for
teachers (low cost, few errors)
• Prevent diagnosis
What is RTI?
• A science of decision making and way of
thinking about how educational resources
can be allocated (or reallocated) to best
help all children learn
• Major premium on child outcomes
STEEP Model
Screening to Enhance Educational Progress
Tier 1: Screening
• Screening
– Math Screening
• 2 minutes. Scored for Digits Correct
– Writing Screening
• 3 Minutes. Scored for Words Written Correctly
– Reading Screening
• 1 Minute. Scored for Words Read Correctly
Class-wide Screening
QuickTime™ and a
Cinepak decompressor
are needed to see this picture.
Feedback to Teachers
Tier 2: Class-wide Intervention
Digits Correct in Two Minutes
Intervention
Baseline
120
100
m
a
s
t
e
r
y
80
60
40
instructional range
20
0
1
2
3
Sessions
4
5
Mary
Chiquita
Randy
Sandy
Brandy
Colvin
Jolisha
Daleesha
Kiera
Bradley
Jared
Alfred
Sienna
Jarian
Trey
Robert
Andrea
Ashley
Jaren
No Class-wide Problem Detected
Tier 2: Can’t Do/Won’t Do
Assessment
• “Can’t Do/Won’t Do”
• Individually-administered
• Materials
3-7 minutes per
child
– Academic material that student performed poorly
during class assessment.
– Treasure chest: plastic box filled with tangible items.
Can’t Do/Won’t Do Assessment
QuickTime™ and a
Cinepak decompressor
are needed to see this picture.
Decision Rule Following Can’t
Do/Won’t Do Assessment
Tier 3: Individual Intervention
QuickTime™ and a
Cinepak decompressor
are needed to see this picture.
#Correct
Response to Intervention
Before
Intervention
During Intervention
Avg. for his Class
Each Dot is one
Day of Intervention
Intervention Sessions
Intervention in Reading
#Correct
Response to Intervention
Before
Intervention
During Intervention
Avg. for his Class
0
Breaux Aisha S
Chapdelain Lily M
Robb Sydney M
Mangione Nikole M
White Bailey V
Ramey Sara B
Littsen Lucas R
Foxhoven Shane A
Gonzales Audryana
Foley Thomas J
Hermes Jay M
Thueson Lila D
Svob Seth H
Salsbury Mariah A
Wills Donna D
Blakeley Brandee E
Kellogg Anthony S
Negrete Sara M
Ashton Julia E
Dailey Brandon L
Showers Phillip J
Counes James G
Hilkemeyer Austin R
Howe Ashley B
Strider Katie N
Santa cruz Daniel R
Gallego Angela M
Lewandowski
Sisk Cody A
Forsyth Ian E
Blake Nicholas K
Hatch Vanessa L
Machain Anthony F
Peterson Tyler L
White Alexa L
Nutbrown Jordan C
Bluemke Megan J
Casamasa Gregory L
Roche Alyssa R
Elias Elizabeth L
Beeston Kristine D
Lopez Theresa A
Pierce Shannon M
Lanier Matthew W
Nanna Caitlin N
Smith Shelby N
Iturralde Jacqueline R
Mcharg Jordan E
Brechbiel Shari L
Cota Alexia K
Jackson Damion M
Lamadrid Leonardo
Oliver Riley W
Layton Marissa M
Mueller Lane E
Turner Alana K
Rowlan Paige E
Dumes Scott M
Riordan Timothy D
Hicks Coltin C
Kenton Chelsee M
Cornwell Kimberly M
Crater Shelbie M
Rytting Ryan C
French Joshua M
Davila Ariel N
Thompson Tasha N
Ryckman Shelby L
Ayers Megan L
Dunham Clayton J
Mausert James R
Morales Eric A
Bain Bryce G
Gryczkowski samluk
Cuff Matthew J
Hackman Lindsey S
Whitlock John C
Benson Brad J
Stanfield Benjamin C
Martinez Nathaniel P
Carrizosa Robert A
Webb Brianna J
Evans Joseph B
Laye Lestot D
Bazzanella Stephen L
Crowl Robert S
Thompson Krysta E
Tipton Emily N
Meyer Sean M
Couture Anne S
Gibbons Cody D
Fuhrman Autum C
Nolen Jayd L
Drake Justin T
Neale Shaine R
Purcell John E
Rugotska Colton J
Bong Samantha L
Johnson Amanda L
Rodriguez Ryan T
Bergstrom Matthew A
Pannell Marina S
Digits Correct Two Minutes
Vehicle for System Change:
System-wide Math Problem
4th Grade Math
Multiplication 0-9
120
100
80
60
Instructional range
40
20
Frustrational range
Each bar is a student’s performance
0
Neale
Frost Joshua
Franklin
Ryckman
Turner Alana
Ayers
Smith
Montano
Kenton
Banken
Mausert
White Alexa
Brechbiel
Hatch
Meyer Sean
Santa cruz
Oliver Riley
Cornwell
Wills Donna
Bluemke
Williams
Cuff Matthew
Nanna
Hilkemeyer
Robb
Gryczkowski
Gonzales
Mangione
Foxhoven
Layton
Negrete
Tipton Emily
Gavino
Lamb Nicole
Chapdelain
Roche
Digits Correct Two Minutes
Re-screening Indicates No
Systemic Problem
Fourth Grade
Fourth Grade Multiplication 0-9
140
120
100
80
60
40
20
Effect on SAT-9 Performance
SAT-9 Standard Scores and t-test Results for Pre- and Post-Implementation Years by Grade
2001-2002
Grade
n
Third
85
Fourth
2002-2003
M
t
SD
n
M
SD
562.06
143.80
129
602.54
35.20
3.07**
116
611.09
120.61
117
638.22
33.39
2.35*
Fifth
113
636.73
109.86
107
659.17
35.77
2.01*
Total
314
607.04
126.83
353
631.53
41.93
3.42**
* p < .05
** p < .01
Cohen’s d (effect size between years)
Third .45
Fourth .35
Fifth
.31
Total .29
VanDerHeyden & Burns, 2005
Effect on CBM Scores
Mean Digits Correct/2 Minutes Scores for Monthly Mathematics Probes
January
February
March
Grade
M
M
SD
M
SD
M
SD
F
Third
27.9
9.9
37.9
13.4
35.3
13.0
38.8
12.3
13.45*
Fourth
39.7
17.6
50.1
22.0
53.4
25.3
58.9
27.0
35.02*
Fifth
47.6
22.9
50.8
22.7
52.0
24.8
59.3
25.1
25.52*
Total
41.1
20.5
48.0
21.5
49.2
24.0
55.1
25.1
64.29*
SD
April
* p < .001
Cohen’s d (effect size between January and April scores)
Third .97
Fourth .86
Fifth
.49
VanDerHeyden & Burns, 2005
Computation Gains Generalized to
High Stakes Test
Improvements
(Gains within Multiple Baseline
shown as pre-post data)
Percent of Students Passing HighStakes Mathematics
3rd Grade Math Performance
100
90
80
70
60
50
40
30
20
10
0
School 1
School 2
School 3
School 4
School 5
Gains within Multiple Baseline
(shown as pre-post data)
Percent of Students Passing HighStakes Mathematics
5th Grade Math Performance
100
90
80
70
60
50
40
30
20
10
0
School 1
School 2
School 3
School 4
School 5
District-wide Implementation Data
• Vail Unified School District
– www.vail.k12.az.us
• Three years, system-wide implementation
of STEEP grades 1-8
System Outcomes
• Referrals reduced greater than half
• % who qualify improved at 4 of 5 schools
• SLD down from 6% of children in district in
2001-2002 (with baseline upward trend) to
3.5% in 2003-2004 school year
• Corresponding gains on high-stakes tests
(VanDerHeyden & Burns, 2005)
• Intervention successful for about 95 to 98%
of children screened
VanDerHeyden, Witt, & Gilbertson, 2007
Cost Reduction
160,000
Cost in Dollars
140,000
120,000
100,000
Baseline
STEEP
80,000
60,000
40,000
20,000
0
Assessment
Placement
VanDerHeyden, Witt, & Gilbertson, 2007
Findings
• Diverse settings, psychologists of diverse
backgrounds and no prior experience with
CBM or functional academic assessment
• Disproportionate representation of males
positively affected
VanDerHeyden, Witt, & Gilbertson, 2007
Team Decision-Making Agreement
RTI + and
Evaluated
RTI- and Did
Not Evaluate
2003-2004
(3 schools)
100%
41%
2004-2005
(5 schools)
100%
87%
VanDerHeyden, Witt, & Gilbertson, 2007
Team Decision-Making
Baseline
STEEP + and Team
STEEP- and Team
Decided to Evaluate
Decided to Evaluate
2003-2004 Cases, Schools 1-3 55%
89%1
50%2
2004-2005 Cases, Schools 1-5 52%
88%3
29%4
VanDerHeyden, Witt, & Gilbertson, 2007
Identification Accuracy
CBA + RTI Criterion
STEEP
WJ-R
Sensitivity
.76
1
.58
Specificity
.89
.99
.77
Positive Predictive Power
.59
.67
.44
.95
1
.86
Sensitivity
.46
.33
.42
Specificity
.69
.94
.85
Positive Predictive Power
.19
.17
.45
.89
.97
.83
Negative Predictive Power
Teacher Referral
ITBS
Negative Predictive Power
VanDerHeyden, et al., 2003
Percent Identified at each Tier
Identified
CBM (Classwide Assessment)
55 (15%)
CBM + Reward (Performance/skill Deficit Assessment)
40 (11%)
CBM + Reward + Instruction
(STEEP +)
22 (6%)
Teacher Referral
32 (19%)
CIBS-R
64 (18%)
DRA
17 (9%)
RTI Criterion Assessment
17 (5%)
WJ-R
ITBS deficit
12
3 (4%)
VanDerHeyden, et al., 2003
Percent of Minority and Caucasian
Students in Risk Category
What Proportion of Ethnicity Represented
Before and After Intervention in Risk
Category?
100
90
80
70
60
Minority
Caucasian
50
40
30
20
10
0
Before
Intervention
After Intervention
Expected
VanDerHeyden & Witt, 2005
Screening tells you
• How is the core instruction working?
• What problems might exist that could be
addressed?
• Most bang-for-the-buck activity
• Next most high-yield activity is classwide
intervention at Tier 2.
Any
Curriculum
Area
Academic Systems
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•High Intensity
•Of longer duration
1-5%
80-90%
5-10%
Students
Universal Interventions
•All students
•Preventive, proactive
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•Intense, durable procedures
1-5%
5-10%
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
Behavioral Systems
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
80-90%
Universal Interventions
•All settings, all students
•Preventive, proactive
Dave Tilly, 2005
“Weighing a cow doesn’t
make it fatter.”
Any
Curriculum
Area
Academic Systems
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•High Intensity
•Of longer duration
1-5%
80-90%
5-10%
Students
Universal Interventions
•All students
•Preventive, proactive
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•Intense, durable procedures
1-5%
5-10%
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
Behavioral Systems
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
80-90%
Universal Interventions
•All settings, all students
•Preventive, proactive
Dave Tilly, 2005
Intervention Plan- 15 Min per Day
• Protocol-based classwide peer tutoring,
randomized integrity checks by direct
observation
• Model, Guide Practice, Independent Timed
Practice with delayed error correction
• Group performance contingency
• Teachers encouraged to
– Scan papers for high error rates
– Do 5-min re-teach for those with high-error rates
– Provide applied practice using mastery-level
computational skill
Math Sample Sequence
3RD GRADE
1. addition and subtraction facts 0-20
2. fact families addition and subtraction 0-20
3. 3 digit addition without and with regrouping
4. 3 digit subtraction without and with regrouping
5. 2 and 3 digit addition and subtraction
6.
7.
8.
9.
with and without regrouping
multiplication facts 0-9
division facts 0-9
fact families multiplication and division 0-9
add/subtract fractions with like denominators
flash cards
practice set – same as skill
practice set – same as skill
practice set – same as skill
practice set – same as skill
flash cards
flash cards
practice set – same as skill
practice set – same as skill
(3rds, 4ths, 8ths, 10ths, no regrouping)
10. single digit multiplied by double/triple digit
practice set – same as skill
without regrouping
11. single digit multiplied by double/triple digit
practice set – same as skill
with regrouping
12. single digit divided into double/triple digit
without remainders
13. add and subtract decimals to the hundredths
practice set – same as skill
practice set – same as skill
Class-wide Math Intervention
QuickTime™ and a
Cinepak decompressor
are needed to see this picture.
Class 1 at Screening
Class 1: Following 10 Days
Intervention
Class 1: Following 15 Days
Intervention
Class 2 at Screening
Class 2: Following 5 Days
Intervention
Class 2: Following 10 days
Intervention
Class 3 at Screening
Class 3: Following 5 days
Intervention
Following 10 Days Intervention
Any
Curriculum
Area
Academic Systems
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•High Intensity
•Of longer duration
1-5%
80-90%
5-10%
Students
Universal Interventions
•All students
•Preventive, proactive
Intensive, Individual Interventions
•Individual Students
•Assessment-based
•Intense, durable procedures
1-5%
5-10%
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
Behavioral Systems
Targeted Group Interventions
•Some students (at-risk)
•High efficiency
•Rapid response
80-90%
Universal Interventions
•All settings, all students
•Preventive, proactive
Dave Tilly, 2005
Tier 3
• Assessment Data
– Instructional level performance
– Error analysis (high errors, low errors, pattern)
– Effect of incentives, practice, easier task
– Verify intervention effect
• Same implementation support as Tier 2
• Instructional-level materials; Criterion-level
materials
Tier 3
• Implement for 5-15 consecutive sessions
with 100% integrity
• Link to referral decision
• Weekly graphs to teacher and weekly
generalization probes outside of
classroom, supply new materials
• Troubleshoot implementation weekly
Tier 3 Intervention
• >5% of children screened (total
population) IF solid Tier 1
• Possibly as low as 2% IF solid Tier 1 and
Tier 2
• About 1-2% failed RTI; 10% of most at-risk
VanDerHeyden et al., 2007
Principal
FILTER-- How much time allocated to instruction? Children actively
engaged? Standards introduced? Effective instruction occurring?
Upset parent
DATA on Learning
Check on health dept
Check on police interview
Goal Setting
Etc.
Teacher Evaluation
Allocation of Instructional Resources
Great Implementers
• Follow the aimline and attend to implementation
integrity
• Understand the variables of effective instruction and
engage in contextualized assessment that is
technically valid for the purposes needed AND has
treatment utility
• Minimize meeting time and avoid “the science of
strange behavior…”
• Provide adequate resources and space for
principals to be effective instructional leaders and
hold them accountable for results
• Evaluate quality of all programs locally and make
decisions about continued use based on DATA.
For More Information
• [email protected]
• www.isteep.com
• Thank you to the US Dept of Education for
providing all film clips shown in this
presentation