Transcript Tips for Using This Template

Results Driven Accountability and
Intensive Intervention:
Using MTSS to Improve Outcomes for
Students with Disabilities
Sarah Arden, Laura Berry Kuchle, Christopher Lemons, T. Chris Riley-Tillman
This document was produced under U.S. Department of Education, Office of Special Education Programs, Award No. H326Q110005. Celia Rosenquist serves as the project officer. The views expressed herein do not necessarily represent the positions
or policies of the U.S. Department of Education. No official endorsement by the U.S. Department of Education of any product, commodity, service or enterprise mentioned in this document is intended or should be inferred.
A note about questions…
Please type
questions related to
technical issues in
the Chat box.
Please type
questions related to
webinar content in
the Q&A box.
Panelists
Sarah V. Arden, Ph.D. – Researcher and Technical Assistance
Liaison at American Institutes for Research
Laura Berry Kuchle, Ph.D. – Researcher and Technical
Assistance Liaison at American Institutes for Research
Chris Lemons, Ph.D. – Senior Advisor to the National Center on Intensive
Intervention and Assistant Professor in the Department of Special Education at
Peabody College of Vanderbilt University
T. Chris Riley-Tillman, Ph.D. – Trainer for the National Center
on Intensive Intervention and Professor at the University of
Missouri College of Education
Session Overview







What is the vision of RDA
How is intensive intervention part of RDA
Overview of National Center on Systemic Improvement
Introduction to DBI
Why Do We Need to Assess DBI Implementation?
DBI Implementation Rubric
Lessons Learned: Strategies for Improving Technical
Assistance and DBI Implementation
 NCII Resources to Support Implementation
Results-Driven Accountability: Vision
All components of an accountability system will be aligned in a
manner that best support states in improving results for infants,
toddlers, children, and youth with disabilities and their families.
Shift from Compliance to Results + Compliance
Slide adapted from: OSEP Slides to
Explain Results Driven Accountability
(RDA) Retrieved from
http://www2.ed.gov/about/offices/list/osers/
osep/rda/index.html
5
• Evaluation of progress annually
• Adjust plan as needed
SSIP
Phase III
SSIP
How well is
the solution
working?
• Initiate Data Analysis
• Conduct broad
Infrastructure Analysis
• Identify problem area
What is the
problem?
SSIP
Phase I
SSIP
SSIP
Phase I and II
• Search/evaluate
evidencebased solutions
(Exploration Phase)
What shall
we do
about it?
• Develop action steps
(address barriers/use
leverage points)
• Develop Theory of Action
• Develop Plan for Improvement
(Implementation Framework)
SSIP
Phase I
• Conduct root
cause analysis
(including
infrastructure) to
identify
contributing
factors
• For each contributing
factor, identify both
barriers and leverage
points for improvement
Why is it
happening?
Slide from: OSEP Slides to Explain Results Driven
Accountability (RDA) Retrieved from
http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html
6
State Systemic Improvement Plan
Year 1—FFY 2013
Delivered by April 2015
Year 2—FFY 2014
Delivered by Feb. 2016
Years 3–6—FFY 2015–18
Feb. 2017–Feb. 2020
Phase I Analysis
Phase II Plan
Phase III Evaluation
 Data analysis
 Multiyear plan addressing:
 Reporting on progress
including:
 Infrastructure
analysis
 State-identified
measureable result
 Coherent
improvement
strategies
• Infrastructure development
• Support early intervening
services program and local
education agencies in
implementing evidencebased practices
• Results of ongoing
evaluation
• Extent of progress
 Revisions to the State
Performance Plan
• Evaluation plan
 Theory of action
Slide from: OSEP Slides to Explain
Results Driven Accountability
(RDA) Retrieved from
http://www2.ed.gov/about/offices/list/oser
s/osep/rda/index.html
7
State-identified Measurable
Result(s)
 State-identified Measurable Result(s) (SiMR)
• A child-level (or family-level, for Part C) outcome
• Not a process or system result
• May be a single result or a cluster of related results
 Identified based on analysis of data
8
On What Are States Focusing?
• Graduation: AK, DC, FL, GA, MN, MT, NC, ND, NJ, PA, RMI, VA, WV
• Reading/ELA: AR, AS, AZ, CNMI, CO, CT, DE, FSM, GU, HI, IA, ID, IL, IN,
KS, LA, MI, MS, NE, NV, NM, NY, OH, OK, OR, Palau, SC, SD, TN, TX, VI,
WA, WI, WY
• Math: KY, MD, ME, PR, RI, UT, VT
• Reading and Math: CA, MO
• Early Childhood Outcomes: MA, NH
• Post-school Outcomes: AL, BIE
Variations: Disability category; race/ethnicity; gender; grades; English learner;
poverty status; subset of districts
9
Collaboration between Centers
 Led by WestEd
 Partners:
•
•
•
•
•
•
AIR
NASDSE
Learn more at http://ncsi.wested.org/
SRI
CCSSO
Ask NCSI: http://ncsi.wested.org/contact/
NCSI Resources: http://ncsi.wested.org/resources/
Parent Center Network
Meadows Center for Preventing
Educational Risk (evaluator)
10
Targeted Outcomes of NCSI
1. Increase the capacity of state education agencies (SEAs) and lead
agencies (LAs) to develop, implement, and evaluate their State
Systemic Improvement Plans (SSIPs)
2. Increase SEAs’ and LAs’ knowledge, selection, and utilization of
evidence-based practices (EBPs)
3. Improve SEA and LA infrastructure and coordination within SEAs
and LAs for delivering effective technical assistance (TA) on
implementing and scaling-up effective strategies, stakeholder
engagement, resource mapping and allocation, and instructional
collaboration
11
Targeted Outcomes of NCSI Cont.
4. Increase the use of effective dissemination strategies by SEAs
and LAs to ensure that local education agencies (LEAs) and
early intervention service (EIS) programs and providers have
access to EBPs and can select and implement those EBPs in a
sustainable manner
5. Increase the effectiveness of SEAs and LAs to meaningfully
engage State and local stakeholders in the development and
implementation of the SSIP
12
NCSI Approach to TA
13
Why Is This Important?
 Meeting SiMR goals will require a focus on improving
instruction.
 States will be in need of support on how to provide
intensive intervention for the kids who need it the most,
including:
• Evidence-based intervention strategies
• Overcoming implementation barriers
• Making connections to other state, district, and school initiatives
14
What Does This Mean for the Field?
 Data Use
• Knowing what data to look at…accessing those data…root cause analysis to make
sense of the data
 Knowledge Utilization
• Selecting coherent/aligned evidence-based practices…implementing them with
fidelity…scaling up and sustaining them
 Systems Change
• Infrastructure considerations…resource mapping and alignment…policies that support
the goals
 Communication & Collaboration
• Problem solving and planning with diverse stakeholders, in the right settings, at the
right time…and working together for implementation to achieve results
15
What is Intensive Intervention
and Why Do We Need to
Assess Fidelity of
Implementation?
16
What is Intensive Intervention?
Intensive intervention addresses severe and
persistent learning or behavior difficulties. Intensive
intervention should be
 Driven by data
 Characterized by increased intensity (e.g., smaller group,
expanded time) and individualization of academic
instruction and/or behavioral supports
17
What Intensive Intervention…
Is…
Is Not…
 Individualized based on
student needs
 More intense, often with
substantively different
content AND pedagogy
 Comprised of more
frequent and precise
progress monitoring




A single approach
A manual
A preset program
More of the same Tier 1
instruction
 More of the same Tier 2
instruction
18
Why Do We Need
Intensive Intervention?
Low academic achievement
Dropout rates
Arrest rates
19
What is NCII’s Approach to
Intensive Intervention?
Data-Based Individualization (DBI): A systematic method
for using data to determine when and how to provide more
intensive intervention:
 Origins in data-based program modification/experimental teaching
were first developed at the University of Minnesota (Deno & Mirkin,
1977).
 It is a process, not a single intervention program or strategy.
 It is not a one-time fix, but an ongoing process comprising intervention
and assessment adjusted over time.
20
DBI: Integrating
data-based
decisionmaking across
academics and
social behavior
21
Is DBI the same as MTSS?
Is DBI special education?
 Many components of DBI are consistent with elements of
special education and tiered service delivery systems
Tiered Interventions (RTI,
MTSS, PBIS)
• Universal, secondary, and
tertiary interventions
• Progress monitoring
• Team-based decisions based
on data
Special Education
• Individualized program
• Progress monitoring
• Team-based decisions
based on data
22
Who Needs DBI?
 Students with disabilities who are not making adequate
progress in their current instructional program
 Students who present with very low academic achievement
and/or high-intensity or high-frequency behavior problems
(typically those with disabilities)
 Students in a tiered intervention system who have not
responded to secondary intervention programs delivered
with fidelity
23
What Is Fidelity?
 Extent to which an intervention’s core components have
been implemented as planned (Nelson et al., 2012)
 For DBI, this includes the instructional platform, adapted
iterations of intervention, ongoing progress monitoring, and
decision-making procedures.
 See IRIS module for additional information:
http://iris.peabody.vanderbilt.edu/module/fid/
24
Levels of Fidelity
 Student level
• For a given student, are assessment and intervention being carried out as
planned?
• Student-level fidelity may be assessed through teacher logs, observations,
or record review
 Systems level (school or team)
• Are essential components of DBI being implemented consistently?
• Are there systems-level problems that hinder DBI implementation?
• A systems-level rubric or interview might examine team meeting checklists,
logs, or other records
25
Why Does Fidelity Matter?
 Fidelity assessment provides evidence that DBI is being
implemented as intended
• Assessment, intervention, and decision-making processes
• Student and systems levels
 Provides guidance on how to improve DBI implementation
• Does the interventionist need additional training or support?
• Is an intervention adaptation needed?
• Is there a systems-level problem? (e.g., scheduling prevents sufficient
intervention time, staff do not have access to evidence-based instructional
platforms)
26
Developing a Rubric to
Measure Implementation
Fidelity
27
Developing a Systems-Level Rubric
 Based on the structure of the Center on Response to
Intervention’s RTI Essential Components Integrity Rubric
http://www.rti4success.org/resource/essential-componentsrti-integrity-rubric-and-worksheet
 Identify structures, resources, and practices needed for
quality school-level implementation of DBI
 Reflect lessons learned from NCII knowledge development
sites
28
Measuring School-Level
Implementation
 NCII’s DBI Implementation Rubric and Interview:
http://www.intensiveintervention.org/resource/dbiimplementation-rubric-and-interview
 Aligned with the essential components of DBI and the
infrastructure that is necessary for successful
implementation in Grades K–6
29
Key Findings From Knowledge
Development Sites
 Purpose: to learn about strategies for implementing
intensive intervention from sites that have demonstrated
positive outcomes for students with disabilities (SWDs)
 Implementing Intensive Intervention: Lessons Learned
From the Field (NCII, 2013)
http://www.intensiveintervention.org/resource/implementing
-intensive-intervention-lessons-learned-field
30
Lessons Learned
1. Intensive intervention is most likely to be facilitated when
implemented as a component of a multi-tiered system of
support (MTSS)
2. Family engagement supports implementation
3. Intensive behavioral intervention is often more
challenging because of limited progress monitoring
4. Inconsistent decision rules for intensifying supports can
hinder intervention planning and resource allocation
5. Hidden inefficiencies may overtax staff unnecessarily
31
Components/Infrastructure Required
for DBI Implementation
1.
2.
3.
4.
5.
System Features
Data and Decision Making
Intervention
DBI Process
DBI Evaluation
32
1. System Features
a)
b)
c)
d)
e)
f)
Leadership
School Schedules
Resources
Cultural and Linguistic Responsiveness
Communication With and Involvement of Parents
Communication With and Involvement of All Staff
33
2. Data and Decision Making
a) Process to Identify Students in Need of Intensive
Intervention
b) Academic Progress-Monitoring Tools
c) Behavior Progress-Monitoring Tools
d) Progress-Monitoring Procedures
e) Diagnostic Assessment
f) Fidelity of Assessment Implementation
34
3. Intervention
a)
b)
c)
d)
e)
Intensive Intervention
Adaptation of Intervention
Fidelity of Implementation of Intervention
Interventionist Characteristics
Relationship to Core Instruction
35
4. DBI Process
a)
b)
c)
d)
e)
f)
Team or Appropriate Support Mechanism
Regular Meetings
Data Review
Documentation
Decision Rules
Overall Implementation of DBI Process
36
5. DBI Evaluation
a) Evaluation
37
Rating Scale
1–5 points with anchors for
 1 point = little or no implementation
 3 points = partial or inconsistent implementation
 5 points = complete and consistent implementation
38
DBI Implementation Interview
 Script and note-taking template for gathering information to
evaluate rubric
 Sample questions for each rubric item
39
Lessons Learned: Initial use in
NCII Intensive Sites
Strategies for Improving DBI Implementation
40
Highest Rated Items
 Trained, experienced interventionists
 Teams to support decision making for DBI, with a regular
meeting schedule
 Aligning intervention to student need and core
expectations, addressing prerequisite skills as appropriate
 District and school leadership support for DBI
 Technically rigorous academic progress-monitoring tools
that are sensitive to student improvement
41
Lowest Rated Items
Evaluation
 Schools may informally review implementation without
setting specific goals
 Evaluation may be difficult or a low priority if other pieces
are not clearly articulated and in place
Behavior progress monitoring
 Fewer validated tools available compared to academics
 New to many schools
42
Lowest Rated Items Continued….
Fidelity
 Few schools systematically monitor
Decision rules
 May not be clearly articulated in one or more areas
 Application may not be consistent
Overall implementation
 May be inconsistent or poorly documented
43
Essential Elements of DBI
Implementation





Staff commitment
Student plans
Student meetings
Valid, reliable data
Inclusion of students with disabilities
44
Staff Commitment
Key Element
Commitment of:
 Principal
 Intervention staff
 Special educators
Flexibility Within
Implementation
 Specific intervention
staff involved including
staff who work with
students with intensive
needs in the area(s) of
concern. (e.g., reading
specialists,
social workers)
Student Plans
Key Element
Student plans are developed and
reflect:
 Accurate and timely student
data
 Goal(s) for the intervention
based on valid, reliable
assessment tools
 Timeline for executing and
revisiting the intervention plan
Flexibility Within
Implementation
 Content area(s)
 Number of student plans
 Grade level(s)
46
Student Meetings
Key Element
 Student meetings are
data driven
 There is a regularly
scheduled time to meet
 Meetings are structured
to maximize efficiency
and focused problem
solving
Flexibility Within
Implementation
 Frequency
 Schedule
 Team members
47
Progress Monitoring
Key Element
Flexibility Within
Implementation
 Valid, reliable progress
 Choice of tool
monitoring tools are used.  Use of progress Data are graphed.
monitoring data at
 Data are collected at
other tiers
regular intervals.
48
Students With Disabilities
Key Element
 Students with disabilities
must have access to
intensive intervention.
Flexibility Within
Implementation
 Who delivers intervention
for students with
disabilities
 Inclusion of students with
and without IEPs
49
NCII & NCSI Resources to
Support Implementation
50
Addressing Common Barriers to
Implementation
 Systems level considerations—infrastructure and staff
commitment
• Module to assess and support readiness to implement DBI
 Identifying intervention and assessment tools for
mathematics and behavior
• Tools charts
• Sample lessons and activities
 Consistent procedures and documentation
• Student meeting tools
51
Getting Ready to Implement Intensive
Intervention: Infrastructure for DBI
 Module introducing schools to DBI and infrastructure
needed for implementation:
http://www.intensiveintervention.org/resource/gettingready-implement-intensive-intervention-infrastructure-databased-individualization
 Handouts and activities to assess readiness and begin
action planning
52
Fidelity Resources
http://www.intensiveintervention.org/fidelity-resources
53
Fidelity Resources Continued
http://www.rti4success.org/resource/essential-componentsrti-integrity-rubric-and-worksheet
54
Tools
Charts
Academic Progress Monitoring
http://www.intensiveintervention.org/char
t/progress-monitoring
Academic Intervention
http://www.intensiveintervention.org/char
t/instructional-intervention-tools
Behavioral Progress Monitoring
http://www.intensiveintervention.org/char
t/behavioral-progress-monitoring-tools
Behavioral Intervention
http://www.intensiveintervention.org/char
t/behavioral-intervention-chart
Mathematics:
Sample
Lessons &
Activities
http://www.intensiveintervention.org/resources/sample-lessonsactivities/mathematics
56
Implementing Behavioral Strategies:
Considerations and Sample Resources
http://www.intensiveintervention.org/implementing-behavioral-strategiesconsiderations-and-sample-resources
57
Individual Student Meeting Tools
http://www.intensiveintervention.org/tools-support-intensive-interventiondata-meetings
58
Learn More About DBI





NCII website
Examples of Standards-Aligned Instruction Across Tiers
DBI Training Series
Webinars
Connect to NCII
59
Find Out What NCII Has to Offer
www.intensiveintervention.org
60
Examples of Standards-Aligned Instruction Across Tiers
http://www.intensiveintervention.org/illustration-standards-relevant-instructionacross-levels-tiered-system
61
DBI Training Series
 Eight training modules focusing on components of DBI for
academics and behavior
 One module focused on
readiness and action
planning
 Include
 Slides and speaker notes
 Activities
 Coaching guides
http://www.intensiveintervention.org/content/dbi-training-series
62
Webinars
View archived
webinars and look for
announcements about
the next live webinar:
www.intensiveinterven
tion.org
63
Connect to NCII
 Sign up on our website
to receive our newsletter
and announcements
 Follow us on YouTube
and Twitter
• YouTube Channel:
National Center on
Intensive Intervention
• Twitter handle: @TheNCII
64
Connect to NCSI
 This website provides immediate, easy
access to foundational NCSI information,
previously developed technical assistance
resources, a snapshot of the NCSI team,
and contact information.
 www.ncsi.wested.org/contact/
65
Ask the NCSI
 Ask the NCSI is intended to support
states to (1) obtain information about
evidence-based practices;
 (2) develop, implement and evaluate
State Systemic Improvement Plans
(SSIPs);
 (3) learn about practices being
implemented in other states; and
 (4) find out what current research
says about “what works” to improve
results for children with disabilities.
 www.ncsi.wested.org/contact/
66
Questions
67
References
National Center on Intensive Intervention. (2013). Implementing intensive
intervention: Lessons learned from the field. Washington, DC: U.S.
Department of Education, Office of Special Education Programs.
http://www.intensiveintervention.org/sites/default/files/Lessons_Learned_Fro
m_Field_0.pdf
Nelson, M. C., Cordray, D. S., Hulleman, C. S., & Sommer, E. C. (2012). A
procedure for assessing intervention fidelity in experiments testing
educational and behavioral interventions. The Journal of Behavioral Health
Services & Research, 39(4), 374–396.
68
NCII & NCSI Disclaimers
 This presentation was produced under the U.S. Department of
Education, Office of Special Education Programs, Award No.
H326Q110005 for which Celia Rosenquist serves as the project officer
and No. H326R140006 for which Perry Williams and Shedeh
Hajghassemali serve as project officers. The views expressed herein
do not necessarily represent the positions or polices of the U.S.
Department of Education. No official endorsement by the U.S.
Department of Education of any product, commodity, service, or
enterprise mentioned in this website is intended or should
be inferred.
69
Although permission to redistribute this webinar is not necessary,
the citation should be: National Center on Intensive Intervention
and National Center on Systemic Improvement (2015). Results
Driven Accountability and Intensive Intervention: Using MTSS to
Improve Outcomes for Students with Disabilities. Washington, DC:
U.S. Department of Education, Office of Special Education
Programs, National Center on Intensive Intervention.
National Center on Intensive
Intervention
1000 Thomas Jefferson Street NW
Washington, DC 20007-3835
www.intensiveintervention.org
[email protected]
@TheNCII
National Center on Systemic
Improvement
730 Harrison St
San Francisco, CA 94107
http://ncsi.wested.org/
[email protected]