Transcript Using Implementation Frameworks to Identify Evidence Based Practices 2011 PBIS National Forum Rosemont IL October 27, 2011 Barbara Sims Debbie Egan Dean L.

Using Implementation
Frameworks to Identify
Evidence Based Practices
2011 PBIS National Forum
Rosemont IL
October 27, 2011
Barbara Sims
Debbie Egan
Dean L. Fixsen
Karen A. Blase
Michelle A. Duda
Implementation
Successful programs do not contain
the seeds of their own replication.
– Lisbeth Schorr, 1993
Implementation Gap
RESEARCH
IMPLEMENTATION
PRACTICE
GAP
Implementation is defined as a specified set
of activities designed to put into practice an
activity or program of known dimensions.
Implementation Gap
Why Focus on Implementation?
RESEARCH
IMPLEMENTATION
PRACTICE
GAP
“Students cannot benefit from
interventions they do not experience.”
Implementation Science
IMPLEMENTATION
INTERVENTION
Effective
Effective
NOT Effective
from Mark Lipsey’s 2009 Metaanalytic overview
of the primary
Inconsistent;
factors that characterize effective
Not Sustainable;
juvenile offender interventions –
Actual
Poor analyses,
outcomesthe
Benefits“. . . in some
quality with which the
intervention is implemented
outcomes;
as strongly
related
NOT Effective Unpredictable orhas beenPoor
poor outcomes;to recidivism
Sometimes
harmful
effects as
the
type of program, so much so
that a well-implemented
intervention of an inherently
less
type can on
(Institute of Medicine, 2000; 2001; 2009;
Newefficacious
Freedom Commission
outperform
a more
Mental Health, 2003; National Commission
on Excellence
in efficacious
that is Services,
poorly 1999)
Education,1983; Department of Health one
and Human
implemented.”
Who’s Accountable for Learning?
Student
Practitioner
System
Active Implementation

Letting “It” happen. . .


Helping “It” happen. . .


Innovation occurs without intervention
Interested innovators figure it out on their own
Making “It” happen. . .

Active use of strategies to support the adoption
of the innovation

Active installation of supports for the
implementation of the innovation
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004

Active Implementation
Frameworks
Successful implementation on a
useful scale requires. . .




Purposeful matching of critical implementation
activities to the stage of the process –
“STAGES OF IMPLEMENTATION”
Active use of implementation core components
“best practices”– “IMPLEMENTATION
DRIVERS”
Organized, expert assistance –
“IMPLEMENTATION TEAMS”
A focus on continuous, purposeful
improvement – “IMPROVEMENT
PROCESSES”
Implementation Stages
2 - 4 Years
Exploration
Installation
• Assess needs
• Examine
Acquire resources
Prepare
innovations
• Examine
Implementation
• Assess fit
organization
Prepare
implementation
Prepare staff
Initial
Implementation
Implementation
drivers
Manage change
Data systems
Improvement
cycles
Full Implementation
Implementation
drivers
Implementation
outcomes
Innovation outcomes
Standard practice
Exploration: The Big Picture
What happens during Exploration?





Formalize Team Structures
Determine Need and Identify Options
Assess “Fit” and Feasibility
Promote “Buy in” for the innovation and for
implementation supports
Re-Assess
Formalize Team Structures


Who will be accountable on a
day-to-day basis for ensuring this
work is done?
How will State leadership be a part
of this process to ensure that
successes are operationalized and
barriers are removed?
Determine Need and
Identify Options


What do your current data suggest
is the most critical or pivotal need?
What is the supporting research or
evidence of the strategies you are
considering?
Assess Fit and Feasibility




What are the priorities of your State?
What is your theory of change (i.e.,
logic model, blueprint, outcome map,
etc.)?
How will you measure progress toward
that goal at the SEA? At the LEA?
Who will do what differently at the SEA
to impact that outcome? At the LEA?
Promote Buy-In


How will readiness be created at the
SEA?
How will readiness be created at the
LEA?
Re-Assess and Decide

What has emerged during Exploration
that impacts your decision?
GUIDING IMPLEMENTATION
QUESTIONS
Assessing Evidence-Based
Programs and Practices
Need in the Educational Setting,
Socially Significant Issues,
Parent & Community Perceptions of Need,
Objective Data indicating Need
Need
Fit with current -
Capacity
Staff meet minimum qualifications
Able to sustain Implementation Drivers
• Financially
• Structurally
Buy-in process operationalized
• Educators
• Administrators
• Families
Fit
•Initiatives
• RtI Implementation
• School and District Priorities
• Organizational structures
• Community Values
Capacity to Implement
Readiness
Qualified purveyor
Expert or TA available
Mature sites to observe
# of replications
How well is it operationalized?
Are Imp Drivers operationalized?
Intervention Readiness
for Replication
EBP:
Resource
Availability
5 Point Rating Scale: High = 5; Medium =
3; Low = 1. Midpoints can be used and
scored as a 2 or 4.
High
Medium
Low
Need
Evidence
Fit
Resources Availability
Resources
Curricula & Classroom
Materials,
IT requirements,
Staffing,
Training and PD,
Data Systems,
Coaching & Supervision,
Administrative & system
supports needed
Evidence
Outcomes – Is it worth it?
Fidelity data
Cost – effectiveness data
Number of studies
Population similarities
Diverse cultural groups
Efficacy or Effectiveness
Evidence
Readiness for Replication
© National Implementation Research Network- 2009
Capacity to Implement
Total Score:
Implementation
Team
Practitioners
Children
Practice Informed
Policy (PIP)
System
Change
Management
Team
Policy Enabled
Practice (PEP)
Organized, “Expert” Assistance
Practice-Policy Feedback Loops
Adaptive Challenges
•RFP methods
• Service Silos
• Salaries
• Funding
• Credentialing
• Licensing
• Time/ scheduling
• Union contracts
• Duplication
• Fragmentation
• Hiring criteria
• Federal/ State laws
Implementation
Review and synthesis of the implementation
research and evaluation literature
(1970 – 2004)
 Multi-disciplinary
 Multi-sector
 Multi-national
www.scalingup.org