Transcript How to implement EBPs

State Readiness CoP:
The Science and Practice
of Implementation
State Readiness
Community of Practice
November, 2009
Melissa Van Dyke
Vicki Hornus
Today’s Plan





Welcome
Roll call – by state
Presentation
Team discussion and report out
Assignment
Evidence Based Movement
The “evidence-based movement” is an
international experiment to make
better use of research findings in
typical service settings.
The purpose is to produce greater
benefits to consumers and society.
Getting Ready to Scale-up
The Goal:
To reliably produce…
…predictable and positive
outcomes for students,
schools, and communities
…that improve every year
…for the next 50 years
Problem

Students and schools cannot
benefit from interventions they
do not experience
What Works
INTERVENTION
IMPLEMENTATION
Effective
NOT Effective
Effective
NOT Effective
Performance
Implementation
(High Fidelity)
Paper Implementation
Procedure
Implementation
(Low Fidelity)
What Works
INTERVENTION
IMPLEMENTATION
Effective
Effective
NOT Effective
Performance
Implementation
(High Fidelity)
Paper Implementation
Procedure
Implementation
(Low Fidelity)
NOT Effective
Getting Here and Staying Here is a Challenge!
Learning About Effective
Implementation

Craft knowledge







National meeting of EBP Program Developers
National meeting of EBP Implementation Sites
National meeting of Implementation Researchers
Interviews with 64 Program Developers
Work with Program Developers, States, Agencies
Formal and scientific information

Program Efforts and Replication Data

Review of evaluation and research literature related to
implementation

Publication of Implementation Research: A Synthesis
of the Literature (2005)
WHAT HAVE WE LEARNED?
There are Challenges
 Science to Service Gap
 What is known is generally not
what is adopted
Implementation Gap
■ There are not clear pathways to
implementation
■ What is adopted often is not used with
fidelity and good effect
■ What is implemented disappears over time
and with staff turnover
There are Shared Challenges!!
Implementation is implementation is
implementation in:
•
•
•
•
•
•
•
•
•
•
Agriculture and Forestry
Manufacturing and Business
Substance abuse
Child welfare
Education
Engineering
Health and Medicine
Juvenile Justice
Mental health
Nursing, …and more
The commonalities are striking.
There Are Shared Challenges!!
Despite tremendous variability in the
content and context across
domains….



They have similar implementation
problems
They have similar implementation
solutions
We can learn more …more
quickly
Positive Intervention Outcomes
≠ Implementation
Implementation has not been achieved by
doing more or better research on
interventions.
The usability of a program or practice has
nothing to do with the weight of the
evidence regarding that program
–“Evidence” on effectiveness helps
you select what to implement for
whom
–“Evidence” on outcomes does not
help you implement the program
Insufficient Methods
■ Diffusion/dissemination of information
by itself does not lead to successful
implementation
■Training alone, no matter how well done,
does not lead to successful
implementation
■Policies and funding alone do not lead to
successful implementation
© Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008
New Knowledge

Successful implementation on a
useful scale requires a purveyor
 An individual or group of individuals
representing a program or practice
who actively work to implement that
practice or program with fidelity and
good effect
 Purveyors accumulate data &
experiential knowledge, & become
more effective and efficient over time
Active Purveyor Role
Simultaneous, Multi-Level Interventions
Purveyor
Teachers
Schools
Management (leadership, policy)
Administration (HR, structure)
Supervision (nature, content)
Regions / Districts
State, County & Local Context
Fidelity Matters

Higher Fidelity is correlated with better
outcomes across a wide range of
programs and practices

Adult Mental Health – ACT, IPS, IDDT

Medicine – DOTS, Texas Algorithm,
OMAP

Children’s Services – FFT, MST,
Wraparound, TFM

Education – HiPlaces, SWPBS, STEP

School-Based MH Prevention Programs PATHS
Implementation


Successful scale-up requires attention
to the science and practice of
implementation
Requires change at the

Practice Level (Teachers/Educational Staff)

Organization Level (Principal, Supervisor,
District Administrators)

System Level (SEA, Funders, Government,
Regulatory Bodies)
Research on Community Context
Attention needs to be paid to:





Providing leadership for the initiative
Creating buy-in from stakeholders
Cultivating and informing champions
Promoting structures and processes for
frequent and accurate communication at
multiple levels with lots of opportunity for
feedback from diverse groups of
stakeholders
And examination of feasibility of
implementation all along the way
Implementation Frameworks
Multi-dimensional, Fully integrated




Implementation Drivers
Implementation Stages
Implementation Teams
Improvement Cycles
Implementation Frameworks
Multi-dimensional, Fully integrated




Implementation Drivers
Implementation Stages
Implementation Teams
Improvement Cycles
Student Benefits
Performance Assessment
(Fidelity)
Coaching
Training
Selection
Implementation Lens
© Fixsen & Blase, 2008
OUTCOMES
(% of Participants who Demonstrate Knowledge,
Demonstrate new Skills in a Training Setting,
and Use new Skills in the Classroom)
Knowledge
Skill
Demonstration
Use in the
Classroom
Theory and
Discussion
10%
5%
0%
..+Demonstrati
on in Training
30%
…+ Practice &
Feedback in
Training
60%
60%
5%
…+ Coaching
in Classroom
95%
95%
95%
TRAINING
COMPONENTS
20%
0%
Joyce and Showers, 2002
Sobering Observations
"All organizations [and systems] are
designed, intentionally or unwittingly,
to achieve precisely the results they
get."
R. Spencer Darling
Business Expert
“The tyranny of the status quo.”
Fritz Oser
Educator
© Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008
Student Benefits
Performance Assessment
(Fidelity)
Coaching
Training
Selection
Systems
Intervention
Facilitative
Administration
Decision Support
Data System
© Fixsen & Blase, 2008
Student Benefits
Performance Assessment
(Fidelity)
Systems
Intervention
Coaching
Adaptive
Facilitative
Administration
Training
Decision Support
Data System
Selection
Technical
Leadership
© Fixsen & Blase, 2008
Student Benefits
Performance
Assessment (Fidelity)
Coaching
Adaptive
Systems
Intervention
Training
Integrated &
Compensatory
Selection
Technical
Facilitative
Administration
Decision Support
Data System
Leadership
© Fixsen & Blase, 2008
Implementation Frameworks
Multi-dimensional, Fully integrated




Implementation Drivers
Implementation Stages
Implementation Teams
Improvement Cycles
Student Benefits
Stages of Implementation
Performance Assessment
•Exploration (Fidelity)
•Installation
Coaching
Adaptive
Systems
Intervention
•InitialTraining
Implementation
•Full
Integrated &
Compensatory
Implementation
Selection
•Innovation
Technical
Leadership
•Sustainability
Facilitative
Administration
Decision Support
Data System
© Fixsen & Blase, 2008
Stages of Implementation
Major Implementation Initiatives occur
in stages:

Exploration

Installation

Initial Implementation

Full Implementation

Innovation

Sustainability
Two to Four Years
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Exploration

Exploration Goals:

Examine degree to which the EBP, Best
Practice, Systems Change meets the
needs in the settings identified

Determine whether moving ahead with the
initiative and implementation is desirable
and feasible

Create readiness for change at many
levels
“Pay now or pay later.”
Installation

Installation Stage Goal

To make the structural and instrumental
changes necessary to initiate services
“If you build it, they will come”. . . but you
actually have to built it.
Initial Implementation

Initial Implementation Goal
 Survive the Awkward Stage!
»
Learn from Mistakes
»
Continue “buy-in” efforts
»
Manage expectations
“Anything worth doing…is worth doing
poorly”
Full Implementation

Full Implementation Goals





Maintaining and improving skills and activities
throughout the system
Components integrated, fully functioning
Skillful practices by front line staff, supervisors,
administrators
Changes in policy that are reflected in practice
at all levels
Ready to be evaluated for expected outcomes
“The only thing worse than failing and not knowing why you failed,
is succeeding and not knowing why you succeeded.”
~ Jane Timmons-Mitchell
Innovation

Innovation

First do it right (high fidelity) or “as intended”

Then do it differently and better

Ability to retain function while changing form
given turnover, changing needs and context

Ability to adhere to the underlying theory of
change
Innovation needs to equal improvement
Sustainability

Goals of Sustainability

Financial:
»

Ensure funding streams for service and infrastructure
Programmatic:
»
Ensure high fidelity and positive outcomes through
infrastructure improvement and maintenance
»
Plan for turnover
The only thing harder than getting there is
staying there.
Stages of Implementation
Major Implementation Initiatives occur
in stages:

Exploration

Installation

Initial Implementation

Full Implementation

Innovation

Sustainability
Two to Four Years
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Putting “It” Into Action


Who will do what differently?
Who will help change occur? AND How
will change happen? At all levels….

Practice

Program

Organization

System
Implementation Frameworks
Multi-dimensional, Fully integrated




Implementation Drivers
Implementation Stages
Implementation Teams
Improvement Cycles
Implementation Science

Letting it happen
 Recipients are accountable

Helping it happen
 Recipients are accountable

Making it happen
 Implementation teams are
accountable
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Implementation Team
■A group that knows the innovation
very well (formal and practice
knowledge)
■A group that knows
implementation very well (formal
and practice knowledge)
■A group that knows improvement
cycles to make intervention and
implementation methods more
effective and efficient over time
Implementation Team
Prepare
Communities
Prepare School &
District Systems
Prepare teachers
and staff
Implementation
Teams
Assure
Student
Benefits
Work with TA & Parents &
Stakeholders
Researchers
Create Readiness
Assure Implementation
Implementation Frameworks
Multi-dimensional, Fully integrated




Implementation Drivers
Implementation Stages
Implementation Teams
Improvement Cycles
PDSA Cycles
Shewhart (1924); Deming (1948); SixSigma (1990)
 Plan – Develop specific things to do
 Do – Do them (make sure)
 Study – See what happens
 Act – Make adjustments
 Cycle – Do over and over again until
the goal is reached (again)
Improvement Cycle Uses

Rapid Cycle Teams
 Problem-solving
 Practice Improvement


Transformation Zones
PEP – PIP Cycles
Practice Feedback Loops
Practice (Do)
Policy
Policy Enabled Practices
(PEP)
Feedback
Practice Informed Policy
(PIP)
Policy (Plan)
Study - Act
“External” System Change Support
Policy
Structure
Procedure
Practice
Form Supports Function at every level
(National, State, District, Region, School, Classroom)
System Alignment
ALIGNMENT
State
Department
Districts/Regions
Schools
Teachers / Staff
Effective Practices
FORM SUPPORTS FUNCTION
Implementation Teams
Federal
Departments
Functional Systems
Policies
Bureaucracy
Schools
Teachers
Outcomes for
Children, Families,
& Communities
Summary









We need to invest in “what works” – science matters
Research results help us “choose” what to implement
But “implementation” is a practice and science unto itself
Fidelity Matters
Selection, Training, Coaching, Fidelity Measures help change
and support new practitioner behavior and skills
Data systems need to be used to make decisions
Facilitative administrative practices & systems interventions
create hospitable environments
Purveyors and Implementation Teams help with system and
service change
Policy enables new practice but practice needs to inform policy
Team Reflection and Discussion


What aspect of the presentation
(on the science and practice of
implementation) seemed most
relevant to your state’s efforts?
What implications does this
information have for your state’s
efforts?
For More Information
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. &
Wallace, F. (2005). Implementation Research: A Synthesis of
the Literature. Tampa, FL: University of South Florida, Louis de
la Parte Florida Mental Health Institute, The National
Implementation Research Network (FMHI Publication #231).
Download all or part of the monograph at:
http://www.fpg.unc.edu/~nirn/resources/publications/
Monograph/
Order monograph at:
https://fmhi.pro-copy.com/
SISEP at www.scalingup.org
Next Steps and Assignment


Develop a “team profile” to be posted on
the website

Who’s on your team – names, position/title

Which research-based strategies does your
state have experience implementing?
Send your team’s profile to Vicky


[email protected]
Join us for our next call

December 17 with Barb Sims, STS from
Illinois