Transcript How to implement EBPs

When "Just Do It" Won't
Do It: Implementing EBPs
in the Real World
April 2007
Karen A. Blase, Ph.D.
Sandra Naoom, MSPH
Melissa Van Dyke, MA
National Implementation Research Network
Louis de la Parte Florida Mental Health Institute
Evidence-Based -It’s All the
Buzz!
Advanced Google Search
Child Abuse Prevention + Evidence
Based = 1,100,000 hits
Child Neglect + Evidence-Based =
487,000
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Overall Puzzle
Each year, more research is done
More careful methods are crafted for
reviewing research and identifying
evidence-based practices and programs
More attention is directed to evidencebased practices and programs in
journals, conferences, and meetings
Yet, it is a challenge to realize sustainable
benefits for children, adolescents, and
families
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Naming the Challenge
Science to Service Gap
What is known is not what is adopted to
help children, families, and caregivers
Implementation Gap
There are not clear pathways to
implementation
What is adopted often is not used
with fidelity and good effect
What is implemented disappears with
time and staff turnover
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Science to Service
SCIENCE
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
IMPLEMENTATION
GAP
SERVICE
The Implementation Gap
It is one thing to say with the
prophet Amos, “Let justice roll
down like mighty waters,” and
quite another to work out the
irrigation system.
William Sloane Coffin
Social activist and clergyman
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
The National Implementation Research
Network (NIRN)
Craft knowledge
EBP purveyors (program developers)
EBP implementation site managers
Implementation researchers
Scientific information
Program development and replication data
Qualitative Study of program developers
Synthesis of the implementation evaluation and
research literature
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
From the Synthesis of the Literature
We Know That…
Implementation issues are common
across widely diverse domains
Human service prevention and intervention
(e.g. substance abuse, mental health,
violence prevention, education)
Advanced Manufacturing technologies
Research-based clinical guidelines
Engineering (e.g. Bridge maintenance)
Hotel management
National Franchise operations
Cancer prevention and treatment
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
From the Synthesis of the Literature
We Know That…
Implementation solutions are
common across widely diverse
domains
HOPE – Implementation research
findings likely have broad practice
implications across domains!
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Ineffective Methods
Excellent experimental evidence for
what does not work
Diffusion/dissemination of information
by itself does not lead to successful
implementation (research literature, mailings,
promulgation of practice guidelines)
Training alone, no matter how well
done, does not lead to successful
implementation
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Ineffective Methods
Excellent evidence for what does
not work
Implementation by edict does not work
Implementation by “following the
money” does not work
Implementation without changing
supporting roles and functions does not
work
Paul Nutt (2002). Why Decisions Fail
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
What Works
Effective intervention practices and
programs
+
Effective implementation practices
= Good outcomes for children, families
and caregivers
No other combination of factors reliably
produces desired outcomes for children,
families, and caregivers
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Only One Combination Works
IMPLEMENTATION
INTERVENTION
Effective
Effective
Performance
Implementation
(High Fidelity)
NOT Effective
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
NOT Effective
Paper & Process
Implementation
(Low or No Fidelity)
EBPs & Implementation
From an implementation
perspective, what do we
need to know about
evidence-based programs?
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
EBPs & Implementation
The usability of a program has little
to do with the quality or weight of
the evidence regarding that program
Evidence on intervention effectiveness for
specific populations helps us choose what to
implement
Evidence on the effectiveness of the
intervention does not help implement the
program or practice successfully
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
EBPs & Implementation
Core intervention components
Clearly described (who/what)
Practical measure of fidelity
Fully operationalized (do/say)
Field tested (recursive revision)
Contextualized (org./systems fit)
Effective (worth the effort)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation
What do we need to know
about approaches to
successful implementation?
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
What Seems to be Required
Successful implementation on a useful
scale requires a purveyor
A group of individuals representing a
program or practice who actively work
with organizations and communities to
help them implement that practice or
program with fidelity and good effect
Purveyors accumulate data & experiential
knowledge to become more effective and
efficient over time
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Framework
Organizational
Structures/Culture
Practitioner
Purveyor
Evidence-based
Practices
Fidelity & Outcome
Measures
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Framework
Organizational
Structures/Culture
Infrastructure
(Train, Coach,
Evaluate)
Purveyor
Evidence-based
Practices
Fidelity & Outcome
Measures
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Active Purveyor Role
Simultaneous, Multi-Level Interventions
Practitioner
Purveyor
Organization
Management (leadership, policy)
Administration (HR, structure)
Supervision (nature, content)
Service Systems
State, County and Local Context
Federal Context
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Purveyor Sources and Strategies
Purveyor group organized and promoted by
the originators
Nationally Based with Outreach
Nationally Based plus Regional T & TA Site
“Local” Implementation Team with the
knowledge, skill, freedom, and authority to act
(e.g. within a larger organization or a collaboration of
agencies)
An Intermediary Purveyor Organization that
becomes expert in implementation and a
“bridge” or expert with multiple EBPs (a new
way of doing T & TA) connecting providers and
purveyors.
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Stages of Implementation
Implementation is not an event
A mission-oriented process
involving multiple decisions,
actions, and corrections
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Stages of Implementation
Implementation occurs in stages:
Exploration
Installation
Initial Implementation
Full Implementation
Innovation
Sustainability
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
2–4
Years
Implementation
What happens during
exploration?
Building a “team”
Assessing community needs and current
performance
Studying and assessing “fit”
Assessing capacity, resources, and “buy-in”
Interviewing the “purveyor”
Deciding to move ahead/or not
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation
What happens during the
initial implementation, full
implementation, and
sustainability stages?
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Installing and Using
Implementation Drivers
Implementation Drivers are
mechanisms that
Help to develop, improve, and sustain
practitioners’ ability to implement an
intervention to benefit children
Help ensure sustainability and
improvement at the organizational level
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Drivers
STAFF
PERFORMANCE
EVALUATION
CONSULTATION
& COACHING
DECISION SUPPORT
DATA SYSTEMS
INTEGRATED &
COMPENSATORY
PRESERVICE
TRAINING
RECRUITMENT
AND SELECTION
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
FACILITATIVE
ADMINISTRATIVE
SUPPORTS
SYSTEMS
INTERVENTIONS
Integrated and Compensatory
Integrated
Consistency in philosophy, goals,
knowledge and skills across these
processes (S/T/C/E/A/SI)
Compensatory
At the Practitioner Level
At the Program Level
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Drivers
STAFF
PERFORMANCE
EVALUATION
SUPERVISION
& COACHING
DECISION SUPPORT
DATA SYSTEMS
INTEGRATED &
COMPENSATORY
PRESERVICE &
INSERVICE
TRAINING
RECRUITMENT
AND SELECTION
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
FACILITATIVE
ADMINISTRATIVE
SUPPORTS
SYSTEMS
INTERVENTIONS
Supervision and Coaching
Ensures implementation
Develops clinical and practice judgment
Ensures fidelity
Provides feedback to selection and
training processes
Grounded in “Best Practices”
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Supervision and Coaching
Implementation Best Practices:
Design a Supervision Service Delivery Plan
Use multiple sources of information for feedback –
Direct observation is critical
Provide regular feedback to all “Drivers”
Develop accountability structures for Supervision –
Supervise the Supervisor!
Regular review of adherence to Supervision Service
Delivery Plan
Multiple sources of information for supervisor feedback
Satisfaction surveys from clinicians
Observations of supervision
Program data (e.g. client retention, improvement)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Integrated and Compensatory
OUTCOMES
% of Participants who Demonstrate Knowledge,
Demonstrate New Skills in a Training Setting,
and Use new Skills in the Classroom
Knowledge
Skill
Demonstration
Theory and
Discussion
10%
5%
0%
..+Demonstration
in Training
30%
20%
0%
…+ Practice &
Feedback in
Training
60%
60%
5%
…+ Coaching in
Classroom
95%
95%
95%
TRAINING
COMPONENTS
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Use in the
Classroom
Integrated & Compensatory
Practitioners impact consumers
It is the job of directors, managers,
and funders to align policies and
structures to facilitate effective
practitioner practices
There is no such thing as an
“administrative decision” – they are
all treatment decisions
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Drivers
STAFF
PERFORMANCE
EVALUATION
CONSULTATION
& COACHING
INTEGRATED &
COMPENSATORY
PRESERVICE
TRAINING
RECRUITMENT
AND SELECTION
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
DECISION
SUPPORT DATA
SYSTEMS
FACILITATIVE
ADMINISTRATIVE
SUPPORTS
SYSTEMS
INTERVENTIONS
Quality Insistence NOT the Path of
Least Resistance
Fidelity Matters
and Outcome Measures Matter
Use Decision Support Data Systems
High fidelity supports good outcomes
Outcome and Process evaluation are
both essential
Detect Drift and Promote Quality
Discriminate effectiveness problems from
implementation problems
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Who does the work?
Implementation Teams
Purveyors
Intermediary Purveyor Organizations
Engaged Communities
Transformed provider organizations
Transformed human service systems
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Team
Those in position at organizational
and system levels to influence
decision-making and change policy
Administrators, practitioners and
others doing the “on the ground”
work of implementation
Consumers (e.g. Parents, Youth)
Community Partners
Active Purveyors of EBPs
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation
What role does
implementation play in overall
organizational change and
system transformation?
“Systems trump programs.”
Patrick McCarthy, Annie E. Casey Foundation
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Organizational Change
Successful and sustainable
implementation of evidence-based
practices and programs always
requires organizational change at
multiple levels.
Changing the behavior of adult human
service professionals
Changing organizational structures,
cultures, and climates
Changing the thinking of system
directors and policy makers
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Getting from Here to There
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Initiate & Manage Change
Innovation Zones
Suspend the usual rules
Establish the operational value of a
system change, EBP, or innovation
A zone may be part of an agency,
an agency, a county, a region, a
part of a system
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Initiate & Manage Change
Cannot change a whole system at
one time
Manage the old while creating the
new
Retain the best (of the old) while
changing the rest
Reduce impact of mistakes
(minimize damage, increase
flexibility, repair rapidly)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Organizational Change:
Top to Bottom
Facilitate or Hinder
Practitioners
Settings
System of care
Active alignment of policies
and coordination of efforts
in support of practitioner’s
use of effective practices to
benefit children, families,
and caregivers
State policies. coordination, funding,
support
Federal policies, coordination, funding,
supports
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Policy - Practice - Feedback
Study - Act
Feedback
Policy (Plan)
Policy
Structure
Procedure
Practice (Do)
Practice
Form follows Function
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
The Bottom Line
Fidelity Counts
The data and the experience match up; first do it as intended then do it
differently
Practitioner Performance Matters
In human services, the PRACTITIONER IS THE INTERVENTION – If their
behavior doesn’t change then the intervention or prevention practices
don’t happen. Selecting, training, coaching, evaluating matter.
Organizational and System Performance Matter
All decisions impact quality of treatment! Alignment of policies, funding,
and practitioner support are critical. Systems and organizations have to
change to support a new way of work.
Preparation and Persistence Pay Off
Implementation is not an event. It is a process that occurs over time, in
stages, - It is most successful when explored fully by the community and
stakeholders. - It requires organizational change and top to bottom
alignment
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Effective Use of EBPs
Stages
RESEARCH IMPLEMENTATION
Drivers
GAP
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
SERVICE
Don’t mistake a clear view for
a short distance.
Grand Canyon Hiking Advice
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Our Thanks To…
We thank the following for their support
Annie E. Casey Foundation
(EBPs and cultural competence)
William T. Grant Foundation
(implementation literature review)
Substance Abuse and Mental Health Services
Administration
(implementation strategies grants; sub-contract
for NREPP reviews; sub-contract for SOC analyses
of implementation)
Centers for Disease Control
(implementation research contract)
National Institute of Mental Health
(research and training grants)
Juvenile Justice and Delinquency Prevention
(program development and evaluation grants)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
For More Information
Karen A. Blase
Dean L. Fixsen
813-974-4463
813-974-4446
[email protected]
[email protected]
At the Louis de la Parte Florida Mental Health Institute
University of South Florida
http://nirn.fmhi.usf.edu
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
For More Information
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman,
R. M. & Wallace, F. (2005). Implementation
Research: A Synthesis of the Literature. Tampa, FL:
University of South Florida, Louis de la Parte Florida
Mental Health Institute, The National Implementation
Research Network (FMHI Publication #231).
Download all or part of the monograph at:
http://nirn.fmhi.usf.edu/resources/publications/Mono
graph/index.cfm
Copyright © Dean L. Fixsen and Karen A. Blase, 2007