Transcript How to implement EBPs

Creating Mechanisms that
Work in Implementing
Evidence-Based Family
Therapy
American Family Therapy
Academy, Inc. – 8th Annual
Research Conference
Evidence-Based Family
Treatments: Improving Family
Therapy by Advancing Clinician
and Researcher Partnerships
February 24, 2007
Karen A. Blase, Ph.D.
National Implementation Research Network
Louis de la Parte Florida Mental Health Institute
Evidence-Based -It’s All the Buzz!
Advanced Google Search
Evidence-Based X Family Therapy =
1,120,000 hits
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Overall Puzzle
Each year, more research is done
More careful methods are crafted for
reviewing research and identifying
evidence-based practices and programs
More attention is directed to evidencebased practices and programs in
journals, conferences, and meetings
Yet, it is a challenge to realize sustainable
benefits for children, adolescents, and
families
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Naming the Challenge
Science to Service Gap
What is known is not what is adopted to
help children, families, and caregivers
Implementation Gap
There are not clear pathways to
implementation
What is adopted often is not used
with fidelity and good effect
What is implemented disappears with
time and staff turnover
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
The National Implementation Research
Network (NIRN)
Craft knowledge
EBP purveyors (program developers)
EBP implementation site managers
Implementation researchers
Survey of EBP program developers
Scientific information
Program development and replication data
Synthesis of the implementation evaluation and
research literature
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
From the Synthesis of the Literature
We Know That…
Implementation issues are common
across widely diverse domains
Human service prevention and intervention
(e.g. substance abuse, mental health,
violence prevention, education)
Advanced Manufacturing technologies
Research-based clinical guidelines
Engineering (e.g. Bridge maintenance)
Hotel management
National Franchise operations
Cancer prevention and treatment
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
From the Synthesis of the Literature
We Know That…
Implementation solutions are
common across widely diverse
domains
HOPE – Implementation research
findings likely have broad practice
implications across domains!
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Ineffective Methods
Excellent experimental evidence for
what does not work
Diffusion/dissemination of information
by itself does not lead to successful
implementation (research literature, mailings,
promulgation of practice guidelines)
Training alone, no matter how well
done, does not lead to successful
implementation
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Ineffective Methods
Excellent evidence for what does
not work
Implementation by edict does not work
Implementation by “following the
money” does not work
Implementation without changing
supporting roles and functions does not
work
Paul Nutt (2002). Why Decisions Fail
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
What Works
Effective intervention practices and
programs
+
Effective implementation practices
= Good outcomes for children, families
and consumers
No other combination of factors reliably
produces desired outcomes for children,
families, and caregivers
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Only One Combination Works
IMPLEMENTATION
INTERVENTION
Effective
Effective
Performance
Implementation
(High Fidelity)
NOT Effective
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
NOT Effective
Paper & Process
Implementation
(Low or No Fidelity)
EBPs & Implementation
From an implementation
perspective, what do we
need to know about
evidence-based programs?
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
EBPs & Implementation
The usability of a program has little
to do with the quality or weight of
the evidence regarding that program
Evidence on intervention effectiveness for
specific populations helps us choose what to
implement
Evidence on the effectiveness of the
intervention does not help implement the
program or practice successfully
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
EBPs & Implementation
Core intervention components
Clearly described (who/what)
Practical measure of fidelity
Fully operationalized (do/say)
Field tested (recursive revision)
Contextualized (org./systems fit)
Effective (worth the effort)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation
What do we need to know
about approaches to
successful implementation?
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
What Seems to be Required
Successful implementation on a useful
scale requires a purveyor
A group of individuals representing a
program or practice who actively work
with organizations and communities to
help them implement that practice or
program with fidelity and good effect
Purveyors accumulate data & experiential
knowledge to become more effective and
efficient over time
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Framework
Organizational
Structures/Culture
Practitioner
Purveyor
Evidence-based
Practices
Fidelity & Outcome
Measures
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Framework
Organizational
Structures/Culture
Infrastructure
(Train, Coach,
Evaluate)
Purveyor
Evidence-based
Practices
Fidelity & Outcome
Measures
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Active Purveyor Role
Simultaneous, Multi-Level Interventions
Practitioner
Purveyor
Organization
Management (leadership, policy)
Administration (HR, structure)
Supervision (nature, content)
Service Systems
State, County and Local Context
Federal Context
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Purveyor Sources and Strategies
Purveyor group organized and promoted by
the originators
Nationally Based with Outreach
Nationally Based plus Regional T & TA Site
“Local” Implementation Team with the
knowledge, skill, freedom, and authority to act
(e.g. within a larger organization or a collaboration of
agencies)
An Intermediary Purveyor Organization that
becomes expert in implementation and a
“bridge” or expert with multiple EBPs (a new
way of doing T & TA) connecting providers and
purveyors.
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Stages of Implementation
Implementation is not an event
A mission-oriented process
involving multiple decisions,
actions, and corrections
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Stages of Implementation
Implementation occurs in stages:
Exploration
Installation
Initial Implementation
Full Implementation
Innovation
Sustainability
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
2–4
Years
Implementation
What happens during the
initial implementation, full
implementation, and
sustainability stages?
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Installing and Using
Implementation Drivers
Implementation Drivers are
mechanisms that
Help to develop, improve, and sustain
practitioners’ ability to implement an
intervention to benefit children
Help ensure sustainability and
improvement at the organizational level
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Drivers
STAFF
PERFORMANCE
EVALUATION
CONSULTATION
& COACHING
DECISION SUPPORT
DATA SYSTEMS
INTEGRATED &
COMPENSATORY
PRESERVICE
TRAINING
RECRUITMENT
AND SELECTION
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
FACILITATIVE
ADMINISTRATIVE
SUPPORTS
SYSTEMS
INTERVENTIONS
Integrated and Compensatory
Integrated
Consistency in philosophy, goals,
knowledge and skills across these
processes (S/T/C/E/A/SI)
Compensatory
At the Practitioner Level
At the Program Level
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Supervision and Coaching
Purposes:
Ensures implementation
Develops clinical judgment
Ensures fidelity
Provides feedback to selection and
training processes
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Supervision and Coaching
Implementation Best Practices:
Design a Supervision Service Delivery Plan
Use multiple sources of information for feedback –
Direct observation is critical
Provide regular feedback to all “Drivers”
Develop accountability structures for Supervision –
Supervise the Supervisor!
Regular review of adherence to Supervision Service
Delivery Plan
Multiple sources of information for supervisor feedback
Satisfaction surveys from clinicians
Observations of supervision
Program data (e.g. client retention, improvement)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Integrated and Compensatory
OUTCOMES
% of Participants who Demonstrate Knowledge,
Demonstrate New Skills in a Training Setting,
and Use new Skills in the Classroom
Knowledge
Skill
Demonstration
Theory and
Discussion
10%
5%
0%
..+Demonstration
in Training
30%
20%
0%
…+ Practice &
Feedback in
Training
60%
60%
5%
…+ Coaching in
Classroom
95%
95%
95%
TRAINING
COMPONENTS
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Use in the
Classroom
Integrated & Compensatory
Practitioners impact consumers
It is the job of directors, managers,
and funders to align policies and
structures to facilitate effective
practitioner practices
There is no such thing as an
“administrative decision” – they are
all treatment decisions
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Who does the work?
Implementation Teams
Purveyors
Intermediary Purveyor
Organizations
Transformed provider
organizations
Transformed human service
systems
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation Team
Those in position at organizational
and system levels to influence
decision-making and change policy
Administrators, practitioners and
others doing the “on the ground”
work of implementation
Consumers
Active Purveyors of EBPs
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Implementation
What role does
implementation play in overall
organizational change and
system transformation?
“Systems trump programs.”
Patrick McCarthy, Annie E. Casey Foundation
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Organizational Change
Successful and sustainable
implementation of evidence-based
practices and programs always
requires organizational change at
multiple levels.
Changing the behavior of adult human
service professionals
Changing organizational structures,
cultures, and climates
Changing the thinking of system
directors and policy makers
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Initiate & Manage Change
Innovation Zones
Suspend the usual rules
Establish the operational value of a
system change, EBP, or innovation
A zone may be part of an agency,
an agency, a county, a region, a
part of a system
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Initiate & Manage Change
Cannot change a whole system at
one time
Manage the old while creating the
new
Retain the best (of the old) while
changing the rest
Reduce impact of mistakes
(minimize damage, increase
flexibility, repair rapidly)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Policy - Practice - Feedback
Study - Act
Feedback
Policy (Plan)
Policy
Structure
Procedure
Practice (Do)
Practice
Form follows Function
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
The Bottom Line
Fidelity Counts
The data and the experience match up; first do it as intended then do it
differently
Practitioner Performance Matters
In human services, the PRACTITIONER IS THE INTERVENTION – If their
behavior doesn’t change then the intervention or prevention practices
don’t happen. Selecting, training, coaching, evaluating matter.
Organizational and System Performance Matter
All decisions impact quality of treatment, alignment of policies, funding,
and practitioner support are critical. Systems and organizations have to
change to support a new way of work.
Preparation and Persistence Pay Off
Implementation is not an event. It is a process that occurs over time, in
stages, - It is most successful when explored fully by the community and
stakeholders. - It requires organizational change and top to bottom
alignment
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
Effective Use of EBPs
Stages
RESEARCH IMPLEMENTATION
Drivers
GAP
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
SERVICE
Our Thanks To…
We thank the following for their support
Annie E. Casey Foundation
(EBPs and cultural competence)
William T. Grant Foundation
(implementation literature review)
Substance Abuse and Mental Health Services
Administration
(implementation strategies grants; sub-contract
for NREPP reviews; sub-contract for SOC analyses
of implementation)
Centers for Disease Control
(implementation research contract)
National Institute of Mental Health
(research and training grants)
Juvenile Justice and Delinquency Prevention
(program development and evaluation grants)
Copyright © Dean L. Fixsen and Karen A. Blase, 2007
For More Information
Karen A. Blase
Dean L. Fixsen
813-974-4463
813-974-4446
[email protected]
[email protected]
At the Louis de la Parte Florida Mental Health Institute
University of South Florida
http://nirn.fmhi.usf.edu
Download all or part of the monograph at:
http://nirn.fmhi.usf.edu/resources/publications/Monograph/index.cfm
Copyright © Dean L. Fixsen and Karen A. Blase, 2007