Transcript How to implement EBPs - Mental Health Connection Home

Implementation and
Systems Change to
Support Effective Service
Mental Health
Connection of
Tarrant County
Bridging the Gap:
Research, Practice,
and Policy Symposium
November 3, 2009
Fort Worth, Texas
Allison J. R. Metz, Ph.D., Karen A. Blase, Ph.D.,
Dean L. Fixsen, Ph.D., & Melissa Van Dyke, LCSW
National Implementation Research Network
Frank Porter Graham Child Development Institute
University of North Carolina-Chapel Hill
The Implementation Gap
As anyone knows who has worked
in the field, implementation of a
new practice is the biggest
challenge of all.
~ Hollin and McMurran,
2001
The failure to utilize research rests in
large part on faulty or non-existent
implementation infrastructure.
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Applying Frameworks
The value of frameworks is
To promote the ability to generalize beyond the
immediate project or initiative
To enhance communication among partners (e.g.
better understanding of one another)
To more easily share and apply improvements
To increase the relevance of the “lessons learned”
So that implementation and systems change
efforts accomplish the common goal of
better outcomes for children and
families.
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Implementation Frameworks
Brief overview of the science of
implementation
Practice, program and systems change
through…
Multi-dimensional, fully integrated use of
….
Implementation Stages
Implementation Drivers
Implementation Teams
Improvement Cycles
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Building the Implementation Platform
Brief overview of the science of
implementation
Multi-dimensional, fully integrated
Implementation Stages
Implementation Drivers
Implementation Teams
Improvement Cycles
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
What Do We Mean by
Implementation?
A specified set of activities designed to
put into practice a policy, activity, or
program of known dimensions.
Implementation processes are
purposeful and defined in sufficient
detail such that independent observers
can detect the presence and strength
of these “specified activities”
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
There are Challenges
Science to Service Gap
■ What is known is generally not
what is adopted
Implementation Gap
■ There are not clear pathways to
implementation
■ What is adopted often is not used with
fidelity and good effect
■ What is implemented disappears over time
and with staff turnover
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
The National Implementation Research
Network (NIRN)
Craft knowledge
EBP purveyors (program developers)
EBP implementation site managers
Implementation researchers
Scientific information
Program development and
replication data
Qualitative study of
program developers
Synthesis of the
implementation
evaluation and
research literature
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
From the Synthesis of the Literature
We Know That…
Implementation issues are common
across widely diverse domains
Human service prevention and intervention
(e.g. child welfare, substance abuse, mental
health, violence prevention, education)
Advanced manufacturing technologies
Research-based clinical guidelines
Engineering (e.g. bridge maintenance)
Hotel management
National franchise operations
Cancer prevention and treatment
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Positive Intervention Outcomes
≠ Implementation
Implementation has not been achieved by
doing more or better research on
interventions or on curricula
The usability of a program or practice has
nothing to do with the weight of the
evidence regarding that program
–“Evidence” on effectiveness helps you
select what to implement for whom
–“Evidence” on these outcomes does not
help you implement the program or
practice
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Problem
Children and families cannot
benefit from interventions they
do not experience
“In a study of real world practice in
addiction treatment settings…found
that many clinicians reported that they
were doing CBT. However, an
independent analysis of treatment
sessions found that CBT strategies
showed up in just 3 percent of them.”
Journal of Substance Abuse Treatment (Vol. 35, No. 4)
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Insufficient Methods
Implementation by laws/ compliance by itself
does not work
Implementation by “following the money” by
itself does not work
Implementation without changing supporting
roles and functions does not work
Fixsen, Naoom, Blase, Friedman, Wallace, 2005
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Insufficient Methods
Diffusion/dissemination of information by
itself does not lead to successful
implementation
Training alone, no matter how well done,
does not lead to successful implementation
Fixsen, Naoom, Blase, Friedman, Wallace, 2005
So what does work?
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
What works?…Fidelity Matters
Higher Fidelity is correlated with better
outcomes across a wide range of
programs and practices
Children’s Services – FFT, MST,
Wraparound, TFM
Education – HiPlaces, SWPBS, STEP
School-Based MH Prevention Programs –
PATHS
Adult Mental Health – ACT, IPS, IDDT
Medicine – DOTS, Texas Algorithm,
OMAP
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
What Works
INTERVENTION
The WHAT
IMPLEMENTATION – The HOW
Effective
Effective
NOT Effective
Performance
Implementation
(High Fidelity)
Paper Implementation
Procedure
Implementation
(Low Fidelity)
NOT Effective
Getting Here and Staying Here is the Challenge!
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Pre-Requisite: Defining “IT”
From what current state to what
future state?
The “it” must be operationalized
whether it is:
An Evidence-Based Practice or Program
A Best Practice Initiative
A Broad Systems Change Initiative
Operationalize
Part of Speech: verb Definition: to define a
concept or variable so that it can be
measured or expressed quantitatively
Webster's New Millennium™ Dictionary of English, Preview
Edition (v 0.9.7)
Copyright © 2003-2008 Lexico Publishing Group, LLC
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
What Works
Successful implementation on a
useful scale requires a purveyor
An individual or group of individuals
representing a program or practice who
actively work to implement that practice
or program with fidelity and good effect
Purveyors accumulate data &
experiential knowledge & become
more effective and efficient over time
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Active Purveyor Role
Simultaneous, Multi-Level Interventions
Practitioner
Organization
Purveyor
Management (leadership, policy)
Administration (HR, structure)
Supervision (nature, content)
Service Systems
State, County and Local Context
Federal Context
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Purveyor Sources and Strategies
Purveyor group organized and promoted by
the EBP originators
Nationally Based with Outreach
Nationally Based plus Regional T & TA Site
Purveyor group organized and/or accessed
through a National Resource Center or other
qualified TA entity
“Local” and “State” Implementation Teams
with the knowledge, skill, freedom, and
authority to act (e.g. within a larger organization or a
collaboration of agencies)
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Change What?….Practice?
Program? Organization? Systems?
From more limited and focused changes (e.g.
targeted new policy, new program implemented
by a provider agency) to more sweeping
practice change (e.g. system-wide differential
response, new practice models, systems of care)
all will require…
Organizational and Systems Change
Organizational and Systems Change are “in
service” to the new or improved “future state” and
“desired outcomes”
The organization changes in order to….
The system needs to change in order to…
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Change “in order to”…
Functional Organizational Change: Purposeful
and iterative change in an individual agency’s
internal structures and procedures and in its
external relationships in order to support,
sustain, and renew effective practices and
services.
Strategic Systems Change: Purposeful and
iterative change among interdependent service,
regulatory, funding, and decision-making bodies
in order to create and maintain hospitable
conditions for effective service delivery.
» National Implementation Research Network, 2009
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Building the Implementation Platform
Overview of the science of
implementation
Multi-dimensional, Fully integrated
Implementation Stages
Implementation Drivers
Implementation Teams
Improvement Cycles
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Stages of Implementation
Major Implementation Initiatives occur
in stages:
Exploration
Installation
Two to Four Years
Initial Implementation
Full Implementation
Innovation
Sustainability
Fixsen, Naoom, Blase, Friedman, & Wallace, 2005
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Exploration
Exploration Goals:
Examine degree to which the EBP, Best
Practice, Systems Change meets the
needs in the settings identified
Determine whether moving ahead with the
initiative and implementation is desirable
and feasible
Create readiness for change at many
levels
“Pay now or pay later.”
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Installation
Installation Stage Goal
To make the structural and instrumental
changes necessary to initiate services
“If you build it, they will come”. . . but you
actually have to build it.
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Initial Implementation
Initial Implementation Goal
Survive the Awkward Stage!
Learn from Mistakes
Continue “buy-in” efforts
Manage expectations
“Anything worth doing…is worth doing
poorly”
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Full Implementation
Full Implementation Goals
Maintaining and improving skills and activities
throughout the system
Components integrated, fully functioning
Skillful practices by front line staff, supervisors,
administrators
Changes in policy that are reflected in practice
at all levels
Ready to be evaluated for expected outcomes
“The only thing worse than failing and not knowing why you failed,
is succeeding and not knowing why you succeeded.”
~ Jane Timmons-Mitchell
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Innovation
Innovation
First do it right (high fidelity) or “as intended”
Then do it differently and better
Ability to retain function while changing form
given turnover, changing needs and context
Ability to adhere to the underlying theory of
change
Innovation needs to equal improvement
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Sustainability
Goals of Sustainability
Financial:
Ensure funding streams for service and infrastructure
Programmatic:
Ensure high fidelity and positive outcomes through
infrastructure improvement and maintenance
Plan for turnover
The only thing harder than getting there is
staying there.
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Building the Implementation Platform
Overview of the science of
implementation
Multi-dimensional, Fully integrated
Implementation Stages
Implementation Drivers
Implementation Teams
Improvement Cycles
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Implementation Drivers
Implementation Drivers are
methods to develop, improve, and
sustain competent uses of
innovations. (Competency Drivers)
Implementation Drivers are
methods to create and sustain
hospitable organizational and
systems environments for
effective services (Organization
Drivers)
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Child and Family Benefits
Implementation
Drivers
Performance Assessment
(Fidelity)
Coaching
Training
Selection
Implementation Lens
© Fixsen & Blase, 2008
Coaching is Key
Joyce and Showers (2002) data
With coaching = 95% use in classrooms
Without = 5% use in classrooms
Rogers, Wellens, & Conner (2002) data
About 10% of what is taught in business
workshops actually is put into practice
Significant data across domains that
feedback improves performance
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Sobering Observation
"All organizations [and systems] are
designed, intentionally or unwittingly, to
achieve precisely the results they get."
R. Spencer Darling
Business Expert
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Child and Family Benefits
Implementation
Drivers
Performance Assessment
(Fidelity)
Coaching
Training
Selection
Systems
Intervention
Facilitative
Administration
Decision Support
Data System
© Fixsen & Blase, 2008
Child and Family Benefits
Implementation
Drivers
Performance Assessment
(Fidelity)
Systems
Intervention
Coaching
Adaptive
Facilitative
Administration
Training
Decision Support
Data System
Selection
Technical
Leadership
© Fixsen & Blase, 2008
Types of Leadership Needed
Different strategies for different
challenges
Technical Leadership
Adaptive Leadership
According to Ron Heifitz and his
colleagues, one of the biggest
mistakes “leaders” make is to
incorrectly identify the type of
challenge they are facing
Using technical approaches for
adaptive issues
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Technical Work
Perspectives are aligned (views, values)
Definition of the problem is clear
Solution and implementation of the
solution is clear
Primary locus of responsibility for
organizing the work is the leader
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Adaptive Work
Legitimate, yet competing,
perspectives emerge
Definition of the problem is unclear
Solution and implementation is unclear
and requires learning
Primary locus of responsibility is not
the leader
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
The Adaptive Work of the Leader
Get on the Balcony
Identify the Adaptive Challenge
Regulate Distress
Maintain Disciplined Attention
Give the Work Back to the People
Protect All Voices
» R. Heifetz and D. Laurie: The Work of
Leadership. Harvard Business Review,
1998.
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Child and Family Benefits
Implementation
Drivers
Performance Assessment
(Fidelity)
Coaching
Adaptive
Systems
Intervention
Training
Integrated &
Compensatory
Selection
Technical
Facilitative
Administration
Decision Support
Data System
Leadership
© Fixsen & Blase, 2008
Integrated and Compensatory
Integrated
Consistency in philosophy, goals,
knowledge and skills across these
processes
Compensatory
Assessment of weaknesses and strengths
in Driver functioning
Installation of Drivers at multiple levels of
the system (practitioner, provider, district,
county, etc)
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Building the Implementation Platform
Overview of the science of
implementation
Multi-dimensional, Fully integrated
Implementation Stages
Implementation Drivers
Implementation Teams
Improvement Cycles
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Implementation Science
Letting it happen
Recipients are accountable
Helping it happen
Recipients are accountable
Making it happen
Implementation teams are
accountable
Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Initiating and Managing System
Change
Implementation Team(s)
General Definition: Core group of individuals,
who are representative of the stakeholders
and “systems” and who are charged with
guiding the overall implementation from
exploration through to full implementation
Benefits: Provides a focused and
accountable structure to increase the
likelihood that this effort will not be
abandoned or derailed.
Scope of the initiative determines the scope
of authority and the need for linked
Implementation Teams
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Core Competency for
Implementation Teams
A Team that:
■Knows the innovation very well (formal
and practice knowledge)
■Knows implementation very well (formal
and practice knowledge)
■Knows improvement cycles to make
intervention and implementation methods
more effective and efficient over time
■ Promotes systems change at multiple
levels to create hospitable cultures,
policies, and funding streams
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Linked Implementation Teams
State
Team
Regional or
Collaborative
Implementation
Team
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Agency
Team
Implementation Teams
Integrated and Interlocking at Multiple Levels
Agency, “Setting-Based” Teams, or EBP Based
Community or Collaborative Teams
State Team and/or Project Design Team
Represents the stakeholders and the
‘system’
Clear purpose through “Terms of Reference”
Focus is on
Engagement of stakeholders
Exploration and Selection of EBPs
Installation, Quality and Sustainability of the Drivers
Attending to fidelity & outcomes,
Promoting Alignment (funding and policy)
Problem-Solving and Promoting Sustainability
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Catawba County Child Well-Being
Project
Consumers
Key DSS Leaders
Design
Team
Sector Leads
Stakeholders
Funders
Providers
Meta- Team
Leads + Design
Team Rep’s
EBP
Area,
Agency,
or Unit
EBP
Area,
Agency,
or Unit
EBP
Area,
Agency,
or Unit
IMPLEMENTATION TEAMS
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Senior Mgmt
Team
EBP
Area,
Agency
or Unit
Coherent Systems Change
Each Implementation Team Needs
Terms of Reference
Linking Communication Protocols
Focus on Overall Systems Change
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Implementation Teams
Establishing Terms of Reference (ToR)
Terms of Reference (ToR) are detailed in a
written document
Outlining the purpose of the project or group,
How it will be structured and implemented.
It is like an internal Memorandum of
Understanding (MOU)
Helps you clarify communication, mandates,
and core features and functions of the group
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Building the Implementation Platform
Overview of the science of
implementation
Multi-dimensional, Fully integrated
Implementation Stages
Implementation Drivers
Implementation Teams
Improvement Cycles
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Why Improvement Cycles?
• It’s a hallmark of a Learning Culture
• Learning is Good!
• Perfection is impossible
• Avoid having the “perfect” become
the enemy of the good
• In a complex environment your
“solution” will change the problem –
You need to see what “emerges” from
implementation
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
PDSA Cycles
Plan – Develop specific things to do
Do – Do them (make sure)
Study – See what happens
Act – Make adjustments
Cycle – Do over and over again until
the goal is reached (again)
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Improvement Cycle Uses
Rapid Cycle Teams
Problem-solving
Practice Improvement
Transformation Zones
PEP – PIP Cycles
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Rapid Cycle Improvement Teams
Problem-solving during early efforts
Team Lead identified
Right people on the team
Time-limited to address the problem
Team disbands
Practice Improvement
On – going efforts to improve practices
and competencies
Use data to achieve better outcomes for
children and “embed” solutions
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Improvement Cycle Uses
Rapid Cycle Teams
Problem-solving
Practice Improvement
Transformation Zones
PEP – PIP Cycles
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
System Change
Demonstrations or “pilots” are a place to
start for innovations (“it’s possible!”)
Don’t usually lead to sustainable service
and system change
Random Acts of Innovation
Person and Passion Dependent
Can “ghost” system its way to success
Executed by the “Extraordinary”
No replicable implementation infrastructure
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Transformation Zone
Focus on innovations
and implementation
infrastructure development
and systems change processes
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Transformation
Zone
Use Innovations
Develop
Implementation
Infrastructure
Change System
Transformation Zone
A TZ can be thought of as a “vertical
slice” of the service system (from the
court to the Capitol)
The “slice” is small enough to be
manageable
The “slice” is large enough to include all
aspects of the system (court, case work,
clinic, service system, urban, rural, frontier)
Manage the change process (small bites)
Manage the risks (most innovations don’t
work at first)
Limit the damage (quick recovery, repair)
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Improvement Cycle Uses
Rapid Cycle Teams
Problem-solving
Practice Improvement
Transformation Zones
PEP – PIP Cycles
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Implementation Science
“In theory there is no difference
between theory and practice;
in practice, there is.”
Jan L. A. van de Snepscheut
Computer Scientist
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Practice
Policy (Plan)
Policy
Policy Enabled Practices
(PEP)
“External” System Change Support
Policy
Practice (Do)
Structure
Procedure
Practice
Practice Feedback Loops
Practice (Do)
Policy
Policy Enabled Practices
(PEP)
Feedback
Practice Informed Policy
(PIP)
Policy (Plan)
Study - Act
“External” System Change Support
Policy
Structure
Procedure
Practice
Form Supports Function at every level
(National, State, District, Region, Agency, Practitioner)
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
System Stability
EXISTING SYSTEM
Effective Innovations are
Changed to Fit the
System
Or Operate in the Shadows
(The Ghost System)
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Effective System Change
EXISTING SYSTEM
EXISTING SYSTEM IS
CHANGED TO SUPPORT
THE EFFECTIVENESS OF
THE INNOVATION
Effective Innovations are
Changed to Fit the
System
Or Operate in the Shadows
(Ghost System)
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
(Host System)
EFFECTIVE INNOVATION
System Alignment
ALIGNMENT
State
Department
Communities,
Regions
Provider
Agencies
Practitioners
Effective Practices
FORM SUPPORTS FUNCTION
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Implementation Teams
Federal
Departments
Summary
We need to invest in “what works” – science matters
Research results help us “choose” what to
implement
But “implementation” and the science of “systems
change” are practices and science unto themselves
Materials and training alone won’t work
Policy and Mandates alone won’t work
Fidelity Matters
When working with “systems change” and best
practices, the “it” needs to be operationalized
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Summary
Stage related activities prepare the system for a successful
change process
Competence needs to be Developed and Sustained
Selection, Training, Coaching, Fidelity Measures help change and
support new practitioner behavior and skills
Organizations and Systems need to change
Data systems need to be used to make decisions
Facilitative administrative practices & systems interventions create
hospitable environments
Policy enables new practice but practice needs to inform policy
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
Summary
Purveyors, Intermediary Organizations, and Implementation
Teams help with system and service change – Knowledge
won’t “get there” of its own volition
You are never done – The environment is in motion
Improvement Cycles are Critical
The ‘right’ leadership strategies are needed for the issues at hand
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
And Don’t Forget the “Big Picture”
Effective Mental Health Services
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
For More Information
Implementation
Research:
A Synthesis of the
Literature
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005).
Implementation Research: A Synthesis of the Literature. Tampa, FL: University of
South Florida, Louis de la Parte Florida Mental Health Institute, The National
Implementation Research Network (FMHI Publication #231).
Download all or part of the monograph at:
http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=31
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
SISEP Center
State Implementation and Scaling up of Evidencebased Practices (SISEP)
Dean Fixsen, Karen Blase, Rob Horner, George Sugai
www.scalingup.org
“Resources”
Concept paper
Annotated bibliography
Scaling up data
Scaling up “Briefs”
Copyright © Dean L. Fixsen and Karen A. Blase, 2009
For More Information
Allison Metz, Ph.D.
Karen Blase, Ph.D.
202-905-6581
919-966-9050
[email protected]
[email protected]
Frank Porter Graham Child Development Institute
University of North Carolina
Chapel Hill, NC
www.scalingup.org
http://nirn.fpg.unc.edu/
http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/
Copyright © Dean L. Fixsen and Karen A. Blase, 2009