Systems and Practice Change Through Implementation Lens

Transcript Systems and Practice Change Through Implementation Lens

System and Practice Change Through an Implementation Lens

Ohio Alternative Response Summit Presentation on Implementation Science May 14, 2010 Allison Metz, PhD Karen Blase, PhD Melissa Van Dyke, LCSW Frank Porter Graham Child Development Institute University of North Carolina at Chapel Hill

Common Goal

To build capacity in child welfare agencies and to support Counties in achieving the sustainable implementation infrastructure and systemic change necessary for an effective Alternative Response model….

. . .

To improve outcomes for families and children

Overall Challenge

Science to Service Gap

What is known often is not what is adopted to help children, families, and caregivers

Implementation Gap

There are not clear pathways to implementation What is adopted often is not used with fidelity and good effect What is implemented often disappears with time and staff turnover

The National Implementation Research Network (NIRN) Craft knowledge

EBP purveyors (program developers) EBP implementation site managers Implementation researchers

Scientific information

From the Synthesis of the Literature We Know That…

Implementation issues are common across widely diverse domains Human service prevention and intervention (e.g. child welfare, substance abuse, mental health, violence prevention, education) Advanced manufacturing technologies Research-based clinical guidelines Engineering (e.g. bridge maintenance) Hotel management National franchise operations Cancer prevention and treatment

Positive Intervention Outcomes ≠ Implementation

Implementation has not been achieved by doing more or better research on interventions or on curricula The usability of a program or practice has nothing to do with the weight of the evidence regarding that program

–

“Evidence” on effectiveness helps you select what to implement for whom

–

“Evidence” on these outcomes does not help you implement the program or practice

Problem

Children and families cannot benefit from interventions they do not experience

“In a study of real world practice in addiction treatment settings…found that many clinicians reported that they were doing CBT. However, an independent analysis of treatment sessions found that CBT strategies showed up in just 3 percent of them.”

Journal of Substance Abuse Treatment (Vol. 35, No. 4)

Insufficient Methods

Diffusion/dissemination of information by itself does not lead to successful implementation Training alone, no matter how well done, does not lead to successful implementation Policies and funding alone do not lead to successful implementation Fixsen, Naoom, Blase, Friedman, Wallace, 2005

What Works?... Fidelity Matters

Effective IMPLEMENTATION – The HOW Effective BENEFITS (High Fidelity) NOT Effective Paper Implementation (Low Fidelity) NOT Effective Getting Here and Staying Here is the Challenge!

What Works?...Purveyor

Successful implementation on a useful scale requires a purveyor

An individual or group of individuals representing a program or practice who actively work to implement that practice or program with fidelity and good effect (Affiliated Private, Funded TA Centers, Intermediaries) Purveyors accumulate data & experiential knowledge & become more effective and efficient over time Purveyors differ in what they offer and how they help Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Purveyor AND Intermediary Structures and Strategies Simultaneous, Multi-Level Interventions

Practitioners Agency/Provider Management (leadership, policy) Administration (HR, structure) Supervision (nature, content) Local, County, & State Context Federal Context

What Works? …Implementation Best Practices

There are intervention and prevention “best practices” and EBPs Practices Motivational Interviewing, Appropriate Risk Assessment, Social Skills Training Programs and Programs Nurse Family Partnership, MST, PCIT, TF-CBT, Strengthening Families, Incredible Years There are implementation best practices to make the principles and programs come to life in communities and agencies.

Change What?….

Practice? Program? Organization? Systems?

From more limited and focused changes

(e.g. targeted new policy, new program implemented by a provider agency

) to more sweeping practice change

(e.g., systems of care)

all will require…

Organizational and systems change Organizational and systems change are “in service” to the new or improved “future state” and “desired outcomes” The organization changes

in order to

….

The system needs to change

in order to

Cascading Logic Model

For systems to change, we need individuals to change their activities, patterns, behavior How do we identify, create change in, and measure changes in people’s behavior, knowledge, and attitudes that relate to better outcomes for children and families?

How do we measure the products and processes that are created to support the service or system change?

Cascading Logic Model

Population

Children at Risk of Child Abuse and Neglect Children and Families with substantiated issues

Intervention Strategies and Measures (WHAT)

Alternative Response Model

Intervention Outcomes

Increased family engagement in services Improved safety at home Reduced incidents of recurrence

Population

Cascading Logic Model

Young Children at Risk of Child Abuse Children and Families with substantiated issues

Populations

Front-line Practitioners and Case Managers

Intervention Strategies and Measures (WHAT)

Alternative Response Model

Practice, Organizational and Systems Change Strategies (HOW)

Carefully selected staff and the provision of skillful, timely training, coaching, performance assessments , use of data in supportive administrative and state /county environments

Intervention Outcomes

Increased family engagement in services Improved safety at home Reduced incidents of recurrence

Implementation Related Outcomes Competent

use of selected Interventions & approaches Alternative Response (fidelity measures or staff performance assessments)

Population

Cascading Logic Model

Young Children at Risk of Child Abuse Children and Families with substantiated issues

Populations

Front-line Practitioners and Case Managers

Intervention Strategies and Measures (WHAT)

Alternative Response Model

Practice, Organizational and Systems Change Strategies (HOW)

Carefully selected staff and the provision of skillful, timely training, coaching, performance assessments , use of data in supportive administrative and state /tribal environments

Intervention Outcomes

Increased family engagement in services Improved safety at home Reduced incidents of recurrence

Implementation Related Outcomes Competent

use of selected Interventions & approaches Alternative Response (fidelity measures or staff performance assessments) Personnel in Child Welfare Training Systems, Supervisors, Data System Managers, Agency Administrators, State/Tribal Administrators Agreements with Program Developers and Consultants Training and coaching for Trainers – capacity building Release time for Supervisors to learn to coach Installation of fidelity monitoring systems Skillful, timely training, coaching, performance assessments , data systems in supportive administrative and regulatory environments

Population

Young Children at Risk of Child Abuse Children and Families with substantiated issues

Populations

Front-line Practitioners and Case Managers

Cascading Logic Model

Intervention Strategies and Measures (WHAT)

Alternative Response Model

Practice, Organizational and Systems Change Strategies (HOW) Intervention Outcomes

Increased family engagement in services Improved safety at home Reduced incidents of recurrence

Implementation Related Outcomes

Carefully selected staff and the provision of skillful, timely training, coaching, performance assessments , use of data in supportive administrative and state /county environments

Competent

use of selected Interventions & approaches Alternative Response (fidelity measures or staff performance assessments) Personnel in Child Welfare Training Systems, Supervisors, Data System Managers, Agency Administrators Program and Agency Administrators, Child Welfare Agency Personnel at the Tribal/ State/County/Local level/External Partners/Courts Agreements with Program Developers and Consultants Training and coaching for Trainers – capacity building Release time for Supervisors to learn to coach Installation of fidelity monitoring systems & outcome data Skillful, timely training, coaching, performance assessments , data use in supportive administrative environments Common mission, vision developed Stages of Implementation utilized Linked Implementation Teams created Formal structures created to develop practice informed policy loops Changes in Funding Streams support new functions and relationships Fidelity and outcome data systems developed and maintained Collaboration among providers to pool funding for infrastructure Sustainable infrastructure developed Relationships with consultants & Program Developers formalized Training and coaching for Trainers (pre/post) Release time for Supervisors to learn to coach Fidelity monitoring systems exist and are used Outcome measures exist and are used Funding and regulatory changes occur to support infrastructure

Population

Young Children at Risk of Child Abuse Children and Families with substantiated issues

Populations

Front-line Practitioners and Case Managers

Cascading Logic Model

Intervention Strategies and Measures (WHAT)

Alternative Response Model

Practice, Organizational and Systems Change Strategies (HOW) Intervention Outcomes

Increased family engagement in services Improved safety at home Reduced incidents of recurrence

Implementation Related Outcomes

Carefully selected staff and the provision of skillful, timely training, coaching, performance assessments , use of data in supportive administrative and state /tribal environments

Competent

use of selected Interventions & approaches Alternative Response (fidelity measures or staff performance assessments) Personnel in Child Welfare Training Systems, Supervisors, Data System Managers, Agency Administrators Program and Agency Administrators, Child Welfare Agency Personnel at the Tribal/ State/County/Local level/External Partners/Courts Agreements with Program Developers and T/TA Centers Training and coaching for Trainers – capacity building Release time for Supervisors to learn to coach Installation of fidelity monitoring systems & outcome data Skillful, timely training, coaching, performance assessments , data use in supportive administrative environments Common mission, vision developed Stages of Implementation utilized Linked Implementation Teams created Formal structures created to develop practice informed policy loops Changes in Funding Streams support new functions and relationships Fidelity and outcome data systems developed and maintained Collaboration among providers to pool funding for infrastructure Sustainable infrastructure developed Relationships with consultants & Program Developers formalized Training and coaching for Trainers (pre/post) Release time for Supervisors to learn to coach Fidelity monitoring systems exist and are used Outcome measures exist and are used Funding and regulatory changes occur to support infrastructure

Stages of Implementation

Two to Four Years

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

Exploration

“ Many implementation efforts fail because someone underestimated the scope or importance of preparation.

Indeed, the organizational hills are full of managers who believe that an innovation’s technical superiority and strategic importance will guarantee acceptance.” Leonard-Barton & Kraus, Harvard Business Review, 1985 Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Exploration

Exploration Goals: Examine degree to which the EBP, best practice, systems change meets the needs in the settings identified Determine whether moving ahead with the initiative and implementation is desirable and feasible Create readiness for change at many levels

“Pay now or pay later.”

Assess Feasibility

Needs Fit Resource availability Evidence Readiness for replication or degree to which it is operationalized Capacity

Assessing Fit and Feasibility of Initiatives

Need Capacity

Staff meet minimum qualifications Able to sustain Imp Drivers • Financially • Structurally Buy-in process operationalized • Practitioners • Families • Agency and Departments

Capacity to Implement

Initiative :

Readiness

Qualified purveyor Expert TA available Mature sites to observe # of replications How well is it operationalized?

Are Imp Drivers operationalized?

Intervention Readiness for Replication

5 Point Rating Scale: High = 5; Medium = 3; Low = 1. Midpoints can be used and scored as a 2 or 4.

High Medium Low Need Fit Resources Availability Evidence Readiness for Replication Capacity to Implement Total Score:

Need

in Tribal/ State, Agency, Setting Socially Significant Issues Parent & Community Perceptions of Need Data indicating Need

Fit Fit

with current •Initiatives • Tribal//State, County, Agency Priorities • Organizational structures • Community Values

Resource Availability Resources

Staffing Training Data Systems Coaching & Supervision Administrative & system supports needed Time

Evidence Evidence – is there any?

Outcomes – Is it worth it?

Fidelity or process data Cost – effectiveness data Number of studies Population similarities Diverse cultural groups Efficacy or Effectiveness © National Implementation Research Network 2009 Adapted from work by Laurel J. Kiser, Michelle Zabel, Albert A. Zachik, and Joan Smith at the University of Maryland

Assess Buy-in and Create Readiness

Determine “buy-in” process and measures Prepare decision makers, agency leads, and partners for implementation and….

Operationalize the policy, evidence-based innovation, or systems change Describe benefits and risks and help interested parties gain confidence in managing risk Assess agencies’ and/or collaborating partners’ comfort with ability to manage risk Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Resistance to Change

There is no such thing – only inadequate preparation It is not “their” problem, it is ours.

Creating Readiness for Change

Individual readiness for change

Transtheoretical Model or

Stages of Change

Stage of Change for Pre-Action Individuals: Precontemplation – 40% Contemplation – 40% Preparation – 20% “If only 20% of employees in organizations are prepared to take action. . . .” Janice M. Prochaska, James O. Prochaska, and Deborah A. Levesque (2001) Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Working with the “Not Quite Ready”

Provide Information What is the change, why now, and what might it mean for me?

“Reflect” Concerns In a neutral way, acknowledge and validate the person’s concerns (e.g. “It sounds like you need more information…”, “It makes sense to me that you are wondering about whether this is a passing fad…”) Roll with Resistance Resistance is a signal to change strategies New perspectives are invited but not imposed (e.g. “Would you like the opportunity to learn more about this?”) The person’s perspective is a valuable resource in learning more about what is concerning or challenging about the change initiative Miller and Rollnick, 1991 Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Engaging Stakeholders: Rationales



A Rationale is a statement of

  Naturally or logically occurring outcomes Associated with current or future goals and/or behavior (“if…..then”).



Benefits

 Data indicate that providing rationales:  Is respectful and supportive of honest exploration of issues   Promotes choice and “buy-in”  Increases confidence in decision-making Promotes understanding of “functions” of the decisions being made rather than fostering compliance Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Developing Rationales

 Brief  Aligned with Stage of Change  Relevant to the Individual  Pre-Contemplation and Contemplation   Promote or provide assurance about the “upside” of taking risks Points out the “downside” of not moving forward – a different view of ‘risk’  Requires a leader to be “on purpose” Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Installation

Installation Stage Goal

To make the structural and instrumental changes necessary to initiate services

“If you build it, they will come”. . . but you actually have to build it!

Installation

Activities Mutual Selection of Provider Organization (Community, Organization, Program Experts) Initial training provided for practitioners and provider organization staff Negotiate and enact structural and instrumental changes needed to initiate the program or practice (write new job descriptions, recruit, hire/redeploy staff, make special purchase of equipment, arrange for space, organize billing procedures) Establish referral flow, develop ancillary contacts Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Initial Implementation

Initial Implementation Goal Survive the awkward stage!

Learn from mistakes Continue “buy-in” efforts Manage expectations

“Anything worth doing…is worth doing poorly.”

Initial Implementation

Activities Provide new services Manage staff turnover Manage change process Change practice Change organizational and community structure and culture Put infrastructure components in place Overcome fear and inertia Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Full Implementation

Full Implementation Goals

Maintaining

and

improving

skills, activities, linkages throughout the system Components integrated, fully functioning Skillful practices by front line staff, supervisors, administrators Changes in policy that are reflected in practice at all levels Ready to be evaluated for expected outcomes

“The only thing worse than failing and not knowing why you failed, is succeeding and not knowing why you succeeded.” ~ Jane Timmons-Mitchell

Innovation

Innovation First do it right (high fidelity) or “as intended” Then do it differently

and better

Consult with content experts Evaluate impact Ability to retain

function

while changing

form

given turnover, changing needs and context Ability to adhere to the underlying theory of change “

Innovation needs to equal improvement.”

Sustainability

Goals of Sustainability Financial: Ensure funding streams for service and infrastructure Programmatic: Ensure high fidelity and positive outcomes through infrastructure improvement and maintenance Plan for turnover

“The only thing harder than getting there is staying there.”

Reflection: Stages

What might this information tell us about how Alternative Response is implemented in the remaining counties? Should we build in time for “exploration” and planning for these change initiatives?

Does this inform our work as we attempt to promote the scaling up of Alternative Response statewide in Ohio? What lessons have pilot counties learned regarding exploration work? installation work? initial implementation?

What might the State or TA provider do to facilitate the new counties’ work during the stages of implementation?

Putting “It” Into Action

Who will do what differently?

Who will help change occur? AND How will change happen? At all levels….

Improved outcomes for children and families

Implementation Drivers

Performance Assessment Training Selection Coaching Adaptive

Integrated & Compensatory

Technical Leadership Systems Intervention Facilitative Administration Decision Support Data System

Graphics by Steve Goodman,2009

Fixsen & Blase, 2008

Implementation Drivers

Competency Drivers

are mechanisms that help to

develop, improve, and sustain one’s ability to implement an

intervention with fidelity and benefits to consumers.

Competency Drivers include: Selection, Training, Coaching, and Performance Assessment

Organization Drivers

are mechanisms to

create and sustain hospitable organizational and systems environments

effective services. Organization Drivers include: Decision for Support Data System, Facilitative Administration, and Systems Intervention

Leadership Drivers

problems

are

methods to manage Technical

where there is high levels of agreement about problems and high levels of certainty about solutions and to constructively

deal with Adaptive challenges

Improved outcomes for children and families Performance Assessment Coaching Training

Implementation Lens

Selection

Implementation Drivers Graphics by Steve Goodman,2009

Fixsen & Blase, 2008

Sobering Observations

"All organizations [and systems] are

designed, intentionally or unwittingly, to achieve precisely the results they

get."

R. Spencer Darling Business Expert

Improved outcomes for children and families

Implementation Drivers

Training Selection Performance Assessment Coaching Systems Intervention Facilitative Administration Decision Support Data System

Graphics by Steve Goodman,2009

Fixsen & Blase, 2008

Decision-Support Data Systems

Measure Fidelity AND Outcomes BECAUSE you need to know: Are we having an implementation problem?

Low fidelity & Poor outcomes = Implementation problem Are we having an effectiveness problem?

Improved outcomes for children and families

Implementation Drivers

Training Selection Performance Assessment Coaching Systems Intervention Facilitative Administration Decision Support Data System ©

Fixsen & Blase, 2008

System Stability

EXISTING SYSTEM Effective Innovations are Changed to Fit the System Or Operate in the Shadows (The Ghost System)

Effective System Change

EXISTING SYSTEM EXISTING SYSTEM IS CHANGED TO SUPPORT THE EFFECTIVENESS OF THE INNOVATION Effective Innovations are Changed to Fit the System Or Operate in the Shadows (Ghost System)

(Host System) EFFECTIVE INNOVATION

Improved outcomes for children and families

Implementation Drivers

Performance Assessment Training Selection Coaching Systems Intervention Adaptive

Integrated &

Compensatory

Technical Leadership Facilitative Administration Decision Support Data System

Graphics by Steve Goodman,2009

Fixsen & Blase, 2008

Reflection: Implementation Drivers

How is the information related to Implementation Drivers relevant to scaling up Alternative Response?

How do you see using the information regarding the Drivers in your work to support the continued implementation of Alternative Response in pilot counties?

How do you see using the information regarding the Drivers in your work to support implementation of Alternative Response in new counties?

Initiating and Managing Change Implementation Team(s)

General Definition : Core group of individuals, who are representative of the stakeholders and “systems” and who are charged with guiding the overall implementation from exploration through to full implementation Benefits : Provides a focused and accountable structure to increase the likelihood that this effort will not be abandoned or derailed Scope of the initiative determines the scope of authority and the need for linked

Implementation Teams

Core Competency for Implementation Teams

A team that:

■

Knows the innovation very well (formal and practice knowledge)

■

Knows implementation very well (formal and practice knowledge)

■

Knows improvement cycles to make intervention and implementation methods more effective and efficient over time

■

Promotes systems change at multiple levels to create hospitable cultures, policies, and funding streams

Implementation Teams

Integrated and Interlocking at Multiple Levels Agency or “setting-based” teams Community or collaborative teams Tribal or state team Represents the stakeholders and the ‘system’ Focus is on Ongoing “buy-in” and readiness Installing and sustaining the drivers Fidelity & outcomes Alignment (funding and policy) Building the new system – linkages Problem-solving and sustainability Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Catawba County Child Well-Being Project

Consumers Sector Leads Funders Design Team Meta- Team Leads + Design Team Rep’s Key DSS Leaders Stakeholders Providers Senior Mgmt Team EBP Area, Agency, or Unit EBP Area, Agency, or Unit EBP Area, Agency, or Unit

IMPLEMENTATION TEAMS

Reflection

– Implementation Teams

What implications does the information regarding implementation teams have for work with pilot and new counties?

How can the State and counties ensure that implementation teams are in place, clear on their roles, and linked to support the implementation of Alternative Response?

Improvement Cycles

Policy to Practice to Policy Cycles Transformation Zones Rapid Cycle Teams

Problem-solving Practice Improvement

Usability Testing

Policy Practice

Policy (Plan) Policy Structure Practice (Do) Procedure Practice

Policy Practice Feedback Loops

Policy (Plan) Policy Structure Procedure Practice

Practice (Do)

Reflection

– Improvement Cycles

Are policy-practice feedback loops currently in place in pilot counties? What form do they take?

How can policy-practice feedback loops be established in each county and statewide?

Summary

Stage related activities prepare the system for a successful change process Competence needs to be developed and sustained

Selection, training, coaching, fidelity measures help change and support new practitioner behavior and skills

Organizations and systems need to change

Data systems need to be used to make decisions Facilitative administrative practices & systems interventions create hospitable environments Policy enables new practice but practice needs to inform policy Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Summary

Purveyors and implementation teams help with system and service change Scaling up requires establishing implementation capacity and tending to necessary “scale-up shifts” You are never done – The environment is in motion

For More Information

Allison Metz, PhD 919-218-7540

[email protected]

At the Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC

www.scalingup.org

For More Information

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005).

Implementation Research: A Synthesis of the Literature.

Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

Download all or part of the monograph at:

http://www.fpg.unc.edu/~nirn/resources/publications/ Monograph/

To order the monograph go to:

Systems and Practice Change Through Implementation Lens

Transcript Systems and Practice Change Through Implementation Lens

System and Practice Change Through an Implementation Lens

Common Goal

To build capacity in child welfare agencies and to support Counties in achieving the sustainable implementation infrastructure and systemic change necessary for an effective Alternative Response model….

. . .

To improve outcomes for families and children

Overall Challenge

Science to Service Gap

Implementation Gap

From the Synthesis of the Literature We Know That…

Positive Intervention Outcomes ≠ Implementation

Problem

Children and families cannot benefit from interventions they do not experience

Insufficient Methods

What Works?... Fidelity Matters

What Works?...Purveyor

Purveyor AND Intermediary Structures and Strategies Simultaneous, Multi-Level Interventions

What Works? …Implementation Best Practices

Change What?….

Practice? Program? Organization? Systems?

Cascading Logic Model

Stages of Implementation

Exploration

Exploration

Assess Feasibility

Assess Buy-in and Create Readiness

Resistance to Change

There is no such thing – only inadequate preparation It is not “their” problem, it is ours.

Creating Readiness for Change

Stages of Change

Stages of Change

Working with the “Not Quite Ready”

Engaging Stakeholders: Rationales

Developing Rationales

Installation

Installation

Initial Implementation

Initial Implementation Goal Survive the awkward stage!

“Anything worth doing…is worth doing poorly.”

Initial Implementation

Full Implementation

Innovation

Sustainability

Reflection: Stages

Putting “It” Into Action

Implementation Drivers

Sobering Observations

"All organizations [and systems] are

designed, intentionally or unwittingly, to achieve precisely the results they

get."

Decision-Support Data Systems

System Stability

Effective System Change

Reflection: Implementation Drivers

Initiating and Managing Change Implementation Team(s)

Core Competency for Implementation Teams

Implementation Teams

Catawba County Child Well-Being Project

Reflection

– Implementation Teams

Improvement Cycles

Policy to Practice to Policy Cycles Transformation Zones Rapid Cycle Teams

Usability Testing

Policy Practice

Policy Practice Feedback Loops

Reflection

– Improvement Cycles

Summary

Summary

For More Information

For More Information

Directory