Transcript How to implement EBPs

Implementation
and Scaling
Literacy Programs
Literacy Funders Network 2011
Dean L. Fixsen, Karen A. Blase,
Leah Bartley, Michelle Duda,
Sandra Naoom, Allison Metz,
Barbara Sims, Melissa Van Dyke
National Implementation Research Network
Frank Porter Graham Child Development Institute
University of North Carolina at Chapel Hill
Education
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Capacity Building
AMOUNTS
Implementation Teams
Organization Change
System Reinvention
YEARS
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Education
60 million students
6 million teachers and staff
90,000 school buildings
3,147 counties
58 federal jurisdictions
1 person in 5 in the USA is in school
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
All Schools Left Behind
5 Years of turnaround work
10% out of improvement status
90% still “in improvement”
Stuit (2011; Are bad schools immortal?)
Secretary Duncan predicts 82%
of all schools in the US will not
meet AYP standards in 2011
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Food for Thought
"All organizations [and systems] are
designed, intentionally or unwittingly, to
achieve precisely the results they get.”
…R. Spencer Darling
National Assessment of Education Progress (NAEP)
WHAT IS IT GOING TO TAKE?
Challenges
Students cannot benefit from
instruction they do not
experience
Teachers and staff have to
actually use effective
instructional methods
skillfully if students are to
benefit
Verbal advocacy ≠ Actual use
Dobson & Cook (1980)
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Teachers and Staff
In education, the TEACHER IS THE
INTERVENTION
Everyone / everything else needs
to be aligned to provide effective
supports so all teachers can
produce desired education
outcomes for all students
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Consider This
To meet the individualized learning
needs of students …
Educators need to “forgo
uniqueness” in favor of consistent
use of effective methods …
That are well supported by district
and building staff.
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Evidence-based
Implementation
Research:
A Synthesis of
the Literature
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005).
Implementation Research: A Synthesis of the Literature. Tampa, FL: University of
South Florida, Louis de la Parte Florida Mental Health Institute, The National
Implementation Research Network (FMHI Publication #231).
Download all or part of the monograph at:
http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Implementation Gap
Interventions are not used as
intended and with good outcomes
Interventions are not sustained
for a useful period of time
Interventions are not used on a
scale sufficient to impact social
problems
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Implementation Science
Longitudinal Studies of a Variety of Comprehensive School Reforms
Evidence-base
Actual Supports
Years 1-3
Outcomes
Years 4-5
Every Teacher
Trained
Fewer than 50% of
the teachers
received some
training
Fewer than 10% of
the schools used the
CSR as intended
Every Teacher
Continually
Supported
Fewer than 25% of
those teachers
received support
Vast majority of
students did
not benefit
Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Implementation Science
Letting it happen
Recipients are accountable
Helping it happen
Recipients are accountable
Making it happen
Implementation Teams are
accountable
Based on Hall & Hord (1987); Greenhalgh, Robert, MacFarlane, Bate,
& Kyriakidou (2004); Fixsen, Blase, Duda, Naoom, & Van Dyke (2010)
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Implementation Science
Best Data Show These Methods, When
Used Alone, Do Not Result In Uses
of Innovations As Intended:
Diffusion/ Dissemination of information
Training
Passing laws/ mandates/ regulations
Providing funding/ incentives
Organization change/ reorganization
About 5% to 20% Return on Investment
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Formula for Success
Effective intervention practices
X
Effective implementation practices
=
Intended outcomes
Disproportional Impact: Students
benefit 8 to 12 times more
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Implementation Science
An intervention is one thing
Implementation is something
else altogether
Like serum and a syringe
Very different evidence bases
Each is necessary
Neither one is useful without the
other
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
2013
www.implementationconference.org
Implementation Capacity
It is not enough to have high
standards, the teachers must be
supported if they are to teach
consistently and effectively
Shifts accountability for outcomes
from the teachers to the staff in
buildings, districts, and supporting
regional and State education
systems
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Literacy Improvement
Improve student outcomes
Improve teacher performance
Improve school supports for teachers
Improve district supports for schools
Improve regional supports for districts
Improve State supports for outcomes
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Reliable Benefits
Consistent uses of
Innovations
Performance Assessment
(fidelity)
Systems
Intervention
Coaching
Facilitative
Administration
Training
Selection
Interventions
meet
Implementation
Integrated &
Compensatory
Decision Support
Data System
Leadership Drivers
Technical
Adaptive
© Fixsen & Blase, 2008
Staff Training & Coaching
OUTCOMES
(% of Participants who Demonstrate Knowledge, Demonstrate
new Skills in a Training Setting,
and Use new Skills in the Classroom)
Knowledge
Skill
Demonstration
Use in the
Classroom
Theory and
Discussion
10%
5%
0%
..+Demonstration
in Training
30%
…+ Practice &
Feedback in
Training
60%
60%
5%
…+ Coaching in
Classroom
95%
95%
95%
TRAINING
COMPONENTS
20%
0%
Joyce and Showers, 2002
Literacy Improvement
Improve student outcomes
Improve teacher performance
Improve school supports for teachers
Improve district supports for schools
Improve regional supports for districts
Improve State supports for outcomes
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Literacy
Outcomes
for Students
Teachers
Staff
Administrators
Boards
HOW
Implementation
Teams
Implementation
Processes
WHO
Exploration
Installation
Initial Implementation
Full Implementation
Implementation
Performance
Assessments
(fidelity)
WHAT
Effective Instructional
Practices
Intervention
Performance
Assessments
(fidelity)
Implementation Science
Letting it happen
Recipients are accountable
Helping it happen
Recipients are accountable
Making it happen
Implementation Teams are
accountable: THEY DO THE
WORK
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Implementation Team
Minimum of three people (four or
five preferred) to promote effective,
efficient, and sustainable
implementation, organization change,
and system transformation work
Tolerate turnover; teams are
sustainable even when the players
come and go
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Implementation Team
Implementation Team
Simultaneous, Multi-Level Interventions
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Teacher/Staff Competence
Building/District Supports
Management (leadership, policy)
Administration (HR, structure)
Supervision (nature, content)
Regional Authority Supports
State and Community Supports
Implementation Team
Prepare
Buildings &
Districts
Prepare Regions
80%
Implementation
Team
Parents and
Work with
Researchers Stakeholders
Create Readiness
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Prepare Teachers
and Staff
Assure
Student
Benefits
20%
Assure Implementation
© Fixsen & Blase, 2009
Implementation Science
IMPLEMENTATION
INTERVENTION
Impl. Team
Effective
80%, 3 Yrs
NO Impl. Team
14%, 17 Yrs
Effective use of
Letting it Happen
Implementation
Helping it Happen
Science & Practice
Fixsen, Blase,
Timbers, & Wolf, 2001
Balas & Boren, 2000
3X to 12X Return on Investment
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Costs and Savings
Change in Budget (Percent)
Implementation Costs & Savings
(Inflation Adjusted)
120
110
100
90
80
70
60
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
1 Yr Pre
During
Post
Year 1
Post
Year 2
Post
Year 3
Statewide Change
Innovative practices do not fare
well in existing organizational
structures and systems
Organizational and system
changes are essential to
successful use of innovations
Expect it
Plan for it
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Compliance and Crises, Urgent, Time Sensitive!!
• Services not meeting Standards
• Deal with urgent and high profile issues
Best Practices
Implemented Fully
With Good Outcomes
System Supports & Stability
System Supports
& Stability
Mandates,
System Supports,
Foundational Polices & Regulations
• Regulatory roles
• Basic Data Systems
• Financing and Fiscal Accountability
• Accreditation/ Licensing Standards
• HR rules and regulations
• Safety Standards
• Work with Legislature
• Inclusion of Stakeholders
Leadership Responsibilities and Leverage Points
Thanks to Tom Bellamy
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
System
Change
♦Look for Faulty
Assumptions &
Errors; ♦Make
Needed Changes;
♦Invite System to
Respond
Implementation
Team
Teachers
Innovations
Students
Practice Informed
Policy
State
Management
Team
Policy Enabled
Practice
“External” System Change Support
System Reinvention
Adaptive Challenges
• Duplication
• Fragmentation
• Hiring criteria
• Salaries
• Credentialing
• Licensing
• Time/ scheduling
• Union contracts
• RFP methods
• Federal/ State laws
Capacity Building
AMOUNTS
Implementation Teams
Organization Change
System Reinvention
YEARS
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
For More Information
Karen A. Blase, Ph.D.
Dean L. Fixsen, Ph.D.
919-966-9050
919-966-3892
[email protected]
[email protected]
Frank Porter Graham Child Development Institute
University of North Carolina
Chapel Hill, NC
http://nirn.fpg.unc.edu/
www.scalingup.org
www.implementationconference.org
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Implementation Science
Implementation
Research:
A Synthesis of the
Literature
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005).
Implementation Research: A Synthesis of the Literature. Tampa, FL: University of
South Florida, Louis de la Parte Florida Mental Health Institute, The National
Implementation Research Network (FMHI Publication #231).
Download all or part of the monograph at:
http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=31
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Thank You for your Support
Annie E. Casey Foundation
(EBPs and cultural
competence)
National Institute of Mental
Health (research and training
grants)
William T. Grant
Foundation
(implementation literature
review)
Juvenile Justice and
Delinquency Prevention
(program development and
evaluation grants
Substance Abuse and
Mental Health Services
Administration
(implementation strategies
grants; national
implementation awards)
Office of Special Education
Programs (Scaling up
Capacity Development
Center)
Centers for Disease Control
& Prevention
(implementation research)
Copyright © Dean L. Fixsen and Karen A. Blase, 2011
Administration for Children
and Families (Child Welfare
Leadership; Capacity
Development)
Duke Endowment (Child
Welfare Reform)