Transcript SCCOE Aug 2012 HANDOUTS SEND How to implement EBPs
Santa Clara County Office of Education
Leading Systems Development for the Implementation of the Common Core State Standards Michelle A. Duda, Barbara Sims, Dean L. Fixsen, Karen A. Blase,
August 2012
National Implementation Research Network FPG Child Development Institute University of North Carolina at Chapel Hill
Goals for Today’s Session •
Introduction and Overview of Implementation Science Four Active Implementation Frameworks
• Implementation Team • Improvement Cycles • Implementation Drivers • Implementation Stages •
Moving from “theory to practice”
– How to use implementation science to create the Infrastructure and readiness for CCSS
Our Broader Goals •
Reliably Produce
•
Predictable outcomes for children, families and communities
– (i.e. achieving outcomes defined by the CCSS) •
That improve every year
•
For the next 50 years
Education
The Challenge
We work in complex environments with: Unpredictable people Competing demands Shifting priorities Various points of view
FACT
Students cannot benefit from interventions they do not experience
FICTION
Good science
(evidence-based practices)
leads to
Good practices (implementation science)
National Assessment of Education Progress (NAEP)
WHAT IS IT GOING TO TAKE?
The Challenge….
• Science to Service Gap – What is known is not what is adopted to help students • Implementation Gap – What is adopted is not used with fidelity and good outcomes – What is used with fidelity is not sustained for a useful period of time – What is used with fidelity is not used on a scale sufficient to broadly impact student outcomes
Implementation is defined as
a specified set of activities
designed to put into practice an activity or program of
known dimensions. RESEARCH PRACTICE Why Focus on Implementation?
SO THAT,
ALL
students
CAN
interventions they
DO
benefit from experience
Effective IMPLEMENTATION Effective
Actual Benefits
NOT Effective Inconsistent; Not Sustainable; Poor outcomes NOT Effective Unpredictable or poor outcomes; Poor outcomes; Sometimes harmful (Institute of Medicine, 2000; 2001; 2009; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education, 1983; Department of Health and Human Services, 1999)
Data Show These Methods, When Used Alone, Do Not Result In Implementation As Intended:
–
Diffusion/ Dissemination of information
–
Training
–
Passing laws/ mandates/ regulations
–
Providing funding/ incentives
–
Organization change/ reorganization
Necessary But Not Sufficient
Longitudinal Studies of a Variety of Comprehensive School Reforms Evidence based Actual Supports Years 1-3 Outcomes Years 4-5 Every Teacher Trained Fewer than 50% of the teachers received some training Fewer than 10% of the schools used the CSR as intended Every Teacher Continually Supported Fewer than 25% of those teachers received support Vast majority of students did not benefit Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006
• It is not a “school problem” • District is the point of entry for school improvement – Use short-term infusion of resources – Establish long-term, district-based capacity for quality
Student Practitioner System SISEP 2012
• • •
Letting it happen
– Recipients are accountable
Helping it happen
– Recipients are accountable
Making it happen
– Purposeful use of implementation practice and science – Implementation system is accountable
Based on Hall & Hord (1987); Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou (2004); Fixsen, Blase, Duda, Naoom, & Van Dyke (2010)
SISEP 2012
• To successfully implement and sustain evidence-based and evidence-informed interventions, we need to know: –
The WHAT -
What is the intervention (e.g. effective instruction, effective assessment) –
The HOW -
Effective implementation and sustainability frameworks (e.g. strategies to change and maintain behavior of adults) –
The WHO
– Organized, expert implementation assistance
Implementation Science
Implementation Research: A Synthesis of the Literature
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005).
Implementation Research: A Synthesis of the Literature.
Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
HTTP://NIRN.FPG.UNC.EDU
• Testable Hypotheses • Organize Implementation Knowledge • Identify Gaps • Guide Development of Implementation Capacity
Work of Implementation
Changing the thinking and behavior of:
• Adult human service professionals (teachers, staff) positive outcomes are more frequently created for students • Administrators in organizations (Principals, Assistant Principals) • System directors, policy makers, and funders (District
So that
Staff, SEA) structures, cultures, and climates change to support new ways of work implementation is supported and outcomes can be achieved
Reflection
Implementation Science Supporting New Ways of Work
•
Why is it important to pay attention to the implementation process as we are creating our action plans for CCSS?
Active Implementation Frameworks
• Organized, expert assistance to develop and sustain an accountable structure
Implementation Teams
Improvement Cycles Implementation Drivers Implementation Stages
WHO will do the work?
•Purveyors good effect – a group of individuals representing a program or practice who actively work with organizations and communities to help them implement that practice or program with fidelity and •An Intermediary Purveyor Organization that becomes expert in implementation and a “bridge” or expert with multiple EBPs (a new way of doing T & TA) connecting providers and purveyors •“Local” Implementation Team a collaboration of agencies) with the knowledge, skill, freedom, and authority to act (e.g. within a larger organization or
Implementation Teams
IMPLEMENTATION Impl. Team NO Impl. Team Effective 80%, 3 Yrs 14%, 17 Yrs Making it Happen Letting it Happen Helping it Happen Fixsen, Blase, Timbers, & Wolf, 2001 Balas & Boren, 2000 Green & Seifert, 2005
Linked Team Structures
State-based Implementation Team Regionally-based Implementation Team District-based Implementation Team School-based Implementation Team “We tend to focus on snapshots of isolated parts of the system and wonder why our deepest problems never seem to get solved. (Senge, 1990)
Implementation Team
:
Sphere of Influence
Prepare & Work with Counties, Communities State, Tribe Prepare Staff & Administrators Create Readiness Implementation Teams Assure Child and Family Benefits T/TA, Purveyors & Researchers Family & Stakeholders Assure Implementation
Implementation Teams
Membership
•
Teams whose members collectively represent the “system as is” and the “system to be”
Know the innovation(s) very well (formal and practice knowledge)
Know implementation very well (formal and practice knowledge)
Know improvement cycles to make intervention and implementation methods more effective and efficient over time
Promote and participate in systems change at multiple levels to create hospitable cultures, policies, and funding streams
What does this mean for us?
• Implementation Teams ensure local capacity • Attend to systems alignment • Representation from those who know the “what” and the “how” • Use improvement cycles for purposeful adjustments
Reflection
Implementation Teams Supporting New Ways of Work
•
Do you currently have an Implementation Team?
•
Who are members you might include on your Implementation Team?
Active Implementation Frameworks
• Changing on purpose to support the new way of work Implementation Teams
Improvement Cycles
Implementation Drivers Implementation Stages
Improvement Cycles
Changing on Purpose People, organizations, and systems
• • • •
Cannot change everything at once Cannot stop and re-tool Cannot know what to do at every Many outcomes are not predictable Trial & Learning
Improvement Cycles PDSA Cycles 1. Rapid cycle problem solving (Shewhart; Deming) 2. Usability testing (Neilson; Rubin) 3. Transformation Zones 4. Practice-policy communication loops
Improvement Cycles Rapid Cycle Problem Solving
SISEP 2012
Improvement Cycles Usability vs. Pilot Testing Usability
• Clear description of the program • Trial and learning approach • Small number of participants with multiple iterations • Rapid cycle problem solving applied to program and system
Pilot
• Clear description of the program • Trial and learning approach • Sufficient number of participants and sufficient time to realize potential results SISEP 2012
Improvement Cycles Transformation Zone
• A “vertical slice” of the service system (from the classroom to the District, Region, State) – The “slice” is small enough to be manageable – The “slice” is large enough to include all aspects of the system – The “slice” is large enough to “disturb the system ” – a “ghost” system won’t work.
Policy Practice Feedback Loops
Policy (Plan) Policy Structure Procedure Practice (Do) FORM SUPPORTS FUNCTION Practice
What does this mean for us?
• Change is difficult • Improvement Cycles provide a process for change • PDSA cycle is used so we get better and better • Practice-Policy Feedback Loops promote hospitable environments for effective practice
Reflection
Improvement Cycles and Communication Loops Supporting New Ways of Work
•
How can we make use of improvement cycles in developing and implementing our improvement activities?
•
Linking Communication Protocols
Active Implementation Frameworks
Common features of Implementation Teams Improvement Cycles successful supports to help make full and effective use of a
Implementation Drivers
Implementation Stages wide variety of innovations
Implementation Drivers
What are Implementation Drivers?
•
Mechanisms to:
Develop, improve, and sustain one’s ability to implement an intervention to benefit students
(Competency Drivers)
• Create and sustain hospitable organizational and systems environments for effective educational services
(Organization Drivers)
• Ensure continuous improvement cycles are moving information forward and information backward to improve alignment overtime
(Leadership Drivers)
Reliable Benefits and Consistent Use of Interventions
Performance Assessment (Fidelity)
Training Selection Coaching
Integrated & Compensatory
Systems Intervention Facilitative Administration Decision Support Data System Technical
Leadership
Adaptive
© Fixsen & Blase, 2007
Build Competency and Confidence • Develop, improve, and sustain competent & confident use of innovations
Competency Drivers
Performance Assessment
• • • •
Measure fidelity Ensure implementation Reinforce staff and build on strengths Feedback to agency on functioning of
– Recruitment and Selection Practices – Training Programs (pre and in-service) – Supervision and Coaching Systems – Interpretation of Outcome Data
Competency Drivers
Selection
•
Select for the “unteachables”
•
Screen for pre-requisites
•
Set expectations
•
Allow for mutual selection
•
Improve likelihood of retention after “investment”
•
Improve likelihood that training, coaching, and supervision will result in implementation
Competency Drivers • • • •
Timeliness “Buy-in” Knowledge acquisition Skill Development
Training
TRAINING COMPONENTS Theory and Discussion ..+Demonstration in Training …+ Practice & Feedback in Training …+ Coaching in Classroom OUTCOMES % of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting, and Use new Skills in the Classroom Knowledge 10% 30% 60% Skill Demonstration 5% 20% 60% 95% 95%
Joyce and Showers, 2002
Use in the Classroom 0% 0% 5% 95%
Competency Drivers
Coaching
• • • • •
Ensures fidelity Ensures implementation Develops clinical and practice judgment Provides feedback to selection and training processes Grounded in “Best Practices”
Change Organizations and Systems • Create and sustain hospitable organizational and system environments for effective services
Implementation Drivers
What do we need to know about successful organizational change and system change methods?
• New innovations do not fare well in old organizational structures and systems • Develop new position descriptions and job functions in State Departments of Education and in Regional and District systems • “Systems trump programs.” Patrick McCarthy, Annie E. Casey Foundation
Organizational Drivers
Decision Support Data Systems
• • • • • • •
Improves student outcomes through data-based decisions Provides information to assess effectiveness of intervention and prevention practices Analyzes the relationship of fidelity to outcomes Guides further program development Engages us in continuous quality improvement Celebrate success Be accountable to consumers and funders
Organizational Drivers
Facilitative Administration
•
Facilitates installation and implementation of the Drivers
•
Aligns policies and procedures
•
Takes the lead on Systems Interventions
•
Looks for ways to make work of practitioners and supervisors more effective and less “burdensome”!!
Organizational Drivers
Systems Intervention
•
Identifies barriers and facilitators for the new way of work
•
Creates an externally and internally “hospitable” environment for the new way of work
•
Contributes to cumulative learning in multi-site projects
What do we know about System Stability?
EXISTING SYSTEM Effective Innovations are Changed to Fit the System Or Operate in the Shadows (The “Ghost” System)
What do we know about Effective System Stability?
EXISTING SYSTEM EXISTING SYSTEM IS CHANGED TO SUPPORT THE EFFECTIVENESS OF THE INNOVATION Effective Innovations are Changed to Fit the System Or Operate in the Shadows (The “Ghost” System) (“Host” System) EFFECTIVE INNOVATION
• Different challenges call for different strategies – Technical Strategies – Adaptive Strategies
Leadership Drivers
Strategies
•
According to Ron Heifetz and his colleagues at Harvard ’s Kennedy School of Government, one of the biggest mistakes “leaders” make is to incorrectly identify the type of challenge they are facing
– Using technical approaches for adaptive issues (and vice versa)
Leadership Drivers
Technical Challenges
• Perspectives are aligned (views, values) • Definition of the problem is clear • Solution and implementation of the solution is relatively clear • There can be a “primary” locus of responsibility for
organizing
the work
Leadership Drivers
Technical Strategies
• Use established norms/ goals • Define problems • Provide solutions • Clarify roles and responsibilities • Assign tasks • Manage conflict • Maintain order
Leadership Drivers
Adaptive Challenges
• Legitimate, yet competing, perspectives emerge • Definition of the problem is unclear • There are different perspectives on the “issue” at hand • Solution and implementation is unclear and requires learning • Primary locus of responsibility is not a single entity or person
Leadership Drivers
Adaptive Strategies
• Get on the Balcony • Identify the Adaptive Challenge • Regulate Distress • Maintain Disciplined Attention • Give the Work Back to the People • Protect All Voices Ron Heifetz, Leadership without Easy Answers, 1996
• Infrastructure needed becomes visible to all • Strengths and progress get celebrated • Next right steps are planned and results measured • Resources can be aligned and re purposed to improve implementation
Reflection
Supporting New Ways of Work Implementation Drivers
•
Do we need to promote ‘competence and confidence’ of educators? How can we build in state capacity to select, train, coach, and assess performance well?
•
How are we engaged in promoting more hospitable organizational environments?
•
Do we have support of leadership? How do we know?
Active Implementation Frameworks
Purposeful matching of critical implementation activities to the stage of the process Implementation Teams Improvement Cycles Implementation Drivers
Implementation Stages
Stages of Implementation
AWARENESS TRANSITION Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 FULL IMPLEMENTATION
Stages of Implementation
Exploration
•
Exploration (Awareness) Stage Goals
– Create readiness for change • Changing hearts and minds – Examine degree to which the proposed strategies and practices meet the needs of our State and our students – Determine whether adoption and implementation are desirable and feasible “Pay now or pay later.”
Stages of Implementation
Exploration
What happens during Exploration (Awareness) Stage?
• Formalize Team Structures • Develop Communication Plan • Determine Need and Identify Options • Assess “Fit” and Feasibility • Promote “Buy in” for the innovation and for implementation supports • Make recommendations
Assessing Evidence-Based Programs and Practices
Capacity
Staff meet minimum qualifications Able to sustain Implementation Drivers • Financially • Structurally Buy-in process operationalized • Educators • Administrators • Families
Capacity to Implement Need
EBP:
Readiness
Qualified purveyor Expert or TA available Mature sites to observe # of replications How well is it operationalized?
Are Imp Drivers operationalized?
Intervention Readiness for Replication
5 Point Rating Scale: High = 5; Medium = 3; Low = 1. Midpoints can be used and scored as a 2 or 4.
High Medium Low Need Fit Resources Availability Evidence Readiness for Replication Capacity to Implement Total Score:
Need
in the Educational Setting, Socially Significant Issues, Parent & Community Perceptions of Need, Objective Data indicating Need
Fit Resource Availability Fit
with current •Initiatives • RtI Implementation • School and District Priorities • Organizational structures • Community Values
Resources
Curricula & Classroom Materials, IT requirements, Staffing, Training and PD, Data Systems, Coaching & Supervision, Administrative & system supports needed
Evidence Evidence
Outcomes – Is it worth it?
Fidelity data Cost – effectiveness data Number of studies Population similarities Diverse cultural groups Efficacy or Effectiveness
© National Implementation Research Network- 2009
Reflection
Creating Readiness Supporting New Ways of Work
•
What role can you play in developing readiness for building in CCSS in your district?
•
What are 2 things your team could to tomorrow to asses your current infrastructure?
•
District Initiative Inventory
•
Analysis of Evidence-based Programs or Practices “aka” hexagon tool
Stages of Implementation
Installation
•
Installation (Transition) Stage Goals
• • • • • •
Structural and functional changes are made Selection protocols developed First implementers selected Define and initiate training of first implementers Develop coaching plans Evaluate readiness and sustainability of data systems
Stages of Implementation
What ’s Needed: Installation
• High-level protection, problem solving, and support • Reduced expectations and higher costs during start up • Help in evolving organizational supports at every level • Help in establishing new school, community, and organizational climate and culture
Stages of Implementation
Initial Implementation Initial Implementation (Transition) Stage Goals -Get started, then get better!
• Learn from mistakes • • • • Celebrate progress Continue “buy-in” efforts Make systemic changes Manage expectations
-All the components of the program or innovation are in place and the implementation supports begin to function
Stages of Implementation
Initial Implementation
•
Work through the Awkwardness
– Managing Change – Managing Expectations •
Provide training and coaching on the evidence-based practice, re-organization of school roles, functions and structures
•
Make use of improvement cycles to resolve systems issues
Stages of Implementation
Full Implementation Full Implementation Stage Goals
– Maintaining and improving skills and activities throughout the system – Components integrated, fully functioning – Skillful practices by front line staff, supervisors, administrators – Changes in policy that are reflected in practice at all levels – Ready to be evaluated for expected outcomes
Stages of Implementation
Full Implementation
•
“What Change?....This is our way of work!
”
– Skillful Teaching and School Practices – Skillful Use of the Drivers • Drivers experience their own Improvement Cycles – Data Systems in use, reliable, efficient, and used for Decision-Making at multiple levels to regenerate and improve – Policy to Practice and Practice to Policy Feedback Cycles
Reflection
Supporting New Ways of Work
•
What are you already doing that is “stage-based”?
•
What are the facilitators and barriers to doing stage-based work?
– Stages of Implementation Analysis
What will it Take?
California Common Core State Standards
•
District level system change
– Information, what it is, is not – Stakeholder buy-in, – Application/selection process •
Informed agreement, understand and defend the initiative
What will it Take?
California Common Core State Standards
•
District level system change
– Give lots of rationales – Not a project, not patchwork – Focus on CCSS functions – Establish a common vocabulary to ease communication – Build on what folks are doing already – help them get ready for change
What will it Take?
California Common Core State Standards
•
Guided development
– Leadership involvement (require more in the future) – Year of training with follow up coaching (require more on-site visits in the future) – Include leaders in the training (well informed, able to explain and defend, willing to do what is required)
What will it Take?
California Common Core State Standards
•
Manage expectations and pace based on Stage of Implementation
• Teachers & Staff directly impact students • It is the job of principals, superintendents, and funders to align policies and structures to facilitate effective teacher & staff practices
There is no such thing as an “administrative decision” – they are all education decisions
The Work of Implementation
• • •
Changing the behavior of education system professionals is hard work It requires a systematic approach to support behavior change of
–
Teachers,
–
School and district personnel
–
TA providers,
–
State department of education personnel It requires both effective practices/programs (the “what”) and effective implementation processes (the “how”)
SISEP
Stay Connected!
www.scalingup.org
@SISEPcenter
For more on Implementation Science http://nirn.fpg.unc.edu
www.implementationconference.org
For More Information
Michelle A. Duda, Ph.D., BCBA-D [email protected]
Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC http://nirn.fpg.unc.edu/