Transcript Document

Effective Implementation as a Way of Work
Minnesota’s Journey
March 22, 2012
State Capital, Bismarck, North Dakota
Cammy Lehr, Ph.D.
Debra Price-Ellingstad, Ed.D.
Minnesota Department of Education
Agenda
• Introductions
• Overview of the Structure within the State
– MDE Organizational Structure
– Developing the infrastructure (teams and functions)
• Description of the Journey
– How it all started
– The birth of CPEP
– Where we are now
• Competency Drivers and Application
– Triangle (getting on the same page)
– State Department Work (across divisions)
– MDE Initiatives (extended example - PBIS)
– District Work
education.state.mn.us
2
MDE Organizational Structure
• What does the organizational structure of the
Minnesota Department of Education look
like?
• See handout
education.state.mn.us
3
Compliance and Crises, Urgent, Time
Sensitive!!
• Assist Districts and Schools not meeting Standards
• Deal with urgent and high profile issues
Practice Improvement – “The Willing”
Educational
Best Practices
Initiatives
• TA targeted at teacher or admin level
• Issues that intersect with ‘local values’
• Broad professional development initiatives
• Pilots and demonstrations
System Stability
System Stability
• Regulatory roles
• Basic Data Systems
• Financing and Fiscal Accountability
• Accreditation and Licensing Standards
• Safety Standards
• Work with Legislature & Stakeholders
Mandates,
System Supports,
Foundational Polices & Regulations
SEA Responsibilities and Leverage Points
Challenges
Educational
Best Practices
Initiatives
• Mandates insufficient
•Training alone not enough
• Pockets of excellence but…
• …Pilots come and go
• Uneven access
• Silos of TA
• Lack of integration
• In classrooms
• At District level
• With broad SEA goals
System Stability
Mandates,
System Supports,
Foundational Polices & Regulations
SEA Responsibilities and Leverage Points
What are we working for?
Education that “works” for ALL students
“from the classroom to the capitol”
Policies
Bureaucracy
Districts and
Schools
Teachers & Staff
Outcomes for
Children, Families,
& Communities
© Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008
Building
Implementation
Teams
Practice- Policy
Communication Cycle
System
Change
State/District
Leadership
Teams
Policy Supports
Effective Practice
MDE Regional Implementation
Team System Support
Fostering Communication Between and
Across Systems
Teachers
Students
SISEP 2012
Developing the Infrastructure
(teams and functions)
• An important step to establish infrastructure that
facilitates communication up, down, and across
• Can we repurpose existing teams?
• Who do we need on the teams in terms of role and
function?
• Takes time to establish trust
• Terms of Reference
– Document outlining the purpose, vision, scope of work,
roles, communication pathways
– Functions as internal memorandum of understanding
– Links the team to broader systems work
education.state.mn.us
8
State
Department
Leadership
SISEP’s Vision of Teams and
Communication Linkages
Regional
Implementation
Teams
1 for every group of 4 “Districts”
“District”
Implementation
Teams
1 for every group of 15-25 Schools
Building
Implementation
Team
School
Teachers and
Staff
Staff with
special
implementation
skills
1 for each School
Adult interactions produce
Student benefits
All Students &
Families
SISEP 2012
© Fixsen & Blase, 2008
State Executive Team
State Implementation Specialists
CPEP State Leadership Team
MDE Teams
and
Communication
Linkages
MDE 1st Generation
Implementation Team
District Leadership &
Implementation Teams
(Assistant Superintendent,
Title I Intervention Specialist
and others)
School Principals, Staff Development, PBIS, RtI , Curriculum and Development
Elementary
Middle
High School
Other/EC
Executive Team
• Provide leadership and facilitate policy level
changes to support implementation of effective
educational practice
– Example: Regional Centers of Support
• Support MDE staff learning about and
incorporating implementation concepts into
MDE work
– Attend/participate/provide words of welcome at
Implementation Forums
– Support Book Club
– Support Team Infrastructure
SISEP 2012
CPEP State Leadership Team
• Existing Structure (RtI Leadership Team)
• N = about 25
• Meets 1X monthly (last Tuesday of the month
9:00-11:00 A..)
• E-mail communication and reminders
• Key Functions
Lead, analyze progress toward statewide
efforts, recommend actions for improvement,
problem solve
CPEP State Leadership Team
Internal MDE
• Assistant Commissioners
• Directors
• Supervisors
• Specialists
External MDE (n = 6)
• Parent, MN Association of School Administrators,
Association of Elementary Principals, Education
Minnesota (teacher union), PACER (parent
advocacy), MN Administrators of Special Education
A Few Thoughts…
•
•
•
•
•
•
Communication, communication, communication
Trust building
Active involvement – solicit input
Revisit accomplishments and progress
Not overwhelming or too time consuming
Value contributions – recognition for participation –
show how their input is being used
• Wide representation – create a welcoming and safe
environment
• Follow up with individuals
Regional Implementation Team
• Know innovations very well (formal and craft
knowledge)
• Know implementation very well (formal and craft
knowledge)
• Know improvement cycles to make interventions
and implementation methods more effective and
efficient over time
• Functional Worker Bees
– Meet twice monthly (formal meeting)
– N = 14
– Represent multiple divisions and content
education.state.mn.us
15
SISEP Invests in the Minnesota Team
Formed in the Spring of 2009
What Are We Learning ?
•
•
•
•
•
Common vocabulary is essential
Continuous learning is expected
Learning curve is steep
Membership turnover is inevitable
Both technical and adaptive skills are
important
• Peer-to-peer coaching improves our collective
performance
– Requires trust
– Requires time to reflect
Description of the Journey
• How it all started…
• The birth of CPEP
• Where we are now!
education.state.mn.us
18
National Center on State Implementation
of Scaling-up of Evidence-based Practices
(SISEP)
• June, 2008, Minnesota was one of four states
selected to work actively with a new national
technical assistance center
• Funded by United States Department of Education
• Based at the University of North Carolina
• Dean Fixsen, Karen Blase (U of North Carolina),
George Sugai (U of Connecticut), Rob Horner (U of
Oregon)
• Goal: Building state capacity for scaling up
evidence-based practices
Capacity Development in Minnesota
• Staff from National Center on State Implementation
and Scaling Up of Evidence-based Practices
provided ongoing technical assistance
– Monthly MN on-site visits from SISEP staff
– MDE Implementation Team to NC Institute in
March of 2009
– Website
– Resource materials (briefs, binder, book)
– Community of Practice (includes all 4 states)
• SISEP intensive technical assistance work ends
September 2012
So…What Do We Scale Up?
aka “the birth of CPEP”
• Which programs and practices?
• Decision is based on student need
• A process or framework is needed that can be
used across programs and practices to guide
scaling up
• Applicable at the
– Classroom
– School
– District
– State Level
Guiding Principles to Inform this Work
•
•
•
•
Not a mandate, offered as a way of work
Work collaboratively with the willing
Must apply to ALL students
Must incorporate communication linkages and
feedback loops, up and down as well as across
(state, regional, local)
• Multidisciplinary
• NOT another program – but a tool to help organize
and integrate
• Must incorporate common principles of effective
practice AND key components of implementation
Development of the Framework
Common Principles of Effective Practice
• Worked with several initiatives in place at MDE
– Positive Behavioral Interventions and Supports,
Dropout Prevention, Response to Intervention,
Reading First
– Identified key elements of each “from the
balcony” necessary for effectiveness
– Synthesized to identify commonalities
– Identified a supporting research base for each
principle of effective practice
– Solicited feedback from multiple stakeholders
Minnesota’s Common Principles of Effective
Practice (CPEP) and Implementation
Working Together to Build Capacity for
Effective Implementation of Effective
Educational Practices
(see handout)
The “Big Eight”
Principles of Effective Practice (“What”)
•
•
•
•
•
•
•
•
Defined and Measurable Goals and Outcomes
Evidence Based Practices
Data Driven Decision Making
Multi-Tiered Systems of Support that Accelerate the
Learning of All Students
Implementation of Instruction/Intervention as Intended
Culturally Responsive Teaching and Learning
Alignment with MN P-12 Academic State and Local
Standards
Student, Parent and Community Engagement
Development of the Framework
Key Components of Implementation
• National Center on State Implementation and
Scaling Up of Best Practices (SISEP)
• National Implementation Research Network (NIRN)
– Mission: to advance the science and practice of
implementation, organizational change, and
system transformation … to effect positive
outcomes for children and youth
Key Components of Implementation
(“How”)
• Stages of Implementation (exploration, installation, initial
implementation, full implementation, sustainability)
• Staff Competence Drivers (selection, training, coaching)
• Organization Drivers (systems intervention, facilitative
administration, data support systems)
• Leadership at ALL Levels (classroom, school, district
state; technical and adaptive)
• Performance Assessment (fidelity checks for effective
implementation)
What We Want to See
Consistently in Minnesota’s Schools to Effect Positive
Student Outcomes
Effective Programs
1.0
X
X
Effective Implementation
=
0.0
________________________________
Outcomes that Benefit Individuals 0.0
and Society
How can the CPEP Framework be used?
• As a framework to review educational initiatives and
innovations
• As a tool to plan professional development
• A lens through which to integrate programs and
initiatives
• A reflection tool for assessing best practice
• A resource to gather basic information on
implementation and effective practice
What Are We Learning?
• Repetition is important and required
• Continuous improvement occurs as more is
understood
• Do not impose or mandate, but invite participation
• Coaching is important
• Do not expect perfection immediately as folks are
exploring concepts
• Build capacity at all levels
• Model and provide opportunities for ongoing
professional development – just in time learning
A word on
Implementation Stages
2 - 4 Years – School Level
Exploration
• Assess needs
• Examine
innovations
• Examine
Implementation
• Assess fit
Installation
Acquire resources
Prepare
organization
Prepare
implementation
Prepare staff
Initial
Implementation
Full
Implementation
Implementation
Implementation
drivers
Manage change
Data systems
Improvement
cycles
drivers
Implementation
outcomes
Innovation
outcomes
Standard practice
Competency Drivers and Application
–
–
–
–
Triangle (getting on the same page)
State Department Work (across divisions)
MDE Initiatives (Example, PBIS)
District Work
education.state.mn.us
32
Implementation Drivers…
Build Competency and Confidence
Develop, improve, and sustain competent & confident
use of innovations.
Change Organizations and Systems
Create and sustain hospitable organizational and
systems environments for effective instructional and
educational services
CPEP Implementation Components (the “how”)
(a.k.a. Implementation Drivers)
Performance Assessment
(Fidelity Checks for Effective Implementation)
Staff Coaching
Systems Intervention
(alignment of policies and practices)
Staff Training
Facilitative Administration
Data Support
System
Staff Selection
Leadership
Adaptive
Technical
© Fixsen & Blase, 2008
State Department Work Across MDE Divisions
Building Impacting District and Schools
General Capacity Building
– Book Club process
– Implementation Forums
– Regional Implementation Team Learning
(coaching, rehearsal)
– Paired learning (across divisions)
– Technical Assistance from SISEP staff
– State Capacity Assessment
– Website with 2 pagers
State Department Work Across MDE Divisions
Building Impacting District and Schools
• Read Well by 3rd Grade
– Implementation Appraisals for each core component
– Implementation Rubric
• State Wide System of Support – new Regional Centers of
Excellence to be implementation informed and trained
– Implementation Specialist
– Utilization of teams
• NCLB Waiver - Principal and Teacher Evaluation Models
– Stage-based approach
– Incorporate Plan Do Study Act feedback
– SEA role in providing ongoing training
• Q-Comp
– Networking sessions incorporated principles of effective practice and
information on implementation components
– Tools developed to reflect on best practice with implementation drivers
education.state.mn.us
36
State Department Work Across MDE Divisions
Building Impacting District and Schools
• Early Childhood Centers of Excellence – TACSEI
– Regional Training Centers
– Communication Feedback Loops
• Statewide roll-out of new manuals (LD manual)
– Training implications
• RFP Processes
– Alternative Delivery of Specialized Instructional Services
(ADSIS) requires all interventions to have common
principles of effective practice
– ADSIS Leadership and Implementation Teams – facilitates
communication and coordination with continuum of
supports available to students
education.state.mn.us
37
What Are We Learning?
• Implementation is universal
– Like gravity, implementation influences are
always at work (not about intention)
– Implementation principles are not
dependent on disciplines, educational
content area, developmental ages, region,
school level, etc.
Example: Minnesota SW-PBIS Initiative
Through an Implementation Lens
Implementation Drivers
• Competency & Organization
• Integrated & Compensatory
SW-PBIS Selection
• Application for training moves schools through
the “Exploration” stage. Requirements include:
– a school leadership team (represents the roles,
functions, and diversity of the school)
– administrator participation
– 80% staff buy-in
– School-wide Information System (SWIS or other)
– Requires an internal coach
– Trained School Evaluation Tool (SET) evaluator
• Trainer Selection (and training)
• Coach Selection
education.state.mn.us
40
SW-PBIS Training
• Two-year team training cohort
– Year 1 = 2 days 3 x per year
– Year 2 = 1 day 3 x per year
• Trainers reflect the diversity of team
membership
• Standardized scope and sequence (revised
based on data)
• Moves schools from installation to initial/full
implementation
• Summer Institute for sustainability
• Online training modules (under development
based on data)
education.state.mn.us
41
Cohort 5 School SET Results
Fall 09 – Spring 11
42
SW-PBIS Coaching
• Coaching at multiple levels
–
–
–
–
Regional Coaches
District Coaches
Building/Program Coaches
All inform training scope and sequence (feedback
loop)
• Coaches Workgroup
–
–
–
–
Wiki site
Coaches calendar
Coaches scope and sequence (under development)
Networking opportunities
education.state.mn.us
43
SW-PBIS Coaching
Coaching
44
SW-PBIS Performance Assessment
• MDE Evaluation Contractor – Wilder Research
– Training evaluations
– State-wide and Regional Evaluation Reports
– SET Evaluation Exchange
• School and District-level Assessment Data
– Office Discipline Referrals (School-Wide Information
System or other)
– Team Implementation Checklist (TIC)
– Self-Assessment Survey (SAS)
– School Evaluation Tool (SET)
– Benchmarks of Quality (under consideration)
education.state.mn.us
45
Data Collection Cycle
education.state.mn.us
46
SW-PBIS Systems Intervention
State
Team
Linked
Implementation
Teams
Informed by
Data
Improvement
Cycles
Regional
Implementation
Project Teams
District
Teams
School
Teams
SW-PBIS Facilitative Administration
• Aspects covered in application
–
–
–
–
Administrator participation in team training
Release time for team member training
Permission to revise office discipline referral form
Data support system (SWIS or other)
• Staff person designated as coach
• Staff person designated as SET evaluator
• Secure resources for implementation
– Team meeting time
– Rewards
education.state.mn.us
48
SW-PBIS Data Support Systems
• Wilder Research – PBIS Evaluation Contractor
• PBIS Assessment
–
–
–
–
TIC
SAS
SWIS
SET
• SET Exchange Program
• Data-based Recognition System
education.state.mn.us
49
SW-PBIS Leadership Teams
Terms of Reference address both technical and
adaptive challenges through shared understanding of:
– Vision/Goals and Objectives
– Scope and Boundaries (what the work is and is not, when are
we done)
– Roles and Responsibilities of individuals (who participates in
what ways)
– Linking Communication Protocols (with whom do we
communicate, how, and how often and for what purpose)
– Resources available to the project
– Authority
– Deliverables
– Implementation Plans
education.state.mn.us
50
What Are We Learning?
SW-PBIS Implementation and Scale-up
• State Implementation Team membership will
expand with scale-up
• Five-year + implementation blue print is essential
• Practice data-based decision making
• Focus on infrastructure and capacity-building
• Anticipate implementation “bumps”
• Prepare for the technical and adaptive challenges
of role release
Work with Districts
District Selection
District Entry
•Forest Lake – approximately 15 schools; 7000
students (learning platform – Multi Tiered Systems of
Support/Literacy) Suburban/Rural
•Mahtomedi – approximately 4 schools; 3000 students
(learning platform – RtI) Suburban
•St. Paul - approximately 69 schools; 39,000 students
(learning platform – PBIS) Urban
education.state.mn.us
52
How Do Regional Implementation Team
Members Work With District Teams?
• Convene, collaborate, partner with districts to build on
strengths
• Provide ongoing technical assistance (training and
coaching) on content and application related to the
Science of Implementation
• Help to analyze the infrastructure needed for sustained
implementation or effective practices
• Help embed sustained use of principles of effective
practice (e.g., use of data systems for decision making,
fidelity of implementation, evidence based practice)
• Listen and communicate needs and feedback across
various teams
Stories Related to Work with the Districts
• Avoid jargon, provide just in time assistance, need
to screen for leadership that is open to being
coachable
• Spending time on exploration will save time down
the road
• Hold accountable to expectations and sequence for
process, watch out for turnover in leadership before
a foothold is established
• Creating a functional leadership and implementation
team – the perfect storm to get the work done
education.state.mn.us
54
Remember what we said about
implementation stages?
2 - 4 Years – School Level
Exploration
• Assess needs
• Examine
innovations
• Examine
Implementation
• Assess fit
Installation
Acquire resources
Prepare
organization
Prepare
implementation
Prepare staff
Initial
Implementation
Full
Implementation
Implementation
Implementation
drivers
Manage change
Data systems
Improvement
cycles
So… How Are We Doing?
drivers
Implementation
outcomes
Innovation
outcomes
Standard practice
Implementation Book Club Feedback
• Implementing the Findings of Research: Bridging
the Gap Between Knowledge and Practice ( Wallace,
Blase, Fixsen, & Naoom, 2008)
•
•
•
•
•
•
•
•
Six one-hour monthly sessions
Sixty-four total participants
Across 8 MDE Divisions
Core group of 21 (attended 4 or more sessions)
Average attended 3 sessions
90% reported content was excellent/good
87% would recommend the book club series to colleagues
90% could apply the CPEP principles to their work at MDE
Overall Implementation Capacity - four states data
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
SMT Investment
SMT Alignment
RIT Functioning
SISEP 2012
DLIT Functioning
What Are We Learning?
Implementation can be done on purpose
•Letting it happen
– Recipients (WHO) are accountable
•Helping it happen
– Recipients (WHO) are accountable
•Making it happen
– Implementation Teams (WHO) are accountable
(programs go where they are most needed)
Based on Hall & Hord (1987); Greenhalgh, Robert, MacFarlane,
Bate, & Kyriakidou (2004); Nord & Tucker (1987)
What Are We Learning?
Active intentional management of
implementation must be done on purpose to
increase likelihood of reaching intended results.
"Successful programs do not contain the seeds
of their own replication“ (Schorr, 1993)
What Are We Learning?
Sustainable benefits require organization and
system change and this is hard work.
•Innovative practices do not fare well in
existing organizational structures and
systems (legacy systems)
•Organizational and system changes are
essential to successful use of innovations
(Institute of Medicine, 2000; 2001; 2009; New Freedom Commission on Mental
Health, 2003; National Commission on Excellence in Education,1983; Department
of Health and Human Services, 1999; Ulrich 2002)
http://education.state.mn.us/MDE/EdExc/Best
Prac/ImpleEffecPrac/index.html
education.state.mn.us
61
http://sisep.fpg.unc.edu
education.state.mn.us
62