Transcript Document

Evidence Informed System
Improvement
A Story From Canada
1
Ontario, Canada
Population:13.5 Million Ontario is the
most populous Province in Canada with
a population of approximately 13.5
million people (nearly 40% of all
Canadians)
Ontario is a large geographic area
1,076,000 square kilometers (larger
than France and Spain combined, more
than 3 times the size of Germany)
Service Collaboratives Context
• Canada has a federal system of Government. The organization,
governance, funding and delivery of mental health services and
supports and addiction treatment in Canada are primarily the
responsibility of provincial and territorial governments.
• Provinces and territories also govern mental health legislation
and set policy in their respective jurisdictions.
• The federal government is responsible for some delivery of
mental health services for certain groups of people (e.g. First
Nations, Military).
• Health services, including mental health and addictions, are
funded by Ontario’s Ministry of Health and Long-term care.
Background:
Open Minds, Healthy Minds
Open Minds, Healthy Minds:
Ontario’s Comprehensive
Mental Health and Addictions
Strategy commits to the
transformation of mental health
and addiction services for all
Ontarians. The Strategy begins
with a three-year-plan that
focusses on children and youth.
4
Ontario’s Comprehensive Mental Health and Addictions Strategy
OVERVIEW OF THE THREE YEAR PLAN
Starting with Child and Youth Mental Health
INDICATORS
THEMES
Our Vision: An Ontario in which children and youth mental health is recognized as a key determinant of
overall health and well-being, and where children and youth reach their full potential.
Provide fast access to high quality
service
Identify and intervene in kids’ mental
health needs early
Kids and families will know where to go to get what
they need and services will be available to respond
in a timely way.
Professionals in community-based child and youth
mental health agencies and teachers will learn how to
identify and respond to the mental health needs of kids.
• Reduced child and youth suicides/suicide
attempts
• Educational progress (EQAO)
• More professionals trained to identify kids’
mental health needs
• Fewer school suspensions and/or
expulsions
• Higher parent satisfaction in services
received
Improve public access to
service information
INITIATIVES
• Higher graduation rates
Pilot Family Support
Navigator model
Y1 pilot
Funding to increase supply
of child and youth mental
health professionals
Increase Youth Mental
Health Court Workers
Reduce wait times for service,
revise service contracting,
standards, and reporting
Outcomes, indicators and
development of scorecard
Implement Working
Together for Kids’ Mental
Health
Amend education curriculum
to cover mental health
promotion and address
stigma
Close critical service gaps for vulnerable
kids, kids in key transitions, and those in
remote communities
Kids will receive the type of specialized service they need
and it will be culturally appropriate
• Decrease in severity of mental health
issues through treatment
• Decrease in inpatient admission rates
for child and youth mental health
Implement standardized
tools for outcomes and
needs assessment
Develop K-12 resource
guide for educators
• Fewer hospital (ER) admissions and
readmissions for child and youth
mental health
• Reduced Wait Times
Enhance and expand
Telepsychiatry model and
services
Hire new Aboriginal workers
Implement Aboriginal Mental
Health Worker Training Program
Provide support at key
transition points
Improve service
coordination for high needs
kids, youth and families
Implement school mental
health ASSIST program and
mental health literacy
provincially
Provide designated mental
health workers in schools
Expand inpatient/outpatient
services for child and youth
eating disorders
Hire Nurse Practitioners for
eating disorders program
Implement Mental Health
Leaders in selected School
Boards
Provide nurses in schools to
support mental health services
Create 18 service
collaboratives
Strategy Evaluation
What Ministries are involved?
The Centre for Addiction and Mental Health (CAMH) is
working with six provincial ministries to ensure the Service
Collaboratives’ success. They are:
•
•
•
•
•
•
Ministry of Health and Long-Term Care
Ministry of Children and Youth Services
Ministry of Education
Ministry of Training, College and Universities
Ministry of the Attorney General, and
Ministry of Community Safety and Correctional
Services.
6
Goal: Improve Processes, Outcomes &
Build on Partnerships
 Early identification of, and equitable access to, appropriate services
and supports for Ontarians with mental health and addictions
issues
 Shorter wait times for community and hospital-based services
 Fewer emergency department visits and unplanned hospital
readmissions
 Improved service linkages and referrals from the justice system
 Better mental health outcomes
 Fewer patients identified as requiring an alternate level of care in
the institutional sector
 Better quality of life for people with mental health and/or
addictions issues and their families.
7
Provincial
PSSP System Support Program
• Supports Ontario’s 10‐year
Comprehensive Mental
Health and Addictions
Strategy
• Offices across Ontario
• Capacity and expertise in
knowledge exchange,
information management,
implementation, and
evaluation
Community-driven, System-level Change
Community
driven
Projectbased
Systemlevel
change
Service
Collaborative:
A group of agencies
and individuals who
work together to
identify and address
system gaps in a local
community.
Consensus
-based
decision
making
Group
action
Multisectoral
Who is participating?
Service
Users
Children &
youth
services
Educational
institutions
Family
health
care
centres
Community-led
Service
Collaborative
Culturespecific
services
Mental
health and
addictions
Justice
Programs
Hospital
services
Community
services
11
Community-led, CAMH facilitated
CAMH…
Service Collaborative
Members…
•
•
•
•
•
•
•
Identify local system gaps
Select and define system intervention
Develop implementation plan
Communicate within their agency
Implement key changes
Monitor process and outcomes
Support evaluation at service level
=
+
• Engages partners
• Supports implementation
and capacity building
• Facilitates knowledge
exchange
• Provides best practices,
evidence and evaluation
expertise
• Acts as a conduit for
information to Ontario
Government
Increased trust and connections between agencies
Formalized collaboration between sectors
Access to innovative processes and tools
A chance to build a broader system understanding
IMPLEMENTATION METHODS
13
Dilemma
People cannot benefit from interventions that
are not being delivered as intended (i.e., in
keeping with evidence)
Adapted with permission M. Duda, 2013
The Challenge….
Science to Service Gap
• What is known is not what is adopted
Implementation Gap
• What is adopted is not used with fidelity
• What is used with fidelity is not sustained
• What is used with fidelity is not replicated broadly
enough to impact outcomes (Pilots vs Scaling-Up)
Our definition of evidence
Multiple forms of knowledge taken together form
“evidence”:
• Research
• Professional expertise
• The lived experience of people and families
• Culture and traditional knowledge
The use of evidence must take into account local
contexts
17
Our Framework for System Change
Deliberate, purposeful action-oriented decision-making and
planning for sustainable implementation.
Implementation Science
+
Continuous analysis of system performance supporting the
implementation of evidence-informed interventions.
Quality Improvement
+
Use of the HEIA tool identifies potential health equity impacts
& informs decision-making on engagement, intervention
selection and implementation.
Health Equity
+
A flexible and responsive approach that supports discussion
and continuous use of evidence for decision-making.
Developmental &
Formative Evaluation
+
Considers research, professional expertise, lived experience,
culture and traditional knowledge throughout
implementation.
Use of Evidence
=
Sustainable Change
18
IMPLEMENTATION
SCIENCE
19
Business as Usual ≠ Impact
Data Show These Methods, When Used Alone, Do
Not Result In Implementation As Intended:
– Diffusion/ Dissemination of information
– Training
– Passing laws/ mandates/ regulations
– Providing funding/ incentives
– Organization change/ reorganization
Necessary But Not Sufficient
NIRN: Active Implementation Frameworks
WHAT
WHEN
HOW
WHO
HOW
Usable Interventions
Implementation Stages
Implementation Drivers
Implementation Teams
Improvement Cycles
Stages of Implementation
Exploration
Sept 2013 – Feb 2014
Installation
March – July 2014
Initial
Implementation
August 2014- Jan 2015
Full
Implementation
Feb - Dec 2015
Decide What to adopt and
implement
How will it happen. Plan what
needs to be in place to implement
the What
Put the plan on the ground and
implement the What (continuous
PDSA cycles)
Make sure it works, then do it
better (PDSA) and make it
“business as usual”.
Stages are iterative and overlap often occurs.
Sustainability planning is important at all stages.
23
Our plan: Service Collaboratives and System
Change
Exploration
• Engage community
• Conduct Needs
Validation
• Identify a system
level gap
• Select a system
level intervention
Installation
• Identify SC
Implementation
Team
• Develop system and
agency level plan
• Develop evaluation
framework
• Include sustainability
planning
Initial
Full
Implementation
Implementation
• Identify Agency
Implementation
Team
• Put intervention into
practice.
• Monitor at agency
and system level.
• Identify barriers and
possible solutions.
• Systems in place for
ongoing data collection
• Coaching well
integrated into
agencies
• Evaluation
• Transition
sustainability
responsibility to
community
Socially Significant Outcomes
Consistent Use of EBPs
Performance Assessment
(Fidelity)
Systems
Intervention
Coaching
Facilitative
Administration
Training
Selection
Integrated &
Compensatory
Leadership
Decision Support
Data System
© Fixsen & Blase, 2008
26
Implementation Teams
• Provide accountable structure to move intervention
through stages of implementation for any new innovation
• Ensure Implementation: Do the work of implementation
(stages, drives, improvement cycles)
• Increase “buy-in” and readiness
• Action and sustain the implementation drivers
• Assess and monitor implementation fidelity and outcomes
• Align systems
• Problem-solve and promote sustainability
• Help create a “hospitable environment”
NIRN Website
PSSP Implementation Teams
Implementation
Coordinators
Engage stakeholders, coordinate and
facilitate implementation.
Evaluation Coordinators
Provide data, planning and evaluation support.
Knowledge Exchange
Leads
Bring in best available research evidence
to support decision-making and share out
what is learned.
Equity & Engagement
Leads
Support a health equity focus on engagement,
intervention selection and planning.
Implementation Coaches
Work directly with agencies to support
implementation of the intervention.
Ex: Service Collaborative Implementation
Structure
CAMH Central
Team
Advice, Communication and Approvals
Services and
Stakeholders
CAMH
Regional
Implementation
Team
Advice & Capacity
Service Collaborative
Building
Accountability & Communication
Advice &
Capacity Building
Service Collaborative
Implementation Team
Accountability & Communication
Advice &
Capacity Building
Agency
Implementation Team
Attending to System Challenges
Practice-Policy Communication Cycle
Practice Informs Policy
Feedback
Do
Practice
Policy Enables Practices
Plan
Study - Act
External Implementation Support
Policy
Policy
Structure
Procedure
Practice
Putting the PIP/PEP loop in place…
Provincial Government Oversight Committee
Advice, Communication and Approvals
System
Improvement
through Service
Collaboratives
(SISC)
Project Sponsor:
CAMH
Accountability
Communication
Ministry of Health and Long-Term Care
Service User Expert Panel
Advice and
Communication
Provincial
Collaborative
Advisory Group
Scientific Expert Panel
Other Expert Panels
Communication
Related Services and Stakeholders
31
KNOWLEDGE EXCHANGE
32
KE
On average, there’s a gap of 17 years between the
production of research evidence and the uptake
and use of that evidence.
The Problem
There are different kinds of gaps and reasons for them. For
example:
• Awareness – people don’t know certain evidence exists
• Comprehension - the research is not presented in a way that is
‘usable’
• Relevance – the research that does exist does not answer the
problems faced by decision-makers
• Recognition – the source of the evidence or knowledge is not
valued
• Implementation – there is awareness, but there are barriers to
change
• Behaviour change – all pieces are in place, but individual behaviour
change is difficult to achieve
Knowledge Exchange in action….
• Bringing evidence to the Service Collaboratives
– Created menu of evidence interventions
– Evidence briefs addressing shared knowledge needs
• Sharing out what is learned
– Developing and implementing a Sharing Out
strategy
Both supported by provincial communication
vehicles: newsletter, website, online community
35
Sharing Out & Scaling-up
Spreading knowledge and learning in order to
extend reach and impact of the Service
Collaboratives. For example:
• Sharing successes and challenges of effecting system
change
• Replicating interventions in other Service Collaborative
communities
• Implementing interventions in communities without
Service Collaboratives.
9 CAMH Regional Knowledge Exchange Leads across the province can link
with other communities and promote sharing and spread.
36
Sharing Out & Scaling-up
37
DEVELOPMENTAL
EVALUATION
38
Balancing Science and
Pragmatism
Pragmatism
Science
39
Approach to Evaluation
 Evaluation of the Service Collaboratives is based on:




Provincial/local approaches
Tools such as logic models and evaluation frameworks
Building performance indicators
Applying newer and combined approaches.
 Guided by a health equity lens through the inclusion of:
 Diverse communities and their self-identified needs
 Community-led approaches.
 Grounded in National Implementation Research
Network (NIRN) implementation stages.
40
Evaluation: Performance Measures
• 2 major levels of analyses:
– Provincial
– Local
Health
Equity
• 5 major provincial domains:
– Health equity
Intervention
Outcomes
– Key transitions
 Between hospital and
community services
 Between health and justice services
 From child to adult services
– Partnership assessment
– Implementation progress
Implementation
– Intervention outcomes
Progress
Key
Transitions
Provincial
Evaluation
Partnership
Assessment
41
Integrating Evaluation & Implementation
Outcome
Evaluation
Process
Evaluation
Needs
Assessment
Program
Planning
(Cunning & Russell, 2013; Fixsen et
al., 2005; Gamble 2008; Love, 2002)
42
Aligning Intervention &
Implementation Outcomes
Effective intervention practices
+
Effective implementation practices
+
Effective performance measurement practices
=
GOOD OUTCOMES
(Adapted from Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005)
43
Integrating Evaluation in all Stages
Outcome
Evaluation
Process
Evaluation
(Cunning & Russell, 2013;
Fixsen et al., 2005;
Gamble 2008; Love, 2002)
Needs
Assessment
Program
Planning
The Work of Implementation
• Changing the behavior of education system
professionals is hard work
• It requires a systematic approach to support behavior
change of
–Teachers,
–School and district personnel
–TA providers,
– State department of education personnel
A formula not yet discovered during
Einstein’s time……
For more information, contact:
Heather Bullock
Director of Knowledge Exchange
Provincial System Support Program (PSSP)
Centre for Addiction and Mental Health (CAMH)
[email protected]
Alexia Jaouich, Ph.D.
Senior Project Manager
Provincial System Support Program (PSSP)
Centre for Addiction and Mental Health (CAMH)
[email protected]
KE
• Knowledge exchange (KE) aims to close the 17-year
gap.
• KE is “a dynamic and iterative process that includes
synthesis, dissemination, exchange, and ethicallysound application of knowledge to improve the health
of Canadians, provide more effective health services
and products, and strengthen the health care system.”
—Canadian Institutes for Health Research, July 2011
The Value of Implementation Teams
48