Transcript CAMH generic slides 3 - Addictions and Mental Health Ontario

Enhancing Evidence Based Services:
Ontario Perception of Care and
Addiction Screening & Assessment
Presentation Outline
 Looking Back
 Results of Drug Treatment Funding Program (DTFP) research,
development and piloting of both the Ontario Perception of
Care tool for Mental Health & Addictions and the new Staged
Screening and Assessment process for addictions
 Current State
 Status of both projects
 Coming Soon!
 Next steps for Ontario wide implementation
Presentation Objectives
• Enhance knowledge regarding the purpose and
impact of the Ontario Perception of Care tool for
Mental Health and Addictions
• Increase knowledge and understanding regarding
the staged screening and assessment protocol for
addictions and concurrent disorders
• Enhance knowledge regarding implementation plans
and system readiness for implementation
What is DTFP?
 Drug Treatment Funding Program (DTFP) is
funded by Health Canada
 Overarching goal of enhancing quality of
addiction treatment systems
 Resources flowed to provinces and territories as
well as selected national projects
 Ministry of Health and Long Term Care funds
projects to enhance the Ontario addiction system
Looking Back…..
5
Drug Treatment Funding Program
Staged Screening
and Assessment for Addictions
Brief Project History
More detail at:
http://eenet.ca/dtfp/best-practice-assessment-procedures-project/
Last Round of DTFP: Informed Process
•
•
•
•
Evaluation of ADAT (pre-DTFP)
Background literature review
Environmental scan of Ontario’s addiction agencies
Consultation with large Advisory Committee and close
engagement of Working Group
• New process piloted within 5 treatment agencies
• Recommendation: to move forward with
implementation of new suite of tools
Conceptual Framework
DIVERSITY LENS
Developmental Perspective
Child
Screening
Stage 1
Stage of Client Engagement
(case finding)
Stage 2
(case definition)
Assessment
Stage 1
(placement matching)
Stage 2
(modality matching)
Treatment &
Support
Stage 1
(within-treatment
monitoring)
Recovery
Monitoring
Stage 2
(post-treatment
monitoring)
Adolescent
Transitional
Youth
Adult
Older Adult
Definitions
Placement Matching
 Initial client assignment to a treatment setting with a
certain resource intensity (therefore important cost
implications)
Modality Matching
 Creation of a case conceptualization and formulation
leading to an individualized and adaptable treatment
plan
 Grounded in person’s overall life situation and
problem-focused, including trauma informed
Selected and Piloted
1st Stage
Screener
2nd Stage
Screener
1st Stage
Assessment
• GAIN-SS
• MMS (Modified Mini Screen)
• PDSQ (Psychiatric Diagnostic Screening Questionnaire)
• POSIT (Problem Oriented Screening Instrument for Teens)
• GAIN Q3 MI ONT
Pilot Sites
 Engaged five treatment agencies to pilot all
components of Staged Screening and Assessment
 Including one youth agency
 234 clients recruited
Feedback Gathered On….
 Feasibility
 Potential usefulness at clinical and agency levels
Perceived STRENGTHS of New Process
• Screeners (stage 1 and 2):
• Quick and easy to administer
• Facilitates a comprehensive look at relevant psychosocial
domains
• Stage 2 screener facilitates referrals for mental health
services
• GAIN-Q3 (stage 1 assessment):
• The assessment is comprehensive and standardized
• Concrete recommendations for treatment planning and
referral
• Reasons and readiness to change questions highly valued
• Strong foundation for outcome determination
Perceived CHALLENGES of New Process
• Screeners:
• Established cut offs for stage 1 screener (GAIN-SS)
• Some redundancy between the screeners
• GAIN-Q3:
• Insufficient coverage of substance use history
• Challenge for administration in a group intake context
• Sensitivity around the perceived invasiveness of some
questions
• Timing of administration for youth (impact on rapport)
• Length of administration
• More training needed to edit and interpret summary reports
Compared to ADAT…..
Reported Value-add:
Possible Value-loss:
• Staged approach seen as efficient
and comprehensive
• Better quality data
• Less variability in clinical
interpretation
• Better treatment planning
• Better support for referrals to mental
health services for clients with
concurrent disorders
• Summary reports for clients provide
better documentation
• Flexibility of group intake
• Concern about potential
impact on wait times –
quality vs. quantity
• Collection of detailed
information regarding
substance use history
Bottom Line from Pilot Work
• Support for staged model
• General support to replace part or all of ADAT with
screening and assessment tools
• Some revisions needed for the GAIN-SS and the
GAIN-Q3 (i.e. more on substance use)
• Need to build on existing infrastructure and
provincial processes underway
Recommendation: Move forward with Ontario
implementation of new suite of tools
Responding to Pilot Feedback
• Addressing the concerns/challenges identified in
advance of broad implementation including:
Only one second stage screener for adults (mitigate
redundancy)
Pilot agency has been successfully engaging in client selfadministration process (allow group intake, mitigate
possible wait times)
Adapting GAIN Q3 MI ONT to include more detailed
substance use and treatment history (respond to need
for more depth of information in this area)
Pilot Sample Data:
Supporting Agency & System
Planning
Analysis of Initial Assessment Data –
combined substance use
Case
Defined
As
Also
Experienced
Heavy
Alcohol Use
(53.5% of
sample)
N = 76
Heavy Alcohol Use
Heavy
Marijuana Use
(30.3% of
sample)
N = 43
Heavy Cocaine/
Crack Use
(10.3% of
sample)
N = 14
Heavy Heroin/
Methadone/
Opioid Use
(12.5% of
sample)
N = 17
48.8%
71.4%
70.6%
28.6%
35.3%
Heavy Marijuana
Use
27.6%
Heavy
Cocaine/Crack Use
13.5%
9.5%
Heavy Heroin/
Methadone/
Opioid Use
16.2%
14.3%
11.8%
14.3%
Analysis of Initial Assessment Data – % severe
problem areas by gender (n=150)
Male
Female
Total
SU
62.8
84.3
70.3
Int. MH
64.9
88.5
73.3
Ext. MH
38.3
55.8
44.5
Physical
35.8
54.7
42.6
Work
7.4
16.7
10.7
Stress
43.2
54.7
47.3
Risk Behav.
17.2
48.1
28.3
Crime-Viol.
2.2
8.3
4.3
90-days Trauma
32.3
57.7
41.4
90-days Victimization
26.9
47.1
34.0
Problem Domain
Analysis of Initial Assessment Data – % using
community services in past 90 days
Analysis of Initial Assessment Data – % using
community services in past 90 days by gender
Value of GAIN-Q3 MI for
Outcome Monitoring
• In 2013/14 the project team also completed analysis of pilot
outcome data
• 117 clients followed for 3 and then 6 months
• Results show the value of the GAIN-Q3 for outcome
monitoring
 90 day re-administration of Q3 MI ONT showed:
 Decrease in substance use
 Positive changes in mental health, stress, physical health and risk
behaviours
 Fidelity use of GAIN Q3 MI linked to better client outcomes
Ontario Perception of Care Tool for
Mental Health and Addictions
(OPOC-MHA)
Brief Project History
More detail at:
http://eenet.ca/dtfp/client-satisfaction-project/
Ontario Perception of Care Project
• Measures of client experience are widely used by
customer-oriented businesses and healthcare services
and settings
• Recognized as an important indicator of the quality of
care as it is a direct measure of whether a client
received services that met expectations and needs
Satisfaction versus Perceptions of Care
• Satisfaction a measure of the reaction to the services
received
• Respondents tend to report high levels of satisfaction even
though dissatisfaction might be voiced in open-ended
questions or other feedback formats such as focus groups
• Measures of perception of care ask more directly about
the care experience in relation to what is expected as
standard practice
• Range of responses likely to be wider as respondents may be
more willing to report infrequent exposure or use of a practice
than express dissatisfaction about it
Previous DTFP Iteration
• Literature review completed
• Environmental scan
– All mental health and addiction agencies in Ontario
using ConnexOntario database (30% response rate)
– Asked about current tools and processes to assess
client perceptions of care
– Most agencies using something
– Majority of tools developed in-house
• Extensive stakeholder consultation through Advisory
Structure and project Working Group
OPOC- MHA
• Developed new tool with support of Working Group
• All tools evaluated in the literature review were assessed
according to validity, usage, length, and relevance
• 8 tools were selected from this process
• All items from these 8 tools were collated according to
domain for review by the Working Group and served as
the foundation for the new tool
• Can be used in addictions, mental health, and concurrent
disorder settings
• Translated into French
Domains of the OPOC-MHA
Domain
Sample Question
Access/Entry
“The location of services was convenient for me”.
Services Provided
“I had a good understanding of my treatment and
support plan”.
Participation/Rights
“I felt comfortable asking questions about my
treatment and support, including medication”.
Therapists/Support
Workers/Staff
“I found staff knowledgeable and competent”.
Environment
“I felt safe in the facility at all times”.
Discharge/Leaving the
Program
“I have a plan that will meet my needs after I leave
the program”.
Overall Experience
“The services I have received have helped me deal
more effectively with my life’s challenges”.
OPOC-MHA Versions
CLIENT Version (38 items)
• Registered clients of the program
– Those receiving services for their own treatment/support
– Family members/significant others/supporters who are receiving
services in their own right)
Note: 6 items specific to inpatient/residential treatment
services only
FAMILY/SUPPORTERS (17 items)
• Family members/significant others/supporters who are not
registered clients but who are also receiving services from
the program (such as parent who has a child in the program)
OPOC-MHA Additional Questions
• Respondent’s age, gender, sexual orientation, ethnic
background, and stage in the treatment process are
included
• Information can be used for subgroup analyses and
from an equity perspective
• Two open-ended questions to allow for comments
about what the respondent found most and least
helpful in their experience with the program, as well
as room for comments throughout the questionnaire
OPOC-MHA Pilot Process
 Data collected April 1/12 through June 30/12
 1, 753 respondents
 Administration process varied by site
 Least disruptive to program
 Administered at various points in time
Pilot Sites
• 23 pilot sites from both mental health & addiction sectors
• Sites represented a diversity of clients and services across
Ontario (i.e. inpatient/community, gender-specific, youth,
Indigenous, ethno-racial and immigrant etc.)
OPOC-MHA Pilot Results
We have a tool and it works!
• Overall feedback from staff and clients about the
OPOC-MHA was positive
• Significant interest from mental health and addiction
agencies in the province (and elsewhere) to
implement the tool
• The OPOC-MHA demonstrated strong psychometrics
OPOC-MHA
Pilot Sample Data
OPOC-MHA: Referral to Services
34
OPOC-MHA: Environment
35
OPOC-MHA: Feeling Safe
36
OPOC-MHA: Effectiveness
37
Sub-group Differences in Responses Some Findings
• Patients and supporters did not differ substantially in
their opinion on most of the statements
• Respondents from MH programs answered strongly
agree less often on some statements compared to
respondents from A and CD programs
• Young respondents (age <=18) appeared to answer
strongly agree less often on most of the statements
Differences in Responses - Some Findings
• LGBQT respondents were less likely to endorse strongly
agree on all the statements
• Respondents who have been mandated by court, medical
authority, etc. appeared to answer strongly agree less
frequently on all statements than respondents who
voluntary participated in the program/treatment
40
Provincial Implementation
• Staged Screening and Assessment Process being
implemented in Ontario MoHLTC funded Addiction
agencies
• OPOC-MHA being implemented in Ontario MoHLTC
funded addiction, mental health and concurrent
disorder programs
• Implementation beginning in about half of the LHINs,
with other half beginning early 2016
Research Based Approach to
Implementation
•
•
•
Implementation Science (IS) is a proven approach to
bringing evidence-based research into practice to
improve client outcomes
IS means purposeful, planned and active
implementation, supporting fidelity and sustainability
Planned and supported implementation results in higher
implementation with fidelity
More info on IS: http://nirn.fpg.unc.edu/learn-implementation
Reference: Greenhalgh et el. 2004.Diffusion of Innovations in Service Organizations: Systematic Review and
Recommendations. The Milbank Quarterly, Vol. 82(4), 581 – 629.
Implementation Science Supports
• Implementation supports increase direct
practice change from 5% to as much as 95%
• Implementation supports (i.e. fidelity monitoring
and supervision/coaching) help staff see the new
evidence based practice is not ‘just another
change/project/model’
• Increases staff’s ability to provide the new
evidence-based practice competently & flexibly
Implementation Planning to Date
• Developed Provincial implementation plan using
implementation science framework
• Fine tuning tools and processes to support
implementation
• Developing infrastructure (i.e. catalyst)
• Capacity building for DTFP implementation team
– Certification to trainer level of GAIN assessment
tool
DTFP Implementation Team
• DTFP Implementation Team includes:
–
–
–
–
–
–
DTFP Implementation Supervisor
Implementation Coordinator (assigned to LHIN)
Implementation Coach (assigned to LHIN)
Evaluation Coordinator
Knowledge Broker
Research Analyst (OPOC-MHA)
• Dr. Brian Rush providing ongoing consultation and
guidance to the team
The Details:
What is Being
Implemented
46
Best Practices for Screening and
Assessment – Key Principles
• Structured information gathering with validated tools
is vital: unstructured interviews miss co-morbidity –
you have to ask!
• Staged approach saves time and resources for the
longer screening and assessment
• Tools that cover both mental health and addictions
enhance communication & relationship building
across sectors
Staged Screening & Assessment Process
If score is 2 or
greater on
internalizing
scale, proceed to
2nd stage
screener.
Stage 1
Screener:
GAIN SS
Stage 2
Screener:
MMS (18+)
POSIT (12-17)
If score is less than
2 on internalizing
scale, proceed to 1st
stage assessment.
Stage 1 Assessment:
GAIN Q3 MI ONT
(with detailed SU & tx. history)
All Tools
• Available in English and French
• Will be accessible via Catalyst/DATIS
– Ease of administration
– Supports development of clinical profile/system level data
Stage 1 Screening: GAIN-SS
• Studied and used in a number of different settings
including Canada
• Valid and reliable down to 10 years of age – Canadian
validation with adults – recommend for age 12 and up
• Cost: $100 agency licensing fee for 5 years unlimited use
• Self- or clinician-administered (via GAIN ABS or paper
and pencil)
• Reported to take 5 - 10 minutes to complete
• Pilot used the CAMH-modified version – 7 additional
questions
Stage 2 Screening (Adults):
Modified Mini Screen (MMS)
• Validated in public sector settings in the U.S.
• No cost
• 22 items divided into 3 sections to capture the
three major categories of mental illness (mood,
anxiety and psychotic disorders)
• Paper and pen: self-/clinician- administered
• Estimated 15 minutes to complete
Stage 2 Screening (Youth): Problem
Oriented Screening Instrument for
Teenagers (POSIT)
• Valid and reliable
• Designed to identify problems and potential
treatment/service needs in 10 areas including
substance abuse, mental and physical health and social
relations
• Estimated 20-30 minutes to administer, 2-5 minutes to
score
• Administered by self/clinician and with
paper/computer
• For use with clients aged 12 - 17
Stage 1 Assessment: GAIN-Q3 MI
(Ontario Version)
• Developed by Chestnut Health Systems in Illinois
• Good psychometric properties
• One of main instruments in the GAIN family of
assessments
• Ontario version was developed to increase the tool’s
relevance to the provincial context
– Incorporated items around trauma and barriers
– Cross-walk with ADAT; mapped to strengths and needs
criteria
Stage 1 Assessment: GAIN-Q3 MI
(Ontario Version)
• Multi-purpose tool that identifies a wide range of life
problems
• For use among adolescents and adults in both clinical
and general populations
• For use in diverse settings
• Established with strong focus on subsequent outcome
monitoring
Content of GAIN-Q3 MI
(Ontario Version)
Basic Domains Covered:
•
•
•
•
•
School Problems
Work Problems
Physical Health
Sources of Stress
Mental Health
• Risk Behaviours for
Infectious Diseases
• Substance Use
• Crime and Violence
• Life Satisfaction
On the GAIN-Q3 MI
(Ontario Version)
• Each domain includes motivational-based questions
specific to readiness for change and reasons for changing
behaviour
• Estimated 45 minutes to complete face-to-face
administration
 Computer/ABS (accessed via Catalyst)
 paper-and-pencil
• Self administration in a group format being used
regularly at one of the pilot sites (quite successfully)
On the GAIN-Q3 MI
(Ontario Version)
• Supports system-level client profile mentioned previously
• A number of reports can be generated to support clinical
decision making and referral process
–
–
–
–
Individual Clinical Profile (ICP)
Personalized Feedback Report (PFR)
Recommendation Referral Summary (RRS)
Validity Report
Examined Complementarities of GAINQ3 MI ONT with other Tools
• Crosswalk between GAIN-Q3 and OCAN, RAI and LOCUS
was conducted to explore relevant contributions of the
GAIN-Q3
• Results:
• Tools can complement each other
• Only the GAIN Q3 provides the level of information
needed for substance use assessment and treatment
planning
Substance Use Grids
Embedded within the GAIN Q3 MI ONT:
• Provide greater depth of information regarding
substance use and treatment history
• Supports referral and/or treatment planning for
clients with more complex issues
• Clinician may administer in same session as Q3 MI
ONT or subsequently if more information deemed
necessary
In Summary, New Staged Process……
• Includes screeners:
 That are quick and easy to administer
 Provide comprehensive look at relevant psychosocial
domains
 Facilitate referrals for mental health services
• Includes assessment that:
 Is comprehensive and standardized
 Provides concrete recommendations for treatment planning
and referral
 Includes reasons & readiness to change questions that are
highly valued
 Provides a strong foundation for outcome determination
Value Add at Multiple Levels
Clinical:
• Assessment a process that should occur over time
• Assessment needs to be motivationally oriented and
comprehensive across bio-psycho-social and spiritual
domains of health
• Assessment alone can contribute to outcomes
• GAIN assessment fidelity is linked to clinical outcomes
• Assessment plus evidence-informed interventions
improves outcomes
Value Add at Multiple Levels
Organizational:
• Analysis/summary of initial assessment data
• Organizational clinical profile informs program
planning and quality improvement
• Electronic administration and printable validity reports
(that notes administration inconsistencies) can
support performance monitoring and coaching
Value Add at Multiple Levels
System:
•
Structured data:
• Basic information on demographic characteristics of
clients that can be compared to the community profile
to identify under-served populations
• Development of clinical/severity-based profiles of
people in treatment (i.e. to confirm that those with the
most severe profiles are being served in the most costintensive services)
• Provide a baseline for follow-up outcome assessment,
which also has value at the individual level
Clinical & Agency Value-Add Summary
•
•
•
•
•
•
Tools with sound psychometrics and track record
Comprehensive assessment and resulting reports
facilitate better treatment planning
Higher concurrent disorder capacity
High level of detail on substance use and overall
severity: holistic view of client need
Fully integrated process incl. report generation
Agency clinical and psychosocial profile = informed
planning
LHIN/System Value-Add Summary
•
•
•
•
Placement matching: Initial client assignment to a
treatment setting with appropriate resource intensity
(important cost implications and better use of existing
resources)
Detailed client profiles for community gap analysis,
project planning (i.e. reducing ER and hospital use)
and performance monitoring
High potential for outcome monitoring
Comparable data with other LHIN’s for potential
benchmarking
OPOC-MHA
Implementation
OPOC-MHA Implementation
Objective:
• Systematically implement OPOC-MHA across all
MoHLTC funded substance use, concurrent disorder
and mental health services
Projected Outcomes:
• Standardized information regarding client
satisfaction/perception of care
• Enhanced quality improvement and accountability
processes at both service and system levels across
Ontario
OPOC-MHA Administration Essentials
• Details of tool administration can and will vary by agency
• Each agency and/or program can determine how and
when the tool should be administered
• Four key requirements:
–
–
–
–
Provide entire questionnaire
Ensure anonymity
Ensure completion is voluntary
Provide facilitation as needed
OPOC-MHA Data Collection
• Variety of ways the OPOC-MHA can be administered. No
prescribed way, as this depends on the practices of each
agency and/or program
• Distributed to clients in many ways, i.e. group setting,
individually, in person at the agency or through email/mail
• Current administration via paper and pencil or electronically
through Fluid Survey database
– Data will be collected and analyzed centrally
• Future administration through DATIS/Catalyst (for both MH
and A)
OPOC-MHA Administration
Timing
• At any point in the person’s treatment process
 One question asks which part of the treatment process
a participant is currently engaged
Scoring
• Likert scale
 strongly agree, agree, disagree strongly disagree, N/A
Analyzing OPOC-MHA Data
• Analysis & interpretation of OPOC-MHA data may involve individual
or grouped items
• Responses to each item may be reported as % OR averages
• “Overall Perception of Care” score
– Plus subscale scores for “Accessing Services” & “Within Services”
Scales
# of
Items
Items
Scoring
Average score of the 23 items
Overall Perception
of Care
23
Accessing Services
6
Within Services
17
1-8, 12-15, 17-18, 20-25, 30-32
1-6
7-8, 12-15, 17-18, 20-25, 30-32
Average score of the 6 items
Average score of the 17 items
OPOC-MHA and Quality Improvement
• OPOC-MHA designed to capture information on
quality improvement indicators such as:
– safety, accessibility, client-centredness, equity,
integration, effectiveness, and appropriate use of
resources
• OPOC-MHA identified by Accreditation Canada as
an instrument approved for use for assessing
client satisfaction/perception of care for
accreditation purposes
Implementation Next Steps
• LHIN Engagement - Scope of the work
–
–
–
# of agencies
key contacts (A and MH networks?)
current process for system access etc.
• Development of LHIN-specific implementation plans
– Using Implementation Science
– Connected to overall Provincial plan
– Contextualized to the LHIN
• Development and implementation of LHIN and
agency supports
Note on Coordinated/Central Access
• Staged process lends itself more to coordinated access
models than previous ADAT assessment process
– who does what component can vary but the package is
well-developed and supported
• LHIN-level implementation plans will be developed for the
specific community context
– Supported and tailored approach
– Implementation will work within access model to
determine how the tools best fit
Key Milestones and Timelines
Milestone
Anticipated Timeline
Engage with early adopter LHINs
May/June 2015
Assemble DTFP Advisory Committee and June 2015
SS&A/OPOC Working Group
Develop LHIN Specific Implementation
July 2015
Plans
LHIN Implementation Team
Development
August 2015
Develop Agency Implementation Plans
(incl. coaching support)
August/September
2015
Key Milestones and Timelines
Milestone
Anticipated Timeline
Training and Capacity Building Begins
September/October
2015
October 2015
Initial Implementation begins in Early
Adopter LHINs
Track implementation, monitor progress, October 2015 – April
engage in developmental evaluation
2016
Engage with remaining LHINs
Full Implementation in Early Adopter
LHINs
February/March 2016
May – December 2016
Training: Staged Screening & Assessment
• Mixed modalities
• Web based training on some elements
• Introduction to the process
• GAIN – SS
• MMS
• POSIT
• Face to face training
• GAIN Q3 MI ONT
• Motivational Interviewing
• Implementation
78
SS & A Training Plan Overview
Chestnut Health
Systems
DTFP Team
Trains DTFP Team to
Trainer Certification
Level on
GAIN Q3 MI ONT
Trains early
adopter/champion
agencies in each LHIN on
entire SS&A process,
including administration
certification on GAIN Q3
MI ONT
Trains Team of LHIN
Based Trainers to
Trainer Certification
Level of GAIN Q3 MI
ONT
Trains Team of LHIN Based
Trainers on all other
elements of the SS&A
process
LHIN Based
Trainers
Trains all other
implementing
agencies on the
entire process
(with DTFP Team
support) and
provides ongoing
training for
sustainability
79
OPOC-MHA Capacity Building
• Web based orientation and training
•
•
•
•
Understand the tool
Administration details
Data gathering
Quality Improvement implications
80
Kim Baker
DTFP Implementation Supervisor
[email protected]
Brian Rush
Project Consultant
[email protected]
Linda Sibley
Addiction Services of Thames Valley
[email protected]
Donna Rogers
FourCast
[email protected]