Transcript Presentation 4 - Addictions and Mental Health Ontario

Impetus
Our current capacity to evaluate our effectiveness among youth addiction
agencies in Ontario is varied and non-systematic
Can’t know if we help or how much
Can’t know if we are getting the right youth to the right program
BUT
Agency level dedication to understanding effectiveness
Ministry will at some point mandate outcome eval
We want to drive the mandate
WHAT WE ARE DOING
‘We’ are a group of Youth Managers
We needed to:
1. Understand what ‘success’ means for youth addiction treatment
2. Measure success without straining therapeutic relationship
3. Measure success in a common way across programs
4. Value our measures at the client-therapist level and for program evaluation
5. Share what we learned with other programs in/out of youth addiction sector
Understanding What ‘Success’ Means
Environmental Scan
Varied agency type
Varied client populations
Talked to clients, families, staff
“What does success mean?”
C: “How do you feel about contributing?”
P:“What are the current evaluation practices”
Common Outcomes & Indicators
SELF
Coping
Hope
Confidence
Insight
Esteem
Emotion
Management
Goals
Motivation
RELATIONSHIPS
Family
Peers
Therapeutic
Emplyers/Tchers
Mentors/Elders
Legal
Leadership
WELL BEING
Physical
Mental
Sexual
Substance Use
Sleep
Daily Functioning
Exercise
Doctor/Dentist
Visits
COMMUNITY/CULTURE
Connection Cmty
Connection Cltr
Volunteer
Cultural Activities
Spirituality
Resource
Knowledge
School/Work
Compliance
Safety
Finding Tools to Measure Success
Screening
GAIN SS + 5 Questions
Assessment
Gain Q3 MI (ON) + WM-Tool
Outcomes
If only GAIN SS + 5 at pre-treatment, GAIN SS + 5 post
If GAIN Q3 MI (ON) + Other at pre-treatment, GAIN Q3 WM-Tool post
Measuring Success Without Strain
Embed evaluation in therapeutic practice
Ensure process is user-friendly for clients and clinicians
Allow for instant access to results for individual evaluation and planning
Allow for instant access to measures of change over time
Allow for results to roll up to group, program, or agency level instantly
Develop standard dashboard for results
Screening
Assessment
Other Indicators GAIN Q3 MI
Work problems
Substance Use
Mental Health
Physical Health
Sources of Stress
Risk Behaviour & Trauma
Crime & Violence
Life Satisfaction
What Would You Like To Talk About?
Where Should We Start?
Physical Health
GAIN – BK – BMI. ………………………………………………………………………17.5
GAIN – BK - Are you pregnant? ………………………………………………..…No
GAIN – PH - Last time you:
gained 10 or more pounds when you were not trying to? ……………….…Past Month
lost 10 or more pounds when you were not trying to? ……………………...Never
were worried about your physical health? …………………………….…......Past Month
had a lot of physical pain or discomfort? ………………………………….…4-12 Mths Ago
had physical health problems kept you from meeting responsibilities? …Past Month
saw a doctor /nurse for physical health or took medication for one?.......Never
Past 90 days, how many days:
have you exercised for at least 20 minutes per day?............................................5 Days
did you have an injury……………………………………………………………..………0 Days
you took prescribed medication for a physical health problem?........................0 Days
you were bothered by any physical health or medical problems?.....................5 Days
have physical health problems kept you from meeting your responsibilities?....10 Days
have you smoked or used any kind of tobacco?.................................................90 Days
GAIN – PH - During the past 90 days, how many:
times have you had to go to emergency for a physical health problem?.........5 Times
nights total did you spend in hospital for a physical health problem?.................5
Nights
times did you see a doctor / nurse for a physical health problem?....................1 Time
times did you have an outpatient surgery for a physical health problem?.......5 Times
Where Should We Start?
Common + Flexible
Basic tools will be provided (GAIN SS, GAIN Q3 MI (ON), WMTool)
Each agency may add tools (e.g., trauma-symptom-specific)
Tablets
Collaborative team will be outfit with tablets for each clinician
Wi-fi enabled (not cell)
Will be loaded with data collection software
Capacity for narrative reports for screening & assessment
Working on capacity for ongoing therapeutic notes
Working on tool for ongoing monitoring (e.g., 9 questions, clinician response)
WHERE DOES THE DATA GO?
Data stored in CATALYST software
Currently an online system
Working on tablet APP
Owned by MOH housed at CAMH
New release Spring 2015
Scoring will be instant
Dashboards instant
Dashboards:
◦ individual
◦ time-bound
◦ provider-bound (group, program, agency)
Dashboards / Output
EVERYDAY WELL BEING
PHYSICAL HEALTH
MENTAL HEALTH
SEXUAL HEALTH
DAILY FUNCTIONING
SUBSTANCE USE
DAILY FUNCTIONING
PRE
3.4
2.0
2.5
1.9
2.0
3.0
6M IN TX
3.5
2.2
2.4
1.8
2.1
3.1
1Y IN TX
3.6
2.8
2.7
2.5
3.0
3.8
3-6M POST 9M-1Y POST
3.8
3.8
3.1
3.2
2.8
2.8
2.1
1.9
3.1
3.2
3.1
3.0
PRE- POST AVERAGES MULTIPLE
RESPONDENTS
PRE POST AVERAGE CHANGE MULTIPLE RESPONDENTS
Vetting
Four ‘new’ agencies will review the Common Evaluation Plan
Critique, suggest, edit
We will revise as necessary
These agencies may later be offered to be one of the pilot agencies
Implementation Proposed Start July 2015
Each team agency will implement the Common Evaluation Plan
CAMH Implementation team will support ‘front end’ Screening & Assessment
EPYC will train on use for interactive & dashboard outcomes
Multi-level implementation plan (board, exec, management, staff)
Implementation will be collaborative with each agency
Project staff will support on site or phone
Our Support
Knowledge Exchange Initiative Grant $19,000
◦ Ontario Centre of Excellence for Child & Youth Health
◦ Started the conversation ‘What Should Define Youth Health’
Addictions Ontario $15,000
◦ Workshops in our team agencies with staff
◦ What defines youth success, what is current evaluative practice
Ontario Trillium Foundation $327,500
◦ Environmental Scan Ontario-wide with clients, parents, staff
◦ What defines youth success, current practice, feelings about doing evaluation
Drug Treatment Funding Program $207,000
◦ Proposed to support tech & population-specific liaison
Alignment with Provincial Initiatives
DTFP (CAMH) – Screening & Assessment, OPOC
DATIS – Involvement with new software platform, possible pilot sites
MCYS – Learning their outcome reporting initiative for YJ
Individual agencies – primarily informative / consultative
Thank you.
QUESTIONS?