Transcript Slide 1

Being Data-Driven
Sheila A. Pires
Human Service Collaborative
[email protected]
Ashley Keenan
Parent Support Network of Rhode Island
[email protected]
Michelle Zabel
University of Maryland
[email protected]
Myra Alfreds
Westchester County, New York
[email protected]
Systems of Care are data driven.
2
Pires, S. (2001). Utilization management concerns. Washington, DC: Human Service Collaborative.
Examples of How to Use Data
• Planning and Decision Support (Day-to-Day and
Retrospectively)
• Utilization Management
• Quality Improvement
• Cost Monitoring
• Research
• Evaluation
• Social Marketing
• Accountability
• Education and Advocacy
Pires, S. 2005. Building systems of care. Human Service Collaborative. Washington, D.C.
3
Utilization Management (UM) Concerns
Who is using services?
What services are being used?
UM
How much service is being used?
What is the cost of the services
being used?
What effect are the services
having on those using them? (i.e.,
are clinical/functional outcomes
improving? Are families and
youth satisfied?)
4
Pires, S. (2001). Utilization management concerns. Washington, DC: Human Service Collaborative.
Continuous Quality Improvement: Utilizing Data to
Drive Quality
Contra Costa County, CA
Internal Evaluators
University-based Evaluator
Evaluation Subcommittee
(diverse partners, including families)
•Developing activities to ensure
CQI for:
-Youth with multiple placements
-Transition-aged youth
-Multi-jurisdiction youth
-Youth at-risk for multiple
placements
•Developing and Tracking Quality
and outcome measures:
I.E. reduction in number of youth
with 3 or more placements;
linkage to needed resources upon
emancipation
Pires, S (2006) Primer Hands On for Child Welfare. From Caliber, Building the Infrastructure to Support Systems of Care.
5
Example of Statewide Quality Improvement Initiative
Michigan: Uses data on child/family outcomes (CAFAS) to:
• Focus on quality statewide and by site
• Identify effective local programs and practices
• Identify types of youth served and practices associated with
good outcomes (and practices associated with bad outcomes)
• Inform use of evidence-based practices
(e.g., CBT for depression)
• Support providers with training informed by data
• Inform performance-based contracting
QI Initiative designed and implemented as a partnership among State,
University and Family Organization
K. Hodges. & J. Wotring. 2005. State of Michigan.
6
Social Marketing
Using commercial marketing practices and techniques to
promote social change
Example: Marketing system of care to legislatures – might
use cost/benefit data
Marketing system of care to diverse families – might use
stories of other diverse families who have experienced the
system as effective
Social Marketing/Communications Activities
and Resources
•
•
•
•
•
•
•
•
On-call/on-site consultation
Communication listserv
Bimonthly conference calls
Resource center
Tip sheets
Workshops
Training academies
Excellence in Community Communications and Outreach
(ECCO) Recognition Program
• Education Products
systemsofcare.samhsa.gov/TechnicalAssistance/smc.aspx
Caring for Every Child’s Mental Health Campaign; NASMHPD/Vanguard Communications/FFCMH;
8
Examples of How to Collect Data
• Questionnaires
• Surveys
• Interviews
• Focus groups
• Clinical outcome data
• Record reviews
• Participatory Action Research
• Network analyses
• Financial analyses
Lazear, K. (2003). “Primer Hands On” A skill building curriculum. Washington. D.C. Quote: Warren Bennis, Leadership Institute, University of
Southern California
9
Example of Quantitative Outcomes Milwaukee Wraparound
•Reduction in placement disruption rate from 65% to 30%
•School attendance for child welfare-involved children improved
from 71% days attended to 86% days attended
•60% reduction in recidivism rates for delinquent youth from one
year prior to enrollment to one year post enrollment
•Decrease in average daily RTC population from 375 to 50
•Reduction in psychiatric inpatient days from 5,000 days to less
than 200 days per year
•Average monthly cost of $4,200 (compared to $7,200 for RTC,
$6,000 for juvenile detention, $18,000 for psychiatric
hospitalization)
Milwaukee Wraparound. 2004. Milwaukee, WI.
10
Example of Qualitative Outcomes:
Family/Caregiver Experience Milwaukee Wraparound
91% felt they and their child were
treated
Not At with
All 4% respect (n=191)
91% felt staff were sensitive to their
cultural, ethnic and religious needs
(n=189)
Somewhat
5%
Not At All 4%
Somewhat
5%
Very Much So
Somewhat
Not At All
Very Much
So 91%
72% felt there was an adequate
crisis/safety plan in place (n=172)
Not At All
15%
Very Much
So 91%
64% reported Wrap Milwaukee empowered them
to handle challenging situations in the future (n=188)
Somewhat
29%
Somewhat
13%
Very Much
So 72%
Not At All
7%
Pires, S. (2006). Primer Hands On – Child Welfare. Washington, D.C.: Human Service Collaborative.
Very Much
So 64%
11
Information Management Systems
Importance of web-based, real time data
to support care managers, administrators,
policymakers, families and youth
Synthesis, The Clinical Manager (TCM),
Others
12
Information and Communications Technology
Information technology – use of electronic computers and software
to store, process and transmit information – e.g., electronic
health records
Communications technology – electronic systems used for
communication between individuals or groups who are not
physically present at the same location – e.g., video conferencing,
Twitter
Telehealth
Using communications technology to provide access to
health/behavioral health assessment, diagnosis, intervention,
consultation, supervision, education, peer support across
distance
Example: Kansas Center for Telemedicine and Telehealth at
University of Kansas Medical Center using technology for • child psychiatric consultation in remote areas of the state
• individual and group therapy; care management
• consultation to schools, group homes, and child care
programs in inner city communities
Youth Participation
• Make the process worthwhile for youth
• Needs to be a priority during all phases of planning
• Access to information in an engaging and
developmentally appropriate way
• Young people need support to be involved
Data-Driven Decision Making
Presented by Michelle Zabel, MSS
Director, Maryland Child & Adolescent Innovations Institute, Mental
Health Institute & Juvenile Justice Institute
Division of Child & Adolescent Psychiatry, School of Medicine,
University of Maryland, Baltimore
Align Outcomes
with Shared Results and Indicators and
Performance Measures Already in Use across
Systems
Overarching Long Term Outcomes for Populations of Focus
• Maryland’s Children’s Cabinet Results for Child Well-Being
• Maryland Child and Family Services Interagency Strategic
Plan
Consistent Performance Measures
• Connect the data requirements across grants and contracts
–
–
–
–
1915(c) Psychiatric Residential Treatment Facilities (PRTF) Waiver
SAMSHA funded SOC grants – MD CARES and Rural CARES
Child Welfare’s Place Matters Group Home Diversion
Other Out-of-Home Diversion using Care Coordination
Demystifying Data:
Using Understandable Language and Structure to
Collect and Analyze Service Data
Performance Accountability
• How much service was provided?
– Number of customers served (by customer characteristic)
– Number of activities (by type of activity)
• How well was the service provided?
– Customer satisfaction, unit cost, percent of staff fully trained
– Activity-specific measures: Percent of clients completing activity,
percent of actions meeting standard
• Is Anyone Better Off? (What effect are the services having?)
– Measurable changes in Skills/Knowledge, Attitude/Opinion,
Behavior, Circumstance
Adapted from Friedman, M. 2005. Trying Hard is Not Good Enough. Trafford Publishing, Victoria, BC
Establishing a CORE Set of Data Elements
Collected across Populations
• Align your data collection efforts where possible to
avoid redundant data collection and reporting
• Federally-funded initiatives will require instruments
and tools (performance measures) around which your
cross-initiative evaluation plans can be built
• Local evaluations can be allowed for collection of
additional measures of interest, such as:
– Education – achievement, completion
– Child Welfare – permanency, child safety
– Juvenile Services – restrictiveness of placement,
recidivism
Making the Data Work for You:
Proving Your Initiative’s Effectiveness to Funders
and the larger Community
• Planning and day-to-day decision making is reasoned.
• Utilization Management changes effectively redirect
resources to where they are needed most.
• Quality improvement efforts can be focused on subgroups of
concern with methods of proven effectiveness.
• Cost monitoring will show return on the dollar in terms of
desired outcomes.
• Research and evaluation questions are answered in clear
measurable terms as well as with effective anecdotal
evidence.
• Use data to strengthen social marketing efforts, to target
training efforts, to educate about and to advocate for your
children, youth and families across venues.
Examples of Statewide Quality Improvement
Efforts in Maryland
• Focus on quality statewide and by site
• Identify local programs and practices
• Identify types of youth served and practices
associated with good outcomes (and practices
associated with bad outcomes)
• Inform use of evidence-based practices (e.g. CBT for
depression)
• Support providers with training informed by data
• Inform performance-based contracting
NEW COMMUNITIES TRAINING
February 10, 2010
Being Data-Driven
Westchester Community Network
Myra Alfreds, Director, Children’s Mental
Health Services
Westchester County, New York
From Data to Sustainability
• Data Collection/Exploration
• Social Marketing
• Cross-System Buy In
• Program Development
• Sustainability
Example #1
• Cost-Benefit analysis of savings from
residential placement
• County Executive’s State of the County
Address
• Single Point of Entry/Single Point of
Return is developed across systems
Residential Placement and Psychiatric Hospitalization
All Residential
Psych Hospital Only
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
Baseline
(n=253)
6 Months
(n=178)
12 Months
(n=144)
18 Months
(n=104)
24 Months
(n=72)
Child Psychiatric Epidemiology Group: Columbia University – MSPH/NYSPI
Example #2
• 10 Kid Study
• Integrated County Planning
• Interdepartmental Agreement
• Sustainability
Example #3
•
•
•
•
•
•
•
•
ER Study of Children Ages 8 and Under
Cross System Planning with New Partners
Early Childhood Networks
Advocacy and County Buy-In
Model Programs and Services
Development of Early Childhood SOC
Foundation Support
New Federal Grant (Project Launch)
• Other Data/Survey/Planning efforts led to
new partnerships, successful county-wide
cross-system approaches that have been
sustained
– Juveniles with Sexually Aggressive/Reactive
Behaviors
– Juveniles with Fire Setting Behaviors
– High Risk Adoptions
– Co-occurring Developmental Disabilities
Lessons Learned
• Need for a dedicated
Researcher/Evaluator in County
• Exclusive focus on the National Evaluation
did not lead to sustainability
• It did lead, however, to a partially funded
position – our System of Care Analyst