Transcript Insert Workshop Title

Global Appraisal of Individual Needs (GAIN):
A Standardized Biopsychosocial
Assessment Tool
Michael Dennis, Ph.D.
Chestnut Health Systems,
Normal, IL
Workshop at 52nd International Conference on Dependencies (Estoril, Portugal, 11 to
16 October 2009). This presentation reports on treatment & research funded by the
Center for Substance Abuse Treatment (CSAT), Substance Abuse and Mental Health
Services Administration (SAMHSA) under contracts 270-2003-00006 and 270-07-0191,
as well as several individual CSAT, NIAAA, NIDA and private foundation grants. The
opinions are those of the author and do not reflect official positions of the consortium
or government. Available on line at www.chestnut.org/LI/Posters or by contacting
Joan Unsicker at 448 Wylie Drive, Normal, IL 61761, phone: (309) 451-7801, Fax:
(309) 451-7763, e-mail: [email protected]
Goals of this Presentation are to
1.
Provide an overview of the role of the GAIN as a piece
of infrastructure in support the move toward both
evidence based practice and practice based evidence
2. Describe each of the measures, the reports that they use
to help the assessment guide clinical decision making
and illustrate how they provide a successively more
detailed picture of client needs
3. Highlight our current work to using actuarial estimates
of outcomes to improve placement decisions
4.
Summarize the status of efforts to make the data
available for secondary analysis and translate the
software, measures and reports from English into
Spanish, French, Portuguese and other languages
Part 1.
Provide an overview of the role
of the GAIN as a piece of
infrastructure in support the
move toward both evidence
based practice and practice
based evidence
The Global Appraisal of Individual
Needs (GAIN) is ..





A family of instruments ranging from screening,
to quick assessment to a full Biopsychosocial and
monitoring tools
Designed to integrate clinical and research
assessment
Designed to support clinical decision making at
the individual client level
Designed to support evaluation and planning at
program level
Designed to support secondary analyses and
comparisons across individuals and programs
As of June 30, 2009, there were 1127 administrative
units (agencies, grantees, counties, states)
collaborating to use the GAIN in the U.S.,
State or County System
GAIN-Short Screener
GAIN-Quick
GAIN-Full
Canada and other countries
1-10 Sites in
Other Countries:
Brazil
China
Mexico
Japan
So what does it mean to move the field
towards Evidence Based Practice (EBP)?

Introducing explicit intervention protocols that are
– Targeted at specific problems/subgroups and outcomes
– Having explicit quality assurance procedures to cause
adherence at the individual level and implementation at the
program level

Introducing reliable and valid assessment that can be used
– At the individual level to immediately guide clinical judgments
about diagnosis/severity, placement, treatment planning, and
the response to treatment
– At the program level to drive program evaluation, needs
assessment, performance monitoring and long term program
planning

Having the ability to evaluate client and program outcomes
– For the same person or program over time,
– Relative to other people or interventions
Key Issues that we try to address with the
GAIN Instruments and Coordinating Center
1. High turnover workforce with variable education
background related to diagnosis, placement,
treatment planning and referral to other services
2. Heterogeneous needs and severity characterized
by multiple problems, chronic relapse, and multiple
episodes of care over several years
3. Lack of access to or use of data at the program
level to guide immediate clinical decisions, billing
and program planning
4. Missing, bad or misrepresented data that needs to
be minimized and incorporated into interpretations
5. Lack of Infrastructure that is needed to support
implementation and fidelity
1. High Turnover Workforce with Variable Education





Questions spelled out and
simple question format
Lay wording mapped onto
expert standards for given area
Built in definitions, transition
statements, prompts, and
checks for inconsistent and
missing information.
Standardized approach to
asking questions across
domains
Range checks and skip logic
built into electronic
applications




Formal training and certification
protocols on administration,
clinical interpretation, data
management, coordination, local,
regional, and national “trainers”
Above focuses on consistency
across populations, level of care,
staff and time
On-going quality assurance and
data monitoring for the
reoccurrence or problems at the
staff (site or item) level
Availability of training resources,
responses to frequently asked
questions, and technical
assistance
Outcome: Improved Reliability and Efficiency
2. Heterogeneous Needs and Severity






Multiple domains
Focus on most common
problems
Participant self description of
characteristics, problems,
needs, personal strengths and
resources
Behavior problem recency,
breadth , and frequency
Utilization lifetime, recency
and frequency
Dimensional measures to
measure change with
interpretative cut points to
facilitate decisions




Items and cut points mapped
onto DSM for diagnosis,
ASAM for placement, and to
multiple standards and
evidence- based practices for
treatment planning
Computer generated scoring
and reports to guide decisions
Treatment planning
recommendations and links to
evidence-based practice
Basic and advanced clinical
interpretation training and
certification
Outcome: Comprehensive Assessment
3. Lack of Access to or use of Data at the Program Level




Data immediately available to
support clinical decision
making for a case
Data can be transferred to
other clinical information
system to support billing,
progress reports, treatment
planning and on-going
monitoring
Data can be exported and
cleaned to support further
analyses
Data can be pooled with other
sites to facilitate comparison
and evaluation




PC and web based software
applications and support
Formal training and
certification on using data at
the individual level and data
management at the program
level
Data routinely pooled to
support comparisons across
programs and secondary
analysis
Over three dozen scientists
already working with data to
link to evidence-based practice
Outcome: Improved Program Planning and Outcomes
4. Missing, Bad or Misrepresented Data





Assurances, time anchoring,
definitions, transition, and
question order to reduce
confusion and increase valid
responses
Cognitive impairment check
Validity checks on missing,
bad, inconsistency and
unlikely responses
Validity checks for atypical
and overly random symptom
presentations
Validity ratings by staff





Training on optimizing
clinical rapport
Training on time anchoring
Training answering questions,
resolving vague or
inconsistent responses,
following assessment protocol
and accurate documentation.
Utilization and documentation
of other sources of
information
Post hoc checks for on-going
site, staff or item problems
Outcome: Improved Validity
5. Lack of Infrastructure
Development
Direct Services
Training and quality assurance
on administration, clinical
interpretation, data
management, follow-up and
project coordination

Clinical Product Development

Software Development

Collaboration with IT vendors
(e.g., WITS)

Data management


Evaluation and data available
for secondary analysis
Over 36 internal & external
scientists and students

Workgroups focused on
specific subgroup, problem, or
treatment approach

Labor supply (e.g., consultant
pool, college courses)


Software support

Technical assistance and back
up to local trainer/expert
Outcome: Implementation with Fidelity
Across measures, the GAIN has a Common
Factor Structure of Psychopathology
Source: Dennis, Chan, and Funk (2006)
CFI=.92, RMSEA=.06 allowing for age
Alcohol and Other Drug Abuse, Dependence and
Problem Use are Age Related
100
90
80
70
Percentage
60
Over 90% of
use and
problems
start between
the ages of
12-20
It takes decades before
most recover or die
Severity Category
Other drug or
heavy alcohol use
in the past year
50
40
30
Alcohol or Drug Use
(AOD) Abuse or
Dependence in the
past year
20
10
0
Source: 2002 NSDUH and Dennis & Scott, 2007
65+
50-64
35-49
30-34
21-29
18-20
16-17
14-15
12-13
Age
Co-occurring Mental Health Problems are Common,
Prevalence
of co-occurring
problems
but the Type
of Problems
also Changes
with Age
by age groups
Internalizing
Disorders go up
with age
100
Prevalence (%)
80
Any
Internalizing
60
Externalizing
Both internalizing
and externalizing
40
20
0
<15
15-17
18-25
25-39
Age groups
40+
Externalizing
Disorders go down
with age (but do
NOT go away)
Source: Chan, YF; Dennis, M L.; Funk, RR. (2008). Prevalence and comorbidity of
major internalizing and externalizing problems among adolescents and adults presenting to
substance abuse treatment. Journal of Substance Abuse Treatment, 34(1) 14-24 .
Progressive Continuum of Measurement
(Common Measures)
Quick
Comprehensive Special
More Extensive / Longer/ Expensive
Screener

Screening to Identify Who Needs to be “Assessed” (5-10 min)
–
–
–
–
–
–
Focus on brevity, simplicity for administration & scoring
Needs to be adequate for triage and referral
GAIN Short Screener for SUD, MH & Crime
ASSIST, AUDIT, CAGE, CRAFT, DAST, MAST for SUD
SCL, HSCL, BSI, CANS for Mental Health
LSI, MAYSI, YLS for Crime

Quick Assessment for Targeted Referral (20-30 min)
– Assessment of who needs a feedback, brief intervention or referral for
more specialized assessment or treatment
– Needs to be adequate for brief intervention
– GAIN Quick
– ADI, ASI, SASSI, T-ASI, MINI

Comprehensive Biopsychosocial (1-2 hours)
– Used to identify common problems and how they are interrelated
– Needs to be adequate for diagnosis, treatment planning and placement
of common problems
– GAIN Initial (Clinical Core and Full)
– CASI, A-CASI, MATE

Specialized Assessment (additional time per area)
–
–
Additional assessment by a specialist (e.g., psychiatrist, MD, nurse,
spec ed) may be needed to rule out a diagnosis or develop a treatment
plan or individual education plan
CIDI, DISC, KSADS, PDI, SCAN
Part 2.
Describe each of the measures,
the reports that they use to
help the assessment guide
clinical decision making and
illustrate how they provide a
successively more detailed
picture of client needs
18
Next slides will

Describe the difference in the breadth of
information you get with different levels of
assessment

Summarize validation studies to date

Illustrate the difference using data from a single
sample (Reclaiming futures project)

Demonstrate that multi-morbidity is the norm
and varies by type of client and program
GAIN-Short Screener (GSS)




Administration Time: A 3- to 5-minute screener
Purpose: Used in general populations to
– identify or rule-out clients who will be identified as
having any behavioral health disorders on the 60-120 min
versions of the GAIN
– triage area of problem
– serve as a simple measure of change
– Easy for administration and interpretation by staff with
minimal training or direct supervision
Mode: Designed for self- or staff-administration, with paper
and pen, computer, or on the web
Scales: Four screeners for Internalizing Disorders,
Externalizing Disorders, Substance Disorders,
Crime/Violence, and a Total
GAIN-Short Screener (GSS) (continued)



Response Set: Recency of 20 problems rated past month (3), 212 months ago (2), more than a year ago (1), never (0)
Interpretation: Combined by cumulative time period as:
– Past month count (3s) to measure of change
– Past year count (2s or 3s) to predict diagnosis
– Lifetime count (1s, 2s or 3s) as a measure of peak severity
– Can be classified within time period low (0), moderate (1-2)
or high (3)
– Can also be used to classify remission as
– Early (lifetime but not past month)
– Sustained (lifetime but not past year)
Reports: Narrative, tabular, and graphical reports built into web
based GAIN ABS and/or ASP application for local hosting
GAIN-Short Screener (GSS)
GAIN SS Psychometric Properties
100%
Low Mod.
High
Prevalence (% 1+ disorder)
90%
Sensitivity (% w disorder above)
80%
Specificity (% w/o disorder below)
(n=6194 adolescents)
70%
60%
50%
40%
20%
At 3 or more
symptoms we get
99% prevalence,
10%
91% sensitivity, &
89% specificity
30%
Using a higher cut
point increases
prevalence and
specificity, but
decreases sensitivity
0%
0
1
2
3
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Total Disorder Screener (TDScr)
Total score has alpha of
.85 and is correlated .94
Source: Dennis et al 2006
with full GAIN version
GSS Performance by Subscale and Disorders
Screener/Disorder
Internal Disorder Screener (0-5)
Any Internal Disorder
Major Depression
Generalized Anxiety
Suicide Ideation
Mod/High Traumatic Stress
External Disorder Screener (0-5)
Any External Disorder
AD, HD or Both
Conduct Disorder
Substance Use Disorder Screener (0-5)
Any Substance Disorder
Dependence
Abuse
Crime Violence Screener (0-5)
Any Crime/Violence
High Physical Conflict
Mod/High General Crime
Total Disorder Screener (0-5)
Any Disorder
Any Internal Disorder
Any External Disorder
Any Substance Disorder
Any Crime/Violence
Prevalence
1+
3+
Sensitivity
1+
3+
Specificity
1+
3+
81%
56%
32%
24%
60%
99%
87%
56%
43%
82%
94%
98%
100%
100%
94%
55%
72%
83%
84%
60%
71%
54%
44%
41%
55%
99%
94%
83%
79%
90%
88%
65%
78%
97%
82%
91%
98%
99%
98%
67%
78%
70%
75%
51%
62%
96%
85%
90%
96%
65%
30%
100%
87%
13%
96%
100%
89%
68%
91%
25%
73%
30%
14%
100%
82%
28%
88%
31%
85%
99%
46%
100%
94%
100%
94%
49%
70%
51%
76%
38%
71%
99%
77%
100%
97%
58%
68%
89%
68%
99%
63%
75%
92%
73%
99%
100%
100%
99%
100%
91%
98%
99%
92%
96%
47%
8%
10%
20%
10%
89%
28%
37%
51%
32%
Moderate
(1+) gives
best result
for
sensitivity
High (3+) gives
best result for
specificity
Recommend
Triage as
0=Not likely
1-2 Possible
3+=Likely
GAIN SS Total Score is Correlated With
Level Of Care Placement: Adolescents
Total Disorder Screener for Adolescents
% within Level of Care and Age Group
11%
Lo
10% w
9%
Mod
High ->
OP/IOP
Median=6.0
Residential
Median=10.5
OP/IOP (n=2499)
Residential (n=1965)
8%
7%
6%
5%
4%
3%
2%
1%
0%
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20
TDScr Score
GAIN SS Total Score is Correlated With
Level Of Care Placement: Adults
GAIN SS Can Also be Used for Monitoring
20
12+ Mon.s ago (#1s)
2-12 Mon.s ago (#2s)
Past Month (#3s)
Lifetime (#1,2,or 3)
16
12
10
11
9
9
10
Track Gap Between
Prior and current
Lifetime Problems to
identify “under
reporting”
8
8
3
4
2
2
0
Intake
3
6
9
12
15
18
21
24
Mon Mon Mon Mon Mon Mon Mon Mon
Track progress in
reducing current
(past month)
symptoms)
Total Disorder Screener (TDScr)
Monitor for Relapse
GAIN Short Screener Profile: Reclaiming Futures
(Range based on 0/1-2/3+ Symptoms)
100%
90%
Low
80%
70%
60%
50%
Mod.
40%
81%
30%
20%
48%
33%
37%
38%
High
10%
0%
Internalizing Externalizing
Disorder
Disorder
Screener
Screener
Substance
Disorder
Screener
Crime/
Violence
Screener
Total
Disorder
Screener
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Short Screener Number of Problems Mod/Hi
in Reclaiming Futures
100%
90%
7%
9%
No SR prob
80%
70%
22%
1 Prob.
22%
2 Probs.
93% endorsed one or
more problems
(40% 4 or more)
60%
50%
40%
30%
40%
3 Probs.
20%
4 Probs.
10%
0%
No. of
Problems
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
Full GAIN measure
Construct Validity of
GSS Internalizing Disorder Screener
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
% Days with MH
problem
Mod/High on
Emotional Problem
Scale (EPS)
Mod/High on
Internal Mental
Distress Scale
(IMDS)
Internalizing Disorder Screener (IDScr)
0
1
2
3
4
5
Source: Education Service District 113 (n=979) and King County (n=1002)
Construct Validity of
GSS Externalizing Disorder Screener
Full GAIN measure
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
% Days with
behavioral
problems
Mod/High on
High on Behavior
Emotional Problem Complexity Scale
Scale (EPS)
(BCS)
Externalizing Disorder Screener (EDScr)
0
1
2
3
4
5
Source: Education Service District 113 (n=979) and King County (n=1002)
Construct Validity of
GSS Substance Disorder Screener
100%
90%
Full GAIN measure
80%
70%
60%
50%
40%
30%
20%
10%
0%
% Days of
AOD use
Past Year Abuse or
Dependence
Past Year
Dependence
Substance Disorder Screener (SDScr)
0
1
2
3
4
5
Source: Education Service District 113 (n=979) and King County (n=1002)
Construct Validity of
GSS Crime/Violence Screener
Full GAIN measure
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
% Days of illegal
activities
Mod/High on
Illegal Activity
Scale (IAS)
High on
Crime/Violence
Scale (CVS)
Crime and Violence Screener (CVScr)
0
1
2
3
4
5
Source: Education Service District 113 (n=979) and King County (n=1002)
Problems could be easily identified
Comorbidity
is common
75%
75%
12%
12%
Substance Abuse Student Assistance
Treatment
Programs
(n=8,213)
(n=8,777)
Either
Juvenile Justice
(n=2,024)
High on Mental Health
Mental Health
Treatment (10,937)
High on Substance
12%
11%
46%
35%
61%
60%
73%
62%
40%
37%
86%
83%
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
77%
67%
57%
47%
Adolescent Rates of High (2+) Scores on Mental Health
(MH) or Substance Abuse (SA) Screener by Setting
in Washington State
Children's
Administration
(n=239)
High on Both
Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders
Among DSHS Clients. Olympia, WA: Department of Social and Health Services.
Retrieved from http://publications.rda.dshs.wa.gov/1392/
4%
3%
17%
17%
18%
17%
Lower than expected
rates of SA in Mental
Health & Children’s
Admin
69%
69%
44%
51%
31%
64%
43%
53%
31%
65%
51%
46%
78%
73%
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
81%
68%
69%
56%
Adult Rates of High (2+) Scores on Mental Health
(MH) or Substance Abuse (SA) Screener
by Setting in Washington State
Substance
Abuse
Treatment
(n=75,208)
Either
Eastern State
Hospital
(n=422)
Corrections:
Community
(n=2,723)
High on Mental Health
Corrections:
Prison
(n=7,881)
Mental Health
Treatment
(55,847)
High on Substance
Childrens
Administration
(n=1,238)
High on Both
Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders
Among DSHS Clients. Olympia, WA: Department of Social and Health Services.
Retrieved from http://publications.rda.dshs.wa.gov/1392/
Adolescent Client Validation of Hi Co-occurring from
GAIN Short Screener vs Clinical Records
by Setting in Washington State
Substance Abuse
Treatment (n=8,213)
Juvenile Justice
(n=2,024)
GAIN Short Screener
Mental Health
Treatment (10,937)
9%
11%
15%
12%
34%
35%
56%
Two page measure closely approximated all found
in the clinical record after the next two years
47%
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Children's
Administration
(n=239)
Clinical Indicators
Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders
Among DSHS Clients. Olympia, WA: Department of Social and Health Services.
Retrieved from http://publications.rda.dshs.wa.gov/1392/
Higher rate in clinical record in Mental
Health and Children’s Administration
(But that was past on “any use” vs.
“abuse/dependence” and 2 years vs. past year
3%
17%
22%
39%
59%
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
56%
Adult Client Validation of Hi Co-occurring from
GAIN Short Screener vs Clinical Records
by Setting in Washington State
Substance Abuse Treatment
(n=75,208)
Mental Health Treatment
(55,847)
GAIN Short Screener
Childrens Administration
(n=1,238)
Clinical Indicators
Source: Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders
Among DSHS Clients. Olympia, WA: Department of Social and Health Services.
Retrieved from http://publications.rda.dshs.wa.gov/1392/
Other Validations
Confirmatory Factor Analysis
 Dennis, Chan & Funk (2006) found that the 20 item GSS and its four
subscales were highly correlated (.84 to .94) with the full scale, had 90%
sensitivity and over 90% area under the curve relative to the full GAIN;
Confirmatory factors analysis also found it to be consistent with the overall
model of psychopathology after allowing for age (CFI=.92; RMSEA=.06).
Substance Disorders:
 McDonnell and colleagues (2009) found that the 5-item GAIN SS Substance
Disorder Screener had 92% sensitivity and 85% correct classification relative
to the Diagnostic Inventory Scale for Children (DISC) Predictive Scales
(DPS; Lucas et al 2001) and 88% sensitivity and 88% correct classification
relative to the CRAFFT (Knight et al 2001)
Internalizing Disorders:
 McDonnell and colleagues (2009) found that the 5-item GAIN SS
Internalizing Disorder Screener had 100% sensitivity and 75% correct
classification relative to the Youth Self Report (YSR; Achenbach et al, 2001)
and that the 5-item GAIN SS Externalizing Disorder Screener had 89%
sensitivity and 65% correct classification to the YSR.
 Riley and colleagues (2009) found that the 5-item GAIN SS’s Internalizing
Disorder Screener had 92% sensitivity and 80% area under the curve relative
to the Structured Clinical Interview for DSM (SCID) and was more efficient
relative to 11 item Addiction Severity Index (ASI) psychiatric composite
score (McLellan et al., 1992), 10 item K10 (Kessler et al., 2002) and the 87
item Psychiatric Diagnostic Screening Questionnaire (PDSQ; Zimmerman
and Mattia, 2001)
GAIN Quick (GQ)

Administration Time: 20-30 minutes (depending on severity
and whether reasons for quitting module used)

Training Requirements: 1 day (train the trainer) plus
certification within 1-2 months

Mode: Generally Staff Administered on Computer (can be done
on paper or self administered with proctor)

Purpose: Designed for use in targeted populations to support
brief intervention or referral for further assessment or behavioral
intervention. Not originally designed for follow-up.

Scales: The GQ has total scale (99-symptoms) and 15 subscales
(including more detailed versions of the GSS scales and
subscales plus scales for service utilization, sources of
psychosocial stress, and health problems).
GAIN Quick (GQ) (Continued)

Response Set: Breadth (past year symptom counts for behavior
and lifetime for utilization) and Prevalence (past 90 days)

Interpretation:
– Items can be used individually or to create specific diagnostic
or treatment planning statements
– Items can be summed into scales or indices for each behavior
problem or and for recent service utilization overall
– All scales, indices and selected individual items have
interpretative cut-points to facilitate clinical interpretation
and decision making
– Some people repeat just days items for follow-up.

Reports: Narrative, tabular, graphical, validity and
“motivational interviewing” reports built into web based GAIN
ABS; Program level reports available in SPSS/Excel
GAIN Quick Profile of Reclaiming Futures Sites
(Range based on 0-24% / 25-74% / 75-100% of Symptoms)
Source: Reclaiming Futures Chicago, IL, Dayton, OH, Portland,
OR and Santa Cruz, CA sites (n=475).
25%
28%
AOD Dependence
Substance Problem
High
(76-100%)
*Total Score
29%
AOD Use & Abuse
29%
General Crime
22%
26%
Conduct Disorder
*Externalizing
24%
Hyper-Inattention
*Internalizing 5%
18%
Anxiety-Trauma Sx
Suicide Risk
Depression
*General Life Prob
Health Distress
24%
Mod
(25-75%)
Sources of Stress
Risk
Stress
Health
Low
(0-24%)
General Factors
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
More
detail
within
each area
* Summary Measure
GAIN Quick Number of Problems Mod/Hi
100%
90%
80%
70%
60%
3%
8%
8%
13%
No SR prob
1 Prob.
69%
2 Probs.
50%
40%
3 Probs.
30%
20%
4 Probs.
10%
0%
No. of
Problems
Source: Reclaiming Futures Chicago, IL, Dayton, OH, Portland,
OR and Santa Cruz, CA sites (n=475).
97% endorsed one or
more problems
(69% 4 or more
problems)
Validations

Titus and colleagues (2008) found that the internal
consistency of the full scales ranged from .82 to .90
among adults and adolescents with alpha above .7 for
7 of 8 subscale scores for adults and 7 of 8 subscale
scores for the adolescents.

Titus and colleagues (2008) found that the mental
health scales from the GAIN quick have good internal
consistency (.86 to .90), are correlated with the full
GAIN dimensional measures (.92 to .97) and .99 to
100% sensitivity relative to the full GAIN.
GAIN Quick (GQ): In Transition

Strengths: Length (20-30min) in desired range, range of
topics, efficiently categorizes, narrative reports to support
screening, brief intervention, and referral to treatment

Problems:
–
–
–
–
Lacks scales or recency to support analyses or outcomes
related to “change over time”
Item response choices do not provide information about
lifetime problems
Current Personal Feedback Report focuses only on substance
use and does not address the other content areas of the
GAIN-Q
Only about 60% of the items can be directly imported into
the GAIN-I
GAIN Quick (GQ): In Transition

Plans for Version 3:
– Keep focus on screening, brief intervention and referral to
treatment
– Break out sections for Crime/Violence, HIV risk, Work and
School problems
– Subsume GSS and add similar screeners in other GAIN Q
areas with recency response to address change and lifetime
issues
– Change measures for each symptom count and days items
– Create reasons for change items in each area to support brief
intervention, reducing number of items in substance use
– Make all questions importable into full GAIN
– Expand narrative report to have more treatment planning
statements and to allow motivational interviewing within
each area

Plans for Version 4: Add computer adaptive testing (CAT)
component to get at more detailed diagnosis where needed
GAIN Initial (GI)




Administration Time: Core version 60-90 minutes; Full version 110140 minutes (depending on severity)
Training Requirements: 3.5 days (train the trainer) plus recommend
formal certification program (administration certification within 3
months of training; local trainer certification within 6 months of
training); Advanced clinical interpretation recommended for clinical
supervisors and lead clinicians
Mode: Generally Staff Administered on Computer (can be done on
paper or self administered with proctor)
Purpose: Designed to provide a standardized biopsychosocial for
people presenting to a substance abuse treatment using DSM-IV for
diagnosis, ASAM for placement, and needing to meet common
(CARF, COA, JCAHO, insurance, CDS/TEDS, Medicaid, CSAT,
NIDA) requirements for assessment, diagnosis, placement, treatment
planning, accreditation, performance/outcome monitoring, economic
analysis, program planning and to support referral/communications
with other systems
GAIN Initial (GI) (continued)

Scales: The GI has 9 sections (access to care, substance use, physical
health, risk and protective behaviors, mental health, recovery
environment, legal, vocational, and staff ratings) that include 103 long
(alpha over .9) and short (alpha over .7) scales, summative indices,
and over 3000 created variables to support clinical decision making
and evaluation. It is also modularized to support customization

Response Set: Breadth (past year symptom counts for behavior and
lifetime for utilization), Recency (48 hours, 3-7 days, 1-4 weeks, 2-3
months, 4-12 months, 1+ years, never) and Prevalence (past 90 days),
patient and staff ratings

Interpretation:
– Items can be used individually or to create specific diagnostic or
treatment planning statements
– Items can be summed into scales or indices for each behavior
problem or type of service utilization
– All scales, indices and selected individual items have interpretative
cut-points to facilitate clinical interpretation and decision making
GAIN Initial (GI) (continued)

Reports:
– Validity Report (VR): identifying missing/bad data and
potentially problematic areas of assessment
– Individual Clinical Profile (ICP): lab report with graphical
and tabular summary with links back to the items
– GAIN Recommendation and Referral Summary (GRRS):
Draft of biopsychosocial narrative for clinician to use for
initial assessment summary, diagnosis, placement and
treatment planning
– Personal Feedback Report (PFR): used to support
Motivational Interviewing (MI) / Motivational Enhancement
Therapy (MET)
– Program Profile: program level report that allows
comparison of client characteristics, services received and
outcomes between programs, cohorts or types of clients.
GAIN Initial Profile: Substance Problems Past Year
(Range based range of clinical/logical/statistical rules)
100%
90%
Low
80%
70%
60%
Mod.
50%
40%
30%
High
20%
19%
Sub. Prob.
Past Year
7%
Sub. Use/
Induced
Prob.
0%
15%
Dependence
10%
Abuse
31%
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Substance Problems by Time
(Range based range of clinical/logical/statistical rules)
100%
90%
Low
80%
70%
60%
Mod.
50%
40%
30%
39%
10%
High
31%
13%
Sub. Prob.
Past Month
Sub. Prob.
Past Year
Sub. Prob.
Lifetime
0%
2%
Withdrawal
Sx Past
Week
20%
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Motivation and Readiness
(Range based range of clinical/logical/statistical rules)
100%
90%
Low
80%
70%
60%
Mod.
50%
40%
76%
30%
High
20%
32%
10%
Problem
Orientation
Treatment
Motivation
Treatment
Pressure
7%
Treatment
Resistance
0%
SelfEfficacy
0%
19%
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Crime/Violence
(Range based range of clinical/logical/statistical rules)
100%
90%
80%
70%
60%
Mod.
51%
8%
Drug Crime
Crime
Violence
5%
Interpersonal
Crime
33%
Property
Crime
25%
ConflictTactic
50%
40%
30%
20%
10%
0%
Low
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
High
GAIN Initial Profile: Environmental Risk
(Range based range of clinical/logical/statistical rules)
Low
Mod.
64%
54%
39%
Environmental
Risk
Social
Environment
Risk
28%
Vocational
Environment
Risk
Living
Environment
Risk
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
High
GAIN Initial Profile: Internalizing Disorders
(Range based range of clinical/logical/statistical rules)
100%
90%
80%
70%
60%
Low
50%
Mod.
40%
30%
20%
Sucide Risk
Depression
1%
9%
9%
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
Internalizing
3%
Truama
15%
Somatic
0%
High
24%
AnxietyFear
10%
GAIN Initial Profile: Externalizing Disorders
(Range based range of clinical/logical/statistical rules)
Low
12%
14%
20%
Conduct
Disorder
Externalizing
20%
Hyperactivityimplusive
Mod.
Inattentiveness
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
High
GAIN Initial Profile: Personality Disorders
(Range based range of clinical/logical/statistical rules)
100%
90%
Low
80%
70%
60%
Mod.
50%
40%
30%
53%
20%
10%
Total
Personality
Worrying
(Cluster C)
Implusive
(Cluster B)
Cautious
(Cluster A)
0%
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
High
GAIN Initial Profile: General Factors / Stress
(Range based range of clinical/logical/statistical rules)
100%
90%
80%
70%
60%
50%
40%
30%
Low
Mod.
44%
20%
10%
0%
High
10%
12%
Person
Axis IV
Other Axis
IV
Victimization
School
Prob.
Employment
Prob.
26%
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Other Problem Scales
(Range based range of clinical/logical/statistical rules)
100%
90%
Low
80%
70%
60%
Mod.
50%
40%
30%
High
20%
2%
4%
Life
Satisfaction
Health
0%
17%
Social
Support
12%
Gambling
10%
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Initial Profile: Measures of Behavior Change
(Range based range of clinical/logical/statistical rules)
100%
90%
80%
Low
70%
60%
50%
Mod.
40%
30%
Work
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
Financial
3%
4%
School
Substance
Use
14%
High
23%
Illegal
Activity
3%
0%
10%
Recovery
Environment
41%
Emotions
41%
Health
20%
10%
GAIN Initial Number of Problems Mod/Hi
100%
90%
1%
0%
98%
No problems
80%
70%
1 Prob.
60%
2 Probs.
50%
99% endorsed one or
more problems
(98% 4 or more)
40%
30%
3 Probs.
20%
10%
4 Probs.
0%
No. of
Problems
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
GAIN Treatment Planning/Placement Grid
Problem Recency/Severity
None
Past
Current (past 90 days)*
Low-Mod
None
1. No Problem
Past
Current
Treatment History
0. Not Logical
Check understanding of
problem or lying
and recode.
2. Past problem
Consider
monitoring
and relapse
prevention.
.
5. No current
problems;
Currently in
treatment
Review for step
down or
discharge.
3. Low/Moderate
problems;
Not in treatment
Consider initial or
low invasive
treatment.
6. Low/Moderate
problems;
Currently in
treatment
Review need to
continue or step up.
* Current for Intoxication & Withdrawal = Past 7 days
| High Severity
4. Severe problems;
Not in treatment
Consider a more
intensive treatment
or intervention
strategies.
7. Severe problems;
Currently in
treatment
Review need
for more intensive or
assertive levels.
Reclaiming Futures ASAM Placement Cells
0%
20%
40%
60%
80%
100%
B1.Intox/Withd.
B2 Biomedical
B3.Psych/Beh
B4.Readiness
B5.Rel. Pot.
B6.Environ.
Inconsistent
High Prob
No problem
No Prob in Tx
Past Prob
L/M Prob in Tx
Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)
Low/Mod Prob
H Prob in Tx
Continuing care
90%
Case management
89%
Recovery Environment Risk
87%
Coping w/ Psychosocial Stressors
80%
Accessing Treatment
76%
Child Maltreatment
74%
Disatisfaction with Environment
73%
Behavior Control
School Problems
70%
66%
Anger Management
62%
Vocational Assistance
61%
Detox or Withdrawal
59%
Recovery Coach
58%
HIV risk reduction (sex)
57%
Tobacco Cessation
56%
Source: Reclaiming Futures (n=192)
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Other Common Treatment Planning Needs:
Reclaiming Futures
Part 3.
Highlight our current work to
using actuarial estimates of
outcomes to improve
placement decisions
64
CSAT Adolescent Treatment GAIN Data
from 203 level of care x site combinations
Levels of Care
Long-term Residential
Moderate-Term Residential
Short-Term Residential
Source: Dennis, Funk & Hanes-Stevens, 2008
Outpatient Continuing Care
Intensive Outpatient
Outpatient
Early Intervention
General Group Home
Corrections
Other
Ratings of Problem Severity (x-axis) by Treatment
Utilization (y-axis) by Population Size (circle size)
Utilization
Average Current Treatment
.
1.00
F. HiHi (CC)
12%
0.80
0.60
0.40
B
Low- Mod
0.20
0.00
C
Mod-Mod
20%
A
Low-Low
D
Hi-Low
8%
12%
-0.20
-0.20
G. Hi-Mod
(Env Sx/
PH Tx)
9%
E
HiMod
14%
14%
H. Hi-Hi
(Intx Sx;
PH/MH Tx)
12%
0.00
0.20
0.40
0.60
Average Current Problem Severity
0.80
1.00
Variance Explained in 10 NOMS Outcomes
Percent of Variance Explained
0%
5%
10%
15%
20%
25%
24%
No AOD related Prob.
11%
No Health Problems
25%
No Mental Health Prob.
15%
No Illegal Activity
33%
No JJ System Involve.
26%
Living in Community
18%
No Family Prob.
14%
Vocationally Engaged
8%
Count of above
\1
35%
26%
No AOD Use
Social Support
30%
\2 Past 90 days *All statistically Significant
Past
month
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
24%
Predicted Count of Positive Outcomes by Level
Predicted Count of Positive Outcomes by Level of Care:
of Care: Cluster
A
Low
Low
(n=1,025)
Cluster A Low - Low (n=1,025)
10
10
9
9
8
8
7
7
6
6
5
5
4
Person “A” does
better in Outpatient
3
Person “B” does
better in Higher
Levels of Care
2
4
3
2
Outpatient
Higher LOC
Best Level of Care*:
Level
of Care*:
Cluster A LowBest
- Low
(n=1,025)
Cluster A Low - Low (n=1,025)
120%
% Best Predicted Outcomes
99.6%
100%
80%
60%
40%
20%
0.4%
0%
Outpatient
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Higher LOC
Best Level of Care*:
Cluster B Low - Mod (n=1,654)
90%
% Best Predicted Outcomes
80%
75.1%
70%
60%
50%
40%
30%
20%
14.1%
10.5%
10%
0.4%
0%
Outpatient
IOP
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
OPCC
Residential
Best Level of Care*:
Best Level
of Care*:
Cluster C Mod-Mod
(n=1209)
Cluster C Mod-Mod (n=1209)
90%
% Best Predicted Outcomes
80%
70%
60%
50%
40%
38.6%
30.2%
30%
23.6%
20%
7.6%
10%
0%
Outpatient
IOP
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
OPCC
Residential
Best Level of Care*:
Cluster D Hi-Low (n=687)
90%
80%
70%
60%
50%
40%
38.3%
33.8%
27.9%
30%
20%
10%
0%
Outpatient
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
IOP/OPCC
Residential
Best Level of Care*:
Best Level of Care*:
Cluster F Hi-Hi
(n=968)
Cluster(CC)
F Hi-Hi
(CC) (n=968)
90%
81.5%
% Best Predicted Outcomes
80%
70%
60%
50%
40%
30%
20%
10%
9.9%
8.6%
0.0%
0%
Outpatient
IOP
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
OPCC
Residential
Best Level of Care*:
Cluster Cluster H Hi-Hi (Intx/PH/MH) (n=1,017)
90%
78.2%
% Best Predicted Outcomes
80%
70%
60%
50%
40%
30%
17.2%
20%
10%
0.0%
4.6%
0%
Outpatient
IOP
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
OPCC
Residential
Best Level of Care*:
Cluster E Hi-Mod (n=1,190)
88.3%
90%
% Best Predicted Outcomes
80%
70%
60%
50%
40%
30%
20%
10.6%
10%
0.0%
1.1%
IOP
OPCC
0%
Outpatient
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Residential
Best Level of Care*:
Best Level
of Care*:
Cluster G Hi-Mod
(Env/PH)
(n=749)
Cluster G Hi-Mod (Env/PH) (n=749)
100%
94.1%
90%
80%
70%
60%
50%
40%
30%
20%
10%
5.9%
0.0%
0%
Outpatient
IOP/OPCC
* Based on Maximum Predicted Count of Positive Outcomes
Source: CSAT 2007 AT Outcome Data Set (n=11,013)
Residential
Best (x) by Actual (y) Level of Care Placement
3500
3132
3000
2339
2500
1968
2000
Higher
Best
1500
Lower
1000
797
500
0
Outpatient
(n=3132)
Intensive Outpatient
(n=797)
OP - Continuing
Care (n=1968)
Residential
(n=2339)
Exploring Need, Unmet Need, & Targeting of
Mental Health Services in AAFT
At Intake . No/Low
After 3 mon
Any Treatment
Mod/High
Need
Need
6
218
Total
224
218/224=97% to targeted
No Treatment
205
553
758
553/771=72%
unmet need
Total
211
771
982
771/982=79% in need
Size of the Problem
Extent to which services are not reaching those in most need
Extent to which services are currently being targeted
Mental Health Problem (at intake) vs.
Any MH Treatment by 3 months
97%
100%
90%
80%
79%
72%
70%
60%
50%
40%
30%
20%
10%
0%
% of Clients With
Mod/High Need
(n=771/982)*
% w Need but No Service % of Services Going to
After 3 months
Those in Need
(n=553/771)
(n=218/224)
*3+ on ASAM dimension B3 criteria
Source: 2008 CSAT AAFT Summary Analytic Dataset
Why Do We Care About Unmet Need?

If we subset to those in need, getting mental
health services predicts reduced mental health
problems

Both psychosocial and medication interventions
are associated with reduced problems

If we subset to those NOT in need, getting mental
health services does NOT predict change in
mental health problems
Conversely, we also care about services being
poorly targeted to those in need.
Residential Treatment need (at intake) vs.
7+ Residential days at 3 months
100%
90%
80%
70%
60%
50%
40%
30%
90%
Opportunity to
redirect
existing funds
through better
targeting
52%
36%
20%
10%
0%
% of Clients With
Mod/High Need
(n=349/980)*
% w Need but No
% of Services Going to
Service After 3 months Those in Need (n=34/66)
(n=315/349)
Source: 2008 CSAT AAFT Summary Analytic Dataset
Part 4. Summarize the status of efforts
to make the data available for
secondary analysis and
translate the software,
measures and reports from
English into Spanish, French,
Portuguese and other
languages
82
Secondary Analysis
 We currently pool data from Center for Substance
Abuse Treatment (CSAT) grantees annual and make
it available for secondary analysis:
–
–
–
–
Requires abstract length proposal/ feasibility
Requires agreement to respect privacy and not attempt to
re-identify
We will get permission from any active grantees
No cost to the end user
 Over 36 scientist and evaluators have already
accessed the data and about 1-2 more come get
approval each month
 We can also negotiate access to additional data from
individual grantees and/or regional projects
83
Status of Translations
Short
Screener
Other
Instruments Software
Reports
English
Spanish
Done
Done
Done
Done
Done
Done
In progress
In progress
French
In progress
In progress
In progress
In progress
Portuguese Done
Starting
Not yet
Not Yet
Done
Japanese
Done
Hmong,
Japanese,
Russian,
Pilipino,
Punjabi,
Vietnamese
Not yet
Not yet
Not Yet
Not yet
Not yet
Not Yet
Language
84
Acknowledgments and Contact Information

This presentation was supported by analytic runs provided by Chestnut Health Systems for the
Substance Abuse and Mental Health Services Administration's (SAMHSA's) Center for Substance
Abuse Treatment (CSAT) under Contracts 207-98-7047, 277-00-6500, 270-2003-00006 and 2702007-00004C using data provided by the following 152 grantees: TI11317 TI11321 TI11323
TI11324 TI11422 TI11423 TI11424 TI11432 TI11433 TI11871 TI11874 TI11888 TI11892
TI11894 TI13190TI13305 TI13308 TI13313 TI13322 TI13323 TI13344 TI13345 TI13354
TI13356 TI13601 TI14090 TI14188 TI14189 TI14196 TI14252 TI14261 TI14267 TI14271
TI14272 TI14283 TI14311 TI14315 TI14376 TI15413 TI15415 TI15421 TI15433 TI15438
TI15446 TI15447 TI15458 TI15461 TI15466 TI15467 TI15469 TI15475 TI15478 TI15479
TI15481 TI15483 TI15485 TI15486 TI15489 TI15511 TI15514 TI15524 TI15524 TI15527
TI15545 TI15562 TI15577 TI15584 TI15586 TI15670 TI15671 TI15672 TI15674 TI15677
TI15678 TI15682 TI15686 TI16386 TI16400 TI16414 TI16904 TI16928 TI16939 TI16961
TI16984 TI16992 TI17046 TI17070 TI17071 TI17334 TI17433 TI17434 TI17446 TI17475
TI17476 TI17484 TI17486 TI17490 TI17517 TI17523 TI17535 TI17547 TI17589 TI17604
TI17605 TI17638 TI17646 TI17648 TI17673 TI17702 TI17719 TI17724 TI17728 TI17742
TI17744 TI17751 TI17755 TI17761 TI17763 TI17765 TI17769 TI17775 TI17779 TI17786
TI17788 TI17812 TI17817 TI17825 TI17830 TI17831 TI17864 TI18406 TI18587 TI18671
TI18723 TI19313 TI19323 TI655374. Any opinions about this data are those of the authors and
do not reflect official positions of the government or individual grantees. It is available at
www.chestnut.org/li/posters. Comments or questions can be addressed to Michael Dennis,
Chestnut Health Systems, 448 Wylie Drive, Normal, IL 61761. Phone 1-309-451-7801; E-mail:
[email protected].

More information on the GAIN is available at www.chestnut.org/li/gain or by e-mailing
[email protected] .
85