Transcript gaincc.org
Chapter 19 Methods Appendix GAIN Coordinating Center (11/21/2012). Normal, IL: Chestnut Health Systems. November 2012. Available from www.gaincc.org/slides Created for: Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) under contract number 270-2012-00001 Methods Appendix 2 This section provides an overview of methodological issues including: – – – – – Administration times Follow-up rates, Months to last follow-ups, Number of follow-ups, Instrument fidelity. Administration Times 3 ‘Core Only’ administration refers to records where the participant was asked only the required GAIN items and none of the optional items. ‘Core + Optional’ (Full) administration refers to records where the participant was asked all required and optional GAIN items. This version includes recommended and treatment items for a broader clinical picture. ‘Total’ indicates the combination of both the records that used a core with the records that used the full. Approximately 8% of cases could not be designated as a Full or a Core only administration. Less than 1% were missing data on interview duration. Administration Times 4 Core Only Core + Optional Total 10,691 (127) 16,870 (206) 29,715 (230) 88 112 103 95% Confidence Interval (in minutes) 87 to 89 111 to 113 102 to 103 St. Deviation (in minutes) 31 43 41 1.37 0.78 1.02 5% 49 55 50 25% 65 80 74 50% 83 105 95 75% 103 135 120 95% 145 193 180 Number of records (sites) Average Minutes Skew Percentiles SAMHSA 2011 GAIN Summary Analytic Data Set (n = 29,715) Valid N for Core and Optional Items 5 The number of cases available for analysis will vary depending upon whether the item of interest is a an optional or required (Core) item. The following slides show the number and breakout characteristics of cases available when the item of interest is Optional or Core. – Optional items: Only those that opted to fill out a full version of the GAIN will have valid answers to optional items, so optional items will have a lower number of valid responses. – Required/Core items: Every record should have a valid answer for a required item, so core items will include all records in the SAMHSA/CSAT Data Set. Valid N for Core and Optional Items – 1 6 Optional Items Core Items 13,575 29,782 206 230 6/1998-11/2011 6/1998-11/2011 Female 29% 27% African American 18% 16% Hispanic 22% 29% Other 4% 4% Caucasian 43% 38% Multi-Racial 12% 13% Young Adolescent (<15) 12% 14% Older Adolescent (15-17) 60% 64% Young Adult (18-25) 15% 13% Older Adult (26+) 12% 9% N Number of sites Start date-End date Gender Race Age SAMHSA 2011 GAIN Summary Analytic Data Set (n = 29,782) Valid N for Core and Optional Items - 2 7 Level of Care Treatment Type Optional Items Core Items EI/OP 70% 71% IOP 16% 13% CC-OP 5% 7% M-LTR 8% 8% STR 1% 1% MET/CBT 38% 35% Other EBTx 12% 11% A-CRA/ACC 24% 28% Other 18% 18% Local Manualized Tx 9% 7% SAMHSA 2011 GAIN Summary Analytic Data Set (n = 29,782) Valid N for Core and Optional Items - 3 8 Program Optional Items Core Items Other 3% 4% TCE 7% 10% EAT 21% 22% DC 3% 6% AAFT 15% 19% SCY 10% 8% ATDC 7% 4% ORP 13% 8% YORP 8% 8% CYT 3% 2% ART 3% 6% ATM 7% 5% SAMHSA 2011 GAIN Summary Analytic Data Set (n = 29,782) Valid N for Core and Optional Items - 4 9 Homeless Risk Mental Health Disorders Substance Severity Optional Items Core Items Housed 73% 75% Group/Institution 12% 11% At Risk for Homelessness 10% 10% Currently Homeless 5% 4% Neither 40% 37% Externalizing Only 19% 21% Internalizing Only 10% 9% Both 31% 33% No Past Year Use 1% 1% Past Year Use 24% 22% Past Year Abuse 30% 29% 46% 47% Past Year Dependence SAMHSA 2011 GAIN Summary Analytic Data Set (n = 29,782) Valid N for Core and Optional Items - 5 10 Intensity of Justice System Involvement Optional Items Core Items PY illegal activity/SA use 15% 15% Past arrest/CJ status 6% 6% Other CJ status 12% 12% Other prob/parole/deten 16% 15% Prob/parole 14+ days 20% 22% Drug Court 11% 11% Deten/jail 14-29 days 4% 5% Deten/jail 30+ days 16% 14% SAMHSA 2011 GAIN Summary Analytic Data Set (n = 29,782) Valid N for Core and Optional Items - 6 11 Victimization Crime and Violence Health Problems Optional Items Core Items Low Severity 36% 35% Moderate Severity 19% 19% High Severity 46% 46% Low Crime 44% 41% Moderate Crime 34% 24% High Crime 32% 35% Low Problems 57% 54% Moderate Problems 34% 37% High Problems 9% 9% SAMHSA 2011 GAIN Summary Analytic Data Set (n = 29,590) Follow Up Rates* 12 100% 90% 80% 89% 79% 70% 77% 69% 60% 50% 40% 30% 20% 10% 0% 1+ Follow Up Rate 3 Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 6 Month 12 Month Follow Up Rates* by Age <15 1+ Follow Up Rate 15-17 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 18-25 6 Month 26+ 12 Month 72% 78% 78% 89% 66% 73% 78% 87% 69% 76% 79% 88% 72% 80% 83% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 91% 13 Follow Up Rates* by Gender Male 1+ Follow Up Rate 71% 78% 81% 89% 68% 76% 79% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 88% 14 Female 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 6 Month 12 Month Follow Up Rates* by Race 91% 79% 82% 76% 89% 81% 77% 70% 88% 79% 75% 65% African American 89% 79% 78% 70% 66% 77% 72% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 87% 15 Hispanic Other White Multi-Racial 1+ Follow Up Rate 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 6 Month 12 Month Follow Up Rates* by Substance Use Severity No PY Use 1+ Follow Up Rate PY Use 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set PY Abuse 6 Month 71% 78% 89% 80% 68% 77% 80% 89% 64% 74% 78% 87% 66% 78% 77% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 89% 16 PY Dependence 12 Month Follow Up Rates* by Risk of Homelessness Housed 1+ Follow Up Rate Group/Inst 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set At Risk of Homelessness 6 Month 66% 72% 86% 77% 65% 75% 75% 87% 66% 74% 74% 86% 70% 78% 81% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 89% 17 Currently Homeless 12 Month Follow Up Rates* by Severity of Victimization Low Severity 1+ Follow Up Rate Moderate Severity 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 6 Month 69% 76% 79% 89% 69% 76% 80% 89% 68% 77% 80% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 88% 18 High Severity 12 Month Follow Up Rates* by Mental Health Disorders Neither Externalizing Internalizing Disorders Only Disorders Only 1+ Follow Up Rate 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 6 Month 72% 78% 81% 90% 70% 77% 78% 88% 70% 76% 80% 89% 66% 75% 78% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 87% 19 Both 12 Month Follow Up Rates* by Health Problems Low Problems 1+ Follow Up Rate Moderate Problems 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 6 Month 71% 77% 82% 90% 72% 79% 82% 90% 67% 75% 78% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 88% 20 High Problems 12 Month Follow Up Rates* by Crime and Violence Low Crime 1+ Follow Up Rate Moderate Crime 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 6 Month High Crime 12 Month 69% 77% 79% 89% 70% 77% 81% 89% 68% 76% 79% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 88% 21 1+ Follow Up Rate 3-Month SAMHSA 2011 GAIN Summary Analytic Data Set 89% 81% 76% 68% 92% 82% 79% 73% 90% 81% 77% 70% Past arrest/ CJ status Other CJ status Other prob/ parole/ deten Prob/parole 14+ days 6 Month *Of those due for that follow-up Deten/jail 30+ days Deten/jail 14-29 days Drug Court 88% 82% 78% 71% 86% 73% 73% 61% 93% 84% 82% 72% 71% 66% 83% 74% 89% 81% 79% 69% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% PY illegal activity/ SA use Follow Up Rates* by Intensity of Justice System Involvement 22 12 Month Follow Up Rates* by Program 23 1+ Follow Up Rate 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 6 Month 12 Month ATM ART CYT YORP ORP ATDC SCY AAFT DC EAT TCE Other 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Follow Up Rates* by Level of Care IOP CC-OP 1+ Follow Up Rate 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set 95% 92% 92% 90% 82% 72% 70% 60% OP 93% 86% 82% 79% 88% 76% 73% 63% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 89% 80% 77% 69% 24 M-LTR STR 6 Month 12 Month Follow Up Rates* by Type of Treatment MET/CBT Other EBTx ACRA/ACC 1+ Follow Up Rate 3-Month *Of those due for that follow-up SAMHSA 2011 GAIN Summary Analytic Data Set Other 6 Month 90% 95% 90% 89% 74% 91% 81% 82% 71% 91% 83% 79% 75% 65% 88% 77% 61% 70% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 84% 74% 25 Tx Man 12 Month Average Number of Follow Ups & Months to Last Wave by Gender 26 Average Number of Follow Ups (n=29,771; mean=1.9) 4 12 3 9 2 6 1 1.9 1.9 Average Months from Intake to Last Wave (n=25,498; mean=8.0) 8.0 8.1 Male Female 3 0 0 Male Female SAMHSA 2011 GAIN Summary Analytic Data Set Average Number of Follow Ups & Months to Last Wave by Age 27 Average Number of Follow Ups (n=29,771; mean=1.9) 4 12 3 9 2 6 1 2.0 2.0 1.8 1.6 0 Average Months from Intake to Last Wave (n=25,508; mean=8.0) 8.2 8.1 7.7 7.6 <15 15-17 18-25 26+ 3 0 <15 15-17 18-25 26+ SAMHSA 2011 GAIN Summary Analytic Data Set Average Number of Follow Ups & Months to Last Wave by Race 28 Average Number of Follow Ups (n=29,767; mean=1.9) Average Months from Intake to Last Wave (n=29,499; mean=8.0) 4 12 3 9 2 6 7.8 1 1.8 1.8 2.0 2.0 1.8 7.9 8.7 8.2 7.8 White MultiRacial 3 0 0 African Hispanic Other American White MultiRacial SAMHSA 2011 GAIN Summary Analytic Data Set African Hispanic Other American Average Number of Follow Ups & Months to Last Wave by Substance Use Severity 29 Average Number of Follow Ups (n=29,358; mean=1.9) Average Months from Intake to Last Wave (n=25,162; mean=8.0) 4 12 3 9 2 6 1 1.8 1.7 2.0 2.0 0 3 7.3 7.3 No PY Use PY Use 8.2 8.3 0 No PY Use PY Use PY PY Abuse Dependence SAMHSA 2011 GAIN Summary Analytic Data Set PY PY Abuse Dependence Average Number of Follow Ups & Months to Last Wave by Risk of Homelessness 30 Average Number of Follow Ups (n=29,646; mean=1.9) Average Months from Intake to Last Wave (n=25,396; mean=8.0) 4 12 3 9 2 6 1 2.0 1.8 1.8 1.7 8.1 7.8 8.0 7.8 Housed Group/ Inst At Risk of Homelessness Currently Homeless 3 0 0 Housed Group/Inst At Risk of Homelessness Currently Homeless SAMHSA 2011 GAIN Summary Analytic Data Set Average Number of Follow Ups & Months to Last Wave by Severity of Victimization 31 Average Number of Follow Ups (n=29,501; mean=1.9) Average Months from Intake to Last Wave (n=25,268; mean=8.0) 4 12 3 9 2 6 1 1.9 1.9 1.9 0 7.9 8.0 8.1 Low Severity Moderate Severity High Severity 3 0 Low Severity Moderate Severity High Severity SAMHSA 2011 GAIN Summary Analytic Data Set Average Number of Follow Ups & Months to Last Wave by Mental Health Disorders 32 4 Average Months from Intake to Last Wave (n=25,452; mean=8.0) Average Number of Follow Ups (n=29,684; mean=1.9) 12 3 9 2 6 1 1.8 2.0 1.8 2.0 7.8 8.2 7.9 8.3 3 0 0 Neither ExternalizingInternalizing Disorders Disorders Only Only Both SAMHSA 2011 GAIN Summary Analytic Data Set Neither ExternalizingInternalizing Disorders Disorders Only Only Both Average Number of Follow Ups & Months to Last Wave by Health Problems 33 4 Average Number of Follow Ups (n=29,477; mean=1.9) 12 3 9 2 6 1 1.9 2.0 1.9 0 Average Months from Intake to Last Wave (n=25,266; mean=8.0) 8.1 8.0 7.9 Low Problems Moderate Problems High Problems 3 0 Low Problems Moderate Problems High Problems SAMHSA 2011 GAIN Summary Analytic Data Set Average Number of Follow Ups & Months to Last Wave by Crime and Violence 34 4 Average Number of Follow Ups (n=29,703; mean=1.9) 12 3 9 2 6 1 1.8 2.0 2.0 0 Average Months from Intake to Last Wave (n=25,468; mean=8.0) 7.8 8.1 8.2 Low Crime Moderate Crime High Crime 3 0 Low Crime Moderate Crime High Crime SAMHSA 2011 GAIN Summary Analytic Data Set 1.5 0 SAMHSA 2011 GAIN Summary Analytic Data Set Deten/jail 30+ days Deten/jail 14-29 days 6.6 8.1 Prob/parole 14+ days Drug Court 8.3 0 Other prob/ parole/deten 3 8.5 8.4 6 Other CJ status 2 8.5 9 Past arrest/ CJ status 3 8.4 12 7.4 2.2 1.7 2.0 2.0 2.0 2.0 Average Number of Follow Ups (n=29,617; mean=1.9) PY illegal activity/SA use Deten/jail 30+ days Deten/jail 14-29 days Drug Court Prob/parole 14+ days Other prob/ parole/deten Other CJ status 1 2.0 4 Past arrest/ CJ status PY illegal activity/SA use Average Number of Follow Ups & Months to Last Wave by Intensity of Justice Involvement 35 Average Months from Intake to Last Wave (n=25,390; mean=8.0) Average Number of Follow Ups & Months to Last Wave by Program 36 Average Number of Follow Ups (n=29,782; mean=1.9) 10.0 7.3 7.0 SCY ATDC ORP 6.2 7.7 8.0 EAT 6.4 7.9 1.4 3 7.1 3.1 2.1 1.5 1.8 2.6 2.0 1.4 1.7 1.8 1.7 ATM ART CYT YORP AAFT ATM ART CYT YORP ORP ATDC SCY AAFT DC EAT TCE Other SAMHSA 2011 GAIN Summary Analytic Data Set DC 0 0 Other 1 TCE 6 2 11.1 9 8.4 3 11.5 12 3.8 4 Average Months from Intake to Last Wave (n=25,508; mean=8.0) Average Number of Follow Ups & Months to Last Wave by Level Of Care 37 Average Number of Follow Ups (n=28,749; mean=1.9) Average Months from Intake to Last Wave (n=24,620; mean=8.0) 4 12 3 9 2 6 3.0 1 1.9 2.3 1.8 1.7 0 9.3 8.0 7.5 OP IOP 3 10.0 7.4 0 OP IOP CC-OP M-LTR STR SAMHSA 2011 GAIN Summary Analytic Data Set CC-OP M-LTR STR Average Number of Follow Ups & Months to Last Wave by Type of Treatment 38 Average Number of Follow Ups (n=28,880; mean=1.9) Average Months from Intake to Last Wave (n=24,700; mean=8.0) 4 12 3 9 2 6 10.4 3.0 1 1.8 1.7 1.9 1.8 8.0 3 7.1 7.7 8.1 0 0 MET/ CBT Other ACRA/ Other Tx Man EBTx ACC SAMHSA 2011 GAIN Summary Analytic Data Set MET/ CBT Other ACRA/ Other Tx Man EBTx ACC GAIN Administration Fidelity Index (GAFI) 39 Average Denial/Misrepresentation (0/1/2-4) 4% High 5% Fidelity Max Breaks (0-1/2/3-high) Context Effects (0/1-2/3-high) 7% Inconsistencies (0-4/5-9/10-high) 11% Administration Duration (low-90/91-149/150-high) Don't Know's in Change Scale Items (0/1/2-high) 13% 14% Total GAINEdits (0-1/2-5/6-high) GAFI* (0-69/70-79/80-100) 0% Moderate Fidelity 18% 33% 20% 35% 40% *Proportional sum of 7 items SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,482) 60% 32% 80% 100% Low Fidelity GAFI by Gender 40 100% 90% 80% 30% 35% Low Fidelity (0-69%) 70% 60% Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% Male Female SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,475) GAFI by Age 41 100% 90% 80% 32% 33% 30% 26% Low Fidelity (0-69%) 70% 60% Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% <15 15-17 18-25 SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,482) 26+ GAFI by Race 42 100% 90% 80% 30% 29% 33% 32% 37% Low Fidelity (0-69%) 70% 60% Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% African Hispanic American Other White SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,468) MultiRacial 42 GAFI by Substance Use Severity 43 100% 90% 14% 20% 28% 80% 39% 70% 60% Low Fidelity (0-69%) Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% No PY Use PY Use PY Abuse SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,207) PY Dependence GAFI by Risk of Homelessness 44 100% 90% 80% 30% 28% 40% 47% Low Fidelity (0-69%) 70% 60% Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% Housed Group/ Inst At Risk of Currently Homelessness Homeless SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,362) GAFI by Severity of Victimization 45 100% 90% 24% 80% 30% 38% Low Fidelity (0-69%) 70% 60% Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% Low Severity Moderate Severity High Severity SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,454) GAFI by Mental Health Disorders 46 100% 90% 22% 80% 32% 36% 42% 70% 60% Low Fidelity (0-69%) Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% Neither Externalizing Internalizing Disorders Disorders Only Only SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,451) Both GAFI by Health Problems 47 100% 90% 80% 32% 30% 35% Low Fidelity (0-69%) 70% 60% Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% Low Problems Moderate Problems SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,289) High Problems GAFI by Crime and Violence 48 100% 90% 25% 80% 30% 41% Low Fidelity (0-69%) 70% 60% Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% Low Crime Moderate Crime SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,460) High Crime GAFI by Intensity of Justice System Involvement 32% 45% 38% 32% 32% 28% 27% Low Fidelity (0-69%) Moderate Fidelity (70-79%) SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,404) Deten/jail 30+ days Deten/jail 14-29 days Drug Court Prob/parole 14+ days Other prob/ parole/deten Other CJ status Past arrest/ CJ status High Fidelity (80-100%) PY illegal activity/SA use 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 26% 49 GAFI by Program 50 70% 48% 33% 24% 41% 33% 34% 35% 28% 80% 26% 90% 28% 100% 60% Low Fidelity (0-69%) Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% SAMHSA 2011 GAIN Summary Analytic Data Set (n=19,482) ART YORP ORP ATDC SCY AAFT DC EAT TCE Other 0% GAFI by Level of Care 51 100% 90% 80% 30% 32% 33% 47% 43% Low Fidelity (0-69%) 70% 60% Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% OP IOP CC-OP M-LTR SAMHSA 2011 GAIN Summary Analytic Data Set (n=18,684) STR GAFI by Type of Treatment 52 100% 90% 80% 29% 28% 34% 32% 37% 70% 60% Low Fidelity (0-69%) Moderate Fidelity (70-79%) 50% 40% 30% High Fidelity (80-100%) 20% 10% 0% MET/ CBT Other EBTx ACRA/ ACC Other SAMHSA 2011 GAIN Summary Analytic Data Set (n=18,907) Tx Man Summary of Psychometrics 53 Normative psychometric data and basic demographic information for GAIN scales and data sets (mean, N, standard deviation, Cronbach's alpha and intraclass correlations are available at http://www.gaincc.org/_data/files/Psychometrics_and_P ublications/Resources/GAIN-I_Norms.xls – This file is broken up into separate worksheets for all records, Adolescents (age 12-17), Young Adults (age 18-25), and Adults (age 18 and higher). – Within each age category, demographics are presented first, followed by overall norms and ICC including across follow-up waves, then intake norms by each of the three main demographic categories of gender, race and age. Accessing GAIN Data 54 Access to the de-identified pooled SAMHSA/CSAT data set is available through request. For more information regarding how to request data and what you will need please visit http://gaincc.org/analysis Summary of what is needed: – Data Sharing Agreement (see DSA Decision Tree at http://www.gaincc.org/_data/files/Psychometrics_and_Publicati ons/Analysis_Publication/DSA_Decision_Tree.pdf) – A 1 - 3 page abstract (see Access training memo at http://www.gaincc.org/_data/files/Psychometrics_and_Publicati ons/Analysis_Publication/Access.pdf) For help please contact [email protected]