Transcript Slide 1
Building Community Collaboration to Promote Healthy Youth Development: J. David Hawkins Ph.D. Social Development Research Group School of Social Work University of Washington www.sdrg.org Melissa Institute May 1, 2008 1 The Challenge for Community Prevention • To address those risk factors most prevalent in a community with tested, effective policies and programs. • To address protective factors most depressed in a community with tested, effective policies and programs that strengthen those protective factors. 2 The Communities That Care Prevention System • Helps communities apply the advances of prevention science to guide youth development and prevention work. • Measures community levels of protection and risk by surveying young people. • Matches the community’s profile of risk and protection with tested, effective programs and policies. 3 The Communities That Care Prevention System • Local control builds ownership to create sustainable change. • Focuses on outcomes to insure success: Are fewer teens using drugs? Fewer smoking? Fewer committing violent acts? 4 The Communities That Care Operating System Get Started Implement and Evaluate Creating Communities That Care Create a Plan Get Organized Develop a Profile 5 The Communities That Care Operating System • Community readiness assessment. • Identification of key individuals, stakeholders, and organizations. Get Started Implement and Evaluate Creating Communities That Care Create a Plan Get Organized Develop a Profile 6 The Communities That Care Operating System • Training key leaders and board in CTC Get Started Implement and Evaluate • Building the community coalition. Creating Communities That Care Create a Plan Get Organized Develop a Profile 7 The Communities That Care Operating System Get Started Implement and Evaluate Creating Communities That Care Create a Plan • Collect risk/protective factor and outcome data. •Collect information on community resources • Construct a community profile from the data. Get Organized Develop a Profile 8 w Ne i gh bo La rh ws Co oo an mm dA dN un tta or i ty ch ms me Di so Fa nt r v g Pe o an rce rabl iza et iv e tio Pe n dA o D rce rug va iv e il a Us dA bi l e va i ty il a of bi l Dr it Po ug or y o f s Ha Fa m n dg Pa il y Fa re M a uns mi nta na ly lA ge Hi Pa ttit s to me re F u a nta ry nt de mi of sF lA ly An Co ttit av ti s ud ora nfl oc es i ct bl e i al Fa T B vo ow eh rab av ard ior le sD to rug An ti s Us oc e i al B Ac eh Lo ad a w em vior Co ic mm Fa itm i lu en re t to Fa Re Sch Ea oo vo be rly l rab lli o Pr u le s E o At ne ar ble ti tu ss ly m In i de Be t i s a h Fa To tio vo n o avi o wa rab r rd fD le An ru At g Us ti tu ti so Lo ci a e de w l s B Pe To eh rce wa av ior i ve rd Dr dR ug isk Us so e Fr fD ien ru gU ds 'U Re s s wa eo e Se rd fD sf ns rug or a ti o s An n ti s Se oc ek i al ing In v olv em Ov ent era ll R i sk Lo Percent At Risk Madison Middle School Risk Profile 8th Grade 2002 100% 90% Community Diffusion Consortium Project Social Development Research Group, University of Washington Family School Peer-Individual 80% Survey Participation Rate 2002: 87.4% 70% 60% 50% 40% 30% 20% 10% 0% School 2002 District 2002 Estimated National Value 9 The Communities That Care Operating System Get Started Implement and • Define outcomes. •Prioritize factors to be Evaluate targeted. • Select tested, effective interventions. • Create action plan. • Develop evaluation plan. Creating Communities That Care Create a Plan Get Organized Develop a Profile 10 Addressing Barriers with Effective Action Factor Addressed Family Management Problems Program Strategy Developmental Period Prenatal/Infancy Programs prenatal-2 Early Childhood Education 3-5 Parent Training prenatal-14 Family Therapy 6-14 11 Diffusion Consortium Project Social Development Research Group, University of Washington Effective Training for Middle School Parents • Guiding Good Choices (Spoth et al., 1998) • Adolescent Transitions Program (Dishion and Andrews, 1995) • Parenting Adolescents Wisely (Gordon et al., 1998) • Creating Lasting Connections (Johnson et al., 1996) • Strengthening Families 10 to 14 Program (Spoth, 1998) • Focus on Families (Catalano et al., 1999; 1997) 12 Diffusion Consortium Project Social Development Research Group, University of Washington The Communities That Care Operating System • Form task forces. • Identify and train implementers. • Sustain collaborative relationships. • Evaluate processes and outcomes. • Adjust programming. Implement and Evaluate Get Started Creating Communities That Care Create a Plan Get Organized Develop a Profile 13 What is required to install CTC? • • • • • A coalition of community stakeholders. A coordinator for the CTC process. Manuals and curriculum materials. Training from certified trainers. Technical assistance when difficulties are encountered. • A monitoring system to provide routine feedback on progress and outcomes. 14 CTC Trainings 1. Key Leader Orientation 2. Community Board Training 3. Community Assessment Training 4. 5. 6. Community Resource Assessment Training Community Planning Training Community Program Implementation Training 15 Communities that Care Process and Timeline Process Assess risk, Implement and protection and evaluate resources tested prevention strategies 6-9 mos. 1 year Measurable Outcomes Increase in priority protective factors Increase in positive youth development Decrease in priority risk factors 2-5 years Reduction in problem behaviors Vision for a healthy community 5-10 years 16 The Community Youth Development Study (CYDS) • A 24 community randomized controlled trial to test the Communities That Care system started in 2003. 17 The Community Youth Development Study Team Robert D. Abbott Michael W. Arthur Megan M. Baldwin John S. Briney Blair Brooke-Weiss Eric C. Brown Rick Cady Richard F. Catalano Abigail A. Fagan John Graham Kevin Haggerty Koren Hanson J. David Hawkins David M. Murray Sabrina Oesterle M. Lee Van Horn Funded by: National Institute on Drug Abuse Center for Substance Abuse Prevention National Cancer Institute National Institute on Child Health and Development National Institute on Mental Health CYDS State Collaborators • Colorado Alcohol & Drug Abuse Division • Illinois Division of Community Health & Prevention • Kansas Alcohol and Drug Abuse Services • Maine Office of Substance Abuse • Oregon Office of Alcohol & Drug Abuse Programs • Utah Division of Substance Use • Washington Division of Alcohol & Substance Abuse 19 CYDS Primary Aim To test the efficacy of the Communities That Care system in • reducing levels of risk • increasing levels of protection • reducing health and behavior problems among adolescents using a true experimental design. 20 STUDY DESIGN Randomized Controlled Trial 2003-2008 2003 2004 1997-2002 CTCYS Randomize CTCYS CKI CRD CTCYS CTCYS CKI CRD CTCYS: Cross-sectional student survey of 6th-, 8th-, 10th-, and 12th-grade students using the CTC Youth Survey CKI: Community Key Informant Interview CRD: Community Resource Documentation measuring effective prevention programs and policies in the community CTC Board: CTC Board Member Interview YDS: Longitudinal Youth Development Survey of students in the class of 2011 starting in 5th grade in spring 2004 CTCYS CTC Board YDS Control 2007 2008 CTCYS CKI CRD CKI CRD Intervention 98 99 ‘00 ‘01 ‘02 2006 Implement selected interventions Planning 5-Year Baseline 2005 CTC Board YDS CTCYS CTC Board YDS CTC Board YDS YDS CTCYS CTCYS CKI CRD YDS YDS CTC Board CKI CRD YDS YDS YDS 21 Demographics of 24 CYDS Communities Mean Minimum Maximum 14,616 1,578 40,787 89.4% 64.0% 98.2% Percent Hispanic Origin 9.6% 0.5% 64.7% Percent African-American 2.6% 0.0% 21.4% 36.5% 20.6% 65.9% Total Population Percent Caucasian Percent Eligible for Free/Reduced Lunch 22 PanelYouth Development Survey (YDS) • Annual survey of panel recruited from the Class of 2011 (5th grade in 2004) • Active, written parental consent 23 Youth Development Survey • Participants recruited in grades 5 and 6. • Final consent rate = 76.4% Sixth Grade Eligible Population Percent Consented Percent Surveyed Total Surveyed Control 3170 2621 76.2% 76.7% 75.4% 76.3% 2391 1999 Total 5791 76.4% 75.8% 4390 Experimental 24 2006 YDS • 96.2% Overall Student Participation • 10.3% (n=454) had moved out of project schools 7th Grade Eligible Population Percent Surveyed Total Surveyed Experimental 2406 95.5% 2298 Control 2001 97.0% 1941 Total 4407 96.2% 4239 25 2007 YDS • 96.2% Overall Student Participation • 11.9% (n=525) have moved out of project schools 8th Grade Eligible Population Percent Surveyed Total Surveyed Experimental 2406 95.6% 2300 Control 2001 96.9% 1940 Total 4407 96.2% 4240 26 Communities That Care Logic Model CTC Training and Technical Assistance Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Decreased Risk and Enhanced Protection Positive Youth Outcomes 27 Adoption of Science-Based Prevention Stage 0: No Awareness Stage 1: Awareness of Prevention Science Terms and Concepts Stage 2: Using Risk and Protection Focused Prevention Approach as a Planning Strategy. Stage 3: Incorporation of Community Epidemiological Data on Risk and Protection in Prevention System. Stage 4: Selection and Use of Tested and Effective Preventive Interventions to Address Prioritized Risk and Protective Factors. Stage 5: Collection and Feedback of Process and Outcome Data and Adjustment of Preventive Interventions Based on Data. 28 Note. Community Key Informant Survey (CKI). Baseline Stages of Adoption by Intervention Status 0.6 Probability 0.5 Control Communities CTC Communities 0.4 0.3 0.2 0.1 0.0 0 1 2 3 4 5 Stage of Adoption in 2001 Note. Community Key Informant Survey (CKI); N = 534; γ001 = .037, SE = .395, df = 20, p > .05 . 29 Post-Intervention Stages of Adoption by Intervention Status 0.6 Probability 0.5 Control Communities CTC Communities 0.4 0.3 0.2 0.1 0.0 0 1 2 3 4 5 Stage of Adoption in 2004 Note. Community Key Informant Survey (CKI); N = 534; γ101 = -1.311, SE = .355, df = 20, p < .002 30 Collaboration on Prevention • Prevention Collaboration Assessed by 9 items measuring prevention-specific collaborative activities. There is a network of people concerned about prevention issues who stay in touch with each other. Organizations in [community] share money or personnel when addressing prevention issues. Organizations in [community] participate in joint planning and decision making about prevention issues. 1=strongly agree, 2=somewhat agree, 3=somewhat disagree, 4=strongly disagree 31 Note. Community Key Informant Survey (CKI). Change in Prevention Collaboration by Intervention Status 0.30 0.20 Factor Score Control Communities CTC Communities 0.10 0.00 -0.10 -0.20 -0.30 Post-Int (2004) Pre-Int (2001) Year Note. Community Key Informant Survey (CKI); N = 530; γ001 = -0.123, SE = .118, df = 20, p > .05; γ101 = 0.237, SE = .094, df = 20, p < .022. 32 Communities That Care Theory of Change CTC Training and Technical Assistance Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Decreased Risk and Enhanced Protection Positive Youth Outcomes 33 Program Selection CTC Community Board members selected prevention programs from a menu* of programs that: ~ Showed significant effects on risk/protective factors, and drug use, delinquency, or violence ~ Involved at least one high-quality research study ~ Targeted children or families in grades 5-9 ~ Provided materials and training * Communities That Care Prevention Strategies Guide 34 Programs Selected in 2004-2007 PROGRAM 2004-05 2005-06 2006-07 All Stars Core 1 1 1 Life Skills Training Lion’s-Quest Skills for Adolescence Project Alert Olweus Bullying Prevention Program Program Development Evaluation Training 2 2 1 4* 3 1 2* 5* 3 1 2* 1 - Participate and Learn Skills (PALS) Big Brothers/Big Sisters Stay SMART Tutoring Valued Youth Tutoring Program 1 2 3 4 1 1 2 3 6 1 Strengthening Families 10-14 Guiding Good Choices Parents Who Care Family Matters 2 6 1 1 3 7* 1 1 1 2 1 6 1 3 8* 2 Parenting Wisely - 1 1 27 38 37 TOTAL *Program funded through local resources in one or two communities 35 Exposure in the Community Program Type 2004-05 2005-06 2006-07 School Curricula 1432 3886 5165 After-school* 546 612 589 Parent Training 517 665 476 Note: Total eligible population of 6th, 7th, and 8th-grade students in 2005-06 was 10,031. *Includes PALS, BBBS, Stay SMART, and Tutoring programs 36 Participant Attendance Percentage attending >60% of the total number of sessions Program Type 2004-05 2005-06 2006-07 School Curricula 96% 91% 95% After-school* 77% 81% 65% Parent Training 79% 78% 79% *Includes PALS, BBBS, Stay SMART, and Tutoring programs Fidelity Assessment Checklists • Obtained from developers (9) or created by research staff (7) • Provided similar information across all programs to measure 4 elements of fidelity • Over 6,000 checklists were completed by program implementers and coordinators ~ Minimal missing data (8.2% in 2004-05 and 2.1% in 2005-06) • Checklists were collected and reviewed by communities, then sent to SDRG Adherence Rates 2004-05 and 2005-06 school years Percentage of material taught or core components achieved 100 90 80 70 60 50 40 30 20 10 0 LST AS SFA Alert BPP PDE SS Tutor VY BBBS PALS SFP GGC PWC FM Adherence • Average adherence rate were high: in 2004-05 and 94% in 2005-06 91% • “Problematic” modifications were infrequent (1.2 reported per cycle) ~ Deletions reported twice as often as additions • Implementer reports of challenges: ~ Lack of time (14-20% of checklists) ~ Participant misbehavior (9-11%) ~ Trouble engaging participants (4-9%) Delivery of Lessons 2004-05 and 2005-06 school years (number, length, and frequency of required sessions) Percentage of delivery requirements met 100 90 80 70 60 50 40 30 20 10 0 LST AS SFA Alert BPP SMART Tutor VY BBBS PALS SFP GGC PWC FM Delivery of Lessons • Delivery scores were high: 91% and 92% in 2004-05 and 2005-06 • “Problematic” deviations in delivery requirements were infrequent • Deviations most likely to occur in school-based programs ~ e.g., 30 vs. 45-minute sessions ~ e.g., deleting 1 of 12 required sessions Program Observations • Observed 10-15% of sessions in 10 of 16 programs • Completed fidelity checklists to verify adherence information ~ Rate of agreement between observers and implementers was 93% (range: 77%-100%) • Observers also rated the quality of delivery and participant responsiveness Quality of Delivery • Observers rated the quality of delivery on 10 items (alpha = .83-.88) using a 5-point scale (higher scores indicate better quality) • Example Items: ~ In general, how clear were the program implementer’s explanations of activities? ~ To what extent did the implementer keep on time during the session and activities? ~ Rate the implementer on the following qualities: • Level of enthusiasm • Rapport and communication with participants • Effectively addressed questions/concerns Quality of Delivery 2004-05 and 2005-06 school years Average score on 10 items reported by program observers 5 4.5 4 3.5 3 2.5 2 1.5 1 LST AS SFA BPP SMART PALS SFP GGC PWC Quality of Delivery Diversity of Teaching Techniques • Observers rated the percentage of the session in which each teaching technique was used Lecture Practice Discussion Video 2004-05 29% 34% 31% 6% 2005-06 30% 28% 36% 4% Participant Responsiveness • Observers rated participant responsiveness on two items, using a 1-5 scale (higher scores indicate better responsiveness): ~ To what extent did the participants appear to understand the material? ~ How actively did group members participate in discussions and activities? • Across all programs, rates were high: 4.38 and 4.52 in 2004-05 and 2005-06 Communities That Care Theory of Change CTC Training and Technical Assistance Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Decreased Risk and Enhanced Protection Positive Youth Outcomes Prioritized Risk Factors in CTC Communities • • • • • • • • • Family management problems Parental attitudes favorable to problem behavior Family conflict Low commitment to school Favorable attitudes toward problem behavior Friends who engage in problem behavior Academic failure Rebelliousness Laws and norms favorable toward drug and alcohol use Average Level of Risk Pre-post Change in Targeted Risk Factors 0.25 0.20 0.15 0.10 0.05 0.00 -0.05 -0.10 -0.15 -0.20 Grade 5 Control Communities Grade 7 CTC Communities Note. Values are model-fitted levels of standardized average risk for students in the Youth Developmental Study panel sample, N = 4404. For Grade 7: γ010 = .111, SE = .043, df = 11, p < .026. 50 Communities That Care Theory of Change CTC Training and Technical Assistance Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Decreased Risk and Enhanced Protection Positive Youth Outcomes 51 Youth Outcomes • Initiation of Delinquent Behaviors Nine items measuring delinquent behaviors. • Initiation of Substance Use Alcohol, marijuana, cigarettes, smokeless tobacco, inhalants, non-prescribed use of prescription drugs, other illicit drugs. 52 Delinquency Items How many times in the past year (12 months) have you... 5th Grade 6th Grade 7th Grade stolen something worth more than $5? √ √ √ purposely damaged or destroyed property that did not belong to you (not counting family property)? √ √ √ taken something from a store without paying for it? √ √ √ attacked someone with the idea of seriously hurting them? √ √ √ been arrested? × √ √ beat up someone so badly that they probably needed to see a doctor or a nurse? × √ √ sold illegal drugs? × × √ stolen or tried to steal a motor vehicle such as a car or motorcycle? × × √ taken a handgun to school? × × √ 53 Delinquency Initiation Between Grades 5 and 7 Control Communities CTC Communities Initiation Rate 0.50 0.36 0.40 0.30 0.21 0.20 0.10 0.00 0.31 0.17 0.00 Grade 5 Grade 6 Grade 7 Note. Youth Developmental Study panel sample. Excludes students who initiated delinquent behavior by Grade 5. ω0001 = .237, SE = .103, df = 11, p < .05. 54 Substance Use Initiation Between Grades 5 and 7 Initiation Rate Control Communities CTC Communities 0.50 0.39 0.40 0.24 0.30 0.20 0.10 0.36 0.22 0.00 0.00 Grade 5 Grade 6 Note. Youth Developmental Study panel sample. Excludes students who initiated substance use by Grade 5. ω0001 = .142, SE = .122, df = 11, p > .05. Grade 7 55 Summary • Adoption of science-based prevention and collaboration are higher in CTC communities than in control communities. • CTC communities and schools are adopting tested & effective programs. • The new programs are being implemented with fidelity. • Targeted risk factors are increasing less rapidly in CTC than in control communities. • The rate of initiation of delinquent behaviors is significantly lower in CTC than in control communities. 56 The Communities That Care Prevention Operating System is available at: http://preventionplatform.samhsa.gov/ 57 Published Papers Arthur, M.W., Briney, J.S., Hawkins, J.D., Abbott, R.D., Brooke-Weiss, B. &. Catalano, R.F. (2007). Measuring community risk and protection using the Communities That Care Youth Survey. Evaluation and Program Planning, 30, 197-211. Brown, E.C., Hawkins, J.D., Arthur, M.W., Briney, J.S., & Abbott, R.D. (2007). Effects of Communities That Care on prevention services systems: Findings from the Community Youth Development Study at 1.5 years. Prevention Science. 8, 180-191. Fagan, A. A., Hawkins, J. D., & Catalano, R. F. (in press). Using community epidemiologic data to improve social settings: The Communities That Care prevention system. In M. Shinn & H. Yoshikawa (Eds.), Toward positive youth development: Transforming schools and community programs. New York: Oxford University Press. Fagan, A.A., Van Horn, M.L., Hawkins, J.D., & Arthur, M.W. (2007). Using community and family risk and protective factors for community-based prevention planning. Journal of Community Psychology, 35(4), 535555. Hawkins, J.D. (2006). Science, social work, prevention: Finding the intersections. Social Work Research, 30(3), 137-152. Murray, D.M., Van Horn, M.L., Hawkins, J.D., & Arthur, M.W. (2006). Analysis strategies for a community trial to reduce adolescent ATOD use: A comparison of random coefficient and ANOVA/ANCOVA models. Contemporary Clinical Trials. 27, 188-206. 58 Papers In Press Brooke-Weiss, B., Haggerty, K. P., Fagan, A. A., Hawkins, J. D., & Cady, R. (in press). Creating community change to improve youth development: The Communities That Care (CTC) system. The Prevention Researcher. Brown, E.C., Hawkins, J.D., Arthur, M.W., Van Horn, M.L., & Abbott, R.D. (in press). Multilevel analysis of a measure of prevention collaboration. American Journal of Community Psychology. Fagan, A.A., Hanson, K., Hawkins, J.D., & Arthur, M.W. (in press). Bridging science to practice: Achieving prevention program implementation fidelity in the Community Youth Development Study. American Journal of Community Psychology. Fagan, A.A., Hawkins, J.D. and Catalano, R.F. (in press). Using community epidemiologic data to improve social settings: The Communities That Care Prevention System. In M. Shinn, and H. Yoshikawa, (eds). Improving Social Settings to Facilitate Positive Development among Adolescents. New York: W.T. Grant Foundation. Fagan, A.A., Hanson, K., Hawkins, J.D., & Arthur, M.W. (in press). Implementing effective community-based prevention programs in the Community Youth Development Study. Youth Violence and Juvenile Justice. Hawkins, J.D., Brown, E.C., Oesterle, S., Arthur, M.W., Abbott, R.D., & Catalano, R.F. (in press). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health. Quinby, R.K., Fagan, A.A., Hanson, K., Brooke-Weiss, B., Arthur, M.W., & Hawkins, J.D. (in press). Installing the Communities That Care prevention system: Implementation progress and fidelity in a randomized controlled trial. Journal of Community Psychology. Van Horn, M.L., Fagan, A.A., Jaki, T., Brown, E.C., Hawkins, J.D., Arthur, M.W., Abbott, R.D., & Catalano, R.F. (in press). The use of mixture models to evaluate differential intervention effects in group randomized trials. Multivariate 59 Behavioral Research. Building Community Collaboration to Promote Healthy Youth Development: J. David Hawkins Ph.D. Social Development Research Group School of Social Work University of Washington www.sdrg.org Melissa Institute: Reducing Violence May 1, 2008 60 Risk Factors Addressed in CTC Communities Community Targeted Risk Factors A Parental attitudes favorable to problem behavior Low commitment to school Favorable attitudes toward problem behavior C Family management problems Low commitment to school Friends who engage in problem behavior Favorable attitudes towards problem behavior G Poor family management Low commitment to school H Poor family management Academic failure Low commitment to school I Low commitment to school Friends who engage in problem behavior J Academic failure Low commitment to school Friends who engage in problem behavior Favorable attitudes toward problem behavior Rebelliousness 61 Risk Factors Addressed in CTC Communities Community Targeted Risk Factors N Academic failure Friends who engage in problem behavior O Laws and norms favorable toward drug and alcohol use Low commitment to school Friends who engage in problem behavior Rebelliousness Q Family conflict Friends who engage in problem behavior Favorable attitudes towards problem behavior Rebelliousness T Family conflict Low commitment to school Friends who engage in problem behavior W Poor family management Academic failure Friends who engage in problem behavior Favorable attitudes towards problem behavior X Academic failure Low commitment to school Friends who engage in problem behavior 62