1 Building Community Collaboration to Promote Healthy Youth Development: Social Development Research Group School of Social Work University of Washington J. David Hawkins Ph.D. Melissa Institute May 1, 2008

2 The Challenge for Community Prevention To address those risk factors most prevalent in a community with tested, effective policies and programs. To address protective factors most depressed in a community with tested, effective policies and programs that strengthen those protective factors.

3 The Communities That Care Prevention System Helps communities apply the advances of prevention science to guide youth development and prevention work. Measures community levels of protection and risk by surveying young people. Matches the community’s profile of risk and protection with tested, effective programs and policies.

4 The Communities That Care Prevention System Local control builds ownership to create sustainable change. Focuses on outcomes to insure success: Are fewer teens using drugs? Fewer smoking? Fewer committing violent acts?

5 The Communities That Care Operating System Creating Communities That Care Get Started Get Organized Develop a Profile Create a Plan Implement and Evaluate

6 The Communities That Care Operating System Creating Communities That Care Get Started Get Organized Develop a Profile Create a Plan Implement and Evaluate Community readiness assessment. Identification of key individuals, stakeholders, and organizations.

7 The Communities That Care Operating System Creating Communities That Care Get Started Get Organized Develop a Profile Create a Plan Implement and Evaluate Training key leaders and board in CTC Building the community coalition.

8 The Communities That Care Operating System Creating Communities That Care Get Started Get Organized Develop a Profile Create a Plan Implement and Evaluate Collect risk/protective factor and outcome data. Collect information on community resources Construct a community profile from the data.

Diffusion Consortium Project Social Development Research Group, University of Washington 9 Estimated National Value Peer-Individual

10 The Communities That Care Operating System Creating Communities That Care Get Started Get Organized Develop a Profile Create a Plan Implement and Evaluate Define outcomes. Prioritize factors to be targeted. Select tested, effective interventions. Create action plan. Develop evaluation plan.

Diffusion Consortium Project Social Development Research Group, University of Washington 11 Addressing Barriers with Effective Action 3-5Early Childhood Education prenatal-2Prenatal/Infancy Programs 6-14Family Therapy prenatal-14Parent Training Family Management Problems Developmental PeriodProgram Strategy Factor Addressed

Diffusion Consortium Project Social Development Research Group, University of Washington 12 Effective Training for Middle School Parents Guiding Good Choices (Spoth et al., 1998) Adolescent Transitions Program (Dishion and Andrews, 1995) Parenting Adolescents Wisely (Gordon et al., 1998) Creating Lasting Connections (Johnson et al., 1996) Strengthening Families 10 to 14 Program (Spoth, 1998) Focus on Families (Catalano et al., 1999; 1997)

13 The Communities That Care Operating System Creating Communities That Care Get Started Get Organized Develop a Profile Create a Plan Implement and Evaluate Form task forces. Identify and train implementers. Sustain collaborative relationships. Evaluate processes and outcomes. Adjust programming.

14 What is required to install CTC? A coalition of community stakeholders. A coordinator for the CTC process. Manuals and curriculum materials. Training from certified trainers. Technical assistance when difficulties are encountered. A monitoring system to provide routine feedback on progress and outcomes.

15 CTC Trainings 1. Key Leader Orientation 2. Community Board Training 3. Community Assessment Training 4. Community Resource Assessment Training 5. Community Planning Training 6. Community Program Implementation Training

16 Assess risk, protection and resources Implement and evaluate tested prevention strategies Increase in priority protective factors Decrease in priority risk factors Increase in positive youth development Reduction in problem behaviors Vision for a healthy community Process Measurable Outcomes 6-9 mos. 1 year 2-5 years 5-10 years Communities that Care Process and Timeline

17 The Community Youth Development Study (CYDS) A 24 community randomized controlled trial to test the Communities That Care system started in 2003.

The Community Youth Development Study Team Funded by: National Institute on Drug Abuse Center for Substance Abuse Prevention National Cancer Institute National Institute on Child Health and Development National Institute on Mental Health Robert D. Abbott Michael W. Arthur Megan M. Baldwin John S. Briney Blair Brooke-Weiss Eric C. Brown Rick Cady Richard F. Catalano Abigail A. Fagan John Graham Kevin Haggerty Koren Hanson J. David Hawkins David M. Murray Sabrina Oesterle M. Lee Van Horn

19 CYDS State Collaborators Colorado Alcohol & Drug Abuse Division Illinois Division of Community Health & Prevention Kansas Alcohol and Drug Abuse Services Maine Office of Substance Abuse Oregon Office of Alcohol & Drug Abuse Programs Utah Division of Substance Use Washington Division of Alcohol & Substance Abuse

20 CYDS Primary Aim To test the efficacy of the Communities That Care system in reducing levels of risk increasing levels of protection reducing health and behavior problems among adolescents using a true experimental design.

21 STUDY DESIGN Randomized Controlled Trial Randomize 5-Year Baseline ‘00 ‘01 ‘02 CKI CRD Control Intervention CTCYS CKI CRD CKI CRD CKI CRD CKI CRD YDS CTC Board CTC Board CTC Board CTC Board CTC Board CTCYS: Cross-sectional student survey of 6th-, 8th-, 10th-, and 12th-grade students using the CTC Youth Survey CKI: Community Key Informant Interview CRD: Community Resource Documentation measuring effective prevention programs and policies in the community CTC Board: CTC Board Member Interview YDS: Longitudinal Youth Development Survey of students in the class of 2011 starting in 5 th grade in spring 2004 Planning Implement selected interventions CTCYS CKI CRD YDS

22 Demographics of 24 CYDS Communities MeanMinimumMaximum Total Population14,6161,57840,787 Percent Caucasian89.4%64.0%98.2% Percent Hispanic Origin Percent African-American Percent Eligible for Free/Reduced Lunch 9.6% 2.6% 36.5% 0.5% 0.0% 20.6% 64.7% 21.4% 65.9%

23 Panel- Youth Development Survey (YDS) Annual survey of panel recruited from the Class of 2011 (5 th grade in 2004) Active, written parental consent

24 Youth Development Survey Participants recruited in grades 5 and 6. Final consent rate = 76.4% Sixth Grade Eligible Population Percent Consented Percent Surveyed Total Surveyed Experimental %75.4%2391 Control %76.3%1999 Total %75.8%4390

YDS 7 th Grade Eligible Population Percent Surveyed Total Surveyed Experimental %2298 Control %1941 Total % % Overall Student Participation 10.3% (n=454) had moved out of project schools

YDS 8 th Grade Eligible Population Percent Surveyed Total Surveyed Experimental %2300 Control %1940 Total % % Overall Student Participation 11.9% (n=525) have moved out of project schools

27 Communities That Care Logic Model Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Positive Youth Outcomes Decreased Risk and Enhanced Protection CTC Training and Technical Assistance

28 Adoption of Science-Based Prevention Stage 0:No Awareness Stage 0:No Awareness Stage 1:Awareness of Prevention Science Terms and Concepts Stage 1:Awareness of Prevention Science Terms and Concepts Stage 2:Using Risk and Protection Focused Prevention Approach as a Planning Strategy. Stage 2:Using Risk and Protection Focused Prevention Approach as a Planning Strategy. Stage 3:Incorporation of Community Epidemiological Data on Risk and Protection in Prevention System. Stage 3:Incorporation of Community Epidemiological Data on Risk and Protection in Prevention System. Stage 4:Selection and Use of Tested and Effective Preventive Interventions to Address Prioritized Risk and Protective Factors. Stage 4:Selection and Use of Tested and Effective Preventive Interventions to Address Prioritized Risk and Protective Factors. Stage 5:Collection and Feedback of Process and Outcome Data and Adjustment of Preventive Interventions Based on Data. Stage 5:Collection and Feedback of Process and Outcome Data and Adjustment of Preventive Interventions Based on Data. Note. Community Key Informant Survey (CKI).

29 Stage of Adoption in 2001 Probability Baseline Stages of Adoption by Intervention Status Control CommunitiesCTC Communities Note. Community Key Informant Survey (CKI); N = 534; γ 001 =.037, SE =.395, df = 20, p >.05.

30 Stage of Adoption in 2004 Probability Post-Intervention Stages of Adoption by Intervention Status Control CommunitiesCTC Communities Note. Community Key Informant Survey (CKI); N = 534; γ 101 = , SE =.355, df = 20, p <.002.

31 Collaboration on Prevention Prevention Collaboration  Assessed by 9 items measuring prevention-specific collaborative activities. There is a network of people concerned about prevention issues who stay in touch with each other. Organizations in [community] share money or personnel when addressing prevention issues. Organizations in [community] participate in joint planning and decision making about prevention issues. 1=strongly agree, 2=somewhat agree, 3=somewhat disagree, 4=strongly disagree Note. Community Key Informant Survey (CKI).

32 Change in Prevention Collaboration by Intervention Status Year Pre-Int (2001) Post-Int (2004) Factor Score Control CommunitiesCTC Communities Note. Community Key Informant Survey (CKI); N = 530; γ 001 = , SE =.118, df = 20, p >.05; γ 101 = 0.237, SE =.094, df = 20, p <.022.

33 Communities That Care Theory of Change Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Positive Youth Outcomes Decreased Risk and Enhanced Protection CTC Training and Technical Assistance

34 Program Selection CTC Community Board members selected prevention programs from a menu* of programs that: ~Showed significant effects on risk/protective factors, and drug use, delinquency, or violence ~Involved at least one high-quality research study ~Targeted children or families in grades 5-9 ~Provided materials and training * Communities That Care Prevention Strategies Guide

35 Programs Selected in PROGRAM All Stars Core111 Life Skills Training2 4* 5* Lion’s-Quest Skills for Adolescence233 Project Alert-11 Olweus Bullying Prevention Program- 2* Program Development Evaluation Training11 - Participate and Learn Skills (PALS)111 Big Brothers/Big Sisters222 Stay SMART331 Tutoring466 Valued Youth Tutoring Program11 1 Strengthening Families Guiding Good Choices6 7* 8* Parents Who Care11- Family Matters112 Parenting Wisely-11 TOTAL *Program funded through local resources in one or two communities

36 Exposure in the Community Program Type School Curricula After-school * Parent Training * Includes PALS, BBBS, Stay SMART, and Tutoring programs Note: Total eligible population of 6 th, 7 th, and 8 th -grade students in was 10,031.

Participant Attendance Program Type School Curricula 96%91%95% After-school * 77%81%65% Parent Training 79%78%79% * Includes PALS, BBBS, Stay SMART, and Tutoring programs Percentage attending >60% of the total number of sessions

Fidelity Assessment Checklists Obtained from developers (9) or created by research staff (7) Provided similar information across all programs to measure 4 elements of fidelity Over 6,000 checklists were completed by program implementers and coordinators ~ Minimal missing data (8.2% in and 2.1% in ) Checklists were collected and reviewed by communities, then sent to SDRG

Adherence Rates and school years Percentage of material taught or core components achieved

Adherence Average adherence rate were high: 91% in and 94% in “Problematic” modifications were infrequent (1.2 reported per cycle) ~ Deletions reported twice as often as additions Implementer reports of challenges: ~ Lack of time (14-20% of checklists) ~ Participant misbehavior (9-11%) ~ Trouble engaging participants (4-9%)

Delivery of Lessons and school years (number, length, and frequency of required sessions) Percentage of delivery requirements met

Delivery of Lessons Delivery scores were high: 91% and 92% in and “Problematic” deviations in delivery requirements were infrequent Deviations most likely to occur in school-based programs ~e.g., 30 vs. 45-minute sessions ~e.g., deleting 1 of 12 required sessions

Program Observations Observed 10-15% of sessions in 10 of 16 programs Completed fidelity checklists to verify adherence information ~ Rate of agreement between observers and implementers was 93% (range: 77%-100%) Observers also rated the quality of delivery and participant responsiveness

Quality of Delivery Observers rated the quality of delivery on 10 items (alpha = ) using a 5-point scale (higher scores indicate better quality) Example Items: ~ In general, how clear were the program implementer’s explanations of activities? ~ To what extent did the implementer keep on time during the session and activities? ~ Rate the implementer on the following qualities: Level of enthusiasm Rapport and communication with participants Effectively addressed questions/concerns

Quality of Delivery and school years Average score on 10 items reported by program observers

Quality of Delivery Diversity of Teaching Techniques Observers rated the percentage of the session in which each teaching technique was used Lecture29%30% Practice34%28% Discussion31%36% Video6%4%

Participant Responsiveness Observers rated participant responsiveness on two items, using a 1-5 scale (higher scores indicate better responsiveness): ~ To what extent did the participants appear to understand the material? ~ How actively did group members participate in discussions and activities? Across all programs, rates were high: 4.38 and 4.52 in and

Communities That Care Theory of Change Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Positive Youth Outcomes Decreased Risk and Enhanced Protection CTC Training and Technical Assistance

Prioritized Risk Factors in CTC Communities Family management problems Parental attitudes favorable to problem behavior Family conflict Low commitment to school Favorable attitudes toward problem behavior Friends who engage in problem behavior Academic failure Rebelliousness Laws and norms favorable toward drug and alcohol use

50 Pre-post Change in Targeted Risk Factors Average Level of Risk Note. Values are model-fitted levels of standardized average risk for students in the Youth Developmental Study panel sample, N = For Grade 7: γ 010 =.111, SE =.043, df = 11, p <.026. Grade 5Grade 7

51 Communities That Care Theory of Change Adoption of Science-based Prevention Framework Collaboration Regarding Prevention Issues Appropriate Choice and Implementation of Tested, Effective Prevention Programs & Adoption of Social Development Strategy as Community’s Way of Bringing Up Children Positive Youth Outcomes Decreased Risk and Enhanced Protection CTC Training and Technical Assistance

52 Youth Outcomes Initiation of Delinquent Behaviors  Nine items measuring delinquent behaviors. Initiation of Substance Use  Alcohol, marijuana, cigarettes, smokeless tobacco, inhalants, non-prescribed use of prescription drugs, other illicit drugs.

53 Delinquency Items How many times in the past year (12 months) have you... 5th Grade6th Grade7th Grade stolen something worth more than $5?√√√ purposely damaged or destroyed property that did not belong to you (not counting family property)? √√√ taken something from a store without paying for it? √√√ attacked someone with the idea of seriously hurting them? √√√ been arrested?×√√ beat up someone so badly that they probably needed to see a doctor or a nurse? ×√√ sold illegal drugs?××√ stolen or tried to steal a motor vehicle such as a car or motorcycle? ××√ taken a handgun to school?××√

54 Delinquency Initiation Between Grades 5 and 7 Note. Youth Developmental Study panel sample. Excludes students who initiated delinquent behavior by Grade 5. ω 0001 =.237, SE =.103, df = 11, p <.05. Initiation Rate

55 Substance Use Initiation Between Grades 5 and 7 Note. Youth Developmental Study panel sample. Excludes students who initiated substance use by Grade 5. ω 0001 =.142, SE =.122, df = 11, p >.05. Initiation Rate

56 Summary Adoption of science-based prevention and collaboration are higher in CTC communities than in control communities. CTC communities and schools are adopting tested & effective programs. The new programs are being implemented with fidelity. Targeted risk factors are increasing less rapidly in CTC than in control communities. The rate of initiation of delinquent behaviors is significantly lower in CTC than in control communities.

57 The Communities That Care Prevention Operating System is available at:

58 Published Papers Arthur, M.W., Briney, J.S., Hawkins, J.D., Abbott, R.D., Brooke-Weiss, B. &. Catalano, R.F. (2007). Measuring community risk and protection using the Communities That Care Youth Survey. Evaluation and Program Planning, 30, Brown, E.C., Hawkins, J.D., Arthur, M.W., Briney, J.S., & Abbott, R.D. (2007). Effects of Communities That Care on prevention services systems: Findings from the Community Youth Development Study at 1.5 years. Prevention Science. 8, Fagan, A. A., Hawkins, J. D., & Catalano, R. F. (in press). Using community epidemiologic data to improve social settings: The Communities That Care prevention system. In M. Shinn & H. Yoshikawa (Eds.), Toward positive youth development: Transforming schools and community programs. New York: Oxford University Press. Fagan, A.A., Van Horn, M.L., Hawkins, J.D., & Arthur, M.W. (2007). Using community and family risk and protective factors for community-based prevention planning. Journal of Community Psychology, 35(4), Hawkins, J.D. (2006). Science, social work, prevention: Finding the intersections. Social Work Research, 30(3), Murray, D.M., Van Horn, M.L., Hawkins, J.D., & Arthur, M.W. (2006). Analysis strategies for a community trial to reduce adolescent ATOD use: A comparison of random coefficient and ANOVA/ANCOVA models. Contemporary Clinical Trials. 27,

59 Papers In Press Brooke-Weiss, B., Haggerty, K. P., Fagan, A. A., Hawkins, J. D., & Cady, R. (in press). Creating community change to improve youth development: The Communities That Care (CTC) system. The Prevention Researcher. Brown, E.C., Hawkins, J.D., Arthur, M.W., Van Horn, M.L., & Abbott, R.D. (in press). Multilevel analysis of a measure of prevention collaboration. American Journal of Community Psychology. Fagan, A.A., Hanson, K., Hawkins, J.D., & Arthur, M.W. (in press). Bridging science to practice: Achieving prevention program implementation fidelity in the Community Youth Development Study. American Journal of Community Psychology. Fagan, A.A., Hawkins, J.D. and Catalano, R.F. (in press). Using community epidemiologic data to improve social settings: The Communities That Care Prevention System. In M. Shinn, and H. Yoshikawa, (eds). Improving Social Settings to Facilitate Positive Development among Adolescents. New York: W.T. Grant Foundation. Fagan, A.A., Hanson, K., Hawkins, J.D., & Arthur, M.W. (in press). Implementing effective community-based prevention programs in the Community Youth Development Study. Youth Violence and Juvenile Justice. Hawkins, J.D., Brown, E.C., Oesterle, S., Arthur, M.W., Abbott, R.D., & Catalano, R.F. (in press). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health. Quinby, R.K., Fagan, A.A., Hanson, K., Brooke-Weiss, B., Arthur, M.W., & Hawkins, J.D. (in press). Installing the Communities That Care prevention system: Implementation progress and fidelity in a randomized controlled trial. Journal of Community Psychology. Van Horn, M.L., Fagan, A.A., Jaki, T., Brown, E.C., Hawkins, J.D., Arthur, M.W., Abbott, R.D., & Catalano, R.F. (in press). The use of mixture models to evaluate differential intervention effects in group randomized trials. Multivariate Behavioral Research.

60 Building Community Collaboration to Promote Healthy Youth Development: Social Development Research Group School of Social Work University of Washington J. David Hawkins Ph.D. Melissa Institute: Reducing Violence May 1, 2008

61 Risk Factors Addressed in CTC Communities CommunityTargeted Risk Factors AParental attitudes favorable to problem behavior Low commitment to school Favorable attitudes toward problem behavior CFamily management problems Low commitment to school Friends who engage in problem behavior Favorable attitudes towards problem behavior GPoor family management Low commitment to school HPoor family management Academic failure Low commitment to school I Friends who engage in problem behavior JAcademic failure Low commitment to school Friends who engage in problem behavior Favorable attitudes toward problem behavior Rebelliousness

62 CommunityTargeted Risk Factors NAcademic failure Friends who engage in problem behavior OLaws and norms favorable toward drug and alcohol use Low commitment to school Friends who engage in problem behavior Rebelliousness QFamily conflict Friends who engage in problem behavior Favorable attitudes towards problem behavior Rebelliousness T Family conflict Low commitment to school Friends who engage in problem behavior W Poor family management Academic failure Friends who engage in problem behavior Favorable attitudes towards problem behavior X Academic failure Low commitment to school Friends who engage in problem behavior Risk Factors Addressed in CTC Communities