Girl’s Study Group Project: Implications and Next Steps Coordinating Council on Juvenile Justice and Delinquency Prevention June 6, 2008 by by Delbert S. Elliott, Ph.D. Delbert S. Elliott, Ph.D. Director, Center for the Study and Prevention of Violence, University of Colorado

Prevention Research-1985* Romig, Justice for Our Children, 1978 Casework: No evidence of effectiveness Casework: No evidence of effectiveness Behavior Modification: Limited success, but should not be used for juvenile offenders Behavior Modification: Limited success, but should not be used for juvenile offenders Teaching Academic Skills: Not effective Teaching Academic Skills: Not effective Work & vocational Training: Not effective Work & vocational Training: Not effective Group Counseling: Not effective Group Counseling: Not effective Individual Psychotherapy: Not effective Individual Psychotherapy: Not effective Therapeutic Camping, Diversion, Probation: Not effective Therapeutic Camping, Diversion, Probation: Not effective See also: Martinson, 1974; Lipton et al., 1975; Sechrest et al., 1979 See also: Martinson, 1974; Lipton et al., 1975; Sechrest et al., 1979 Wright and Dixon, Wright and Dixon, 1977.

Prevention Research Better theory development and evaluation methodology & practice Better theory development and evaluation methodology & practice A growing number of programs demonstrated to be effective A growing number of programs demonstrated to be effective Increasing public & government support for evidence-based programs Increasing public & government support for evidence-based programs Confusion over scientific standard for evidence- based certification Confusion over scientific standard for evidence- based certification Limited dissemination of EB programs Limited dissemination of EB programs Relatively little attention to fidelity Relatively little attention to fidelity Promise for EB program effects on reducing crime rates when taken to scale Promise for EB program effects on reducing crime rates when taken to scale

Prevention Research- Agenda for Next 20 Years Establish consensus on scientific standard for certifying effective programs Establish consensus on scientific standard for certifying effective programs Upgrade program evaluation design, methodology and reporting Upgrade program evaluation design, methodology and reporting The new research frontier: dissemination and implementation The new research frontier: dissemination and implementation Address the barriers to dissemination & implementation of evidence-based programs Address the barriers to dissemination & implementation of evidence-based programs Finding stable sustained funding streams Finding stable sustained funding streams

Confusion over Scientific Standard Defining “Evidence-Based”

Federal Working Group Standard for Certifying Programs as Effective* Experimental Design/RCT Experimental Design/RCT Effect sustained for at least 1 year post- intervention Effect sustained for at least 1 year post- intervention At least 1 independent replication with RCT At least 1 independent replication with RCT RCT’s adequately address threats to internal validity RCT’s adequately address threats to internal validity No known health-compromising side effects No known health-compromising side effects *Adapted from Hierarchical Classification Framework for Program Effectiveness, Working Group for the Federal Collaboration on What Works, 2004.

Hierarchical Program Classification* I. Model: Meets all standards I. Model: Meets all standards II. Effective: RCT replication(s) not indep. II. Effective: RCT replication(s) not indep. III. Promising: Q-E or RCT, no replication III. Promising: Q-E or RCT, no replication IV. Inconclusive: Contradictory findings or non-sustainable effects IV. Inconclusive: Contradictory findings or non-sustainable effects V. Ineffective: Meets all standards but with no statistically significant effects V. Ineffective: Meets all standards but with no statistically significant effects VI. Harmful: Meets all standards but with negative main effects or serious side effects VI. Harmful: Meets all standards but with negative main effects or serious side effects VII Insufficient Evidence: All others VII Insufficient Evidence: All others *Adapted from Hierarchical Classification Framework for Program Effectiveness, Working Group for the Federal Collaboration on What Works, 2004.

Defining “Evidence- Based” Programs classified as Model, Effective, or Promising on Federal Hierarchy Programs classified as Model, Effective, or Promising on Federal Hierarchy Consistently positive effects from Meta Analyses Consistently positive effects from Meta Analyses Only Model programs should ever be taken to scale Only Model programs should ever be taken to scale

Federal Working Group Classification of Top Programs on EB Lists Ctr. For MH Services: Effective (14/34) Ctr. For MH Services: Effective (14/34) –Most have not yet been rated on FWG standard NREPP: Model & Effective (18/21) NREPP: Model & Effective (18/21) –Mod-4%; Effec-16%; Prom-16%; Incon/Insuff- 64% NIDA: Effective (20/21) NIDA: Effective (20/21) –Mod - 10%; Effec-25%; Prom- 25%; Incon/Insuff- 40% Blueprints: Model (11/11) Blueprints: Model (11/11) –Mod- 27%; Effec- 64%; Prom- 9%; Incon/Insuff- 0%

Federal Working Group Classification for Top Programs on Other Lists OJJDP-Title V: Exemplary (33/40) OJJDP-Title V: Exemplary (33/40) –Mod- 9%; Effec- 30%; Prom- 15%; Ineff/Incon- 45% OSDFS: Exemplary (9/9) OSDFS: Exemplary (9/9) –Mod- 11%; Effec- 23%; Prom- 33%; Ineff/Incon- 33% HAY: Level 1 (12/12) HAY: Level 1 (12/12) –Mod-25%; Effec- 30%; Prom- 0%; Ineff/Incon- 42%

Violence, Drug and Delinquency Prevention Programs: Overview* Most Programs Have No Credible Evaluation Most Programs Have No Credible Evaluation Those With Credible Evaluations: Those With Credible Evaluations: –Most Don’t Work –35 to 40 have been certified as EB –A Few Appear to be Harmful Most Model Programs Don’t Have Capacity to Go to Scale Most Model Programs Don’t Have Capacity to Go to Scale *Center for the Study and Prevention of Violence *Center for the Study and Prevention of Violence

Girl’s Study Group Findings

Serious Limitations in Evaluation Evidence Few Universal Programs have considered sex/gender effects Few Universal Programs have considered sex/gender effects Few Girls-Only Programs have been evaluated Few Girls-Only Programs have been evaluated Few Girls-Only Programs that are evaluated can be certified as EB Few Girls-Only Programs that are evaluated can be certified as EB

Implications Difficult to Identify Common Elements/Components in EB Programs for Girls Difficult to Identify Common Elements/Components in EB Programs for Girls Gender Similarities Hypothesis Rules Gender Similarities Hypothesis Rules Differences in Exposure Levels to Specific Risk/Protective Factors and Unique Risk Factors Provide Focus for Girls-Only Programs Differences in Exposure Levels to Specific Risk/Protective Factors and Unique Risk Factors Provide Focus for Girls-Only Programs

Implications (Cont’d) Currently, Practical Choice is Between Universal EB Programs and Non-EB Girls-Only Programs Currently, Practical Choice is Between Universal EB Programs and Non-EB Girls-Only Programs Problems with Adaptation of Universal EB Programs Strategy Problems with Adaptation of Universal EB Programs Strategy Consider Costs Associated with Inflated Claims of Gender Differences Consider Costs Associated with Inflated Claims of Gender Differences

Next Steps Implement Rigorous Evaluations of a Few Selected Girls-Only Programs Implement Rigorous Evaluations of a Few Selected Girls-Only Programs Complete Cost-Benefit Analyses of These Selected Programs Complete Cost-Benefit Analyses of These Selected Programs Develop Dissemination Capacity for EB Programs Develop Dissemination Capacity for EB Programs Re-assess Both the Theoretical Causal Rationales and the Change Strategies for Girls programs Re-assess Both the Theoretical Causal Rationales and the Change Strategies for Girls programs

THANK YOU Center for the Study and Prevention of Violence Center for the Study and Prevention of Violence

Referenced Websites NREPP: NREPP: Blueprints: Blueprints: OSDFS: l.html OSDFS: l.html l.html l.html NIDA: NIDA: OJJDP Title V: OJJDP Title V: CMHS: html CMHS: html Surgeon General: ault/html Surgeon General: ault/html