1. Governor’s Prevention Initiative for Youth: Evaluation Jane A. Ungemack, Dr.P.H. Evaluator University of Connecticut Health Center

2. Evaluation Systematic efforts to collect and use information: –Document program implementation –Describe target populations/participants –Inform and improve program performance –Access program effectiveness –Increase accountability –Increase understanding

3. Governor’s Prevention Initiative for Youth Long-term Goal: Reduce substance use among adolescents Intermediate Goal: Reduce risk factors and increase protective factors for substance use in the individual, peer, family, school and community domains Target Group: 12-17 year old youth

4. Evaluation Framework Overview

5. Evaluation Approach Process evaluation: documents program implementation and activities Outcome evaluation: assesses program effects or impacts

6. Capacity-Building for Evaluation Science-based approach Evaluation and assessment as integral parts of the program design –Community-level –Program-level

7. Evaluation Team Relationship with Grantees Training and technical assistance Instruments and administration protocols Consultation and collaboration Statewide coordination

8. Grantee Responsibilities Develop a program plan based on the logic model Specify measurable program objectives Cooperate and collaborate with UConn Evaluation Team Coordinate community survey Collect and submit process data Collect and submit outcome data Commit time and effort to evaluation activities

9. Assessing Community-Level Outcomes: CSAP Requirements School survey Use of core substance use, risk and protective factor measures

10. Community-Level Assessment: School Survey Mandated 7th-10th grade students Representative sample (minimum n=500; 125/grade level) Year 1 and Year 3 First administration: February-April, 2000

11. School Survey Self-administered during a classroom period Anonymous and confidential Parental consents Sampling, instrument and administration protocols provided by UConn Evaluation Team

12. School Survey Measures –Demographic characteristics –Lifetime and current use of ATOD –Risk and protective factors –Limited community-specific items

13. School Survey Grantee responsibilities: –Planning/coordination with UConn Evaluation Team –Planning/coordination with school personnel –Instrument duplication –Data cleaning –Data entry –Analysis

14. Assessing Program-Level Outcomes: CSAP Requirements Select a minimum of three programs (for each of three domains) Measure program outcomes using core measures Include a sufficient sample size for analysis Collect pre- and post-test data

15. Program-Level Evaluation Process evaluation: Document program implementation and activities Outcome evaluation: Assess program outcomes

16. Program-Level Evaluation: Process Evaluation Each program will be responsible for reporting: –Prevention strategies –Types of activities –Dosage –Number served –Participant characteristics (age, gender, race/ethnicity, etc.).

17. Process Evaluation Minimum Data Set (MDS) Instruments, protocols, and training provided by the UConn Evaluation Team

18. Program-Level Evaluation: Outcome Evaluation Based on the logic model, identify measurable objectives that you will address in your program Program objectives should be selected from one or more of the risk/protective factors included in the RFP list of Connecticut Intermediate Outcomes

19. Outcome Evaluation UConn Evaluation Team staff will work with each grantee to finalize program- specific objectives and measures All grantees will be asked to participate in a pre/post-test assessment of age-eligible participants as appropriate

20. Outcome Evaluation Pre- and Post-Test Assessments Youth participation will be voluntary Confidential Informed consents Standardized instrument plus optional program-specific items Minimum sample size = 50

21. Considerations for Estimating Evaluation Costs Personnel (.25 FTE minimum recommended) Computer equipment Photocopying Office supplies Data collection and cleaning Data entry

22. Evaluation Themes for the Governor’s Prevention Initiative Evaluation at all levels Science-based Capacity-building Collaboration Coordination Sustainability