Governor’s Prevention Initiative for Youth: Evaluation Jane A. Ungemack, Dr.P.H. Evaluator University of Connecticut Health Center

Evaluation Systematic efforts to collect and use information: –Document program implementation –Describe target populations/participants –Inform and improve program performance –Access program effectiveness –Increase accountability –Increase understanding

Governor’s Prevention Initiative for Youth Long-term Goal: Reduce substance use among adolescents Intermediate Goal: Reduce risk factors and increase protective factors for substance use in the individual, peer, family, school and community domains Target Group: year old youth

Evaluation Framework Overview

Evaluation Approach Process evaluation: documents program implementation and activities Outcome evaluation: assesses program effects or impacts

Capacity-Building for Evaluation Science-based approach Evaluation and assessment as integral parts of the program design –Community-level –Program-level

Evaluation Team Relationship with Grantees Training and technical assistance Instruments and administration protocols Consultation and collaboration Statewide coordination

Grantee Responsibilities Develop a program plan based on the logic model Specify measurable program objectives Cooperate and collaborate with UConn Evaluation Team Coordinate community survey Collect and submit process data Collect and submit outcome data Commit time and effort to evaluation activities

Assessing Community-Level Outcomes: CSAP Requirements School survey Use of core substance use, risk and protective factor measures

Community-Level Assessment: School Survey Mandated 7th-10th grade students Representative sample (minimum n=500; 125/grade level) Year 1 and Year 3 First administration: February-April, 2000

School Survey Self-administered during a classroom period Anonymous and confidential Parental consents Sampling, instrument and administration protocols provided by UConn Evaluation Team

School Survey Measures –Demographic characteristics –Lifetime and current use of ATOD –Risk and protective factors –Limited community-specific items

School Survey Grantee responsibilities: –Planning/coordination with UConn Evaluation Team –Planning/coordination with school personnel –Instrument duplication –Data cleaning –Data entry –Analysis

Assessing Program-Level Outcomes: CSAP Requirements Select a minimum of three programs (for each of three domains) Measure program outcomes using core measures Include a sufficient sample size for analysis Collect pre- and post-test data

Program-Level Evaluation Process evaluation: Document program implementation and activities Outcome evaluation: Assess program outcomes

Program-Level Evaluation: Process Evaluation Each program will be responsible for reporting: –Prevention strategies –Types of activities –Dosage –Number served –Participant characteristics (age, gender, race/ethnicity, etc.).

Process Evaluation Minimum Data Set (MDS) Instruments, protocols, and training provided by the UConn Evaluation Team

Program-Level Evaluation: Outcome Evaluation Based on the logic model, identify measurable objectives that you will address in your program Program objectives should be selected from one or more of the risk/protective factors included in the RFP list of Connecticut Intermediate Outcomes

Outcome Evaluation UConn Evaluation Team staff will work with each grantee to finalize program- specific objectives and measures All grantees will be asked to participate in a pre/post-test assessment of age-eligible participants as appropriate

Outcome Evaluation Pre- and Post-Test Assessments Youth participation will be voluntary Confidential Informed consents Standardized instrument plus optional program-specific items Minimum sample size = 50

Considerations for Estimating Evaluation Costs Personnel (.25 FTE minimum recommended) Computer equipment Photocopying Office supplies Data collection and cleaning Data entry

Evaluation Themes for the Governor’s Prevention Initiative Evaluation at all levels Science-based Capacity-building Collaboration Coordination Sustainability