TGYS Orientation Day Get ready for evaluation!

ROADMAP TGYS Goals Reasons for Evaluation Evaluation Plans Design Measurement Data Collection Timelines Annual Results

TGYS GOALS REDUCE YOUTH CRIME AND VIOLENCE PREVENT CHILD ABUSE AND NEGLECT Improve school performance Increase adult bonding Increase self-esteem Increase life skills Decrease bullying Decrease ATOD use Decrease delinquency Decrease recidivism Improve quality of early care and educational programs Achieve developmental milestones Increase positive parenting skills TGYS Programs and Activities

REASONS FOR EVALUATION I. Your Participants II. Reporting III. Funding

REASON I What’s working well?What isn’t? Program refinement

REASON I GPA School Engagement

REASON II Package and streamline reporting MeanN Std. Deviation Desired Direction of Change? Sig. Effect Size Perceived relevance of school work - PRE Yes Perceived relevance of school work - POST Enjoyment of school - PRE Yes Enjoyment of school - POST Effort in school - PRE Yes Effort in school - POST Educational aspirations - PRE Yes0.04*0.42 Educational aspirations - POST School Bonding Total - PRE Yes0.04*0.39 School Bonding Total - POST

REASON III Competitive edge for funding

I NDIVIDUAL EVALUATION PLANS Step 1: Choose an evaluation design Step 2: Decide on the best measurement tool Step 3: Design a data collection and management protocol

EVALUATION DESIGN Pre- Test Your Program Post- Test

EVALUATION DESIGN Pre-Test Your Program Post-Test Pre-TestPost-Test Group One: Program Participants Group Two: Waiting-list Participants

MEASUREMENT Anticipated Outcomes Validated, Reliable, & Practical Tools

MEASUREMENT Outcomes: Corresponding Scale Life Skills: Resilience Adult Bonding: Perceived Social Support School Performance: Direct School Records School Performance: School Engagement Self-Efficacy: Self Efficacy Bullying: Bullying, Fighting, & Victimization Alcohol, Tobacco, and Other Drug Use: Use & Attitudes Delinquency: Attitudes toward Delinquency Recidivism: Colorado Criminal Contacts/Re-offenses Quality of Early Care and Education: Teaching Strategies GOLD Developmental Milestones: Social Competence Parenting Skills: Parenting Practices Interview

DATA COLLECTION It’s “FUN”damental! Electronic Surveys Paper Surveys Data Management

DATA COLLECTION ELECTRONIC SURVEYS Obtain a Link CompleteSubmit Example survey link: Alternatives 4 Youth_COSchBond. Example survey link: Alternatives 4 Youth_COSchBond.

DATA COLLECTION ELECTRONIC SURVEYS

DATA COLLECTION PAPER SURVEYS Step 1: Ordering Step 2: Preparation Step 3: Submission

Double-check your order Note the return address Note the ID format Double-check your order Note the return address Note the ID format DATA COLLECTION PAPER SURVEYS

Step 1: Ordering Step 2: Preparation Step 3: Submission Survey condition Administrative Section Instructions Confidentiality Survey condition Administrative Section Instructions Confidentiality DATA COLLECTION PAPER SURVEYS

Useful Data Condition Using originals Stapling or hole punching Using pencils DATA COLLECTION PAPER SURVEYS

Fill in pre, post or follow-up test Fill out Org ID and Local ID from top to bottom Date DATA COLLECTION ALL SURVEYS Useful Data Condition Admin. Section

DATA COLLECTION ALL SURVEYS Useful Data Condition Admin. Section Instructions Read thoroughly Pick best answer Answering questions Birthdate

DATA COLLECTION ALL SURVEYS Useful Data Condition Admin. Section InstructionsConfidentiality Testing space Survey handling

Step 1: Ordering Step 2: Preparation Step 3: Submission Julie Maertens Colorado State University Campus Delivery 1876 Fort Collins, CO Julie Maertens Colorado State University Campus Delivery 1876 Fort Collins, CO DATA COLLECTION PAPER SURVEYS

Assign unique IDsKeep track of each IDKeep your log confidential D ATA MANAGEMENT

Pre-testPost-test Sample Size = 3 D ATA MANAGEMENT

DeadlineTaskPerson in Charge August 16Order pre-test surveys (expect 2-week turn-a- round) Mark V. September 3Organize surveys: count, fill out admin section Susan C. September 22Administer surveys to youth Laura F. September 29Check surveys: bubbles filled-in, admin complete Susan C. December 1Mail pre-test surveys to CSU (2 weeks prior to Dec 15 deadline) Mark V. D ATA MANAGEMENT

Pretest Deadline December 15, 2011 Posttest Deadline June 30, 2012

I NDIVIDUAL EVALUATION PLANS Evaluation Design Measurement Tool Data Collection Protocol & Timeline

RESULTS

MeanN Std. Deviation Desired Direction of Change? Sig. Effect Size Perceived relevance of school work - PRE Yes Perceived relevance of school work - POST Enjoyment of school - PRE Yes Enjoyment of school - POST Effort in school - PRE Yes Effort in school - POST Educational aspirations - PRE Yes0.04*0.42 Educational aspirations - POST School Bonding Total - PRE Yes0.04*0.39 School Bonding Total - POST

THANK YOU ! Also: CSU Evaluation Office (970) main (877) fax Questions?