1. TGYS Orientation Day Get ready for evaluation!

2. ROADMAP TGYS Goals Reasons for Evaluation Evaluation Plans Design Measurement Data Collection Timelines Annual Results

3. TGYS GOALS REDUCE YOUTH CRIME AND VIOLENCE PREVENT CHILD ABUSE AND NEGLECT Improve school performance Increase adult bonding Increase self-esteem Increase life skills Decrease bullying Decrease ATOD use Decrease delinquency Decrease recidivism Improve quality of early care and educational programs Achieve developmental milestones Increase positive parenting skills TGYS Programs and Activities

4. REASONS FOR EVALUATION I. Your Participants II. Reporting III. Funding

5. REASON I What’s working well?What isn’t? Program refinement

6. REASON I GPA School Engagement

7. REASON II Package and streamline reporting MeanN Std. Deviation Desired Direction of Change? Sig. Effect Size Perceived relevance of school work - PRE3.42260.98 Yes0.260.21 Perceived relevance of school work - POST3.63260.86 Enjoyment of school - PRE3.26250.47 Yes0.200.29 Enjoyment of school - POST3.40250.53 Effort in school - PRE3.53250.61 Yes0.170.24 Effort in school - POST3.68250.63 Educational aspirations - PRE2.92251.34 Yes0.04*0.42 Educational aspirations - POST3.48251.22 School Bonding Total - PRE 3.36250.53 Yes0.04*0.39 School Bonding Total - POST3.57250.58

8. REASON III Competitive edge for funding

9. I NDIVIDUAL EVALUATION PLANS Step 1: Choose an evaluation design Step 2: Decide on the best measurement tool Step 3: Design a data collection and management protocol

10. EVALUATION DESIGN Pre- Test Your Program Post- Test

11. EVALUATION DESIGN Pre-Test Your Program Post-Test Pre-TestPost-Test Group One: Program Participants Group Two: Waiting-list Participants

12. MEASUREMENT Anticipated Outcomes Validated, Reliable, & Practical Tools

13. MEASUREMENT Outcomes: Corresponding Scale Life Skills: Resilience Adult Bonding: Perceived Social Support School Performance: Direct School Records School Performance: School Engagement Self-Efficacy: Self Efficacy Bullying: Bullying, Fighting, & Victimization Alcohol, Tobacco, and Other Drug Use: Use & Attitudes Delinquency: Attitudes toward Delinquency Recidivism: Colorado Criminal Contacts/Re-offenses Quality of Early Care and Education: Teaching Strategies GOLD Developmental Milestones: Social Competence Parenting Skills: Parenting Practices Interview

14. DATA COLLECTION It’s “FUN”damental! Electronic Surveys Paper Surveys Data Management

15. DATA COLLECTION ELECTRONIC SURVEYS Obtain a Link CompleteSubmit Example survey link: Alternatives 4 Youth_COSchBond. Example survey link: Alternatives 4 Youth_COSchBond.

16. DATA COLLECTION ELECTRONIC SURVEYS

17. DATA COLLECTION PAPER SURVEYS Step 1: Ordering Step 2: Preparation Step 3: Submission

18. Double-check your order Note the return address Note the ID format Double-check your order Note the return address Note the ID format DATA COLLECTION PAPER SURVEYS

19. Step 1: Ordering Step 2: Preparation Step 3: Submission Survey condition Administrative Section Instructions Confidentiality Survey condition Administrative Section Instructions Confidentiality DATA COLLECTION PAPER SURVEYS

20. Useful Data Condition Using originals Stapling or hole punching Using pencils DATA COLLECTION PAPER SURVEYS

21. Fill in pre, post or follow-up test Fill out Org ID and Local ID from top to bottom Date DATA COLLECTION ALL SURVEYS Useful Data Condition Admin. Section

22. DATA COLLECTION ALL SURVEYS Useful Data Condition Admin. Section Instructions Read thoroughly Pick best answer Answering questions Birthdate

23. DATA COLLECTION ALL SURVEYS Useful Data Condition Admin. Section InstructionsConfidentiality Testing space Survey handling

24. Step 1: Ordering Step 2: Preparation Step 3: Submission Julie Maertens Colorado State University Campus Delivery 1876 Fort Collins, CO 80523 Julie Maertens Colorado State University Campus Delivery 1876 Fort Collins, CO 80523 DATA COLLECTION PAPER SURVEYS

25. Assign unique IDsKeep track of each IDKeep your log confidential D ATA MANAGEMENT

26. Pre-testPost-test 10011001 10031005 20022002 20032003 20142112 Sample Size = 3 D ATA MANAGEMENT

28. DeadlineTaskPerson in Charge August 16Order pre-test surveys (expect 2-week turn-a- round) Mark V. September 3Organize surveys: count, fill out admin section Susan C. September 22Administer surveys to youth Laura F. September 29Check surveys: bubbles filled-in, admin complete Susan C. December 1Mail pre-test surveys to CSU (2 weeks prior to Dec 15 deadline) Mark V. D ATA MANAGEMENT

29. Pretest Deadline December 15, 2011 Posttest Deadline June 30, 2012

30. I NDIVIDUAL EVALUATION PLANS Evaluation Design Measurement Tool Data Collection Protocol & Timeline

31. RESULTS

33. MeanN Std. Deviation Desired Direction of Change? Sig. Effect Size Perceived relevance of school work - PRE3.42260.98 Yes0.260.21 Perceived relevance of school work - POST3.63260.86 Enjoyment of school - PRE3.26250.47 Yes0.200.29 Enjoyment of school - POST3.40250.53 Effort in school - PRE3.53250.61 Yes0.170.24 Effort in school - POST3.68250.63 Educational aspirations - PRE2.92251.34 Yes0.04*0.42 Educational aspirations - POST3.48251.22 School Bonding Total - PRE 3.36250.53 Yes0.04*0.39 School Bonding Total - POST3.57250.58

34. THANK YOU ! Also: jmtgys@lamar.colostate.edu CSU Evaluation Office (970)491-3830 main (877)724-4111 fax Questions?