1. SVP Data Collection Orientation Get ready for evaluation!

2. ROADMAP SVP Goals Reasons for Evaluation Evaluation Plans Measurement Data Collection Timelines Annual Results

3. SVP GOALS Prevent sexual violence in adolescence & young adulthood Prevent sexual violence in communities of color Risk & Protective Factors at 3 Levels Individual Relationship Community & Society SVP Programs and Activities Promote social norms & policies salient to SVP

4. REASONS FOR EVALUATION I. Your Participants II. Reporting III. Funding

5. REASON I Gender Stereotyping Conflict Resolution

6. REASON II Package and streamline reporting MeanN Std. Deviation Desired Direction of Change? Sig. Effect Size Conflict Resolution - PRE3.42260.98 Yes0.260.21 Conflict Resolution - POST3.63260.86 Resilience - PRE3.26250.47 Yes0.200.29 Resilience - POST3.40250.53 Healthy Sexuality - PRE3.53250.61 Yes0.170.24 Healthy Sexuality - POST3.68250.63 Violent Behavior - PRE2.92251.34 Yes0.04*0.42 Violent Behavior - POST3.48251.22 SVP Total - PRE 3.36250.53 Yes0.04*0.39 SVP Total - POST3.57250.58

7. REASON III Competitive edge for funding

8. I NDIVIDUAL EVALUATION PLANS Step 1: Decide on best measurement tool(s) Step 2: Data collection and management protocol Step 3: Testing timeline

9. M EASUREMENT Anticipated Outcomes Validated, Reliable, & Practical Tools

10. M EASUREMENT Reliability Does the tool produce consistent results Validity Does the tool measure what it’s meant to

11. M EASUREMENT Sexual Violence Prevention Survey Please read the following statements, and then choose the answer that best describes your feelings. For example, if you feel the statement is not like you at all, choose '1' and if you feel it is exactly like you choose '6', etc. 1. I resolve my conflicts with other people. Conflict Resolution 2. I avoid unnecessary conflicts with others. Conflict Resolution 3. Other people look up to the way I handle conflict. Conflict Resolution 4. If I choose to do something dangerous or unhealthy, I think carefully about the consequences. Healthy Risk-Taking 5. I usually know how much personal space people need to feel comfortable around me. Social Competence 6. People are usually comfortable about the way I behave around them. Social Competence 7. I feel that I can handle many things at a time. Resilience 8. My belief in myself gets me through hard times. Resilience 9. My life has meaning. Resilience

12. M EASUREMENT Sexual Violence Prevention Survey Based on your experience, mark whether you think it is really wrong, sort of wrong, sort of ok, or perfectly ok to: 10....grab, shove, punch or kick someone. Antisocial, delinquent, violent behavior 11....threaten to hurt someone else. Antisocial, delinquent, violent behavior 12....make unwanted physical sexual advances (grabbing, touching, kissing) toward someone else. Antisocial, delinquent, violent behavior 13....make unwanted sexual comments toward someone else, for example, repeatedly asking to get together with someone who has already said no, publicly "rating" someone else's attractiveness, making jokes about someone's sexuality (like "that girl is a slut" or "that guy is a fag")? Antisocial, delinquent, violent behavior 14.... push or hit someone who is trying to kiss or grab your boyfriend/girlfriend? Conflict Resolution Style

13. M EASUREMENT Sexual Violence Prevention Survey Now please read the next statements, and then choose how much you agree with each statement. 15. If someone says "no" to sex they may not really mean it. Sexual Consent 16. Magazines and music videos show disrespectful sexual images of women. Stereotypes 17. It doesn’t hurt anyone to tell dirty jokes about women or men. Violent Behaviors 18. A person is not really abusive as long as they don't physically harm anyone. Violent Behaviors 19. Most of my friends believe people my age should wait until they are older before they have sex. Healthy Sexuality 20. I believe condoms should always be used if people my age have sex, even if they know each other very well. Healthy Sexuality 21. I believe people my age should talk with their boyfriend/girlfriend about safety (avoiding STDs and pregnancy) before having sex. Healthy Sexuality

14. M EASUREMENT Sexual Violence Prevention Survey Please read the following statements, and then choose the answer that best describes you. 22. Telling your dating partner that he or she is doing something that embarrasses you. Interpersonal Competence – Negative Assertion 23. Telling your dating partner that he or she has done something to hurt your feelings. Interpersonal Competence – Negative Assertion 24. Telling your dating partner that he or she has done something that made you angry. Interpersonal Competence – Negative Assertion

15. M EASUREMENT Sexual Violence Prevention Survey Now please read the next few statements, and then choose how much you agree with each statement. 25. If a guy or girl gets really drunk at a party and has unwanted sex, it is partly their fault. Sexual Consent 26. It is ok for a guy or girl to force someone to have sex, if that person has been flirting and ‘leading them on’. Sexual Consent 27. It is not rape if dating partners have had sex before. Sexual Consent

16. M EASUREMENT Sexual Violence Prevention Survey Based on your experience, mark whether you think it is really wrong, sort of wrong, sort of ok, or perfectly ok to: 28....question your boyfriend/girlfriend about other romantic relationships they have had? Behavioral Jealousy 29....question my boyfriend/girlfriend about their telephone calls. Behavioral Jealousy 30....question your boyfriend/girlfriend about where they have been. Behavioral Jealousy 31....say something mean and insulting to your boyfriend or girlfriend if they were dancing really close with someone else at a party, and totally ignoring you? Behavioral Jealousy

17. M EASUREMENT Sexual Violence Prevention Survey Please read the next 2 questions, and based on your best memory, choose how often these things have happened. 32. In the past 12 months have you ever teased, threatened or harassed someone who you heard others say was gay, lesbian, bisexual or transgendered? Homophobic Teasing 33. How often do you hear people say mean things about someone because of their: Homophobic Teasing a. sexual orientation? b. gender? c. race? d. religion?

18. Removed Items: Men and women are equal and should be treated the same way. Carrying on conversations with someone new whom you think you might like to get to know. Introducing yourself to someone you might like to get to know (or date). Being able to admit that you might be wrong when a disagreement with someone begins to build into a serious fight. When angry with someone, being able to accept that she or he has made an important point even if you don't agree with their view. Added Items: If a guy or girl gets really drunk at a party and has unwanted sex, it is partly their fault. It is ok for a guy or girl to force someone to have sex, if that person has been flirting and ‘leading them on’. It is not rape if dating partners have had sex before. 18 M EASUREMENT

19. DATA COLLECTION It’s “FUN”damental! Electronic Surveys Paper Surveys Data Management

20. DATA COLLECTION QuestionResponse Organization ID# My 3-digit Org ID# is _________ Survey Type  Paper  Electronic Surveys Used Pretest Data Collection Date ________________________________ Posttest Data Collection Date ________________________________ Person who collects data: Name ______________________________ Trained?  Yes  No Backup person who collects data: Name ______________________________ Trained?  Yes  No Where survey copies are stored: Person who tracks ID Assignment Name ______________________________ Where ID list is stored:

21. DATA COLLECTION ELECTRONIC SURVEYS Get the Link CompleteSubmit

22. DATA COLLECTION ELECTRONIC SURVEYS

23. DATA COLLECTION PAPER SURVEYS

24. Step 1: Ordering Step 2: Preparation Step 3: Submission Survey condition Administrative Section Instructions Confidentiality Survey condition Administrative Section Instructions Confidentiality DATA COLLECTION PAPER SURVEYS

25. Useful Data Condition Using originals Stapling or hole punching Using pencils DATA COLLECTION PAPER SURVEYS

26. Fill in pre- or post-test Fill out Org ID and Local ID from top to bottom Dates DATA COLLECTION ALL SURVEYS Useful Data Condition Admin. Section

27. DATA COLLECTION ALL SURVEYS Useful Data Condition Admin. Section Instructions Basic Script Read thoroughly Pick best answer Answering questions

28. DATA COLLECTION ALL SURVEYS Useful Data Condition Admin. Section InstructionsConfidentiality Testing space Survey handling

29. Step 1: Ordering Step 2: Preparation Step 3: Submission Julie Maertens Colorado State University Campus Delivery 1876 Fort Collins, CO 80523 Julie Maertens Colorado State University Campus Delivery 1876 Fort Collins, CO 80523 DATA COLLECTION PAPER SURVEYS

30. Assign unique IDsKeep track of each IDKeep your log confidential D ATA MANAGEMENT

31. Pre-testPost-test 10011001 10031005 20022002 20032003 20142112 Sample Size = 3 D ATA MANAGEMENT

33. DeadlineTaskPerson in Charge Determine which youth and/or adults are eligible and will be surveyed Arrange a date for pretesting or post-testing Order surveys or survey link (expect 2-week turn-a-round for paper surveys) Organize surveys: count, fill out administrative section Administer surveys to youth and/or adults Check surveys: bubbles filled- in, administrative section complete Mail surveys to CSU (1 week prior to deadlines)

34. D UE D ATES Pretest Deadline January 31, 2014 Posttest Deadline May 30, 2014

35. RESULTS

36. ItemsResponse Scale About how often (if ever) do you: Behavior is rated on a 5-pt scale as: 1=Never, 2=A few times a year, 3=Once a month to a few times a month 4=Once a week to a few times a week 5=Once a day to more than once a day. Attitudes are rated on a 4-pt scale as: 1=No risk 2=Slight risk 3=Moderate risk 4=Great risk. 1. Drink beer, wine, wine coolers, or liquor (more than just a few sips)? 2. Drink until you get drunk? 3. Smoke cigarettes? 4. Smoke marijuana (grass, pot) or hashish (hash)? 5. Take prescription drugs that aren’t yours? How much do you think people risk harming themselves (physically or in other ways) if they: 6. Smoke one or more packs of cigarettes per day? 7. Try marijuana once or twice? 8. Smoke marijuana regularly? 9. Take one or two drinks of an alcoholic beverage (beer, wine, liquor) nearly every day?

37. RESULTS Pretest Mean (N=203) Posttest Mean (N=138) 1. Drink beer, wine, wine coolers, or liquor (more than just a few sips)? 1.181.13 2. Drink until you get drunk? 1.051.09 3. Smoke cigarettes? 1.041.09 4. Smoke marijuana (grass, pot) or hashish (hash)? 1.031.06 5. Take prescription drugs that aren’t yours? 1.02 6. Smoke one or more packs of cigarettes per day? 3.343.61 7. Try marijuana once or twice? 2.913.14 8. Smoke marijuana regularly? 3.283.61 9. Take one or two drinks of an alcoholic beverage (beer, wine, liquor) nearly every day? 3.073.45

38. RESULTS NMeanStd. Deviation Range Min - Max Desired Direction of Change? Sig.Effect Size ATOD Attitudes – PRE 362.881.171.00-4.00 Yes0.450.11 ATOD Attitudes - POST 363.011.081.00-4.00 ATOD Use – PRE 371.010.051.00-5.00 No0.32N/A ATOD Use - POST371.010.031.00-5.00

39. 39 RESULTS

40. THANK YOU ! Email: jamaertens@gmail.com CSU Evaluation Office (970)491-3830 main (877)724-4111 toll free Questions?