Includes: an example of a logic model evaluation questions from a logic model learning percent change writing SMART objectives analyzing fake data using that feedback to make changes

Welcome to beautiful City-A

 We want our governmental leaders to be leaders in sexual assault prevention!  So...  We’re going to do a yearlong campaign targeted to governmental leaders in City-A accompanied by individual trainings.

Staff Time Poster Materials Training Materials Computer City Officials Develop Posters Distribute Posters Develop Training Hold Trainings s Sent 50 Posters Displayed Pre and post Evaluations 50 Officials Trained Monthly s Sent To Listserv Decrease Rape myth Acceptance Among Officials Language And actions Support Victims of SA And promote Social norm change City Officials Become Agents of Change in SV Prevention & Intervention InputsActivities Immediate Outputs Long-term Impact Program Outcomes

 So we know WHAT we’re going to do now...  How will we know if we accomplish our goals??  What questions might we have????

Staff Time Poster Materials Training Materials Computer City Officials Develop & Distribute Posters Develop Training Hold Trainings s Sent 50 Posters Displayed Pre and post Evaluations 50 Officials Trained Monthly s Sent To Listserv Decrease Rape myth Acceptance Among Officials Language And actions Support Victims of SA And promote Social norm change City Officials Become Agents of Change in SV Prevention & In supporting victims InputsActivities Immediate Outputs Long-term Impact Program Outcomes What do you want to know??? How much Time & money Are Invested? Quality of Posters & Training? Content? Who attended Trainings? Did officials See posters and Read s? Who Decreased Rape myths? What actions Were seen? Do city Officials Feel like Agents of Change in SV?

 How much time and money is invested?  What is the quality of the posters and training?  What is the content of the s?  Who attended the trainings?  Did the officials see the posters and read the s?  Which officials decreased rape myths? Were they decreased overall?  What language and actions were reported by officials?  Do city officials feel like leaders in SV prevention and in supporting victims?

Evaluation Question Can the program answer this? Which Stake- Holder Cares? How Import- ant? New Data Collect- Ion? Time And Resources Needed? Priority: High, Medium, Low, Bye How much time and money is invested? Yes!Funders, finance, we! Cost benefit idea NoLowPretty high Did the officials see the posters and read the s? ProbablyWe do!Very- pretty importa nt Yes, question on reliability ? ModerateHigh / medium What is the quality of the posters and training? YesWe, funders, others Very importa nt YesVaryHigh (very high) Who attended the trainings? Which officials decreased rape myths? Were they decreased overall? YesEveryon e Very importa nt YesDepends- low resources High What language and actions were reported by officials?

 Which officials decreased rape myths? Were they decreased overall?  2 approaches from here!  Research approach*- Ask a question  Program approach*- Set a goal and see if we can answer it  Let’s frame as a SMART Objective

What exactly are we going to do for whom? Is it clear and well defined? Is it clear to anyone that has a basic knowledge of the work area? S.M.A.R.T. Decrease rape myth acceptance in governmental leaders in Madison, WI.

Is it quantifiable and can we measure it? Think indicators! Which indicators? S.M.A.R.T. You bet we can (try to) measure this!!! Selected indicator: Illinois Rape Myth Acceptance Scale (IRMA)

How much change do we expect to see??? Can we accomplish this (in our time frame with the resources we have available)? Do stakeholders agree this is something we should be striving for? S.M.A.R.T. How will we determine what is achievable???

How much change do we expect to see??? Can we accomplish this (in our time frame with the resources we have available)? Do stakeholders agree this is something we should be striving for? S.M.A.R.T. Let’s say I’m going to assume that the average scale is “neutral” across all items. And my goal is to get the average one “column” closer to strongly disagree.

22 x 3 = x 4 = 88

How much change do we expect to see??? Can we accomplish this (in our time frame with the resources we have available)? Do stakeholders agree this is something we should be striving for? S.M.A.R.T. I want to see an average increase in rape myth “disagreement” from 66 to 88. How much of a change is it to go from 66 to 88???? This difference is called percent change.

 Open a new web browser and enter:   Enter Y1 (1 st value) as 66  Enter Y2 (2 nd value) as 88  Percent Change =  [(New Value-Old Value)/(Old Value)] *100%  For us Percent change =  =[(88-66) / 66]* 100%  = 33% (positive value indicates an INCREASE)

 Gas increases/decreases  $3.49 yesterday (1 st value)  $3.79 (2 nd value)  Difference in coffee sizes at starbucks  Tall-12oz  Grande- 16 oz  Venti- 24oz  Average temperature in Milwaukee  January: 22.5 F  June: 66 F  Batting averages - Ryan Braun  average in 2012  on 4/24/13  Age differences between co-workers

Will this objective have an effect on the desired goal or strategy? S.M.A.R.T. We expect to see a 33% change. Is that 33% change meaningful to us?

When will this objective be accomplished? Is there enough time to achieve the goal? Is there a time limit? S.M.A.R.T. As stated, we’ll have one year. Let’s say June, 2013 to June, 2014

Fill in the blanks! What do you think? S.M.A.R.T. From date1 to date2, who/target will change anticipated as indicated by amount of change anticipated (from X to Y) on indicator. From June 2013 until June 2014, government officials in Madison will decrease rape myth acceptance as indicated by a 33% increase from 66 to 88* on the Illinois Rape Myth Acceptance Scale. *Guestimated- Baseline data needs to be collected

Not-so-SMART objective 1a: Decrease rape myth acceptance in governmental leaders in Madison, WI Key ComponentObjective Specific - What is the specific task? Decrease rape myth acceptance in governmental leaders in City-A through training and a media campaign Measurable - What are the indicators and what change is expected? Pre and post scores on the Illinois Rape Myth Acceptance Scale. We predict a 33% change in average scores based on an estimated baseline score of 66, which indicates neutral responses. Achievable - Is the task feasible? Enough time and resources? Yes! In one year with four trainings, monthly s, and posters we believe we can shift social norms to this regard. Relevant – Does meeting this objective create practical change? Yes! How great to have city officials who are well educated on rape myths and can assist in social norms change. Time-Bound - What are the start and end dates? One year- between June, 2013 and June, 2014 Template: From date1 to date2, who/target will change anticipated as indicated by amount of change anticipated (from X to Y) on indicator. SMART objective 1a: From June , City-A governmental leaders will decrease rape myth acceptance as indicated by a 33% increase (from a predicted baseline score of 66 to 88) in rape myth disagreement on the Illinois Rape Myth Acceptance Scale.

 Let’s collect our baseline data!  Pretend you hold the average views of someone on your local city council.  Complete the IRMA and when you finish put your total number in the box

 Our “guesstimated” values were 66 to 88, 33% change.  Our “real” baseline average is: ____  So now what???  Need to know what ending average will give a 33% increase is if our starting value is ____.  OR excel calculator  (open) June 2013 until June 2014, government officials in Madison will decrease rape myth acceptance as indicated by a 33% increase from X to Y on the Illinois Rape Myth Acceptance Scale.

 33 survivors of sexual assault in the support group will decrease their PTSD scores by 18% from 45 to 30 on the UCLA PTSD Index.  Timely  From September to December, 2013, walk-in clients for sexual assault services will increase 650% monthly from a current average of 20 per month to 150 per month.  Attainable/Achievable  By October, 2013, ELC SASPs will finalize a 40-page manual on gardening techniques.  Relevant  From June, 2013 to June, 2018, 75% of the counties in WI will decrease the number of sexual assault victimizations by 10%.  Measurable (~Attainable/Achievable)  During April, 2013, a campaign will decrease gender stereotypes by 25% on the Male Role Attitudes Scale (baseline yet to be determined).  Specific- who will be impacted by the campaign? Specific Measurable Attainable/Achievable Relevant Timely

 You want to increase the number of clients who use your walk-in sexual assault services. Right now you have about 25 per month.  You plan to distribute flyers at all grocery stores and place an add in the local paper next month.  How many more clients might you expect to see AFTER the outreach?  Number: ____  So if we increase from 25 to ____, what is our percent change???  Percent change: ____  What is our SMART objective?  (Write it in the chat) SMART Format: From date1 to date2, who/target will change anticipated as indicated by amount of change anticipated (from X to Y) on indicator.

Evaluation Question Can the program answer this? Which Stake- Holder Cares? How Import- ant? New Data Collect- Ion? Time And Resources Needed? Priority: High, Medium, Low, Bye How much time and money is invested? What is the quality of the posters and training? Who attended the trainings? Did the officials see the posters and read the s? Which officials decreased rape myths? Were they decreased overall? What language and actions were reported by officials? Go back to slide 9- for which other question do we want to write an objective???? Use next slide SMART Format: From date1 to date2, who/target will change anticipated as indicated by amount of change anticipated (from X to Y) on indicator.

Not-so-SMART objective 1a: *Place here* Key ComponentObjective Specific - What is the specific task? Measurable - What are the indicators and what change is expected? Achievable - Is the task feasible? Enough time and resources? Relevant – Does meeting this objective create practical change? Time-Bound - What are the start and end dates? Template: From date1 to date2, who/target will change anticipated as indicated by amount of change anticipated (from X to Y) on indicator.

 At the end of the City-A intervention you compile your final evaluation results.

IRMA SectionPrePost Subscale 1: She asked for it 1628 Subscale 2: He didn’t mean to 1213 Subscale 3: It wasn’t really rape 1824 Subscale 4: She lied 2025 Totals6690 *Higher score indicates higher disagreement with rape myths. Qualitative Training Feedback: -4 trainings were created to target each “subscale” of the IRMA -Attendance was similar across trainings -Training for “subscale 2: He didn’t mean to” was canceled due to snow and a shorter segment was adapted and added to the training for “subscale 3”

 What would you tell your funders?  Imagine you have an extended 3 months to continue your intervention with city officials... What would your objective be?  Since you achieved so much success, you are asked to help a nearby community set up a similar program. What would you tell them?

Happy Evaluating!