Approaches for Evaluating Local Environmental Efforts Massachusetts Statewide Substance Abuse Prevention Conference At the Educational Development Center October 30, 2014 – Waltham, MA

The Wayne Harding Slide Massachusetts Wyoming November 15, 2018

Environmental Mind Experiment

Challenges Who do we count? How do we know things are changing? Processes Outcomes Who do we count? When do we count them? For how long do we count them? What else do we count? How do we know things are changing? What caused the change? Did we reach our outcomes? November 15, 2018

Discussion: What are some of the challenges you have faced when trying to evaluate environmental prevention Strategies? November 15, 2018

Tectonic Shift in Thinking Serving People Changing Communities November 15, 2018

New Goals and Objectives Provide programs to bring about individual changes, or … … create changes in the system to bring about population-level change November 15, 2018

New Unit of Analysis From Individuals To Populations November 15, 2018

New Discipline Program Evaluation Policy Analysis November 15, 2018

Process Evaluation November 15, 2018

Outcome Evaluation Models versus Individual Variables Ingredients Dessert November 15, 2018

Outcome Evaluation Designs Interrupted Time Series Analysis November 15, 2018

Outcome Evaluation Designs Control Series Analysis November 15, 2018

Outcome Evaluation Designs Interrupted Time Series Analysis with Forecast November 15, 2018

Choosing Comparison Communities Demographics Educational Systems Economic Base Not receiving the intervention Common Baseline Data Example: School Random Drug Testing Policy November 15, 2018

Choosing Comparison Communities November 15, 2018

Questions & Answers

Question: What is the best coffee shop within walking distance of EDC Question: What is the best coffee shop within walking distance of EDC? (and don’t say Dunkin Donuts)

Utah Parents Empowered Evaluating Media Utah Parents Empowered Media Campaign https://www.youtube.com/watch?v=k2ORL9tEh1k November 15, 2018

Evaluation Starts with a Question http://www.ers.usda.gov/media/1118789/err149.pdf November 15, 2018

Awareness? Enrollment? Knowledge? Attitudes? Behavior? 1. Ask Questions Awareness? Enrollment? Knowledge? Attitudes? Behavior? November 15, 2018

2. Decide if its Whole or Stand Alone? November 15, 2018

3. Collect and Analyze Data Quantitative Qualitative Archives Observation Surveys Archives Observation Interviews Focus Groups November 15, 2018

Evaluating Media Example Perkins, W.H., Linkenbach, J.W., Lewis, M.A., & Neighbors, C. (2010). Effectiveness of social norms media marketing in reducing drinking and driving: A statewide campaign. Addictive Behaviors, 35, 866-874. November 15, 2018

Evaluating Media Example Television Radio Local Newspaper College Newspaper Promotional Items Theater Ads Billboards Indoor Ads January 2002 through March 2003 November 15, 2018

Target population of 21 to 34 year-olds surveyed four times Evaluating Media Example Target population of 21 to 34 year-olds surveyed four times n=1669 n=765 n=1047 November 15, 2018

Evaluating Media Example Differences for Intervention and Comparison Counties between November 2001 and June 2003 November 15, 2018

Questions & Answers

Discussion: Let’s do this together Who has a media campaign that needs evaluating? November 15, 2018

Consider the media campaign goals and evaluation research question Step One Consider the media campaign goals and evaluation research question What are you trying to change? Is the media part of a larger effort or does it stand alone? November 15, 2018

Decide on a methodology/ Step Two Decide on a methodology/ research design Can you find a comparison community? How much time do you have? November 15, 2018

Identify indicators or create data collection instruments Step Three Identify indicators or create data collection instruments How will you tell the story (processes)? How will you measure change (outcomes)? November 15, 2018

Step Four Improve the Media Create a Report Make Decisions Create a Presentation November 15, 2018

Questions & Answers

Final Question: Just to be clear, the best nearby coffee shop is?

Contact Me! Slides available at… rodneywambeam.com presentation tab or (307) 760-8928 rodney@uwyo.edu rodneywambeam.com Slides available at… rodneywambeam.com presentation tab or from MassTAPP November 15, 2018

Thank You! November 15, 2018