1. 1 SNAP-E D E VALUATION F RAMEWORK : T HE PAST, P RESENT, AND FUTURE

2. Framework Overview Andrew (Riesenberg) Naja-Riese FNS, Western Region Program Integrity Branch Chief Supplemental Nutrition Assistance Program

3. The ABCDs of PSEs Assessments of need and organizational readiness Buy-in from key decision-makers, leaders, champions, and partners Changes take effect and are evaluated for effectiveness, mid-course changes occur Disparities based on race/ethnicity, income, geography, etc. reduced Sustainability to endure new leadership and resource availability

4. THE PAST “Study the past if you would define the future.” ― Confucius

5. WRO SNAP-Ed Evaluation Framework

7. 9 Western States Piloted Framework

8. Reporting PSE Outcomes From October 1, 2013 – September 30, 2014, an estimated 923 policy, systems, and environmental activities were reported across Western States* * AK, AZ, CA, GU, HI, ID, NV, OR, WA FY 2014 Year-End Reports

9. Lessons Learned 1.Framework indicators provide common direction 2.Using indicators requires buy-in at many levels 3.States interpret measures differently 4.Need to balance consistency with flexibility 5.Not everything can be measured 6.Tension between attribution and contribution 7.Difference between academic evaluation and ‘real world’ evaluation

10. FFY 2016 Priority Indicators Indicator Code Indicator Name Individual Level MT1MyPlate Behaviors MT2Shopping Behaviors MT3Physical Activity Behaviors Environmental Settings Level ST4Identification of Opportunities ST6Partnerships MT4Nutrition Supports Adopted MT5Physical Activity Supports Adopted

11. THE PRESENT “The secret of health for both mind and body is not to mourn for the past, worry about the future, or anticipate troubles, but to live in the present moment wisely and earnestly.” – Buddha

12. NCCOR Workshop

14. Global Framework Changes Dropped WRO from title New numbering system and headings  Short-term = Readiness and capacity building  Medium-term = Changes  Long-term = Effectiveness and Maintenance Replaced Impacts with Population Results Dropped social and cultural norms Combined Physical Activity with Reduced Sedentary Behaviors

15. Which level is it? A school has installed raised garden beds growing vegetables and herbs, which are maintained by student volunteers during the school year, and by district grounds staff during the summer. Environmental Settings

16. A church bishop is concerned that the weekly food pantry needs to offer a variety of leafy green vegetables. In response, the SNAP-Ed creates a partnership with a local farm that is gleaning dark green vegetables. Environmental Settings Which level is it?

17. The food policy council is working with the city manager’s office to raise cash for a city-wide program called FreshApproach to provide free fruits and vegetables for parents and children at- risk youth sports leagues. Sectors of Influence Which level is it?

18. In response to the passage of Assembly Bill 242, all elementary schools will be offering 30 minutes of in-class physical activity per week. SNAP-Ed educators developed an activity kit for teachers to meet the unfunded mandate. Sectors of Influence Which level is it?

19. A small, rural town has one community recreation room that is only open during the school day. A group of parents work with the school health advisory committee to extend the hours of the recreation room into the evenings for unstructured playtime. Environmental Settings Which level is it?

20. Changes - Individual Level Changed short-term indicators to “motivators” Added Food Safety Indicator Changed long-term to minimum of 6-month follow-up on medium-term indicators

21. Changes – Environmental Settings Identification of Opportunities becomes Readiness Changed the denominators for Nutrition/PA Supports Adopted Refined Maintenance indicators Partnerships focuses on organizational partnerships

22. Changes – Sectors of Influence Refined Multi-Sector Collaboration indicator Realigned Medium-Term and Long-Term indicators  MTs are community or jurisdiction policy, systems, and environmental changes that take effect  LTs are the results of these changes Updated social marketing and media practices

23. Changes – Population Results Created a new section to demonstrate impacts from all three levels of the Framework Focuses on nutrition, physical activity, and obesity prevention status  Population surveillance or program evaluation Incorporated indicators for Breastfeeding Family Meals, Active Commuting

24. Changes - Social and Cultural Norms Dropped indicators (except for Family Meals and Active Community, which were moved to Population Results) Will approach this section from a qualitative/case study perspective

25. THE FUTURE “The best way to predict your future is to create it.” - President Abraham Lincoln

26. 36 States, Guam, & D.C. are Using the Framework and/or Involved in Development of IG IG Authors/Reviewers Using Framework

27. ElementDefinitionSample IndicatorMT9 – Agriculture Outcome measures Desired benefit, improvement, or achievement of a specific program or goal Number, or %, of farmers markets or direct marketing farmers with public- private partnerships that provide bonus incentives programs for SNAP EBT What to measure Guidance on surveys or other data collection methodologies Farmers markets with established bonus bucks programs How to measure Data collection tools or instruments with live links and sample survey questions Survey of farmers market associations, State ag depts, AMS, Wholesome Wave PopulationSub-group for which this indicator is relevant Farmers markets or farmers ResourcesLimitations, guidance, and key citations Double bucks programs are seasonal, funding may run out Interpretive Guide – 51 indicators

28. IG Review Process Draft IG Release for Reviewers - Feb 23  Three opportunities for review: High-level (1.5 – 2 hours) Chapter-level (2 – 3 hours) Full guide review (8 – 10 hours) Final IG Release ~ Approximately Mar 31

29. Next Steps 8-hour interactive training program to be updated: Strengthening SNAP-Ed Programs, Policies, and Partnerships for Obesity Prevention Training webinars and sharing success Identifying FY 2017 Priority Indicators