October 23, 2018 Amy DeLisio MPH, RD Suzanne Ryan-Ibarra PhD, MPH SNAP-Ed Works: A Coordinated Regional Program Evaluation Approach and Results Housekeeping notes: all phones are on mute We will have a question and answer section at the end Feel free to type questions or comments into the chat box as they arise We will address chat box questions and comments during the Q&A session At that time, you can also use the “raise hand” function and we will unmute your line for you to speak October 23, 2018 Amy DeLisio MPH, RD Suzanne Ryan-Ibarra PhD, MPH

Our Mission The Public Health Institute Center for Wellness and Nutrition (CWN) is committed to increasing capacity through community change leadership for equity and wellness through research, policies, and practices for all. CWN uses an integrated wellness model to address the Social Determinants of Health.  

The Beginning The Southeast Learning Community project was originally funded by CDC and implemented by Public Health Institute (PHI) in partnership with Southeast Regional Office Currently funded through SERO SNAP-Ed A needs assessment was conducted to prioritize regional opportunities and support The majority of respondents were interested in using the SNAP-Ed Evaluation Framework SERO Evaluation Work Group

Who is familiar with the SNAP-Ed Evaluation Framework Who is familiar with the SNAP-Ed Evaluation Framework? Have you used the Framework?

SERO Regional Evaluation Project: Partnerships and Collaboration USDA Southeast Region PHI CWN RNECE-South States Implementing Agencies Met Throughout FFY 2016 to plan SERO Regional Evaluation Project

SERO Regional Evaluation Project: Reasonable Expectations SERO showed great leadership Supportive and flexible Measuring SERO Common Indicators across the region was mandatory for FFY 2017 Every state had to collect all selected indicators Not every IA needed to collect every indicator

SERO Regional Evaluation Project: Strong Participation and Engagement PHI CWN facilitated the process 20 consistent participants from all eight states Share survey tools and resources– Google Drive Opportunities to share best practices Highlights regional similarities and strengths

SERO Regional Evaluation Project: SERO FFY 2017 Regional Common Indicators MT1 Healthy Eating Behaviors MT2 Food Resource Management Behaviors MT4 Nutrition Supports Adopted

SERO Regional Evaluation Project: Aims To measure outcomes of SNAP-Ed programs in the Southeast Region for healthy eating behaviors (MT1); food resource management behaviors (MT2); and Nutrition Supports (policy, systems, and environmental (PSE) changes)(MT5) during Federal Fiscal Year 2017.

Methods Standardize responses from multiple survey instruments Create analytical dataset (combined data from 25 implementing agencies) Standard meta-analysis – pooled relative risk, 95% confidence intervals Subgroup analyses – state, age group

Methods – Survey Instrument Standardization (MT1c - Example) 24-Hour Recall WebNEERS Food Behavior Checklist Question n/a Do you eat more than one kind of fruit each day? Answers Cups N/A Never Seldom Sometimes Most times Always No Yes, sometimes Yes, often Yes, always Answer Standardization 1 = Yes, ate more than 1 kind of fruit 0 = No, did not eat more than 1 kind of fruit 1 = Always, Most times 0 = Sometimes, Seldom, Never, NA 1 = Yes, always/Yes, often 0 = Yes, sometimes/No We used the Dietary Guidelines for Americans 2015-2020 to standardize responses across survey instruments. For example, IAs reported MT1c using 18 different instruments. 24-hour recall (n=1, Food Behavior Checklist (n=19) WebNEERS (n=11) Other (n=15)

Results 25 implementing agencies submitted data All age groups represented – children (38% of programs), teens (13% of programs), adults (38% programs), and seniors (11% of programs) Data was submitted for all MT1, MT2, and MT5

Results – Healthy Eating Behaviors (MT1) Indicator Description # Pre Total # Post Total MT1c More than 1 kind of fruit 36,833 32,317 MT1d More than 1 kind of vegetable 37,814 33,247 MT1g Drink water frequently 40,378 35,885 MT1h Fewer sugar-sweetened beverages 40,788 36,473 MT1i Choose low-fat/fat-free milk, milk products 38,094 33,719 MT1l Cups of fruit 8,048 6,867 MT1m Cups of vegetables 8,062 6,877 # of participants in 2 columns on the right

Results – Healthy Eating Behaviors (MT1) (continued) Indicator Description Pooled Relative Risk 95% Confidence Interval MT1c More than 1 kind of fruit 1.10 (1.09-1.11)* MT1d More than 1 kind of vegetable 1.14 (1.12-1.15)* MT1g Drink water frequently 1.04 (1.03-1.04)* MT1h Fewer sugar-sweetened beverages (1.08-1.12)* MT1i Choose low-fat/fat-free milk, milk products 1.09 (1.08-1.11)* *Statistically significant (p-value < 0.05). Statistically significant and positive improvements in all MT1 indicators

Results – Healthy Eating Behaviors (MT1) (continued) Indicator Description Standardized Mean Difference 95% Confidence Interval MT1l Cups of fruit 0.36 (0.33-0.39)* MT1m Cups of vegetables 0.22 (0.19-0.26)* *Statistically significant (p-value < 0.05) Statistically significant and positive improvements in cups of vegetables (about 1/3rd cup more) and fruit (about 1/4th cup more)

Results – Food Resource Management Behaviors (MT2) Indicator Description # Pre Total # Post Total MT2a Choose healthy foods on a budget 14,571 10,773 MT2b Read nutrition labels 14,897 11,007 MT2g Not run out of food 14,427 10,657 MT2h Compare prices 14,195 10,501 MT2i Identify foods on sale 7,981 5,013 MT2j Shop with a list 14,155 10,490

Results – Food Resource Management Behaviors (MT2) (continued) Indicator Description Pooled Relative Risk 95% Confidence Interval MT2a Choose healthy foods on a budget 1.42 (1.39-1.45)* MT2b Read nutrition labels 1.74 (1.69-1.79)* MT2g Not run out of food 1.27 (1.25-1.30)* MT2h Compare prices 1.29 (1.27-1.31)* MT2i Identify foods on sale 1.11 (1.08-1.14)* MT2j Shop with a list 1.47 (1.43-1.50)* *Statistically significant (p-value < 0.05). Significant and positive changes in the MT2 indicators from pre to post. 

Results – Policy, Systems, Environmental Changes (MT5) Indicator Description Number MT5b Total number of policy changes 95 MT5c Total number of systems changes 225 MT5d Total number of environmental changes 322 Total PSE adopted 642 MT5e Total number of promotional changes for PSE 469 MT5f Reach of PSE changes 782,231 Reach counts do not include reach reported for promotional efforts (MT5e) only. 642 PSE changes reaching 782,231 people (3/4 million)

Results – Total PSE adopted compared by setting Most common PSE setting is Learn (58% of PSE changes), followed by Shop (19% of PSE changes)

Questions

Reflections Lets share out a few reflections. Take a minute to reflect: What are my personal takeaways? What can you take back to your organization and community? How can this support the evaluation of your obesity prevention efforts? Lets share out a few reflections.

Thank you Brittany Souvenir and Veronica Bryant, USDA FNS SERO Evaluation Workgroup State Implementing Agency partners Sharon Sugerman, MS, RD, FAND, Center for Wellness and Nutrition, Public Health Institute

Follow us on LinkedIn and Twitter! www.centerforwellnessandnutrition.org Follow us on LinkedIn and Twitter! @phi_wellness Suzanne Ryan-Ibarra sryan@s-r-g.org Amy DeLisio Amy.delisio@wellness.phi.org