1. October 23, 2018 Amy DeLisio MPH, RD Suzanne Ryan-Ibarra PhD, MPH
SNAP-Ed Works: A Coordinated Regional Program Evaluation Approach and Results
Housekeeping notes:
all phones are on mute
We will have a question and answer section at the end
Feel free to type questions or comments into the chat box as they arise
We will address chat box questions and comments during the Q&A session
At that time, you can also use the “raise hand” function and we will unmute your line for you to speak
October 23, 2018
Amy DeLisio MPH, RD
Suzanne Ryan-Ibarra PhD, MPH

2. Our Mission
The Public Health Institute Center for Wellness and Nutrition (CWN) is committed to increasing capacity through community change leadership for equity and wellness through research, policies, and practices for all.
CWN uses an integrated wellness model to address the Social Determinants of Health.

3. The Beginning
The Southeast Learning Community project was originally funded by CDC and implemented by Public Health Institute (PHI) in partnership with Southeast Regional Office
Currently funded through SERO SNAP-Ed
A needs assessment was conducted to prioritize regional opportunities and support
The majority of respondents were interested in using the SNAP-Ed Evaluation Framework
SERO Evaluation Work Group

4. Who is familiar with the SNAP-Ed Evaluation Framework
Who is familiar with the SNAP-Ed Evaluation Framework? Have you used the Framework?

5. SERO Regional Evaluation Project: Partnerships and Collaboration
USDA Southeast Region
PHI CWN
RNECE-South
States
Implementing Agencies
Met Throughout FFY 2016 to plan SERO Regional Evaluation Project

6. SERO Regional Evaluation Project: Reasonable Expectations
SERO showed great leadership
Supportive and flexible
Measuring SERO Common Indicators across the region was mandatory for FFY 2017
Every state had to collect all selected indicators
Not every IA needed to collect every indicator

7. SERO Regional Evaluation Project: Strong Participation and Engagement
PHI CWN facilitated the process
20 consistent participants from all eight states
Share survey tools and resources– Google Drive
Opportunities to share best practices
Highlights regional similarities and strengths

8. SERO Regional Evaluation Project: SERO FFY 2017 Regional Common Indicators
MT1 Healthy Eating Behaviors
MT2 Food Resource Management Behaviors
MT4 Nutrition Supports Adopted

9. SERO Regional Evaluation Project: Aims
To measure outcomes of SNAP-Ed programs in the Southeast Region for healthy eating behaviors (MT1); food resource management behaviors (MT2); and Nutrition Supports (policy, systems, and environmental (PSE) changes)(MT5) during Federal Fiscal Year 2017.

10. Methods Standardize responses from multiple survey instruments
Create analytical dataset (combined data from 25 implementing agencies)
Standard meta-analysis – pooled relative risk, 95% confidence intervals
Subgroup analyses – state, age group

11. Methods – Survey Instrument Standardization (MT1c - Example)
24-Hour Recall
WebNEERS
Food Behavior Checklist
Question
n/a
Do you eat more than one kind of fruit each day?
Answers
Cups
N/A Never Seldom Sometimes Most times Always
No Yes, sometimes Yes, often Yes, always
Answer Standardization
1 = Yes, ate more than 1 kind of fruit 0 = No, did not eat more than 1 kind of fruit
1 = Always, Most times 0 = Sometimes, Seldom, Never, NA
1 = Yes, always/Yes, often 0 = Yes, sometimes/No
We used the Dietary Guidelines for Americans to standardize responses across survey instruments. For example, IAs reported MT1c using 18 different instruments.
24-hour recall (n=1,
Food Behavior Checklist (n=19)
WebNEERS (n=11)
Other (n=15)

12. Results 25 implementing agencies submitted data
All age groups represented – children (38% of programs), teens (13% of programs), adults (38% programs), and seniors (11% of programs)
Data was submitted for all MT1, MT2, and MT5

13. Results – Healthy Eating Behaviors (MT1)
Indicator
Description
# Pre Total
# Post Total
MT1c
More than 1 kind of fruit
36,833
32,317
MT1d
More than 1 kind of vegetable
37,814
33,247
MT1g
Drink water frequently
40,378
35,885
MT1h
Fewer sugar-sweetened beverages
40,788
36,473
MT1i
Choose low-fat/fat-free milk, milk products
38,094
33,719
MT1l
Cups of fruit
8,048
6,867
MT1m
Cups of vegetables
8,062
6,877
# of participants in 2 columns on the right

14. Results – Healthy Eating Behaviors (MT1) (continued)
Indicator
Description
Pooled Relative Risk
95% Confidence Interval
MT1c
More than 1 kind of fruit
1.10
( )*
MT1d
More than 1 kind of vegetable
1.14
( )*
MT1g
Drink water frequently
1.04
( )*
MT1h
Fewer sugar-sweetened beverages
( )*
MT1i
Choose low-fat/fat-free milk, milk products
1.09
( )*
*Statistically significant (p-value < 0.05).
Statistically significant and positive improvements in all MT1 indicators

15. Results – Healthy Eating Behaviors (MT1) (continued)
Indicator
Description
Standardized Mean Difference
95% Confidence Interval
MT1l
Cups of fruit
0.36
( )*
MT1m
Cups of vegetables
0.22
( )*
*Statistically significant (p-value < 0.05)
Statistically significant and positive improvements in cups of vegetables (about 1/3rd cup more) and fruit (about 1/4th cup more)

16. Results – Food Resource Management Behaviors (MT2)
Indicator
Description
# Pre Total
# Post Total
MT2a
Choose healthy foods on a budget
14,571
10,773
MT2b
Read nutrition labels
14,897
11,007
MT2g
Not run out of food
14,427
10,657
MT2h
Compare prices
14,195
10,501
MT2i
Identify foods on sale
7,981
5,013
MT2j
Shop with a list
14,155
10,490

17. Results – Food Resource Management Behaviors (MT2) (continued)
Indicator
Description
Pooled Relative Risk
95% Confidence Interval
MT2a
Choose healthy foods on a budget
1.42
( )*
MT2b
Read nutrition labels
1.74
( )*
MT2g
Not run out of food
1.27
( )*
MT2h
Compare prices
1.29
( )*
MT2i
Identify foods on sale
1.11
( )*
MT2j
Shop with a list
1.47
( )*
*Statistically significant (p-value < 0.05).
Significant and positive changes in the MT2 indicators from pre to post. 

18. Results – Policy, Systems, Environmental Changes (MT5)
Indicator
Description
Number
MT5b
Total number of policy changes 95
MT5c
Total number of systems changes
225
MT5d
Total number of environmental changes
322
Total PSE adopted
642
MT5e
Total number of promotional changes for PSE
469
MT5f
Reach of PSE changes
782,231
Reach counts do not include reach reported for promotional efforts (MT5e) only.
642 PSE changes reaching 782,231 people (3/4 million)

19. Results – Total PSE adopted compared by setting
Most common PSE setting is Learn (58% of PSE changes), followed by Shop (19% of PSE changes)

20. Questions

21. Reflections Lets share out a few reflections.
Take a minute to reflect:
What are my personal takeaways?
What can you take back to your organization and community?
How can this support the evaluation of your obesity prevention efforts?
Lets share out a few reflections.

22. Thank you Brittany Souvenir and Veronica Bryant, USDA FNS
SERO Evaluation Workgroup
State Implementing Agency partners
Sharon Sugerman, MS, RD, FAND, Center for Wellness and Nutrition, Public Health Institute

23. Follow us on LinkedIn and Twitter!
Follow us on LinkedIn and Twitter!
@phi_wellness
Suzanne Ryan-Ibarra
Amy DeLisio