1. AEA Conference October 16, 2014 Strategies for evaluating multilevel interventions: lessons learned from two obesity prevention efforts Sarah Welch, MPH

2. Obesity Prevention Efforts 2 http://www.shiftn.com/obesity/Full-Map.html

3. Evaluation of Obesity Prevention Efforts 3 http://www.shiftn.com/obesity/Full-Map.html

4. Two Case Studies Large obesity prevention and reduction efforts Located in Chicago area Multiple organizations Multiple municipalities/local governments Multiple strategies – including policy, systems, environmental and some individual change components. 4

5. Major Lessons Learned Make data collection hassle free. Incorporate multiple types of data to inform each other and provide a bigger picture/story. Keep in mind the unique factors of your project/community/participants etc. 5

6. Suburban Cook County Communities Putting Prevention to Work (SCC CPPW)

7. Suburban Cook County 400+ Square miles 2.5 million residents 129 municipalities Huge income inequalities and health disparities 7

8. SCC CPPW Topics and Strategies Focus on obesity prevention and reduction –Policy and environmental change –Reducing health inequities –Creating coalitions County-wide efforts where that made sense: breastfeeding promotion, transportation planning Local community strategies where that made sense: schools, food access, physical activity, etc. –Model Communities 8

9. Model Communities –One component of SCC CPPW –12-month mini-grants –Menu of evidence-based interventions –Capacity building and goal-specific TA available to grantees –High need communities/regions prioritized –Communities, schools, CBOs, health care organizations eligible –38 awards 9

10. SCC CPPW Program Components Capacity building Technical assistance from expert partners Strategy workshops Coalition building 10

11. Evaluation Measures and Methods SurveyInterviewWork PlanMonthly Reports CapacityX Facilitators/ Barriers XX ImplementationXX OutcomesXXX SustainabilityX 11

12. SCC CPPW Eval Strategy Benefits and Limitations PROS Survey/online survey/phone calls = low burden for participants. Multiple strategies to collect different aspects of a larger effort created a comprehensive picture of success of overall project. CONS Hard to generalize results. Staff turnover/knowledge loss. 12

13. Model Communities Enhanced Evaluation

14. Enhanced Evaluation in Schools Deep dive focus with Model Communities focusing on school based efforts –Implementation data – work plans, menu audits, school observations for environmental changes –Student level outcomes – PA measures (accelerometry), PA/Nutrition surveys 14

15. MC EE Eval Strategy Benefits and Limitations PROS Multiple strategies to collect different aspects of the larger effort created a comprehensive picture of success of overall project. CONS Hard to generalize results. Time/$$/Staff intense. 15

16. Suggested Strategies – Beyond Obesity Prevention Low participant burden. Tailoring of strategies is important for intervention success but complicates evaluation efforts. Have an evaluation strategy that can adapt and assess effectiveness and success of what might look like completely different efforts – but are working toward the same larger goal. Layering of outcomes data from various levels (individual, institutional, community) is important for a meaningful and comprehensive evaluation. Collect implementation data to inform the outcome data. 16

17. Sarah B. Welch, MPH Evaluation Manager Consortium to Lower Obesity in Chicago Children (CLOCC) 225 E. Chicago Ave., Box 157 Chicago, IL 60611-2605 312-227-7045 swelch@luriechildrens.org www.clocc.net Facebook Twitter: @CLOCC_Chicago