1. Implementation, Monitoring, and Evaluation CHIP Learning Session # 2 March 11,2014 Sarah Hartsig, M.S. Kansas Health Institute

2. PHAB Requirements 5.2.2: CHIP  Includes measureable and time-framed targets 5.2.3: Implementation  Implement the CHIP in partnership with others  Tracking process to document progress in implementation (action plan) 5.2.4: Monitoring  Monitor and revise as needed the strategies in the CHIP with participation from stakeholders and partners

3. Crosswalk: Implementation

4. Crosswalk: Monitoring and Evaluation

5. Implementation: Where the ‘rubber hits the road’ Strong CHIP and action steps are a prerequisite for successful implementation  But they DON’T guarantee success Must maintain active engagement with partners Keeping momentum can be aided by strong communication

6. Monitoring Tracking progress of your outcome measures and process measures/activities Need to determine:  Data Source  Responsible organization/individual  Monitoring frequency

7. Monitoring (continued) Frequency is determined by  Sensitivity of measure (i.e. obesity rate vs. attendance at farmers’ market)  Time/resources  Availability of data (some statewide and national measures are only updated yearly) Monitoring more frequently can allow responsiveness in strategies, if needed Allows for adaptation to challenges  Remember to document!

8. Full Detailed Plan Example Priority Area- Decrease Obesity Goal 1. Increase consumption of healthy foods by County residents Objective 1.1 Youth Healthy Food Consumption: Increase the percent of children and teens (ages 2-17) who consume five or more servings of fruits and vegetables daily to 22% by 2017. Outcomes Measures By 2017 22% of youth ages 2 – 17 consume 5 or more servings of fruit and vegetables daily. 100% of schools have a healthy vending policy by 2015 Increase sales of healthy foods in vending machines from 10% to 30% by 2017 Children and adolescents who consume 2 or more glasses of soda/sugary drinks on average daily equal to/less than 15% by 2017 Intervention Strategy 1.1.1. Healthy Vending Machines: Work with school board to create and implement healthy vending policy. Actions/Process Measures 1.1.1.1. Research and write model policies 1.1.1.2. Adapt policy for our community 1.1.1.3. Peer review and corrections made Responsible Party 1. County Commission + KHI 2. Communications lead + County Commission, KHI, KDHE 3. LHD + KHI Date Range 1. April ‘14 2. June ‘14 3. Jul ‘14 Resources 1. 20 hrs staff time 2. 10 hrs staff time 3. 5 hours staff time Objective: 1.2. Adult Healthy Food Consumption: By 2016, 35% of County adults are consuming the recommended amount of fruits and vegetables. Outcome Measures By 2016, 35% of County adults are consuming the recommended amount of fruits and vegetables Increase in population who buy food at farmers' market from 20% to 50% -Increase participation in healthy food classes at local community center to 10 participants per 6-week session Intervention Strategy 1.2.1. Increase hours of operation and awareness of Farmers market Actions/Process Measures 1.2.1.1 Develop marketing materials to increase awareness of farmers market 1.2.1.2 Investigate the optimal hours for expansion of operation 1.2.1.3. Implement expanded hours & Security Responsible Party 1. Farmers Market managers + LHD 2. Farmers Market managers + County Commission 3. Farmers Market Managers + PD Date Range Resources 1. 1Q14 1. $10,000 materials 2. 1Q14 2. 6 hours staff 3. 2Q14 3. $20,000 security

9. What if we’re not on track? PHAB anticipates that you may need to change course That is OKAY!  Document what you did and why  Include in monitoring and evaluation plan

10. Worksheet #9

11. Worksheet #9: Example

12. Practice Use Worksheet #9 Develop a Monitoring/Evaluation plan  Fill out the first 6 columns Take a look at your Outcome Measure(s)  Where will you get the data?  How frequently will you monitor it?  Who will be responsible?

13. Monitoring and… Evaluation Identify and understand the results of the program Helps to form the decision about whether your activities produced the change you set out to make Remember this?

14. Evaluation: Types Impact/outcome evaluation  Review outcomes measures  “Did the community’s health change in the way we set out to change it?” Process evaluation  Review completion of process measures/action steps  “Did we do what we said we were going to do?”

15. Developing vs. Evaluating PriorityGoalObjective Intervention Strategy ActivityImpact Long-term Outcome Medium-Term Outcome Short-tem Outcome Output What will help us meet our…? Were we successful? Did it help mae progress toward our…? Impact/Outcome Process

16. Why Evaluate? Remember: Activities Outcomes Impact What do your stakeholders care about? And what really matters at the end of the day? How will you know when you have made a difference? Evaluation is about telling and learning from your story.

17. Evaluating for Success: Example Goal: Increase the consumption of healthy foods by County residents  Activities : Craft a healthy vending policy for schools  Outcomes:  22% of youth ages 2-17 consume 5 or more servings of fruit and vegetables daily  Increase sales of healthy foods in vending machines from 10% to 30%  Impact: By 2020, obesity in County is under 28% (baseline: 32% in 2012)

18. More on Evaluation For our purposes, evaluation is the end-result of your outcomes measures (and activities/process measures) However, evaluations can include many parts:  Contextual narrative  Focus groups  Surveys  Advanced Statistical Analysis  External evaluation of program and related activities  Financial assessment If you are receiving a grant, your funder may require a more extensive evaluation than a list of outcome measures

19. Tools and Resources http://kansashealth.org/grantmaking/e valuation/resources http://kansashealth.org/grantmaking/e valuation/resources

20. Worksheet #9

21. Worksheet #9: Example

22. Discussion Use Worksheet #9 Review your Monitoring/Evaluation plan  Discuss: What will you do when you’re not on track to reach the goal?

23. Questions? Sarah Hartsig, M.S. Analyst, Kansas Health Institute shartsig@khi.org 785-233-5443

24. Resources NACCHO CHA-CHIP resources  http://www.naccho.org/topics/infrastructure/CH AIP/upload/Final-Planning-for-CHIP-Monitoring- and-Evaluation-Participants-Slides.pdf http://www.naccho.org/topics/infrastructure/CH AIP/upload/Final-Planning-for-CHIP-Monitoring- and-Evaluation-Participants-Slides.pdf  http://www.naccho.org/topics/infrastructure/CH AIP/upload/Unlocking-the-Secrets-of-CHIP- Implementation-Final-Slides.pdf http://www.naccho.org/topics/infrastructure/CH AIP/upload/Unlocking-the-Secrets-of-CHIP- Implementation-Final-Slides.pdf KU Community Toolbox: http://ctb.ku.edu/en/table-of- contents/evaluate/evaluation/framework- for-evaluation/main http://ctb.ku.edu/en/table-of- contents/evaluate/evaluation/framework- for-evaluation/main

25. Information for policy makers. Health for Kansans. Kansas Health Institute