1. Dancing Your Way through Program Evaluation with the CDC 6-Step Evaluation Framework
MaryCatherine Jones, MPH, Public Health Consultant, CVH Team
Zarina Fershteyn, MPH, Director of Program Evaluation, NACDD

2. Learning Objectives
To identify the steps associated with the CDC Framework for Evaluation
To discuss the intersection between logic models and the framework
To engage peers in discussing common roadblocks and solutions in evaluating chronic disease and health promotion programs

3. Course Overview
What we will accomplish

4. Getting Started with Evaluation
Common types of evaluation in public health
Implementation
Effectiveness
Efficiency
Cost Effectiveness
Attribution
Others

5. What is the value in evaluation?
Why do we evaluate public health programs?
Monitor progress toward goals
Determine whether program is producing desired progress on outcomes
Permit comparisons across groups
Justify need for funding/support
Improve program quality
Ensure effective allocation of resources

6. What is the CDC 6 Step Evaluation Framework?
Program evaluation is 1 of 10 essential public health services. CDC published this framework in 1999 to help public health programs produce results that get used. Key concepts behind the framework include knowing your audience for your evaluation results and considering their needs in developing your evaluation plan. In other words:
Who needs the evaluation results (utility)?
What evaluation activities are realistic given our resources and opportunities? (feasibility)
Are the key people affected by the program involved in its evaluation (propriety)
Will the evaluation results be valid and reliable—can our users TRUST them (accuracy)?

7. Your Evaluation Team
An evaluation team ensures that you have a variety of skills and perspectives that will make your evaluation useable and ensure that your results drive action.
Evaluation team members can include:
Program staff
Partners
Funders
Policymakers
Community members
Subject matter experts
Others

8. Your Lead Evaluator Able to work with a variety of stakeholders
Able to balance potentially competing needs of stakeholders
Able to prioritize and deliver in the face of limited resources
Able to innovate

9. Why Are Stakeholders Important?
Who are typical stakeholders?
-Those involved in program operations
-Those served/affected by the program
-The intended users of the evaluation findings—able to take action on the results
Stakeholders can help (or hinder) an evaluation before it is conducted, while it is being conducted, and after the results are collected and ready for use. Because so many public health efforts are complex and because public health agencies may be several layers removed from frontline implementation, stakeholders take on particular importance in ensuring that the right evaluation questions are identified and that evaluation results will be used to make a difference. Stakeholders are much more likely to support the evaluation and act on the results and recommendations if they are involved in the evaluation process. Conversely, without stakeholder support, your evaluation may be ignored, criticized, resisted, or even sabotaged. In addition, to be proper/ethical and accurate, you need to include those who participate in the program and are affected by the program or its evaluation.
Shortly, we’ll be sharing a worksheet with you to practice identifying stakeholders and understanding their investment in your evaluation results. You will use the worksheet today during our small group activity, but we also encourage you to take it home and try it with your program.

10. Stakeholder Roles
Evaluation Team (planning and implementing the evaluation)
Dissemination of evaluation findings
Contributing data
Analyzing and drawing conclusions from data

11. Considerations for Stakeholder Engagement
Standard
Questions
Utility
Who will use these results?
Feasibility
How much time and effort can we expect from the stakeholders? How much time/effort do we have to engage them?
Propriety
Which stakeholders can help our evaluation to be ethical?
Accuracy
How broadly do we need to engage stakeholders to paint an accurate picture of the program?

12. Describing the Program

13. Considerations for Your Description
Need
Target Population
Outcomes
Activities
Outputs
Resources/Inputs
Relationship of activities and outcomes
Stage of development
Context/environmental factors

14. Sample Logic Model Format
Investment
Outcomes
Inputs
Activities
Outputs
Short-Term
Intermediate Term
Long Term

15. Considerations for Your Program Description
Standard
Questions
Utility
Is the level of detail appropriate for the intended use?
Feasibility
Are the activities and at least some of the outcomes within the program’s control?
Propriety
Does the description include enough detail to examine strengths and weaknesses of the program?
Accuracy
Is the program description comprehensive?
Are likely influences on the program identified?

16. Case Study and Small Group Activity?

17. Group Discussion

18. Focusing the Evaluation Design
Types of Evaluation
Implementation/Process
Effectiveness/Outcome

19. Focusing the Evaluation Design
Considerations for Design
WHO
HOW
WHAT (we can reasonably evaluate)

20. Big Details for Your Evaluation Plan
Availability of data
Timing of data collection
Data analysis (who, when, how)
Types of conclusions we can make
By using a logic model in our evaluation planning, we can focus on these activities, outputs, and outcomes to develop measurements

21. Case Study (Continued)?
With your team, complete Worksheets 3A and 3B
You have 15 minutes for this activity

22. Overcoming Trouble Spots
Umm…Sorry, this made sense until I started explaining it.
What are areas that could potentially be tricky for an evaluator to navigate? (list all options)
Group vote: Select 2 of the areas identified as trouble spots
Group discussion on how to prevent, manage, or otherwise address these.

23. Summary
Worksheets adapted from U.S. Department of Health and Human Services Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation. Introduction to program evaluation for public health programs: A self-study guide. Atlanta, GA: Centers for Disease Control and Prevention, 2011.
Visit the self-study guide for worksheets related to the other Steps!

24. Our Challenge to You

25. Our contact information
MaryCatherine Jones,
Zarina Fershteyn,