1. Evaluation Approaches, Frameworks & Designs HSC 489 – Chapter 14

2. Approaches to Evaluation Each approach: Represents a certain way of thinking Which defines the evaluation questions to ask

3. Frameworks & Designs Evaluation Framework: The skeleton of a plan that can be used to conduct an evaluation (steps to be followed) Evaluation Design: Used to organize the evaluation Provides for planned, systemic data collection, analysis, and reporting

4. Evaluation Approaches Systems Analysis Objective-Oriented Goal–Free Management-Oriented Others

5. Systems Analysis Approach Determines most effective programs Focuses on organization of resources to goal and non-goal activities Economic evaluations Cost identification Cost-benefit Cost-effectiveness Cost utility

6. Objective Oriented Approach Data is collected to determine if the predetermined goals have been met Success or failure is based on if goals are met or not Based on behaviors participants should exhibit at end of intervention

7. Goal-Free Approach Evaluator searches for outcomes and is unaware of program goals Evaluator presents a judgment of what actually happens in the program Mostly qualitative methods, interviews (staff/clients), visits program sites Most useful in combination with other approaches

8. CDC Evaluation Frameworks for Public Health Programs 1.Engage the Stakeholders 2.Describe the Program 3.Focus the evaluation design 4.Gathering credible evidence 5.Justifying conclusions 6.Ensuring use & shared lessons learned

9. Standards from Framework Utility – information needs of eval users are satisfied Feasibility – eval is viable & pragmatic Propriety – eval is ethical Accuracy – eval produces findings that are considered correct

10. Selecting an Evaluation Design 1: Orientation to the situation 2: Defining the problem 3: Basic design decision 4: Plans for: Measurement, data collection, analysis & reporting of results