Download presentation
Presentation is loading. Please wait.
Published byAnnice Stone Modified over 9 years ago
1
Simon Hearn, ODI, London Part 2 of 8 AEA Coffee Break Webinars 2013 DEFINE what is to be evaluated
2
The Rainbow Framework
3
Why do we need to start with a clear definition? Source: Hobbies on a Budget / Flickr
4
Three evaluation tasks in DEFINING Develop initial description Develop program theory or logic model Identify potential unintended results
5
Develop initial description 1.
6
Thumbnail description Vision & Mission Peak Experience Aims & Objectives
7
Develop program theory or logic model 2.
8
Options for representing logic models Pipeline / results chainLogical frameworkOutcomes hierarchy / theory of changeRealist Matrix
9
Options for representing logic models Pipeline / results chainLogical frameworkOutcomes hierarchy / theory of changeRealist Matrix
10
INPUTS Apples People at risk of poor health IMMEDIATE RESULTS Apples eaten SHORT-TERM RESULTS Improved nutritional status LONGER-TERM RESULTS Improved health
11
Options for representing logic models Pipeline / results chainLogical frameworkOutcomes hierarchy / theory of changeRealist Matrix
12
RESULTS AREAINDICATOR MEANS OF VERIFICATION ASSUMPTIONS/ RISKS GOAL: Improved health Decreased sick days School and work records Good status largely due to nutrition PURPOSE: Improved nutritional status Adequate levels of Vitamin C Sample blood tests Apples retain Vitamin C OUTPUT: Apples eaten Apples replaced by apple cores Visual inspection Apples eaten on location ACTIVITIES: Deliver apples Required numbers delivered Delivery receipts ACTIVITIES: Buy apples Required numbers bought Purchase receipts Visual inspection
13
Options for representing logic models Pipeline / results chainLogical frameworkOutcomes hierarchy / theory of changeRealist Matrix
14
Outcomes chain showing possible alternative causal paths Improved health Improved vitamin C levels Apples eaten (whole or juice) Apples delivered to schools Improved quercetin levels Red apple skin eaten Decreased Body Mass Index Apples eaten instead of junk food snacks
15
Options for representing logic models Pipeline / results chainLogical frameworkOutcomes hierarchy / theory of changeRealist Matrix
16
ContextMechanismOutcome Healthy people at high risk of gastric cancer Quercetin from red apple skin (or red onion) protect cells from free radicals Reduced risk of cancer Smokers or people with regular exposure to radiation Chemicals in cigarette smoke or radiation damages DNA in healthy cells Continued increased risk of cancer Individuals with vitamin deficiency through malnutrition Sufficient vitamin C absorbed when juice from apples (or oranges) consumed Stronger immune system Individuals with vitamin deficiency through excessive alcohol Alcohol destroys vitamin CNo change in vitamin C level Individuals with obesity due to excessive snacking on high fat, high calorie foods Decreased calorie intake due to substitution of apples (or carrot sticks) Reduced obesity and related conditions Individuals with obesity due to genetic condition Decrease sugar intake through substitution of apples No change in obesity but lower risk of diabetes
17
ContextMechanismOutcome Healthy people at high risk of gastric cancer Quercetin from red apple skin (or red onion) protect cells from free radicals Reduced risk of cancer Smokers or people with regular exposure to radiation Chemicals in cigarette smoke or radiation damages DNA in healthy cells Continued increased risk of cancer Individuals with vitamin deficiency through malnutrition Sufficient vitamin C absorbed when juice from apples (or oranges) consumed Stronger immune system Individuals with vitamin deficiency through excessive alcohol Alcohol destroys vitamin CNo change in vitamin C level Individuals with obesity due to excessive snacking on high fat, high calorie foods Decreased calorie intake due to substitution of apples (or carrot sticks) Reduced obesity and related conditions Individuals with obesity due to genetic condition Decrease sugar intake through substitution of apples No change in obesity but lower risk of diabetes
19
http://betterevaluation.org/resource/example/rubrics-oakden
20
Identify potential unintended or negative results 3. Source: www.smarttoolkit.net
21
Negative program theory Risk assessment Key informant interviews Six Hats Thinking
22
Defining – Framing iteration Initial description Discuss evaluation purpose Refine description Define evaluation questions Develop logic model Agree scope of evaluation DEFINE FRAME
23
Simon Hearn Research Fellow, ODI s.hearn@odi.org.uk http://betterevaluation.org/plan/define facebook.com/betterevaluation @bettereval
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.