Download presentation
Presentation is loading. Please wait.
1
Theory of change Francis Rathinam, Evaluation Specialist
Stuti Tripathi, Senior Policy and Evidence Uptake Officer 16 June 2016
2
What is a theory of change?
Maps out how an intervention is supposed to deliver the desired results Spells out the causal links between inputs, processes, outputs and intermediate and final outcomes, Identifies the underlying assumptions. Provides the causal logic of how and why a particular project, program, or policy will reach its intended outcomes in a given context Spells out the conditions and assumptions needed for the change to take place Activities, procedures, people have to be in place, and in what sequence Resources are required for implementation – and are available? Data are required for M&E – and are available? Roadmap on how inputs, activities bring about desired outputs, outcomes Includes logframe or impact table components, with intermediate steps Inputs, activities, outputs, short-term outcomes, long-term outcomes… Includes assumptions on why implementing agents will do their job Includes assumptions on why intended beneficiaries will sign-up, stay in programme Includes assumptions on context – what will facilitate or impede change? Includes assumptions on time horizon over which outcomes realised
3
Example ToC for school feeding
Nutritious meals prepared and served in correct portions Children eat the meals Children more attentive Better learning outcomes Higher attendance Assumptions Food available in sufficient quantity and quality at right time Other inputs (e.g. teachers) are present
4
Steps in building theory of change
5
Clearly define the intervention
Step One Clearly define the intervention
6
Define the intervention: school feeding
Which meal? How is ingredients to be procured? What is planned to ensure parents know about it? Do guidelines for cooking the meal exist? What training will be provided to those preparing food? Express
7
Map out Steps Two input, activities, outputs, outcomes
the causal links assumptions underpinning these links
8
Identify project school
Theory of change Identify and train trainers Better learning outcomes Higher attendance Children more attentive Staff Money Identify project school Develop training and guidance materials Kitchens equipped Set up procurement, Production and distribution systems Foodstuffs produced, procured or received Nutritious meals prepared and served in correct portions Trained school- level staff in place Children eat the meals Assumptions Equipment is distributed and usable Food available in sufficient quantity Data available for school selection Those trained understand tasks Parents are aware of school feeding and value it Food used for school meals and properly prepared Meals are palatable Other inputs (e.g. teachers) are present Funds available on time Appropriately Qualified and Motivated Project staff
9
What to watch out for?
10
A ‘silo’ theory of change for school meals
Inputs Money Staff Activities Identify schools Provide guidance and training Set up procurement and distribution Distribute cooking equipment Distribute food stuffs Outputs Staff equipped to prepare meals School-based facilities for meal preparation Nutritious meals prepared with correct portions Children eat the meals Outcomes Higher attendance Children more attentive in class Better learning outcomes Impact Higher productivity Better labour market outcomes Better life
11
Looking out for the unanticipated
Better learning outcomes Higher attendance Children more attentive Nutritious meals prepared and served in correct portions Children eat the meals
12
Lesser time spent in classroom
Looking out for the unanticipated Lesser time spent in classroom Better learning outcomes Higher attendance Children more attentive Nutritious meals prepared and served in correct portions Children eat the meals
13
Looking out for the unanticipated
Lesser time spent in classroom Lower learning outcomes Better learning outcomes Higher attendance Children more attentive Nutritious meals prepared and served in correct portions Children eat the meals
14
Add a temporal dimension
Step three Add a temporal dimension Define project time line accurately and realistically Think through time required to observe change in outcomes that the programme cares about
15
School feeding programme impact timeline
0-3 months 0-6months 6-36 months Identification of schools Training Procurement Provision of midday meals 12-24 months 18-36months 24-36months School enrolment School attendance Better health outcomes Learning outcomes
16
Identify the evaluation questions
Step Four Identify the evaluation questions Steps: Interrogate the casual chain Potential weak links What to do differently to improve programme effectiveness What kind of evidence is best suited to answer those questions Factual Counterfactual How best to measure them?
17
Factual questions or process evaluation
Staff Identify projects school Kitchens equipped Nutritious meals prepared and served in correct portions Set up procurement, production and distribution systems Foodstuffs produced, procured or received Develop training and guidance materials Trained school- level staff in place Identify and train trainers Money Factual Assumptions Equipment is distributed and usable Food used for school meals and properly prepared Parents are aware of school feeding and value it Other inputs (e.g. teachers) are present Funds available on time Data available for school selection Those trained understand tasks Food available in sufficient quantity and quality at right time Appropriately Qualified and Motivated Project staff
18
Factual questions or process evaluation
Staff Money Identify projects school Identify and train trainers Develop training and guidance materials Kitchens equipped Set up procurement, production and distribution systems Foodstuffs produced, procured or received Nutritious meals prepared and served in correct portions Trained school- level staff in place Assumptions Funds available on time Appropriately Qualified and Motivated Project staff Data available for school selection Those trained understand tasks Equipment is distributed and usable Food available in sufficient quantity and quality at right time Food used for school meals and properly prepared Parents are aware of feeding and value it Other inputs (e.g. teachers) are present Factual
19
Factual questions or process evaluation
Staff Money Identify projects school Identify and train trainers Develop training and guidance materials Kitchens equipped Set up procurement, production and distribution systems Foodstuffs produced, procured or received Nutritious meals prepared and served in correct portions Trained school- level staff in place Assumptions Funds available on time Appropriately Qualified and Motivated Project staff Data available for school selection Those trained understand tasks Equipment is distributed and usable Food available in sufficient quantity and quality at right time Food used for school meals and properly prepared Parents are aware of feeding and value it Other inputs (e.g. teachers) are present Factual Written or verbal test of representative sample of those trained
20
Factual questions or process evaluation
Staff Money Identify projects school Identify and train trainers Develop training and guidance materials Kitchens equipped Set up procurement, production and distribution systems Foodstuffs produced, procured or received Nutritious meals prepared and served in correct portions Trained school- level staff in place Assumptions Funds available on time Appropriately Qualified and Motivated Project staff Data available for school selection Those trained understand tasks Equipment is distributed and usable Food available in sufficient quantity and quality at right time Food used for school meals and properly prepared Parents are aware of feeding and value it Other inputs (e.g. teachers) are present Factual
21
Nutritional analysis of school meals
Factual questions or process evaluation Staff Money Identify projects school Identify and train trainers Develop training and guidance materials Kitchens equipped Set up procurement, production and distribution systems Foodstuffs produced, procured or received Nutritious meals prepared and served in correct portions Trained school- level staff in place Assumptions Funds available on time Appropriately Qualified and Motivated Project staff Data available for school selection Those trained understand tasks Equipment is distributed and usable Food available in sufficient quantity and quality at right time Food used for school meals and properly prepared Parents are aware of feeding and value it Other inputs (e.g. teachers) are present Factual Nutritional analysis of school meals
22
Nutritional analysis of school meals
Factual questions or process evaluation Staff Money Identify projects school Identify and train trainers Develop training and guidance materials Kitchens equipped Set up procurement, production and distribution systems Foodstuffs produced, procured or received Nutritious meals prepared and served in correct portions Trained school- level staff in place Assumptions Funds available on time Appropriately Qualified and Motivated Project staff Data available for school selection Those trained understand tasks Equipment is distributed and usable Food available in sufficient quantity and quality at right time Food used for school meals and properly prepared Parents are aware of feeding and value it Other inputs (e.g. teachers) are present Factual For example, India midday meals programme: As per guidelines set by the Union Government, the midday meal programme for primary students must have 450 Kcal and 12 gm of protein. However, in a survey conducted from May 13, 2014, to July 22, 2015, 51 meals were tested and 27 of the samples had less than the required 12 gm of protein. Nutritional analysis of school meals
23
Process evaluation: What it can help us learn
Impact evaluation of inventory credit and storage support provided to small-scale palm oil producers Farmers benefit from inter-seasonal fluctuations in agricultural prices
24
The intervention did not have an impact
Process evaluation: What it can help us learn Impact evaluation of inventory credit and storage support provided to small scale palm oil producers Farmers benefit from inter-seasonal fluctuations in agricultural prices The intervention did not have an impact
25
Interventions fall at the first hurdle
Process evaluation: What it can help us learn Interventions fall at the first hurdle Take-up of the intervention was quite low. 30 per cent for storage support and 25 per cent for inventory credit. Farmers benefit from inter-seasonal fluctuations in agricultural prices Focus group discussions showed take-up was low due to high interest rates on the loans lack of info on price fluctuations Lack of trust in bank officials Programme disturbed pre-existing relationship with traders
26
Starting point, an over simplistic theory
Communities provided with community storage containers Increased revenue for farmers benefiting from seasonal price fluctuations Inventory credit loans
27
Developing the theory a bit…
Project provides storage space and containers Project increases liquidity through loans Palm-oil farmers store oil and use the loan Palm-oil farmers don’t sell during peak times Farmers sell during off season and make profits
28
Developing the theory a bit…
Project provides storage space and containers Project increases liquidity through loans Palm-oil farmers store oil and use the loan Palm-oil farmers don’t sell during peak times Farmers sell during off season and make profits Containers are cheap, interest rates are low
29
Developing the theory a bit…
Project provides storage space and containers Project increases liquidity through loans Palm-oil farmers store oil and use the loan Palm-oil farmers don’t sell during peak times Farmers sell during off season and make profits Containers are cheap, interest rates are low Farmers trust the bank, other key players support
30
Developing the theory a bit…
Project provides storage space and containers Project increases liquidity through loans Palm-oil farmers store oil and use the loan Palm-oil farmers don’t sell during peak times Farmers sell during off season and make profits Containers are cheap, interest rates are low Farmers trust the bank, other key players support Stored palm oil easy to access and use
31
What is envisioned What actually happens INPUTS OUTPUTS OUTCOMES
IMPACT What actually happens INPUTS OUTPUTS OUTCOMES IMPACT
32
The funnel of attrition
33
Counterfactual question or impact evaluation
Higher attendance Children more attentive Better learning outcomes Staff Identify projects school Kitchens equipped Nutritious meals prepared and served in correct portions Set up procurement, production and distribution systems Foodstuffs produced, procured or received Develop training and guidance materials Trained school- level staff in place Identify and train trainers Money Factual Counterfactual Assumptions Data available for school selection Equipment is distributed and usable Food used for school meals and properly prepared Parents are aware of school feeding and value it Other inputs (e.g. teachers) are present Funds available on time Those trained understand tasks Food available in sufficient quantity and quality at right time Appropriately Qualified and Motivated Project staff
34
Counterfactual question or impact evaluation
Staff Identify projects school Kitchens equipped Nutritious meals prepared and served in correct portions Higher attendance Better learning outcomes Set up procurement, production and distribution systems Foodstuffs produced, procured or received Children more attentive Develop training and guidance materials Trained school- level staff in place Identify and train trainers Money Factual Counterfactual Assumptions Funds available on time Data available for school selection Equipment is distributed and usable Food used for school meals and properly prepared Parents are aware of school feeding and value it Other inputs (e.g. teachers) are present Those trained understand tasks Food available in sufficient quantity and quality at right time Appropriately Qualified and Motivated Project staff
35
Second generation questions
Staff Money Identify projects school Identify and train trainers Develop training and guidance materials Kitchens equipped Set up procurement, production and distribution systems Foodstuffs produced, procured or received Nutritious meals prepared and served in correct portions Trained school- level staff in place Higher attendance Children more attentive Better learning outcomes Assumptions Funds available on time Appropriately Qualified and Motivated Project staff Data available for school selection Those trained understand tasks Equipment is distributed and usable Food available in sufficient quantity and quality at right time Food used for school meals and properly prepared Parents are aware of feeding and value it Other inputs (e.g. teachers) are present Can we be thinking of take home rations instead?
36
Step five Validate and revise (theory of change walk)
Consult programme managers, intended beneficiaries
37
Sketch out the theory of change for your intervention
Exercise (20 mins) Sketch out the theory of change for your intervention Define intervention Lay out main steps in causal chain, identifying underlying assumptions Add a temporal dimension Identify key evaluation questions
Similar presentations
