Recent Experiences In Monitoring and Evaluation of Nutrition Related Projects in Developing Countries: Nine Lessons Learned James Levinson Training on Comparative Review of the Nutrition Situation and Policies in Selected Countries and with particular reference to Bangladesh 27 March to 6 April 2014

Monitoring & Evaluation Lessons from Tamil Nadu Integrated Nutrition Project (TINP) and Integrated Child Development Service (ICDS) -India Bangladesh Integrated Nutrition Project Iringa Nutrition Project-Tanzania Dular project (UNICEF)-Bihar, India HKI Nutrition-Focussed Child Survival Project-Mali Enhanced Outreach Strategy-Ethiopian Child Survival Project Save the Children US, Positive Deviance -Vietnam Progresa Project -Mexico

Monitoring and Evaluation Lessons

Monitoring and Evaluation Lesson 1 Monitoring and evaluation is greatly facilitated by a Conceptual Framework and attention to Behavioral Outcomes Conceptual framework: “Map” of variables for development of M&E system

Conceptual Framework of a Simplified Nutrition Counseling Project Inputs Outputs Outcomes Impacts Benefits In-service training of Community Nutrition Workers (CNWs) Design, production, and delivery of behavioral change communication materials Assumptions CNWs understand concepts and are motivated to participate Provision of behavioral change communications to the target population Assumptions Target population understands concepts, is able, willing, and motivated to participate Targeted behaviors are a constraint to child growth. Improved childcare practices (e.g. increased duration of breast feeding, timely introduction of complementary foods, continued feeding during infection) Reduced malnutrition in children under 2 years Reduced childhood mortality and morbidity Increased long-term productivity

Conceptual Framework of a Nutrition Counseling Project: Backward Mapping Inputs Outputs Outcomes Impacts Benefits In-service training of Community Nutrition workers (CNWs) Design, production, and delivery of behavioral change communication materials Assumptions CNWs understand concepts and are motivated to participate Provision of behavioral change communications to the target population Assumptions Target population understands concepts, is able, willing, and motivated to participate Targeted behaviors are a constraint to child growth. Improved childcare practices (e.g. increased duration of breast feeding, timely introduction of complementary foods, continued feeding during infection) Reduced malnutrition in children under 2 years Reduced childhood mortality and morbidity Increased long-term productivity

Conceptual Framework of a Nutrition Counseling Project:Backward Mapping Inputs Outputs Outcomes Impacts Benefits In-service training of Community Nutrition Workers (CNWs) Design, production, and delivery of behavioral change communication materials Assumptions CNWs understand concepts and are motivated to participate Provision of behavioral change communications to the target population Assumptions Target population understands concepts, is able, willing, and motivated to participate Targeted behaviors are a constraint to child growth. Improved childcare practices (e.g. increased duration of breast feeding, timely introduction of complementary foods, continued feeding during infection.) Reduced malnutrition in children under 2 years Reduced childhood mortality and morbidity Increased long-term productivity

Conceptual Framework of a Nutrition Counseling Project:Backward Mapping Inputs Outputs Outcomes Impacts Benefits In-service training of Community Nutrition Workers (CNWs) Design, production, and delivery of behavioral change communication materials Assumptions CNWs understand concepts and are motivated to participate Provision of behavior change communication to the target population Assumptions Target population understands concepts, is able, willing, and motivated to participate Targeted behaviors are a constraint to child growth. Improved childcare practices (e.g. increased duration of breast feeding, timely introduction of complementar y foods, continued feeding during Infection) Reduced malnutrition in children under 2 years Reduced childhood mortality and morbidity Increased long-term productivity

Monitoring and Evaluation Lesson 2 “Process Evaluation” is usually a poor substitute for Project Monitoring Primary value of project monitoring : Project management Addressing implementation problems as they arise

Monitoring and Evaluation Lesson 3 Monitoring data must be used locally

Monitoring reports If trigger levels for growth monitoring coverage = 80% Village 1 Village 2 Village 3 GM coverage= 94% GM coverage= 76% GM coverage = 89%

Action Taken Village 2 Growth Monitoring Coverage =76% Village Union /Ward Office

Agenda for Next Meeting Growth Monitoring Village 2

Monitoring and Evaluation Lesson 4 Large projects require an ongoing External Monitoring and Evaluation presence

Advantages of Ongoing External Monitoring and Evaluation Presence : Easier to maintain the same survey methodology Evaluation team can develop broad understanding of project Ongoing quality checks, periodic disaggregation, and review of monitoring data possible Quickly mounted operations research informed by monitoring

Monitoring and Evaluation Lesson 5 Control groups are often inadequately comparable

Why no control groups? 1.Projects with funding limitations unconvinced of necessity of control group 2.Proliferation of NGO activity or national coverage of programs resulting in no true control areas

3.Selection of Project Areas Based on Political Grounds

Why use control groups? Allows assessment of extent to which change observed can be attributed to project

Random selection of project areas

Monitoring and Evaluation Lesson 6 There needs to be Baseline and Evaluation Data collected for each Project Objective specified

Monitoring and Evaluation Lesson 7 We need to evaluate the Distribution of Benefits in projects

Are the poor/vulnerable benefiting ? Resources

Monitoring and Evaluation Lesson 8 Measuring cost effectiveness is essential Examples Cost per child removed from moderate and severe malnutrition categories Cost per case of child stunting averted Cost per child death averted Cost per 100 gram increase in birth weight Cost per 1 gram/litre increase in haemoglobin concentration

Monitoring and Evaluation Lesson 9 Exit Strategies and Sustainability also need to be evaluated

What happens when the project terminates?