1. Monitoring Development Projects in a Messy World Guy Sharrock AEA Conference October 28 th, 2012

2. Why monitoring? Year 1Year 2Year 3Year 4Year 5 Lots going on in between ‘deep-dive’ evaluations! BaselineMonitoringImpact

3. Outline 1. Monitoring approach 2. Project illustration 3. Challenges to monitoring in a complex ecology

4. SMILER Simple Measurement of Indicators and Learning from Evidence-based Reports

5. Overview

6. Process and Products Coaching session Selected participants Draft M&E system M&E Ops Manual Documentation manager ICT4D integration

7. Malawi: Food Security Project, FY2009-14 Title II (food assisted) program Nine consortium partners Multi-year and multi-$m Multiple program areas each with multiple interventions Overlap with PEPFAR program

8. Malawi: Indicators reported to USAID/FFP Data GatheringNumber of ‘Indicator Performance Tracking Table’ indicators Surveys (baseline, endline)22 Annual Survey (collects 15 indicators common to the baseline and endline surveys) ‘Routine’ monitoring18 Total Indicators reported to USAID/FFP40

9. Malawi: Indicators used by project management Program areasIndicators for internal use only Indicators for FFP (and internal) use Indicators in the PVO indicator tracking table Maternal and Child Health and Nutrition 24327 Conservation Agriculture426 Irrigation415 Livestock505 Village Savings and Loans527 Agribusiness628 Disaster Risk Reduction033 Food Distribution213 Other ‘Cross Cutting’347 TOTAL531871

10. Malawi: overall Upside Routine Monitoring System – Data flow maps – Data gathering forms and report formats (with instructions) Monitoring tools – Data Quality Assessment – Quality Improvement Verification Checklist Assessments and evaluations – Baseline and final survey – Mid-term evaluation – Annual surveys Information and Communication Technology – Excellent database for beneficiary numbers and PVO indicator tracking table indicators – Piloting DataWinners mobile solution for monitoring HH hunger Downside Hard Covers – Missing information – Quality – Inconsistent, though some good work being done Reports – Not always verifiable – Missing information – Timeliness PVO – Filing system – Organization

11. Monitoring in a complex ecology Project monitoring impacted by… Relationships that support monitoring Relevance of monitoring Responsibilities for monitoring

12. Final thoughts In a complex operating environment… 1.Good evaluative project monitoring presents an opportunity to learn 2.Even more vital to have a coherent approach to monitoring 3.Setting-up monitoring is more than just ‘technical’ 4.Establish structures and processes to provide continuity and navigate complexity 5.Monitor the monitoring system