1. Carlo Azzarri, Beliyou Haile, Cleophelia Roberts, Melanie Bacou IFPRI/HarvestChoice London, July 17, 2014 M&E of Africa RISING

2.  Africa RISING M&E Objectives and Challenges  ARBES  Project Mapping and Monitoring Tool (PMMT) −Purpose and scope −Major components  Next Steps Agenda 2

3. Africa RISING M&E Objectives and Challenges 3

4.  Support effective project management  Fulfill FtF reporting requirements  Generate knowledge on what works and what doesn’t work  Data/analytical components of the M&E system: −Delineation and characterization of target farming systems −Inventory of technologies −Ex-ante evaluation* −Attribution assessment* M&E of Africa RISING Learning about climbing beans in Linthipe, Dedza 2014.

5. 5  Mega-site stratification by drivers of SI and creation of “development domains”  Identification of representative action research and control sites  Open access M&E data management platform (PMMT)  FtF indicators reporting  Household and community surveys (ARBES)  Site visits and focus groups  Meetings (survey design, project review & planning, annual M&E) M&E Activities Closely watching the fava beans, Sinana, Oct. 2013.

6.  Compiling and verifying data on FtF indicators (suitable level of detail?)  Compiling and verifying data on AR beneficiaries  Sequencing: selection of action sites before setting up specific research activities  Engagement and communication (e.g. lengthy recruitment of local M&E coordinators)  Delays in baseline surveys  Sustainability and its dimensions: indicators and benchmarks? A few M&E Challenges… 6

7. Evaluation questions: Did the program contribute to program goal and objectives? Did it increase agricultural productivity? Are the program’s approaches replicable/scalable? Did farmers adopt new/improved technology(s) and practices? What are the tradeoffs among different impacts? How does impact vary across different domains, geographies, household types, and gender? Are the program-induced improvements sustainable? Did the program demonstrate implementation methods that other projects can use? Impact Evaluation (IE) 7

8.  Practicality of / approaches to attribution of cause and effect for an AR4D program  (Random) selection of action sites and/or subjects  Conflicting incentives among actors  Attribution with small N (action sites, beneficiary households) – generalizability of results  Impact of a “bundle of interventions”  Confounding factors  Moving targets A few E Challenges… 8

9. Africa RISING Baseline Evaluation Surveys (ARBES) 9

10.  Malawi 2013; Tanzania, Ghana, Mali, and Ethiopia 2014 ARBES. In 2013 we started an ambitious workplan on conducting household and community surveys in five of the six AR countries in less than a year time (mega-site representative; N=700-1300; 50-page Qx).50-page Qx  AR Evaluation. We would like to assess the overall impact of the program on different outcomes (poverty, nutrition, health,…) using this approach. this approach  Babati (Tanzania) impact evaluation. The team will continue and refine the impact assessment research project in Babati. Comments welcome! Data and Analysis

11. Feb 2013 July 2013 Aug 2013 Feb 2014 Feb 2016 Initial planting at demonstration plots Follow-up field day: farmers rank preferred seeds Randomization 996 farmers in 25 villages 205 receive fertilizers and improved seeds 210 receive AR intervention End-line survey: measure impacts 105 farmers non- beneficiaries elsewhere 270 controls 206 Babati farmers non- beneficiaries Tanzania IE timeline and design

12. Project Mapping and Monitoring Tool 12 Scope and Purpose Major Components http://dev.harvestchoice.org/africarising/

13. 13 Evaluation is hard. Monitoring is painful… Aligning project goals, 2012.

14. 14  MSExcel revisions no more!  Grow organically based on evolving M&E needs and partners’ feedback and capacity – do not overbuild  Flexible, adapted to most M&E designs  Simple to deploy in the field, no strong dependence on high- speed Internet connectivity  Leverage existing CGIAR tools and repositories  Openness (to and from 3 rd party applications and databases) PMMT Design Principles Interviewing a farmer, 2013.

15. M&E site stratification & selection. Powerful spatial visualization features to provide rich contextual information, and overlays of local biophysical characteristics with socio- economic data in support of action/control site selection process. 15 Powerful Spatial Visualization

16. Streamlined indicator data entry and reporting. Familiar, secure interface for data entry with pre- set list of indicators help clarify reporting requirements. Data import/export tools for multiple end-uses. 16 Simplified Project Performance Monitoring

17. 17  Series of in-country M&E and PMMT training planned through October 2014.  On-line PMMT User GuidePMMT User Guide  Video tutorial Video tutorial Additional Training Resources Interviewing a farmer, 2013.

18. Next Steps 18

19. M&E Roadmap 19

20. The End Thanks! 20

21. ARBES Qx household summary contents TLocation info, GPS TRoster nEducation nLabor nHealth nWomen and child anthropometrics  Agriculture -general-  Crop inputs (Conservation Agr.)  Crop production  Crop inputs (costs)  Crop inputs (labor time use)  Crop inputs (seeds)  Crop sales  Crop storage  Livestock ownership and income  Livestock feed SECOND VISIT  Problems and coping strategies  Agricultural extension and AR program  Other income  Credit  Housing, utilities, assets, distance to services  Subject welfare and food security  Food Expenditures / Consumption  Non-Food Expenditures  Shocks  Re-contact info FIRST VISIT Head Individual Best Informed

22. ARBES Qx community summary contents (5 to 8 informants)  Location info, GPS  Informants’ roster  Access to basic services  Agricultural labor, extension services, agricultural problems  Land use  Demographics, cooperatives, migration  Water access, shocks, food consumption  Market prices  Conversion of non-standard units

23. IE Design Beneficiary HHs Non-beneficiary HHsControl HHs Action Sites Control Sites Spillover effects Program impact