1. Feed the Future MEL webinar series
Introduction to the revised Feed the Future Monitoring, Evaluation, and Learning (MEL) system: More than just standard indicators!
Feed the Future MEL webinar series

2. Presenters Jessica Cagley Anne Swindale Tatiana Pulido Zachary Baquet
Julie MacCartee

3. Feed the Future MEL Webinar Series
Intro to the MEL System – April 19, 2018
Standard Indicator Overview – May 15, 2018
New Indicators 1: Application of improved practices and technologies
New Indicators 2: Yield and geospatial
New Indicators 3: Sales and investment indicators
Learning Agenda
Market Systems Measurement
Annual FTFMS users webinar

4. Survey responses

5. Objectives (and outline)
For participants to understand:
The purpose of the Feed the Future Monitoring, Evaluation, and Learning (MEL) system
Major revisions made to reflect lessons learned
The components of the MEL system
The importance of logic models in addressing multiple MEL challenges

6. The Purpose of the MEL System

7. Accountability Learning
The Global Food Security Act’s Call for Monitoring, Evaluation, and Learning (MEL)
The President shall seek to ensure that assistance to implement the Global Food Security Strategy is provided under established parameters for a rigorous accountability system to monitor and evaluate progress and impact of the strategy, including by reporting to the appropriate congressional committees and the public on an annual basis. (Sec. 6.c)
The President shall submit … reports that describe the status of the implementation of the Global Food Security Strategy …, which shall… indicate how findings from monitoring and evaluation were incorporated into program design and budget decisions; (Sec. 8.a.6)
Accountability
Learning

8. Purposes of the MEL System
Accountability
Learning
Strengthening national data systems

9. Vision of the MEL System
Our vision is that…
Quality evidence is used appropriately to
Design, manage and adapt programs to achieve Feed the Future goals and objectives,
Inform initiative strategic and budget decisions, and
Communicate that what we do matters.

10. MEL System Stakeholders
Congress
Think tanks
Advocacy Orgs
Academia
USG Staff
GAO & IGs
Implementing Partners
Leadership of USG agencies
Other donors
Private sector
Program participants
Partner governments
American taxpayers
White House & OMB
Multilateral organizations

11. Lessons Learned from Feed the Future, Phase 1

12. Revisions to the MEL System Based on Lessons Learned
1. Better capture of systems-level work 2. Linking interventions to higher level impacts 3. Improving the use of evidence generated through MEL system 4. Leveraging national data systems to measure population-level results

13. Components of the MEL System

14. MEL System Components Analysis & Learning Results Framework Monitoring
Evaluation
Learning Agenda
National data systems

15. Component 1. Results framework

16. Component 1. Results framework (p.2)

17. Component 2. Monitoring
Standard indicators: New and improved Indicator Handbook!
Types of standard indicators:
Performance
Context New!
Sources of indicator data:
Zone of Influence (ZOI) surveys: Target countries only, population-based, every 3 years, one survey per country to cover all USG contributions
Implementing partner reporting: Annual, all countries, all activities, all agencies New!
Other: Remotely sensed data, national surveys, global sources etc.
54 Performance Indicators
27 Implementing mechanism
20 ZOI
6 National
25 Context Indicators
5 ZOI
17 National
2 Recurrent crisis area/national
1 Global

18. Component 2. Monitoring (other changes)
Importance of custom indicators and disaggregates… standard indicators are not enough!
Monitoring market system changes through:
Clear and detailed logic model
Standard indicators
Custom indicators
Qualitative measures

19. Component 3. Evaluations
Performance evaluations
Impact evaluations
Rigorous counterfactual
Examines attribution
Lessons learned
Increase use of multi-activity, high-level evaluations
Use balance of evaluation methods and types

20. Component 4. Learning Agenda
Source: USAID/PPL

21. Component 4. Learning Agenda
Revised learning agenda will:
Use a variety of learning activities (e.g., research, evaluations, monitoring, assessments) to address questions
Be co-owned by technical experts
Have more frequent learning products to synthesize and disseminate evidence
Source: USAID/PPL

22. Component 5. National data systems
Piloting use of the World Bank LSMS-ISA for Feed the Future surveys
Support to the Global Rural and Agricultural Integrated Surveys (GRAInS) Partnership to harmonize national data system initiatives
Supporting pilots of the farm-based Agricultural Integrated Survey (AGRISurvey)
Strengthening national data systems

23. Component 6. Analysis and Learning
Dynamic, practical revised Learning Agenda
Increased analysis of existing data
Innovative evaluations and analysis methods
Testing technologies
Improving use of M&E data

24. Logic Models: A Crucial Tool

25. What is a logic model?
Visual representation of a theory of change, detailing the expected causal pathways linking our activities to our ultimate objectives and goals
Used to:
Design programs, projects, and activities
Track whether results and assumptions are happening as expected
Adaptively manage based on data
Tell your story!

26. Why are high-quality, detailed, and dynamic logic models needed?
To clearly articulate the hypotheses and assumptions around what is expected to happen step-by-step
To identify the appropriate data sources (standard and custom indicators + qualitative data) needed to manage programs and tell your story
To set ourselves up to have the data needed to tell a plausible story (if it exists!) connecting USG activities to changes observed at the population or systems level
We are not building widgets!

27. Logic model example Interventions/ Outputs Impacts across population
Tracked through custom indicators or qualitative methods
Tracked through standard indicator
Key:
Business development training offered to export crop processing firms
Increased organizational performance of targeted firms
Increased sales for targeted firms
Increased export of targeted commodities
Decreased national hunger
Growth in targeted crop sub-sector
Increased government revenue from targeted crops
Gov’t budgets and spends more on social safety nets
Partial loan guarantee
Increased financing accessed
Decreased national poverty
Targeted firms increase employment
PPP e-verification program for seeds, fertilizer
Stronger relationships between input wholesalers & retailers
Greater trust in quality of inputs
Reduced counterfeit inputs
Increased employment in the ag system
Increased income
Increased purchases/ investment of crop from smallholders
Decreased ZOI poverty
Greater access to inputs
Increased sales for producers
Increased incomefor producers
Decreased ZOI hunger
Targeted firms train farmer orgs
Procuers adopt key practices and technologies
Producers increase quality of produce
Producers increase ag productivity
Interventions/ Outputs
Impacts across population
Outcomes for participants or system

28. Logic model reminders Not every result necessarily needs an indicator
Think about scale and space in addition to the direction of results
Time element needs to be considered.

29. Take-home messages
Standard indicators are not enough! Use custom indicators, disaggregates, and other measures for adaptive management and telling your story.
Good MEL starts with a theory of change, ideally illustrated through a detailed, dynamic logic model tied to indicators and other measures.
Generating data without analyzing and using it is a waste of resources. Build in time and resources to analyze, learn, and apply the evidence.

30. Questions and discussion

31. Feed the Future MEL Webinar Series
Intro to the MEL System - April 19, 2018
Standard Indicator Overview – May 15, 2018
New Indicators 1: Application of improved practices and technologies
New Indicators 2: Yield and geospatial
New Indicators 3: Sales and investment indicators
Learning Agenda
Market Systems Measurement
Annual FTFMS users webinar