Monitoring and Evaluation for the result oriented management of projects & programmes Land Governance Capitalisation Meeting Addis Ababa, 10 November 2017 Karen McHugh M&E adviser FANSSA Unit C1 DEVCO

The whole political context is pushing for a focus on results Global priorities/SDGs EU priorities & rules EURF Citizens and tax payers

Why do we need indicators/monitoring? To: Improve quality of actions by providing key data to inform management decision making (on/off track) Learn, by identifying what works and what doesn’t Be accountable, by demonstrating in an objective way how public funds have been used and what they have achieved and Bring clarity to the dialogue between project partners

The LFM A Resilient and Versatile Tool

Uses of the LFM Synthesis Planning Communication Monitoring Evaluation… …i.e. your road map! Where are you going? How will you get there? How will you know that you have arrived?

The Logical Framework Matrix (LFM) Results chain Indicators Baselines Targets Sources of verification Assumptions Impact (Overall objective) 1 3 2 Outcome(s) (Specific objective) Outputs Activities “Nested LFMs” how project LFMs fit into a Programme LFM

Indicators, baseline & target Sources of verification Vertical Logic Indicators, baseline & target Results chain Sources of verification Assumptions Impact Outcome Outputs Activities « For each level of the results chain, if results are achieved and assumptions hold true, then ... » 13

Indicators, baseline & targets Sources of verification Horizontal logic Results chain Indicators, baseline & targets Sources of verification Assumptions Impact Outcome Outputs Activities How to verify ‘success’ or progress in achieving results?

In a logframe every result needs What is to be measured (not what is to be achieved; that’s the job of targets) Indicators The desired value or direction for progress Targets On the path towards your target Milestones The starting point Crucial for target setting Baselines Where the information/data comes from Sources

How to measure change? Set the baseline Set the target Set milestones Step 1 Set the baseline Step 2 Set the target Step 3 Set milestones Golden RULE 2: BE REALISTIC

Defining indicators – Focus on what will be measured not by how much Long term goal to which the project contributes e.g. SDG indicators, EURF level 1 e.g. prevalence of stunting (moderate or severe) of children under five years of age) Impact What will change for the beneficiaries/target groups – behaviour/ perceptions/ systems e.g. Proportion of total adult population with secure tenure rights to land, with legally recognised documentation Outcome The services or goods delivered to the target groups e.g. # of seedlings/tools provided to target group/level of awareness of TP of land rights Output

OBJECTIVES/RESULTS OR INDICATORS? Improved levels of nutrition All large-scale agricultural investments meet principles of VGGT and RAI by 2017 Increasing productivity Prevalence of stunting in children under 5 years of age Number (%) of new large-scale investments meeting VGGT and RAI principles Rice yields per hectare

Rule No. 1: BE CLEAR What indicators might measure these results: “Reduce poverty by transforming agriculture into a competitive, sustainable, and inclusive agriculture that brings economic growth, improved livelihoods and job creation, food and nutrition security” “Increased efficiency and effectiveness of research and extension systems, reaching more to smaller farmers and leading to a more diversified production”

Good indicators depend on clearly stated objectives/results on all levels: OUTCOME level Not so good  Better  Improved strategic linkages for food security and nutrition, and support to local market sales in each State in order to improve diets of local population Increased sales of locally produced vegetables

Good & not so good objective/result statements: OUTPUT level Not so good  Better  Increase the number of HH who have increased their production levels thanks to the provision of water harvesting and/or irrigation systems Increased area of arable land under irrigation

Possible indicators outcome and output levels Increased sales of locally produced vegetables Volume (or value) of locally produced vegetables sold Increased area of arable land under irrigation Area of arable land under irrigation

Sources of verification Routine statistics Administrative records Management reports Special survey Technical or specialised reports Observation / Site inspection The source of verification should be explicit! A line Ministry is not a source of verification, it is rather the survey conducted by a given Directorate/Unit within this Ministry…

Data Sources List the specific data sources (do not just list the organisation, mention also the specific data collection method e.g. survey or report) Frequency should be clear and consistent with milestones and targets Provide disaggregated data as required Data collection and reporting responsibilities are clearly specified In your LFM: Avoid « project report » in the source of verification column !

Exercise on objectives, indicators & sources of verification exercicse Exercise on objectives, indicators & sources of verification Take a card and write down a result (output, outcome or impact). Pass to your colleague on the left. On the card you received from your neighbour, write down an indicator you think could be used to measure this result. Pass to your colleague on the left. On the card you received write down a source of verification you think could be used to collect the information related to this result and indicator. Pass to your colleague on the left. Read the objectives, indicator and proposed source of verification you have received and decide if you agree with the choices made.

With indicators we can compare Baseline situation vs. actual Actual vs. planned Trends over time Comparisons between periods Geographical variability Variability between groups Standards and benchmarks

"In an increasingly performance-oriented society, metrics matter "In an increasingly performance-oriented society, metrics matter. What we measure affects what we do. If we have the wrong metrics, we will strive for the wrong things." J.E. Stiglitz, A. Sen, J.P. Fitoussi, "Mis-Measuring our Lives Monitoring systems

Steps for designing a monitoring system Define the objective and scope of the monitoring system (always keep end users in mind- not information for information sake) Identify performance-related questions, information needs and indicators (drawn from your LFM) Plan for data collection and the organisation of information Plan the activities and processes needed for an efficient linkage between information and decision making (with a view to improving and managing the project/programme) Plan the information flow and reporting process (how and to whom?) Plan for the necessary means and skills (budet for M&E) Combination of stakeholders, approaches, methods, tools, indicators… M & E system

Monitoring system Key messages Focus on outcomes and not just outputs to assess the project/ programme performance Build on existing systems and capacities Identify indicators and the source of information at the same time (reality check). Keep it simple and realistic!! When it comes to indicators less is better Involve responsible managers & stakeholders Balance quantitative and qualitative indicators

Monitoring and Evaluation Strategic Evaluations Project & programme evaluations Development Partner monitoring (Field and HQ) Internal project monitoring by implementing partners

Q & A