M & E: WHAT, WHY A ND HOW ? S. Venkatraman UIS-AIMS, UNESCO Bangkok

W HAT IS M&E? Monitoring and evaluation (M&E) of development activities provides government officials, development managers, and civil society with better means for learning from past experience, improving service delivery, planning and allocating resources, and demonstrating results as part of accountability to key stakeholders

W HAT IS M&E? It provides the information needed to answer areas of concern (needs and gaps) about the system used to determine, using reliable and valid evidence, the extent to which goals/targets have been achieved Must be demand-driven and based on information need by country Both Need-based and evidence-based Necessary process that helps decision makers/managers to manage, plan and make sound policy decisions

T HE M ONITORING The monitoring process looks at what is being done and how it is being done. It could involve continuous tracking of activities, review (measure) of the flow of services and activities provided by the program, compliance with laws, regulations, guidelines, etc. Much of the information desirable to monitor a program is also valuable for the evaluation component.

T HE E VALUATION Evaluation looks at performance against goals. This can and should take place while the program or research project is underway, and is concerned with evaluating how the intervention is meeting its performance goals. It overlaps with monitoring to the extent that it uses the data gathered during monitoring that is specifically performance related.

T OP DOWN & C OUNTRY SPECIFIC Top-down approach to M&E results in narrow perception of process and lack of commitment at country level, sub-national level resulting in questions about sustainability Country driven approach in M&E indicates commitment towards improving delivery of services and ownership of process. It is more based on the real need (contextual) & demand (problems & gaps) Having M&E or data or tools alone does not guarantee progress

M&E: T HE REAL V ALUE M&E true benefit comes when information is used at all levels & all stages resulting in progress M & E is used for measuring the quantity, quality. It looks at input, process and outputs, and the outcomes. Impact of the programme can then be assessed Information collected through M&E process - > to identify and understand ‘why’, ‘where’, ‘what’-> need and gaps of performance, assess progress and shortfalls

M&E could be carried out at project, program or sector level, Also at national, sub-national or local level

A CTIVITIES - OUTCOMES - IMPACT

I MPLEMENTING M&E Ownership of M&E is important in ensuring sustainability and progress rather than simply satisfying external need/requirement or physical deliverables Need a efficient system in place to ensure collection of relevant and appropriate information and appropriate skills built at all levels so that such information is used effectively -(eg) ‘down the drain’!

D OWN THE DRAIN ?

W ELL CONNECTED ?

Without sound M & E system in place -> difficult to manage and measure progress & achieve goals M & E system does not guarantee improvement but necessary basis to improve

INSTITUTIONALIZING M & E M & E not only to monitor progress for donor funded projects, instead should be part of regular process of trying to improve overall performance-> need to internalize Need to ensure that M & E system in place is sustainable, and information collected is relevant, timely* (SL), used in all aspects & all levels of operation; policy formulation/ revision, planning, for better transparency and accountability, and management including resource allocation

Effectiveness of Institutionalization of M & E depends on how countries internalize-> internal capacity to collect & use information Use of information depends on -> analysis and evaluation of information to identify key issues relating to progress & performance (evaluation) and determining further info need (continuous process) Current practice of M&E on techno-centric approach and on information collection -> little effort on identifying “what is the emerging, trend, the gaps & messages (stats: stories)? it conveys

I NDENTIFYING RELEVANT INDICATORS Selecting appropriate Indicators. Underpinning all M&E activities is the need to determine what it is that is being measured, and for what purpose. The identification, selection and prioritization (in some cases) of both quantitative and qualitative indicators are therefore critical before M&E can be undertaken The need for specific indicators may also evolve and change over time, as circumstances change.

I NDICATORS : Indicators are measures of inputs, processes, outputs, outcomes, and impacts for development projects, programs, or strategies. When supported with sound data collection— analysis and reporting, indicators enable managers to track progress, demonstrate results, and take corrective action to improve service delivery

W HAT CAN WE USE I NDICATORS FOR ? Setting performance targets and assessing progress toward achieving them Identifying problems via an early warning system (potential problems)to allow corrective action to be taken. Indicating whether an in-depth evaluation or review is needed

A DVANTAGE Effective means to measure progress toward objectives. Facilitates benchmarking comparisons between different organizational units, divisions/province, and over time. Poorly defined indicators are not good measures of success.

D ISADVANTAGE Tendency to define too many indicators, or those without accessible data sources or non- availability of data, making system costly, impractical, (often ‘shopping list’) and likely to be underutilized. Often a trade-off between picking the optimal or desired indicators and having to accept the indicators which can be measured using existing data

C OUNTRY DRIVEN M&E Clear Goals & targets with well-defined indication of what is/are to be achieved Indicators identified and targets set for each goal with proper tools & methodologies for collecting, processing and analysis of information Establish system to monitor (regularly) and evaluate progress made in each goal and reported Key stakeholders involved in addressing each goal (progress and planning) Identification, role clarifies to each stakeholder for collecting relevant information in relation to each indicator (eg: Early childhood)

W HY E VALUATE P ERFORMANCE ? Helps implementation, rethink strategy and re- designing programs, review existing tools (eg. expensive data collections) and resource allocations to improve program/sector’s chances of achieving target Encourages comparison among programs, coordination (eg. Literacy-> Census, NFE, MOE) and supports efficient and effective use of resources Promotes transparency and accountability in implementation and use of resources Encourages good practice in results-based management

I MPACT E VALUATION : WHAT IS IT ?  Impact evaluation is the systematic identification of the effects – positive or negative, intended or not – on individual households, institutions, and the environment caused by a given development activity  Impact evaluation helps us better understand the extent to which activities reach the poor and magnitude of their effects on people’s welfare. They can range from large-scale surveys, to small-scale rapid assessment and participatory appraisals

M AINSTREAMING M&E Realizing and internalizing the need for information at all levels of decision making Making the commitment to set up an effective sustainable system for M&E Ensuring sustainability of M & E system through capacity building, allocating necessary resources Move towards results-based management

N EVER HAVE THIS SITUATION !

T HANK Y OU ! V INAKA !