Monitoring & Evaluation System “Learning to Improve” Making evidence work for development
Contents Rationale of the M&E Users and needs Principles of the M&E Units of analysis and dimensions of study Elements of the M&E Products of the M&E M&E share of responsibilities M&E constraints M&E linkages: KM, decision making and learning
Rationale: Why should the MDGF have a M&E system? It is a requirement included in the legal agreement between the Donor (Spain) and UNDP It is an obligation, included in all signed joint programs It is necessary: if we want, scaling up programs into policies and spread solutions to achieve MDGs at Global level It is useful: As it is part of the program managing cycle and is the best way to measure progress, detect problems, correct them, improve performance and learn at local and global level
Who are the users of the M&E system what are their needs in terms of Information?
For 5 minutes discussion: Are we missing anyone? Are most of the views of stakeholders included? This is an attempt to let your tacit knowledge flow
What kind of principles should the MDGF’s M&E system incorporate? PRINCIPLES
What kind of principles should the MDGF’s M&E system incorporate? Accordance with UNEG and DAC/OECD standards Oriented to well balanced learning and accountability purposes Evidence-based: consistent data, information or knowledge to support judgments and conclusions of monitoring and evaluation
What kind of principles should the MDGF’s M&E system incorporate? Built on an aggregation scheme: Elements of M&E (indicators + evaluations, etc) in lower levels add up in higher level of inquire Measure (DELTA), describe, analyze understand the object of study (JP+C+W+MDGs) and use results to improve program and policy performance
What dimensions should the MDGF M&E system cover? What questions should it answer?
Joint ProgramsCountries 1 st M&E Level Joint programs 2 nd M&E Level Countries Environment and Climate Change Gender Equality Women’s empowerment Culture & development Economic democratic Governance Youth, Employment and Migration Conflict prevention & Peace Building Children Food Security and Nutrition Development and the Private Sector 3th M&E Level Windows Monitoring Aspects Input-Products-Results-Processes Evaluation Dimensions -Quality of the program formulation -Program objectives attained -Contribution to MDGs & other development indicators, gender -Replication: Scale up -Innovation -UN system Coordination -Delivering as One -Ownership -Alignment -Harmonization -MFDR -Mutual Accountability -Delta change, effects in citizens’ life Monitoring Aspects -Results (UNDAF+PRS) -Processes (Coordination) Evaluation Dimensions Induced effects on: -MDG at country level -Other Development indicators -UN system Coordination -Delivering as one -Ownership -Alignment -Harmonization -MFDR -Mutual Accountability -UN country pilots 4 th M&E Level MDG Achievement Fund Monitoring Aspects -Results Evaluation Dimensions Quality of partnership Spain/UNDP: Added Value as a mechanism to Progress MDGs achievement The Secretariat’s role and added value Induced effects on: -Linkage of windows and MDGs -Other Development indicators -UN system Coordination -Delivering as one -Ownership -Alignment -Harmonization -MFDR -Mutual Accountability -Effects on citizens' life Joint Programs Countries Monitoring Aspects -Results as an aggregate of the JP -HR/Gender/Environment Evaluation Dimensions New themes:: Culture & migration Induced effects on: -MDG at country level -Other Development indicators (Peace and Culture) -UN system Coordination -Delivering as one -Ownership -Alignment -Harmonization -MFDR -Mutual Accountability
Please in 5 minutes What are the 3 most burning questions an M&E system should it answer?
What elements comprise the MDGF M&E system Monitoring indicators: to measure progress and trends in the short and medium-term at the (inputs, activities, outputs, outcomes) Field visits: to monitor JP in depth and prepare and manage evaluations as well as disseminate results and provide feedback from recommendations Evaluations: to review programs and value the worth of the dimension of study JP, at country level, in thematic windows and the MDGF as a whole Meta-evaluations: to review the quality of the evaluations (JP + Country) conducted and produce robust evidence at window level to link these evidence to MDG achievement Desk reviews and data collection & analysis: from a variety of sources to contribute with information and knowledge to the M&E+KM system
What products will the MDGF M&E system create and offer? Field monitoring reports Country monitoring reports Mid-term evaluations reports Impact evaluation reports Country evaluations (case studies)reports Meta-evaluations reports In depth review reports MDGF global reports (midterm + final) Special activities under the M&E+ Information & Advocacy focus country initiative
MDG F Achievement Fund M&E USER Responsibilities to be takenDeliver the products MDG-F Secretariat Support the UN country teams in the collection of the information and the reporting procedure for monitoring indicators and activities Design 24 thematic and process indicators to feed MDGF M&E system. Design generic terms of reference for mid-term evaluations. Work with UN country teams customizing mid-term evaluation’s terms of reference Design 8 meta evaluations studies 1 for each window of activity Support and facilitate the ongoing evaluations, assuring quality and participation through all their phases: desk review, field work report drafting, publication and dissemination. Assure that recommendations from monitoring system and evaluations mainstream the manager’s decisions and correct programs’ deviations, guarantee that knowledge created serves as a input for the knowledge management system. Implement a calendar of field missions to carry out M&E activities Elaborate a protocol guide to monitoring visits Implement the calendar of field visits to carry out the above mentioned activities The strategy of M&E for MDGF Country monitoring reports+ special countries with high learning and knowledge transfer potential The MDGF mid-term evidence-based report Mid-term evaluations Country evaluations At least 3 Impact evaluations ( experimental or/and quasi- experimental) meta evaluation studies The MDGF evidence-based final report
MDG F Achievement Fund M&E USERResponsibilities to be takenDeliver the following products UN Country Team Collect and report information on the following indicators: On joint program implementation at 3 levels :(3 input, 3 output, 3 outcome) (best, medium and worst indicators, to observe trends) (quarterly reported) 3 indicators on joint program coordination (supplied by MDGF Secretariat) (quarterly reported) 3 thematic indicators per active window in the country (every six months, supplied by MDGF Secretariat) Report on all Joint Program indicators designed by themselves ( annually) Facilitate the M&E processes and activities according to the guidance and principles supplied by the MDGF Secretariat Integrate recommendations from M&E to improve programs Adapt the templates for TOR for final evaluations joint program and country evaluations as well as financing t final evaluations Special activities under the initiative focus countries (Ethiopia, Mauritania, Morocco,Timor Leste, Philippines, Bosnia Herzegovina, Brazil, Ecuador, Honduras) Monitoring Quarterly reports Final evaluations M&E framework activities Report in coordination indicators And thematic indicators
MDG F Achievement Fund M&E Timeframe
What are the constraints we would face in such a enterprise? budget, time, data, political constraints What level of involvement and what work load can the staff from the JP take? Organizational issues, Who is responsible for what part of the system? Joint program, for real? Do joint programs have the necessary coordination mechanisms to allow for joint monitoring and evaluation? Will the users of the M&E system willing to collaborate to implement it ? Is data available at a reasonable cost and on time? Will we have the political support to implement it?
M&E linkages: KM, decision making and learning M&E is an enormous source of knowledge and will generate an extraordinary amount of valuable explicit and tacit knowledge as well as organizational and general knowledge that will have to be delivered at the right moment, to the right persons in the adequate format for the purposes needed. KM system could also become a source of data, information and knowledge for M&E system, is a reciprocal 2 way relation. The ultimate goal of evaluation is recommend actions to improve decision making and improve performance of programs and policies so all evaluations should focused on utilization of their findings and recommendations