Evaluation Office EVALUATION THE INDEPENDENT EVALUATION OFFICE OF THE UNITED NATIONS DEVELOPMENT PROGRAMME REFLECTIONS ON CONTRIBUTION TO TRANSPARENCY, LEARNING AND ACCOUNTABILITY June 2013 Indran A. Naidoo Director

Demand from Members states and management. The Evaluation Policy of the UNDP clarifies roles and responsibilities for evaluation The Evaluation Office (EO) of the UNDP, the largest in the UN system, produces independent evaluations at the corporate, programme and country level. It extends its influence by managing the UNEG Secretariat, producing guidance and standards, engaging with networks and supporting evaluation capacity building across the globe Setting the stage 2

Evaluation Office 1. Mandate and functions 3  Reports directly to the Executive Board of the UNDP  Supports the Administrator in her substantive accountability function  Operates within the UNEG Norms and Standards and ethical guidelines, and engages stakeholders in the conduct of evaluations to ensure transparency, learning and accountability 1999 EO was established 2004 GA 59/250 Resolutions for UN System 2005 UNEG Norms & Standards 2006 UNDP Evaluation Policy and Independence 2011 UNDP Evaluation Policy (revised)

Evaluations are planned and answer questions within the Strategic Plan - has been RELEVANCE, EFFECTIVENESS, EFFICIENCY and SUSTAINABILITY DO UNDP INTERVENTIONS MAKE A CHANGE? EO staff lead all evaluations, drawing on advisory panels and using engagement processes with evaluands and stakeholders as a means to enhance credibility. As an independent office, the Director signs off on all evaluations. In practice this means… 4

Evaluation Office 2. Contributions to other organisational tier support and professionalasation) 5 Setting standards; Guides; Assessment of evaluation quality UNDP: Support to decentralized evaluation function and products Norms and standards; Guidelines; Methodology; Peer Reviews; UNEG Secretariat. ( United Nations Evaluation Group: Coherence in UN system-wide evaluation Regional: AfrEA, RELAC, Malaysia Eval Society, IPEN International: NONIE; ECG; IDEAS; IOCE; Eval Partners Professional Networks: Advancing Development Evaluation

Commitment to evaluator professionalisation Ability to engage in a supportive yet independent manner High levels of methodological skill and content expertise Strong strategic and communication skills For evaluators it requires … 6

Has produced over 100 evaluations since 2000, and 80 country level evaluations. All have a management response. Management uses evaluations to review policies, programmes and approaches – shown in 92% uptake on recommendations. Independence respected, and evaluation on the agenda at key top management meetings. Board allocates significant time to engage with evaluation findings, management responds fully. The Evaluation Office 7

The EO engages fully as evaluation processes are as important as the “big report” Learning can come through accountability processes, these are not opposed to each Stakeholder workshops held at country level, led by EO, with government as key stakeholder. Transparency and stakeholders 8

Globally, across a broad mandate In complex situations where interventions are difficult to embed and hard to measure Implications for evaluation: difficult to work out additionality in terms of attribution or contribution Context plays an important role, not easy to superimpose systems to measure results, but not impossible. Involves high levels of engagement to agree on what constitutes success Where UNDP operates 9

Evaluation Office UNDP Programmes and Operations 177 countries in 5 Regions 10 National Goals and Priorities United Nations Development Framework UNDP Focus Areas  Poverty and MDG  Democratic governance  Crisis prevention and recovery  Environment and sustainability development Programs Global Regional Country Others Projects Non-Project operations (advisory, advocacy, standard setting/ normative, coordination, mobilization)

Evaluations need to be context specific, yet meet evaluation norms and standards. At the country level they inform the next UNDP programme, and draw on lessons from the last programme period Aggregation for synthesis needs to consider the question of scale, variability and the challenges of validity Stakeholder workshops demonstrate to government UNDP commitment to transparency and accountability What does this mean? 11

Credibility goes beyond the tools and methods. It rests on the leadership that directs evaluation processes. This comes about through transparent and logical evaluation plans and processes which ensure engagement opportunities throughout and across the spectrum to reduce bias Results must be engaged publically Independence is central for the reasons above, to ensure credibility and authenticity of reports Independence, a credibility question 12

Evaluation Office Transparency  Defined in UNEG norms and standards  Consultations with Stakeholders TORS > Inception Report on Scope, Design and plan for data collection and analysis > Stakeholder Meetings > Draft Reports The Audit Trail <> significance for mutual understanding and final decisions by EO  Public Access All UNDP plans and evaluations in the Evaluation Resource Center (l) EO evaluations in the EO websiteEO website Management response and tracking system (ERC)ERC Ratings on quality of decentralized evaluations 13

Evaluation Office Independence - Structural Functions, and staff - organizationally independent from operations and policy units and decision making.  Executive Board  Director reports to the Executive Board (2 terms and no re-entry into UNDP)  Board approval of programme of work and budget (independent of programme budget)  Reporting  Evaluation reports are the responsibility of the Director  Transmitted directly to the Board following review and comment by management  Senior managers safeguard the independence: EO has access to all records and information 14 Executive Board UNDP Administrator Evaluation Office Director

Evaluation Office Examples from … As of June 2013 the Evaluation Office of UNDP presented eight independent evaluations to the UNDP executive board, which will contribute towards the development of the new UNDP Strategic Plan. Five Regional Programme Evaluations Evaluation of UNDP Strategic Plan Evaluation of South-south and Triangular Cooperation Evaluation of Global Programme of UNDP

Evaluation Office Country Programme Evaluation (Assessment of Development Results – ADRs) 16 Evaluation Office has conducted 80 ADRs since 2002

Engaging with decision-makers – engagement for quality and credibility (management responses and update) Statistics on uptake – 92% Acting on evaluations 17

Support of professional networks, associations and events, to profile evaluation: Supportive of IDEAS, Barbados 2013 (key notes, panels courses) Supportive of IPDET – course presentation Supportive of EvalPartners – management board Support to continental and regional networks (AFREA, APEN, … Hosting of the National Evaluation Capacity workshops, next San Paulo, September Global influence 18

Evaluation is not an event, but a difficult journey requiring constant push, reappraisal, strategising to ensure relevance. It does not occur naturally and requires drivers An evaluation function must be independent, which can be supported when it is both inward and outward focused. The outward – through events like these brings in critical insights necessary for ongoing revision. An outward orientation is the only way that one get professionalisation that is necessary for evaluation. Conclusions 19

Evaluation Office Thank You