Proposed Approach to Strengthening Information on Development Effectiveness of Multilateral Organizations Presented by: Goberdhan Singh for the Task Team on Multilateral Effectiveness Presented to: The DAC Network on Development Effectiveness February 2010

Background and Developments CIDA paper arguing for a new approach to augmenting performance information on Multilateral Organizations (MOs) presented to the to the DAC Network on Development Evaluation in June 2009. Network established a Task Team to clarify and explore issues including rationale for a new approach and interconnections with existing initiatives (MOPAN and DAC/UNEG Evaluation Peer Reviews). Task Team Meeting of October 15/16 2009 established Management Group and called for development of proposal for submission to Network Meeting of February 2010. 2

The Work of the Task Team Discussed the need for information on MO performance, especially re Development Effectiveness (DE). Reviewed performance information available from MOPAN, COMPAS, DAC/UNEG Peer Reviews. Developed a Draft Framework describing scenarios for the availability of Development Effectiveness information and identified options for augmenting information where necessary. Established principles for pilot testing approaches to augmenting available information. Established an informal Management Group including CIDA, DFID, UNEG, SADEV, US-AID, and the World Bank to support the work of the Task Team. Oversaw and reviewed the development of this proposal.

Primary Rationale: The Need for Information on Development Effectiveness Task Team identified diverse sources of information on elements of Organizational Effectiveness including MO’s own reporting systems and reports of MOPAN, DAC/UNEG Evaluation Peer Reviews, COMPAS Etc. Agreed that the most significant information gap concerns systematic, credible information on Development Effectiveness of MOs at field level in partner countries. Agreed that any effort to address this gap must recognize the roles of MO reporting systems and be complementary and coordinated with the MOPAN and the DAC/UNEG Evaluation Peer Review processes. Noted that decisions on supplementing available information required development of a framework for reviewing available information and selecting options for engagement with each MO. 4

Framework of MO Effectiveness Information Sources and Options Information Sources: MO Effectiveness Scenarios Regarding Information on Development Effectiveness Options: Development Effectiveness Information MO Reporting on Its Own Organizational Effectiveness Scenario (A): MO Reporting on Development Effectiveness Adequate If MO reports on development effectiveness rigorous, evidence-based and of sufficient coverage and frequency, especially if verified by evaluation data. Implement Option 1. Option 1: Rely on MO Reporting Systems (Medium to Longer Term Goal) MOPAN COMPAS DAC/UNEG Evaluation Peer Review MO and External Evaluation Reports Scenario (B): MO Effectiveness Reporting Insufficient but Evaluation Function Adequate If MOPAN and DAC/UNEG Evaluation Peer Review indicate evaluation function adequate and sufficient evaluation data available, implement Option 2. Option 2. Conduct a systematic synthesis of information from available evaluations as they relate to MO Development Effectiveness. Joint Evaluations Scenario (C) MO Effectiveness Reporting and Available Evaluations Inadequate for Reporting on Development Effectiveness If MOPAN and DAC/UNEG Evaluation Peer Review indicate the evaluation function is inadequate and/or sufficient evaluation material is not available, then implement elements of option 3. Option 3: Implement actions aimed at strengthening MO development effectiveness reporting. Including (among others): Joint evaluation Direct support of MO results reporting systems and evaluation systems Support MO efforts to strengthen decentralized evaluation systems

Task Team Recommends DAC Network endorse proposal. Task Team/Management Group be mandated by the DAC Network to pilot the proposed approach with two to three MOs selected from the MOPAN Survey for 2010 (with a focus on Option 2, Synthesis of Evaluation Results). Pilot to have the characteristics and follow the schedule indicated in the following two slides. Results of the Pilots to be reported to the first meeting of the DAC Network on Development Evaluation for 2011.

Characteristics of the Proposal While coordinated and complementary, the two processes remain distinct. For each pilot, two or more bilateral agencies to enter into a burden sharing agreement for financial and direct management responsibilities. Management Group (MG) responsible for further refinement of the design and for overall management of the process with reporting to the Network through the Task Team. CIDA Evaluation willing to serve as chair of MG. Engagement with MOs chosen for review to include evaluation unit, secretariat and governing body.

Pilot Test Schedule Approval by DAC Network on Development Evaluation – Feb. 2010. Identification of potential MOs and interested bilateral agencies for collaboration on pilot tests – March 2010. Design Phase for refinement of proposed synthesis evaluation methodology including sampling, scoring criteria, reporting formats etc. – April to June 2010. Meeting of the Management Group on design of synthesis evaluation methodologies – April 2010 Review Phase – May to November 2010 Meeting of Management Group to review results and lessons learned – December 2010 Reporting to DAC Network on Development Evaluation – Spring 2011