Improving development evaluation product quality in Africa: Experience from the African Development Bank Foday TURAY; James EDWIN; Mohamed MANAI Colin KIRK OPERATIONS EVALUATION DEPARTMENT AFRICAN DEVELOPMENT BANK For American Evaluation Association’s 2010 Annual Conference

OUTLINE Evaluation function & products in AfDB What does evaluation quality mean in AfDB? Who is driving evaluation quality in AfDB? What has been AfDB’s approach to evaluation quality improvement? What are the challenges in improving evaluation quality in AfDB? Toward improving evaluation quality in AfDB?

DUAL & COMPLEMENTARY EVALUATION FUNCTION INDEPENDENT SELF

OPERATIONS EVALUATION Diverse Multi-level Products OPERATIONS Self Evaluations OPERATIONS EVALUATION Independent Evaluation Project Completion Reports Country Strategy Paper Completion Report Sectoral Strategy and Policy Reviews Country Performance Portfolio Review Annual Portfolio Performance Review Project and Programme level Project Performance Evaluation Reports (PPER) Expanded Supervision Report Reviews (XSRs) Project Completion Report Reviews Multi-country level Sector and Thematic Evaluations Country-level Country Assistance Evaluation (CAE) Sectoral Evaluations Synthesis of PPERs Annual Review of Evaluation Results Corporate Evaluations and Business Process Reviews PROJECT COUNTRY CORPORATE

EVALUATION QUALITY AS MULTI-DIMENSIONAL Evaluation Producer OPEV Adapted DAC Evaluation Quality Standards Operations Operations Manual Guidelines Evaluation Users Board – Implicit and explicit perceptions Management – Implicit and explicit perceptions

Key Quality Drivers Evaluation users Evaluation Producers Board Management Operations Staff Donors Client Governments Evaluation Producers Operations Evaluation Department (OPEV) Operations

Quality Drivers Users as Quality Drivers OPEV Need for credible and useful information OPEV Oversight function Continuous relevance Evaluation Cooperation Group

MULTI-PRONGED AND EVOLVING APPROACH Process changes –e.g. Government participation Institutional changes –e.g. Logical framework; RMF Project evaluations Structural changes OPEV Quality & results

CHALLENGES FOR EVALUATION QUALITY IMPROVEMENT Usefulness of evaluations to client countries Complicated-complex evaluation object Objective-based to whole results-based Data quantity and quality Evaluation capacity Incentive structure

TOWARD IMPROVING EVALUATION QUALITY Methodological approach-eva. object fit -Mixed methods approach: Evaluation questions Results chain Qualitative and quantitative evidence Institutional changes Capacity: internal & client countries Incentives for results culture

Thank You!