History of Evaluation at the World Bank Group Patrick G. Grasso American Evaluation Association Orlando, November 2009

World Bank Group International Bank for Reconstruction and Development (IBRD) International Development Association (IDA) International Finance Corporation (IFC) Multilateral Investment Guarantee Agency (MIGA) International Centre for Settlement of Investment Disputes (ICSID)

World Bank Group Governance International cooperative with 185 member countries (varies among WBG institutions) Board of Governors, made up of finance or development ministers of member governments 24-member Board Executive Directors, (the Board) each with an Alternate Director, resident in Washington Board of Executive Directors approves budgets, policies and loans/credits Management headed by Board-appointed President Five-year renewable term Sits ex officio as Chairman of the Board Broad discretion in managing WBG Executive VPs head IFC, MIGA

The Concept of Development Has Changed Physical Social Human Natural Development Decades Major Objective Main Instruments Dominant Discipline Changes in the world over time have also influenced thinking about development. This slide is an over simplification—sort of a cartoon—but it tries to highlight major decadal themes in development. 50s and 60s: Technical information and economic resources. Provide the blueprints, increase domestic savings and investment, fill the remaining gaps. 70s: Basic needs, sectoral investment, linear programming applied to industry planning, urban and rural development linked to poverty reduction. 80s: Energy crises; recycling petrodollars; liberalization and adjustment; the Washington consensus. 90s: Country focus and CAS; private sector development; financial reform; governance; legal reform; Transition economies. 00s: Emergence of global programs; focus on results; Millennium Development Goals; reconsideration of private/public division of effort, emphasis on partnerships. Fifties Sixties Seventies Eighties Nineties Oughts Reconstruction Growth Basic Needs Adjustment Capacity Building Results Technical Assistance Projects Sector Investment Adjustment Loans Country Program Partnerships Engineering Finance Planning Macroeconomics Multidisciplinary Political economy

Initiating performance measurement at the World Bank Ex-post evaluation inaugurated in a series of stages in 1970s, leading to independent evaluation unit Performance indicators for ongoing operations developed at about the same time Bank policy documents promoting monitoring and evaluation from the late 1970s

Development Thinking, Evaluation, and WBG Outcomes, 1970-2008 IEG-WB Independent DGE Wapenhans Report IEG-IFC IEG-MIGA One IEG Large dams report/ commission Country and sector evaluations Evaluations Influencing Strategy Focus on project evaluations First steps on ECD Project Bank Policy Bank Poverty Bank I Poverty Bank II New Multilateralism

The quality crisis and the Bank’s response Dramatic decline in performance of lending operations during 1980s Bank developed indicators and tracking system to monitor progress, but Evaluation Department found lax enforcement Quality Assurance Group established in 1995 Working Group on Monitoring and Evaluation, 1999 Monitoring and Evaluation Improvement Program, 2001

Policies and guidelines Requirement for monitoring and evaluation, but with latitude in implementation Guidelines set out information requirements Detailed instructions on how to apply ratings standards Management and the Evaluation Department have worked since the mid-1990s to reach consensus on standards; more consistent application of criteria

Moving to the higher plane of country, sector, global evaluation Shift from project to country as unit of account meant changing evaluation focus Country program evaluation beginning in 1990s Adapted project evaluation standards in new context Sector/thematic evaluation program to assess WBG policies expanded Evaluation of global/regional programs in 2000s as these proliferated WBG led in developing international guidance for evaluating global/regional programs

Methodological development Moving from objectives-based to results-based evaluation Increased use of impact evaluation IEG impact evaluation program DIME initiative, Spanish Trust Fund Mixed methods Better integration of quantitative and qualitative methods

Evaluation Now Matters More For accountability: Shareholders want to know whether they are getting “value for money” For World Bank management and staff: Need to enhance professional excellence and get results on the ground through organizational learning Move to results focus and concentration on effectiveness means that evaluation matters more than ever. Concern about impact of aid. Waste of money. Results-orientation. Shareholders and taxpayers want to know what they are buying. For MDBs, focus moves to lessons of experience, best practice, systematic learning through evaluation.

External demand for evaluation Pressures growing for more systematic evaluation of the impacts of aid Donor country parliaments/voters want to see evidence of results for money invested Especially evident in US from both government and research/evaluation community Pending legislation requiring evaluation of all foreign aid 2009 OMB memo on evaluation Often linked to demand for external, impact evaluation Meltzer Commission in 1990s 3IE, Poverty Action Lab, Coalition for Evidence-Based Policy

Impediments to improving monitoring and evaluation Poor incentives to conduct good monitoring and evaluation Diffused accountability, unclear roles and responsibilities within the Bank and between the Bank and borrowers Weak capacity for monitoring and evaluation both in the Bank and among borrowers.

What has been constant in this history? A premium on quality Unique value of independence Need for engagement with Management Scrutiny from donors Drive for development results 14

What has changed? Responding rapidly to emerging issues Inclusion of prospective evaluations Inclusion of advisory and analytical work, technical assistance Forging stronger links with WBG directions Input for sector strategies Aspects of WBG organization Full IEG integration Links across WB, IFC, MIGA Work with public and private partners

Thank you For more information go to: http://www.worldbank.org/ieg/ http://www.ifc.org/ieg http://www.miga.org/ieg