1. How (Does) the World Bank Learn from its Operations? John Heath Independent Evaluation Group Washington, DC July 2015 Connect with us online: ieg.worldbank.org @WorldBank_IEG #WBLearns #WhatWorks /IndependentEvaluationGroup

2. 2 Overview of the Presentation I.Learning>Results: Evidence II.Evaluation Approach III.Findings IV.Recommendations V.Questions

3. How (Does) the World Bank Learn from its Operations? I. Learning>Results: Evidence Are there credible proxies?  Denizer et al, 2013 & Geli et al, 2014 examine effect of TTL Quality on IEG Outcome rating of project.  QUERY: Is TTL Quality a proxy for learning?  Haas, 2006 examines effect of project team member’s N of Years the World Bank on the QAG Quality at Entry rating of project.  QUERY: Is N of Years a proxy for tacit knowledge?  IEG, 2014 content analyzes IEG project assessments to assess link between use of lessons learned during preparation and IEG Quality at Entry rating.  QUERY: Is Use of Lessons a proxy for tacit knowledge?

4. How (Does) the World Bank Learn from its Operations? I.Learning>Results: Evidence TTL Quality Drives the IEG “Outcome Rating Source: Denizer et al, 2013 (N=6,569 World Bank projects) +/- (Positive or Negative Correlation/Magnitude) * (Statistical significance) STRONGER PREDICTORSWEAKER PREDICTORS TTL Quality +++*** Real GDP per capita growth +* CPIA Rating ++*** Multisector +* Repeater project + Size of loan/credit -* Preparation cost - Supervision cost -* Lag between approval and effectiveness -* Length of implementation - Problem flags -* TTL Turnover -* Restructuring +*

5. How (Does) the World Bank Learn from its Operations? I. Learning>Results: Evidence Years of WB Staff Experience Influences Quality at Entry

6. 6 How (Does) the World Bank Learn from its Operations? I. Learning>Results: Evidence Use of Lessons Influences Quality at Entry Source: IEG, Results and Performance of the World Bank Group 2014 QAE (MU- Projects) N=32 QAE (MS+ Projects) N=100 Top 3 positive attributes 1. Active use of lessons 25%50% 2. Adequate technical analysis 22%50% 3. Good relations with stakeholders 25%37%

7. How (Does) the World Bank Learn from its Operations? I. Learning>Results: Evidence Lessons are Context Specific  “Most of the well-designed projects that were found to use past lessons more frequently draw on lessons embedded in specific country context and less frequently on generalized (i.e., regional and global) lessons.”  “Lessons from experience or analytical work that are specific and easier to operationalize appear to be most relevant.” (IEG, Results and Performance of the World Bank Group 2014)

8. How (Does) the World Bank Learn from its Operations? I. Learning>Results: Evidence Drivers of Quality at Entry (IEG, Results and Performance of the World Bank Group 2014)

9. How (Does) the World Bank Learn from its Operations? II. Evaluation Approach The Influence Nexus

10. How (Does) the World Bank Learn from its Operations?

23. Thank you! Don’t forget to connect with us online for more information: Website: ieg.worldbank.org @WorldBank_IEG #WBLearns #WhatWorks /IndependentEvaluationGroup