1. Helena Tang Lead Evaluation Officer September 2008
Using Knowledge to Improve Development Effectiveness: An Evaluation of World Bank Economic and Sector Work and Technical Assistance,
Helena Tang
Lead Evaluation Officer
September 2008

2. Why did we evaluate ESW and TA?
The Bank considers knowledge important for development
ESW and TA are the Bank’s main knowledge products
Knowledge: long-standing agenda for the Bank
“Global knowledge bank” (1996)
One of 6 strategic pillars going forward
First comprehensive evaluation of these products
Inform Bank’s future strategy on knowledge and learning

3. What are ESW and TA? ESW Economic reports (53 types)
Inform Bank activities (strategy and lending)
Influence client’s policies and/or programs TA
Technical Advice
Implement reforms and strengthen institutions (drafting legislation, training in data analysis, knowledge sharing, etc.)

4. 85 percent of AAA, FY00-06

5. One-quarter of spending on country services, FY00-06

6. What questions did we ask?
To what extent did ESW and TA meet their objectives?
To what extent did the following affect the achievement of ESW and TA objectives?
Origination (client-requested or not)
Partnership in production with local institutions (government or others)
Technical quality
Dissemination

7. What evaluation tools did we use?
Five Sets of Evidence
12 Country Reviews

8. Country Review Selection
Serbia
Romania
Jordan
Bangladesh
Mali
Guyana
Vietnam
Malaysia
Democratic Republic of Congo
Peru
Mauritius
Lesotho

9. What evaluation tools did we use?
Five Sets of Evidence
12 Country Reviews
Electronic Surveys of in-country stakeholders
Specific ESW
Specific TA
General
Electronic Surveys of all ESW and TA TTLs
Electronic Survey of loan TTLs
Statistical and econometric analysis

10. MAIN FINDINGS

11. Client views on ESW and TA
Clients find Bank ESW and TA more useful than those provided by other institutions
Clients value Bank ESW and TA for their high quality, objectivity, and provision of international perspectives
Clients generally prefer TA over ESW (IBRD & IDA)
Middle income country (MIC) clients prefer TA and ESW over lending
Some MIC clients clearly prefer de-linking TA and ESW from lending

12. Client country respondents had a range of views on the effectiveness of ESW and TA
At least two-thirds gave an above average rating

13. Examples of effective ESW: Vietnam PER
Budget legislation
MTEF
Capacity

14. Examples of effective ESW: ICAs
Privatization (Serbia)
Competitiveness strategy (Guyana)
Labor law, property registration, deregulation of public service delivery (Malaysia)

15. Example of effective TA: Mauritius Aid for Trade
Just-in-time advice on trade reform program
Analysis of reform scenarios and effects
Recommendations incorporated in government reform program

16. ESW improved Bank activities
Shaped country assistance strategies
Improved lending quality
Presence of relevant ESW associated with better loan quality at entry
Around 90 percent of DPL but only around 60 percent of investment loans

17. What made ESW and TA effective?
Technical Quality
Good quality ESW requires resources
ESW better resourced in IBRD than in IDA countries
Bank budget and not trust-fund matters
Origination
Client interest and buy-in essential but products can be originated by the Bank

18. What made ESW and TA effective?
Partnership
Close collaboration with clients throughout the process but not necessarily co-production
Collaboration takes time (completion of forest sector review in DRC delayed nearly 2 years)
Dissemination
Sustained engagement beyond one-off dissemination
Broad vs targeted
Language and translation

19. What made ESW and TA effective?
Government capacity
Lower in post-conflict and some low income countries (DRC, Lesotho)
Lower in countries with high turnover of senior government officials (Jordan, Serbia, and in the sector ministries in Peru)

20. RECOMMENDATIONS

21. Recommendation #1
Reinvigorate the mandate (underpinned FY99 ESW reforms) of a strong knowledge base on countries where Bank is providing (planning to provide) funds

22. Recommendation #2
For IDA countries, ensure ESW is adequately resourced, even if it means fewer ESW in some countries

23. Recommendation #3
Enhance institutional arrangements: substantive task team presence in country offices, and include a clear strategy for sustained post-delivery engagement

24. Recommendation #4 Tell Us What You Think Feedback
Recognize and build on client feedback to help counter-balance current Bank incentives for lending over non-lending, and ESW over TA
Tell Us
What
You
Think
Feedback

25. Recommendation #5
Take results tracking framework more seriously, including systematizing client-feedback