1. EuropeAid Evaluation of Budget Support: Methodological Approach and Results Jurgen Lovasz DEVCO B/2 EU Expert Group on Results, April 2012

2. EuropeAid 1.Issue paper (May 2008); 2.Common methodological documents approved by the OECD/ DAC Steering Group; 3.Three tests of the methodology in Tunisia, Mali (finalised) and Zambia (under finalisation); 4.Short synthesis of the findings of the 3 tests for Busan HLF; 5.First International Workshop in Brussels on the refined methodological approach in March 2012 6.Dutch Government finances drafting of Standard Methodological Approach for Evaluation of BS based on former docs and results of pilot studies. Finalisation in April 2012 Where we are

3. EuropeAid Next Steps Pilot 2 days internal EC Familiarisation Workshops (June/July 2012) 3 days Training Workshops mainly for potential evaluators of BS Evaluations; as of October 2012 Next joint evaluations of budget support: Confirmed: Tanzania (EC in lead, launched in Jan.2012); Burundi (Belgium in lead); To be decided: Asia: Nepal, Philippines (?) (EC in lead) Morocco (EC in lead) Mozambique (interest from EC, Italy, Sweden, Finland and Netherlands); Ghana (in 2013?) Sierra Leone (in 2013;EC/DFID in lead?)

4. EuropeAid EXISTING BUDGET SUPPORT EVALUATIONS 2004 Tanzania 2006 Joint Evaluation of 7 countries from 1994 to 2004 budget support 2007 Ghana Evaluations addressing mainly THE PROCESS And not enough the CHAIN OF RESULTS IMPACTS AT THE GLOBAL LEVEL IMPACTS ON SECTOR POLICIES

5. EuropeAid

7. What can we see behind, what is easily visible? Main Findings and Results

8. EuropeAid Tunisia 7 out of 12 operations financed by the European Commission over the period 1996- 2009. Four GBS and three SBS, covering secondary and tertiary education and vocational / educational training GBS: European Commission, World Bank, AfDB SBS: European Commission

9. EuropeAid 32 out of 33 operations financed by 11 donors over the period 2003-09, GBS: AfDB, WB-IDA, Canada, Denmark, European Commission, France, Netherland, Sweden SBS - PFM: Canada. SBS – social sectors: Belgium, Canada, France, The Netherlands, Sweden-Norway, Spain SBS – decentralisation: EC Mali

10. EuropeAid Zambia Poverty Reduction Strategy Budget Support and 3 Sector Budget Support (roads, public financial management, health) over the period 2002-2009 PRSBS: European Commission, Finland, Germany, The Netherlands, Norway, Sweden and United Kingdom SBS: European Commission and United Kingdom

11. EuropeAid Contribution of Budget Support to the aid framework and partner country policies Contribution to the implementation of the Paris Declaration and AAA Increased financial resources and strengthened budget management Adapted design, positive dialogue frameworks and diversified use of TA Main findings

12. EuropeAid But... Although harmonisation and alignment improved, they far from completed : - donors often support less coordinated initiatives - unrealistic PAF conditions and targets (donor driven indicators) - multiplication of coordination structures Main findings

13. EuropeAid BS increased discretionary expenditure and allocative efficiency And Effective support to implementation of Public Financial Management & other reforms when government and society are committed: Budget Support cannot buy reforms Main findings

14. EuropeAid BS increased public investment and reform implementation that produced significant outcomes in education and health. In the three countries, the reforms and related investments supported by higher levels of public expenditure produced significant outcomes and some measurable impacts as well Main findings

15. EuropeAid BS contributed partly to sound macro economic policies, growth and private sector development Tunisia: Strong Opportunity Framework including BS contributed to growth and private sector development Mali/Zambia: While BS funds contributed to sound macro economic policies, other important factors (regional integration, export prices, debt relief, etc) also played an important role Main findings

16. EuropeAid BS contribution to growth and social policies improved income and non-income poverty But Rural poverty has not been tackled adequately by government (Zambia) Main findings

17. EuropeAid Key Lessons -BS is an effective tool in a country with robust policies -BS cannot determine major policy changes but can provide sound support for policy reforms, that is not possible with only projects -BS is more beneficial with high degree of harmonisation and alignment -BS is strengthened and multiplied when it is linked to wider political and economic partnerships

18. EuropeAid Cont.: Key Lessons -BS has been an instrument conducive to increasing transparency and accountability of budget processes and public expenditure management and may be used to support anti-corruption policies -BS benefits if it is complementary with other forms of aid, especially for capacity building -The choices between GBS and SBS should be flexible and based on the context. -Improved documented policy dialogue, data production and policy assessments of development results is required

19. EuropeAid THE METHODOLOGY AND FINDINGS ON

20. EuropeAid THANK YOU!

21. EuropeAid

22. Mali: Example of application of methodology

23. EuropeAid Mali: Example of application of methodology

24. EuropeAid GBS/SBS COMPREHENSIVE EVALUATION FRAMEWORK Step 1 Step 2 Step 3

25. EuropeAid 1.Inventory of the inputs provided 2.Assessment of the performance of these inputs in terms of direct outputs and induced outputs. 3.Identification of the changes which have occurred during the period in the partner country and the key causal factors. 4.Assessment of the extent to which budget support (may) have contributed to the results identified at the outcome and impact levels. Expected Results of the Budget Support Evaluation

26. EuropeAid Formulation /Design Evaluation period « Planning » for Evaluation as regards: -Data devel. -- surveys -- indicator devel. Dev + gather info on: -data -PFM assessments -Suveys Document PD Evaluation based on material developed and gathered before; Additional material may be required Learn from evaluation results Follow up recommend ations How to reflect Evaluations in PCM Implementa tion