Detecting Leakage, Identifying Inefficiencies, and Assessing the Outcomes of PFM Reforms: Public Expenditure Tracking Surveys (PETS) Doris Voorbraak, Kai Kaiser Public Sector Governance Group Poverty Reduction and Economic Management The World Bank IMF- FAD December 19, 2007 The World Bank PREM Public Sector Governance

The World Bank PREM Public Sector Governance Page 2 Improving Service Delivery Developing countries have increased spending- with support of donor funding- to fulfill their commitments to quality health and education services for all Problem remains that evidence shows that there is a weak association between spending and outcomes Increased pressures to evaluate modalities by which resource reach the frontline (including country systems, special programs, etc.)

The World Bank PREM Public Sector Governance Page 3 Why are services failing poor people? Increased public resources do not always translate into better outcomes  Governments do not spend enough on public goods or poor people  The money does not always reach frontline service providers  Money does reach frontline service providers but service delivery can still be inefficient  There is no demand for better services

The World Bank PREM Public Sector Governance Page 4 Spending wisely When services fail poor people, a good place to start looking for the underlying problem is the budget: Budgets are misallocated Budgets are misappropriated

The World Bank PREM Public Sector Governance Page 5 Public expenditure management Strengthening PEM is essential to improve service delivery Formulating, implementing and reporting annual budgets is a challenging task when capacities are limited and the broader governance environment weak But what about operational impact of budgets? Even the best budget allocations are only as good as their impact on desired outcomes for poor people

The World Bank PREM Public Sector Governance Page 6 PETS can help you judging operational impact- the quality and quantity of service delivery, and where and how, and to what effect allocated funds are spent It therefore helps to shed light on budget allocation and execution issues Their common theme: the relationship between public spending and development outcomes Judging operational impact of budgets

The World Bank PREM Public Sector Governance Page 7 What is PETS? Surveys that track, locate and quantify the flow of public resources across various administrative levels

The World Bank PREM Public Sector Governance Page 8 PETS-Experience to Date Pioneer was Uganda PETS (1996) Majority of the studies so far cover African countries. Most PETS cover health and education. Some are joint studies Exceptions fiscal sustainability (Honduras), transport, agriculture, rural development (Sierra Leone) PETS prepared on justice sector (Burundi) PETS team leaders (TTLs) as far as the WB is concerned represent different professional affiliations-research (DEC), Public sector governance (PREM) sectors (HD). PETS has demonstrated to provide a good platform for collaboration between sector-and governance specialists In some countries, follow-up PETS exercises (Uganda, Tanzania, Sierra Leone) but not necessarily with same research questions and scope Linked with Quantitative Service Delivery Surveys (QSDS) (Zambia, Nigeria) or other facility-related surveys

The World Bank PREM Public Sector Governance Page 9 PETS Objectives? Identify/Quantify problems in expenditure items Cash Leakages In-kind Leakages (textbooks, drugs) Absenteeism/Ghost Workers Investigate specific issues (equity, transparency, adequacy, timeliness of resources, regional disparities) Increase supply of and demand for information: information is crucial as stimulus for public action, as catalyst for change, as input to make reforms work

The World Bank PREM Public Sector Governance Page 10 Resource Flows to Frontline Facilities Vary

The World Bank PREM Public Sector Governance Page 11 PETS-stocktaking Region/ CountryYearSectors Africa Cameroon2003Health/Education Chad2004Health, Service Delivery DRC2007Education Ghana2000Health/ Education Kenya2004Health/ Education Madagascar2003, 2005, 2006Health Mali2005Education Mozambique2001, 2004Health Namibia2004Health, Education Nigeria2004Health Rwanda2003, 2004Education, Health/ Education Senegal2002Health Sierra Leone2000/01, 2003Agric. /health/ educ./water & san. Tanzania1999, 2001, 2003,2004Health, Educ., Pov. Reduction Uganda1996, 2001,2003,2004Health, Education Zambia2001,2002, 2004Education, Service Delivery

The World Bank PREM Public Sector Governance Page 12 PETS stocktaking-cont… Region/ CountryYearSectors EAP Cambodia2005Health, Education Indonesia1998,2000Education Mongolia2006Education Papua New Guinea2002Health Europe and Central Asia Albania2004Health, Education Azerbaijan2006Education Latin America and Caribbean Brazil2006Health Honduras2001Pov. Reduc./Fiscal Sust. Peru2002Pov.Reduction, Education South Asia Bangladesh2003Health Mid. East & N. Africa Yemen2006Education

The World Bank PREM Public Sector Governance Page 13 The Scale of PETS Vary Schools Sampled Clinics Visited

The World Bank PREM Public Sector Governance Page 14 PETS reveal a range of leakage estimates

The World Bank PREM Public Sector Governance Page 15 Headline Leakages The initial Uganda PETS in the mid-1990s estimated that 87% of non-wage capitation grants in education did not reach its destination. Tanzania and Ghana: leakage on non-wage education expenditures was 41% and 50% respectively Chad : leakage at the regional level is estimated at 73% and only 1% of non-wage health expenditures to regional health administrations was estimated to arrive at the health facility level.

The World Bank PREM Public Sector Governance Page 16 Challenges to Quantifying Leakages In other surveys, no firm conclusions on leakage could be made. For instance: Cameroon, Mozambique, Nigeria, Namibia, Rwanda, Senegal, Tanzania ( 2001 ) Factors explaining this situation: No rules based allocation “benchmarks” (in contrast to e.g. Uganda’s capitation grants) Poor or non existing official records, or inconsistent records between levels Inadequate design of instruments that did not match administrative levels or the facilities’ record practice Implementation problems

The World Bank PREM Public Sector Governance Page 17 Staff Related Leakages Absenteeism PETS/QSDS have also allowed to measure provider absenteeism, which affects service quality. For instance: Uganda education: 27%, health 37%, Mozambique (health) 19% Supports findings of other studies (eg. Chaudhury et al, 2006) Ghost Workers In a few countries, allowed to measure number of staff on official payroll who do not really work in facilities In Uganda for instance, it was estimated that “ghost workers” represented about 20% of the workforce

The World Bank PREM Public Sector Governance Page 18 Ghost Worker Estimates

The World Bank PREM Public Sector Governance Page 19 PETS highlight problems beyond basic leakage… Delays and bottlenecks in the allocation of resources have also been measured Salaries, allowances, financing, material, equipment, drugs and vaccines These issues could have important effects on the quality of services, staff morale and the capacity of providers to deliver services. In some countries, such as Nigeria, in one state, 42% of the health staff respondents reported not receiving salaries for the 6 months or more.

The World Bank PREM Public Sector Governance Page 20 Different problems for different flows The measurement of delays proved sometimes easier to estimate than quantitative data on the same flows ( which would have allowed measurement of leakage levels ). For instance in Zambia (as in most countries), staff compensation flows were not tracked Instead, information was collected on delays (and arrival time) in the reception of salaries and four types of allowances at the school level. While salary payments were found to be efficient, for not well defined allowances, more than 75% of recipients of “double- class allowances” (additional amount paid for overtime, etc) experience at least 6 month’s overdue pay.

The World Bank PREM Public Sector Governance Page 21 Facilities continue to rely on direct user fees Tracking surveys have produced findings on the importance and impact of user fees. In several countries (e.g. Chad, Mozambique), user fees are sometimes the only source of revenues for service providers. On the demand side, however, they often constitute a very important part of service costs for users and could have considerable negative effects on accessibility. For instance in Uganda, although the government was trying to re-establish free education, the survey found that private contributions represented more than 60% of education costs at the school level. In Chad, for the poorest quintile, the average out-of-pocket medical expenses accounted for 21% of their monthly incomes.

The World Bank PREM Public Sector Governance Page 22 PETS perspectives on equity issues In several countries, large variability of health and school spending across regions and districts, as well as within districts, was observed For instance: Mozambique, nine-fold variation of per capita health spending at the district level Chad: non-wage per capita public health spending showed important variations across districts (in a 16 to 1 ratio) Zambia: the most funded school district received 8 times more average per-student public resources than the least funded district. Uganda: poorer schools faced greater leakages

The World Bank PREM Public Sector Governance Page 23 Policy Reforms PETS/QSDS have led to the identification of areas where reforms in information gathering and supervision systems should be introduced. In Uganda in particular, the policy reform following the PETS targeted information flows through an information campaign, in order to give clients potential power over service providers. Results have been spectacular as leakage rate was reduced very substantially. In Tanzania also, such an information campaign approach to reducing corruption has been used. No formal impact evaluation, but reports indicate that the information campaign has reduced leakage

The World Bank PREM Public Sector Governance Page 24 PETS as one means to an end to strengthen sectoral public financial managent and outcomes PETS are powerful tools to assess efficiency, equity and corruption issues in public services. A sound methodological basis is required for survey implementation, In particular with respect to samples, proper preliminary institutional and data assessment, adequate choice of tracking flows, use of efficient and relatively common instruments, etc. Tracking surveys are a means to an end. The information on incentives and deficiencies in service delivery organizational structures should be used to help implement a reform agenda

PETS for Strengthening Human Development Outcomes: PEM/PFM Perspectives Presentation for IMF/FAD 19 December, 2007 Kai Kaiser, Senior Economist, Public Sector Group, World Bank, Washington, DC

The World Bank PREM Public Sector Governance Page 26 Three Levels of PFM Concerns Macroeconomic Management Allocative Priorization Operational Efficiency See: Pradhan and Campos (1996)

The World Bank PREM Public Sector Governance Page 27 Various Instruments Public Expenditure Reviews (PERs) Increasing Number Conducted by Sectors, including HD Sometimes include PETS …also policy note/programmatic approach PEIRs, CFAA, Portfolio Reviews International Benchmarking HIPC Indicators PEFA Indicators

The World Bank PREM Public Sector Governance Page 28 Key Stakeholders in PFM Cross-Cutting Ministries Ministry of Finance Ministry of Planning/Economics Ministry of Interior/Local Government Sectoral-Cutting Ministries Ministry of Education (various levels), Health, Social Protection Sub-National Government Devolved and Deconcentrated Structures World Bank Networks PREM, HD, SDV

The World Bank PREM Public Sector Governance Page 29 Challenges Cross-cutting PFM dialogue to high level Remains abstract, to focused on supply side strategies …enhanced focus on actionable PFM Indicators Dialogue with sectors/line agencies will vary Sectors may be tempted to vertical lens Emphasis on earmarking funds/vertical programs Ambivalent about budgeting autonomy for sub-national governments Community Participation Advocates Neglect Supply Side May create parallelism to get funds to frontline Focuses on narrow volume of expenditures Fiscal flow mapping and tracking inherent in PETS can serve to center these discussion

The World Bank PREM Public Sector Governance Page 30 Fiscal Architectures Vary Institutional Architecture How to funds get prioritized/budgeted, executed, monitored for frontline service delivery? Intermediate/Decentralized Levels Sub-National Governments  Does the center and or both allocate? Facility Treasury Design

The World Bank PREM Public Sector Governance Page 31 International Initiatives for PFM Performance Benchmarking HIPC Indicators Improved Monitoring of PFM systems as part of debt reduction Public Expenditure & Financial Accountability (PEFA) Indicators 28 Performance Indicators + 3 for donors Covers 8 and 23 cover fiscal decentralization and front-line facility information

The World Bank PREM Public Sector Governance Page 32 PEFA Indicators Focused on Facility Financing (PI-23) ScoreMinimum requirements (Scoring methodology: M1) A(i) Routine data collection or accounting systems provide reliable information on all types of resources received in cash and in kind by both primary schools and primary health clinics across the country. The information is compiled into reports at least annually. B(i) Routine data collection or accounting systems provide reliable information on all types of resources received in cash and in kind by either primary schools or primary health clinics across most of the country with information compiled into reports at least annually; OR special surveys undertaken within the last 3 years have demonstrated the level of resources received in cash and in kind by both primary schools and primary health clinics across most of the country (including by representative sampling). C(i) Special surveys undertaken within the last 3 years have demonstrated the level of resources received in cash and in kind by either primary schools or primary health clinics covering a significant part of the country OR by primary service delivery units at local community level in several other sectors. D(i) No comprehensive data collection on resources to service delivery units in any major sector has been collected and processed within the last 3 years.

The World Bank PREM Public Sector Governance Page 33 PEFA Indicators Focused on Sub-National Transfers (PI-8) PI-8 Transparency of inter-governmental fiscal relations (i) Transparent and rules based systems in the horizontal allocation among SN governments of unconditional and conditional transfers from central government (both budgeted and actual allocations); (ii) Timeliness of reliable information to SN governments on their allocations from central government for the coming year; (iii) Extent to which consolidated fiscal data (at least on revenue and expeniture) is collected and reported for general government according to sectoral categories.

The World Bank PREM Public Sector Governance Page 34 PEFA Fiscal Decentralization Outcomes Vary Slide includes finalized and draft reports

The World Bank PREM Public Sector Governance Page 35 As do information on frontline service delivery facility financial flows

The World Bank PREM Public Sector Governance Page 36 Facility and Local Government Frontline Financing Indicators

The World Bank PREM Public Sector Governance Page 37 How can PFM diagnostics and information initiatives have impact? PETS Can Promote Evidence Policy Dialogue Better understanding of prevailing institutional arrangements Range of contexts (post-conflict to middle income) “Hard” numbers on leakages underscore gap between formal and informal practices Supplement or substitute for audit functions? Highlight Supply-Demand Side Interactions Rules Based Allocations Prerequisite to enable demand side accountability Claims against budgetary entitlements at frontline

The World Bank PREM Public Sector Governance Page 38 PETS Process Challenges Managing Expectations Ability to quantify leakages in general Specific flows versus general leakages Catering method to context How to interpret findings Implications for supply side versus demand side reform/capacity building… How to disseminate findings Mainstreaming in PFM and Sectoral Dialogues…

The World Bank PREM Public Sector Governance Page 39 Key Resources PETS Database/Website PREM Fiscal Leakages Note HD PER Guidelines Using Information to Enhance Demand Side Accountability Stocktaking