AMREF International Training Center, Nairobi Assessing GAC risk and emerging good practices: Public Expenditure Tracking Surveys March 21- 25, 2011 AMREF International Training Center, Nairobi Sahr J. Kpundeh

The presentation Why new tools for public expenditure analysis? What are Public Expenditure Tracking Surveys (PETS)? Examples (Uganda, Mozambique, and more) Strengths and limitations of PETS

Why new tools to analyze public spending and service delivery? Limited impact of public spending and external aid on growth and human development (WDR 2004) Inappropriate allocation of resources Resources do not reach service delivery units Poor quality of service delivery Services are not used by the population New approaches to aid delivery (budget support, PRSCs, etc) => fiduciary and accountability concerns Are budget allocations pro-poor? Are budget outturns consistent with allocations? Do expenditures result in intended outputs and outcomes? Lack of reliable data in many developing countries

Public Expenditure Tracking and Service Delivery Surveys First Public Expenditure Tracking Survey (PETS) carried out in Uganda in 1996 Since then, a large number of PETS and related surveys have been implemented Scope and nature of surveys have differed, but common theme is link between public spending and development outcomes Surveys to date Completed Uganda (3) Tanzania (2) Ghana Rwanda Mozambique Zambia PNG Honduras Peru Bangladesh and more… Ongoing Ethiopia Nigeria Brazil Laos

The Approach Approach has varied considerably depending on context and focus Multilevel focus, but frontline providers (schools or health facilities) as main unit of observation Multi-angular approach for validation of data Representative sample Data collected through interviews and record reviews Some surveys include detailed surveys of frontline provider, including availability/adequacy of inputs, quality, staff and user interviews, etc.

What a PETS can and cannot do Resource flow Money Drugs Equipment Schoolbooks etc. Issues that PETS can address Outturn consistent with allocation (leakage?) Consistency of records between different levels (leakage?) - Actual allocation of resources across districts and facilities (equity and efficiency?) Delays in financial transfers or distribution of material Central “Paper trail” Province District Budget records Stock cards Payrolls Student lists etc. Fee revenues School / Facility Issues that PETS can’t address “Cooked” books Services

Uganda Education PETS (1995) Education sector Data from 250 schools and administrative units Only 13 percent of capitation grant entitlement actually reached schools (1991-95). Mass information campaign by Ministry of Finance (the press, posters) Follow-up surveys (PETS, provider surveys, integrity surveys, etc.) High leakage has also been found in other countries (Tanzania, Ghana, Zambia, Peru) Many improvements since 1992 macroeconomic stability and growth shift of public resources from defense to roads and social sectors decentralization Poverty Eradication Action Plan (PEAP) Poverty reduced from 56% in 1992/93 to 35 % in 2000 Strong budget management MTEF, Poverty Action Fund (PAF) Sector level performance did not keep up SYSTEMATIC PATTERNS IN LEAKAGE: Large schools with wealthier parents and qualified teachers were able to obtain more of their budget allocation.

Mozambique Health ETSDS (2002) No budget allocations to individual primary health facilities => focus on: Consistency of records between different levels Equity and efficiency in allocation of resources Timeliness of resource transfers Perceptions of staff and users Sampled 90 health facilities in 35 districts nationwide

Mozambique ETSDS findings Weak record keeping and “leakage” District-level financial records for recurrent budget inconsistent with province-level records for 75% of districts, but no systematic pattern Discrepancies in drug records and some evidence of leakage Inconsistent HR records (provincial admin., district admin, facilities) Facility reporting of user fee revenues approx. 70% of expected amounts Considerable inequities in allocation of resources (recurrent budget, drugs, HR, etc) across districts and facilities

Nigeria: Health PETS Motivation: To assess a decentralized delivery of health services under a federal system. Findings: Large-scale leakage of public resources in Kogi State. 42 percent of health staff had not received salaries in over six months, despite the fact that budget allocations were sufficient to cover costs. To supplement salaries, public health providers charged for services illegally, while expropriating drug supplies and selling them to patients. Follow Up: Though the study suggested that providing service users and citizens with more information about budget resources so they could hold providers accountable, no documented post-PETS experiences currently exist. 11/24/2018 Sahr J. Kpundeh

The strengths of the approach Useful tool for diagnosing and understanding problems in budget execution and service delivery District and frontline provider perspective often forgotten at central level Representative sample provides credibility not achieved through small-sample studies or institutional reviews Validation of administrative data (financial and output) Can provide basis for monitoring of changes over time Surveys provide data for research that can improve our understanding of the determinants of poor service delivery Process of designing and implementing survey is useful for understanding PEM and service delivery system

Some limitations Surveys only provide part of the answer What about inter- and intra-sectoral allocations? Link with outcomes? Surveys should supplement rather than supplant routine information, control, and integrity systems Surveys are not audits Enumerators do not have appropriate skills Too time consuming for large sample Focus on system-diagnostic rather than particular districts or facilities Surveys provide information but don’t necessarily result in change A lack of information may not be primary constraint to improving PEM and service delivery – findings need to be used strategically

PETS and the demand-side Dissemination is a often the most critical part of converting PETS findings into reforms For example, Ugandan government subsidized information campaign The media can be a strong ally Sustained dissemination is important Implementation CSOS can be involved in different stages of PETS implementation, and also sometimes are enlisted to carry out monitoring PETS Monitoring PETS, which are also analytical in nature, ask whether an intervention is working as intended and other narrow questions about service delivery at the local level. Institutionalization For sustainability, it is important to think about how PETS can become a CSO led or collaborative exercise Tracking exercises institutionalized at the community level can also be tied to performance-based budgeting of service providers Institutionalization can take a variety of forms depending on country circumstances. Models can be: (i) independent civil society organizations undertake in partnership with external organizations (Uganda), (ii) governments monitor their own performance as part of ongoing public sector reforms, and (iii) an oversight agency such as the Auditing Bureau undertakes the ICT offers opportunities to track and disseminate expenditures on a more real-time basis as well as to extend dissemination of traditional PETS findings

Tanzania: Primary Education Project A pilot PETS was conducted on the PEDP in 2003. The pilot which covered a limited number of schools and local authorities, suggested that leakage was at less than 5 percent. This meant for every Tshs. 100 disbursed to schools from Central Government, an average of more than Tshs. 95 made it to the schools. It turned out this finding was mistaken since the survey had only registered funds disbursed from the Ministry of Finance. The Consultants had not realized, nor were they told that both the Ministries of Education and Local Government had also disbursed significant portions of the funding to the PEDP. This meant that the survey only captured about a third of the money going into the PEDP, and what the pilot identified as “minimal” leakage was, in fact, a considerable leakage. 11/24/2018 Sahr J. Kpundeh

Tanzania Cont’d. After this on the surface, successful pilot, the GoT decided to run a nation-wide Tracking Survey on the PEDP, and hired a prominent research organization, REPOA. REPOA presented its first draft of the findings at the CG in May 2004. Among other things, the findings documented that approx. 40% of the grant was not accounted for. The MoF quickly issued a rebuttal questioning the methodological authenticity of the REPOA study. REPOA made clarifications to the methodological questions raised in their final report which did not affect any of their earlier findings. REPOA did not receive any reactions from the GoT on their response to the rebuttal or the final draft of the report. Neither has the draft PETS ever been officially acknowledged, not is it available on any website (except for a brief summary on the U4 website www.u4.no/themes/pets/. 11/24/2018 Sahr J. Kpundeh

Conclusions A key lesson from the successful application of PETS in Uganda was that the empowerment of communities to claim monetary entitlements was the main driver of improvements and the significant reductions in leakages. It appears that in Tanzania, informing communities of their entitlements was done poorly and patchily, and this left people with limited and unreliable information for follow-up. The story of the Tanzanian PETS experience clearly shows that PETS cannot be seen as a silver bullet. The application of PETS is only likely to help bring about successful reform if it is seen as part of a comprehensive drive to streamline and simplify what is a complex and opaque system of financial management. Although learning from “best practice” is a good thing, being informed about “poor practice” is also important when learning from previous experience. The Uganda PETS demonstrated the potential impact of a very attractive approach for improving efficiency in service delivery. The question then is why are there so few examples of successful replication of the Uganda accomplishments elsewhere? 11/24/2018 Sahr J. Kpundeh