1. Measurement Matters: The Use of PETS and QSDS
Public Expenditure Analysis and Management Course Ritva Reinikka
Development Research Group (DEC)
Public Services Research Team
January 13, 2004

2. Increasing public spending is not enough
to reach MDGs
* Percent deviation from rate predicted by GDP per capita
Source: Spending and GDP from World Development Indicators database. School completion from Bruns, Mingat and Rakatomalala 2003

3. Similar changes in public spending can be associated with vastly different changes in outcomes
What this means is that countries that spend the same amount could have vastly different outcomes (e.g., Ethiopia and Malawi).
Sources: Spending data from World Development Indicators database. School completion from Bruns, Mingat and Rakatomalala 2003

4. and vastly different changes in spending can be associated with similar changes in outcomes.
Conversely, countries with different spending patterns have similar outcomes.
Sources: Spending data for 1990s from World Development Indicators database. Child mortality data from Unicef Other data from
World Bank staff

5. Unit cost and performance in primary education: Mauritania

6. Expenditure incidence tends to favor the better-off even in health and education
If you look at health and education spending—usually thought to be pro-poor. In Guinea, 48 percent of public spending on health goes to the richest quintile, while less than 8 percent goes to the poorest.
This is one of the areas where we updated the data (most recently available, credible studies)
This anti-poor expenditure incidence is reflected in data about access to public facilities.
Source: Filmer 2003b.

7. Short and long routes of accountability in service delivery

8. The relationship of accountability has five features

9. Why do we need new tools?
Limited impact of public spending on growth and human development – to answer why?
New demands for evidence on efficiency of spending and performance in service delivery
Lack of reliable data on finance and performance: obtain them from sample survey  PETS and QSDS
New approaches in aid delivery
Move towards budget support (e.g., PRSC)
Related fiduciary and accountability concerns

10. Public expenditure tracking surveys PETS
Diagnostic and monitoring tool to understand problems in budget execution
delays / predictability
leakage / capture
discretion in allocation of resources
Data collected from different levels of government, including service delivery units
Data from record reviews and interviews
Variation in design depending on perceived problems, country, and sector

11. Quantitative service delivery surveys QSDS
Focus on frontline service providing unit, e.g. health facilities and schools
Inspired by multi-purpose micro-level household and firm surveys
Resource flows (financial and in-kind)
Availability/adequacy of inputs
Service outputs and efficiency
Quality of service
Focus on costs, dimensions of performance in service delivery, ownership categories

12. Hybrid approaches
Link facility surveys with surveys of administrative levels “upstream” (public officials; PETS)
Why different performance in the same system?
Link facility surveys with household surveys
Effect of school/facility characteristics on household behavior and outcomes?
Mix quantitative and perception-based approaches (e.g., exit polls, staff interviews, focus group discussions)
Relationship between perceptions and observable characteristics of schools or facilities?

13. Nonwage funds not reaching schools: evidence from PETS
Country
Mean percentage
Ghana 2000 49
Madagascar 2002 55
Peru 2001 (utilities) 30
Tanzania 1998 57
Uganda 1995 78
Zambia 2001 (discretion/rule)
76/10
Source: Ye and Canagarajah (2002) for Ghana; Francken (2003) for Madagascar; Instituto Apoyo and World Bank (2002) for Peru; Price Waterhouse Coopers (1998) for Tanzania; Reinikka and Svensson 2002 for Uganda; Das et al. (2002) for Zambia.

14. Capture of public funds (Uganda PETS)
Large variations in receipts across schools
Bargaining between local officials and schools over nonwage spending
Election finance and elite capture
When using actual spending data from PETS, neutral benefit incidence became highly regressive
Leakage endogenous to school characteristics
Parents’ income most important determinant
Size of school, teacher qualifications significant, too
Sparked an information campaign which increased client power and reduced capture

15. Schools in Uganda received more of what they were due
First, strengthen poor people’s voice in policymaking by increasing their access to information.
In the early 1990s, primary schools in Uganda were receiving less than 13 percent of the money they were allocated for textbooks, equipment, and the like. Poor schools were receiving nothing.
To improve this dismal performance, the Ministries of Finance, Education and Local Government jointly launched a public information campaign on fiscal transfers. This created a groundswell of activity to increase the share going to primary schools. The newspapers published the funds flowing to schools on a monthly basis (they still do), and school principals had to post the entire school budget on the schoolroom door. The result was that the share increased from 13 percent to 80 percent.
Source: Reinikka and Svensson (2001), Reinikka and Svensson (2003a)

16. Impact evaluation of information campaign
Repeat PETS shows huge reduction in capture of capitation grants
From 80% to 20%
Schools that have access to a newspaper received 14 percentage points more of their entitlement
Information campaign was an effective and cheap way of reducing capture of funds

17. Ghost workers on payroll (percent)
Country
Education
Health
Honduras 2000 5
8.3
Uganda 1993 20 -
Source: World Bank 2001; Reinikka, 2001.

18. Nigeria QSDS: Problems with local government accountability
Pervasive non-payment of salaries of primary health workers in some states
Percent of staff respondents
20%
80%
KOGI (total=240)
LAGOS (total=495)
15%
10% 5% 0% 1 2 3 4 5 6 7 8 9 10 11 12
Months salary not paid

19. Nigeria QSDS in health care
Non-payment of staff salaries cannot be explained by lack of resources available to local governments
Even when local government spending on staff salaries is sufficient to cover actual staff costs, survey of staff revealed extensive non-payment
General problem of local accountability in the use of public resources transferred from higher tiers of government, about which local citizens may not be well informed, as they are not the tax payers

20. Frontline provider surveys 2002: Absence rates (percent) among teachers and health-care workers
Country
Primary schools
Health facilities
Ecuador 16 -
Honduras 2000 14 27
Peru 13 26
India (19 states) 25 43
Indonesia 18 42
Uganda 35
Zambia 17

21. Percent of staff absent in primary schools and health facilities50
Primary schools
Primary health facilities 40 30 20 10
For example – staff are often absent.
And average masks discrepancies within countries:
For example:
-In Bangladesh the rate for DOCTORS in RURAL areas was close to 75%
-In PNG the teacher absenteeism rate in Enga Province in community schools was almost 35%
Bangladesh
Ecuador
India
Indonesia
Papua New
Peru
Zambia
Uganda
Guinea

22. Good reasons for doing PETS/QSDS
Diagnosing problems – shaping the reform agenda
Analysis: guiding reform
Monitoring over time/benchmarking
Understanding systems – useful for donors and governments
Research – collaboration between practitioners and researchers
A good basis for information campaigns to increase “client power”

23. Survey Design: Survey what? Why?
What are the problems? Research question and hypothesis? Are there important gaps in understanding of the nature, extent, and sources of problems?
Is a quantitative survey the appropriate tool? Stand-alone or as a complement? Worth the cost ($50-150K)?
Is it feasible? How is the budget structured and implemented so that relevant data can be collected?
Who is the audience? Is there a political demand for new information (often “bad news”)?
Will the information be used? By whom? How to ensure impact?

24. Implementation issues: Who? How?
Requires skills similar to other micro surveys
Steps in implementation
Concept document
Buy-in across the board: Ministry of Finance, sector ministry, local governments, frontline, donors, etc.
Rapid data assessment
Questionnaire design
Identifying and contracting implementing agency
Pilot questionnaires
Enumerator training
Field work (quality control and data management)
Data analysis
Dissemination  impact on policies