Power to the People Evidence from a Randomized Field Experiment on Community-Based Monitoring in Uganda Martina Björkman, IGIER, University of Bocconi, & CEPR Jakob Svensson, IIES, Stockholm University, NHH, & CEPR

Background Millions of children die from easily preventable causes Weak incentives for service providers Top-down approach to monitoring also lacks appropriate incentives Recent focus on strengthening providers’ accountability to citizen-clients Beneficiaries lack information Inadequate participation by beneficiaries

Research Questions Can an intervention that facilitates community-based monitoring lead to increased quantity of health care? Increased quality of health care? Did the intervention increase treatment communities’ ability to exercise accountability? Did the intervention result in behavioral changes of staff?

Intervention 50 rural dispensaries in Uganda Drawn from 9 districts Households w/in 5 km catchment area 18 local NGOs Provide communities with information on relative performance Encourage beneficiaries to develop a plan that identified steps the provider and community should take to improve service performance and ways to get the community more actively involved in monitoring

Intervention Specifics Pre-intervention survey data used to compile unique “report card” for each facility Translated into community’s main language Posters by local artist for non-literate Information provided to community through participatory / interactive meetings Community: suggestions summarized in action plan Staff: review & analyze performance Interface: contract outlining what needed to be done, how, and by whom

Timing Intervention intended to “kick-start” community monitoring Mid-term review after 6 months, but no other outside presence in communities Not able to document all actions taken by communities

Data Pre-intervention survey to collect data for report cards Quantitative service delivery data from facilities’ own records Households’ health outcomes, perceptions of health facility performance parameters Whenever possible supported by patient records Post-intervention survey 1 year after intervention Child mortality (under 5) Weight of all infants Roughly 5000 randomly-sampled households in each survey round

Evidence of Increased Monitoring More than 1/3 of Health Unit Management Committees in treatment communities reformed or added members; no change in control communities 70% of treatment communities had some sort of monitoring tool (such as suggestion boxes, numbered waiting cards, duty rosters); only 16% in control communities Performance of staff more often discussed at local council meetings in treatment communities NGO reports suggest that discussions shifted from general to specific issues regarding community contract

Treatment Practices At facilities in treatment communities significantly: More likely to have equipment used during exam (19% increase) Shorter wait times (10% decrease) Less absenteeism (14%age points lower) More on-time vaccinations Larger share received information on dangers of self- treatment and family-planning Also possibility of less drug-leakage

Utilization At facilities in treatment communities significantly: Higher utilization of general outpatient services (16%) More deliveries at the facility (68%) From household surveys: Consistent increases in use of treatment facilities Reduction in visits to traditional healers & the extent of self-treatment

Health Outcomes Child mortality 3.2% in treatment communities 4.9% in control communities 90% confidence interval for difference ranges from 0.3%-3.0% Corresponds to roughly 540 averted deaths (per 55,000 households in treatment communities) Infant weight Compare distributions of weight-for-age (z score) Difference in means is 0.17 z score Reduction in average risk of mortality based on risk of death from infectious disease among underweight children estimated to be 8%

Institutional Issues Did district or sub-district management react to intervention? Check that treatment & control communities have comparable: Monthly supply of drugs Funding Construction or infrastructure improvements Visits from government or Parish staff Employment (dismissals, transfers, hiring)

External Validity Idiosyncratic process differed from community to community in experiment In another context, process could play out entirely differently Cultural factors key

Scaling Up What actually caused the observed effects? How to replicate the intervention? Process dependent on NGO facilitators No way to know which components of monitoring were influential

An Alternative Explanation Possible (but unlikely) that intervention directly influenced providers’ behaviors Outcomes not necessarily result of increased monitoring Considered additional treatment of staff meetings only but decided against it Financial reasons Ethical reasons

Conclusion Impressive effects, but intervention difficult to replicate Important piece of causal chain undocumented ?