1. Power to the People Evidence from a Randomized Field Experiment on Community-Based Monitoring in Uganda
Martina Björkman, IGIER, University of Bocconi, & CEPR
Jakob Svensson, IIES, Stockholm University, NHH, & CEPR

2. Background Millions of children die from easily preventable causes
Weak incentives for service providers
Top-down approach to monitoring also lacks appropriate incentives
Recent focus on strengthening providers’ accountability to citizen-clients
Beneficiaries lack information
Inadequate participation by beneficiaries

3. Research Questions
Can an intervention that facilitates community-based monitoring lead to increased quantity of health care?
Increased quality of health care?
Did the intervention increase treatment communities’ ability to exercise accountability?
Did the intervention result in behavioral changes of staff?

4. Intervention 50 rural dispensaries in Uganda
Drawn from 9 districts
Households w/in 5 km catchment area
18 local NGOs
Provide communities with information on relative performance
Encourage beneficiaries to develop a plan that identified steps the provider and community should take to improve service performance and ways to get the community more actively involved in monitoring

5. Intervention Specifics
Pre-intervention survey data used to compile unique “report card” for each facility
Translated into community’s main language
Posters by local artist for non-literate
Information provided to community through participatory / interactive meetings
Community: suggestions summarized in action plan
Staff: review & analyze performance
Interface: contract outlining what needed to be done, how, and by whom

6. Timing Intervention intended to “kick-start” community monitoring
Mid-term review after 6 months, but no other outside presence in communities
Not able to document all actions taken by communities

7. Data Pre-intervention survey to collect data for report cards
Quantitative service delivery data from facilities’ own records
Households’ health outcomes, perceptions of health facility performance parameters
Whenever possible supported by patient records
Post-intervention survey 1 year after intervention
Child mortality (under 5)
Weight of all infants
Roughly 5000 randomly-sampled households in each survey round

8. Evidence of Increased Monitoring
More than 1/3 of Health Unit Management Committees in treatment communities reformed or added members; no change in control communities
70% of treatment communities had some sort of monitoring tool (such as suggestion boxes, numbered waiting cards, duty rosters); only 16% in control communities
Performance of staff more often discussed at local council meetings in treatment communities
NGO reports suggest that discussions shifted from general to specific issues regarding community contract

9. Treatment Practices
At facilities in treatment communities significantly:
More likely to have equipment used during exam (19% increase)
Shorter wait times (10% decrease)
Less absenteeism (14%age points lower)
More on-time vaccinations
Larger share received information on dangers of self-treatment and family-planning
Also possibility of less drug-leakage

10. Utilization At facilities in treatment communities significantly:
Higher utilization of general outpatient services (16%)
More deliveries at the facility (68%)
From household surveys:
Consistent increases in use of treatment facilities
Reduction in visits to traditional healers & the extent of self-treatment

11. Health Outcomes Child mortality Infant weight
3.2% in treatment communities
4.9% in control communities
90% confidence interval for difference ranges from 0.3%-3.0%
Corresponds to roughly 540 averted deaths (per 55,000 households in treatment communities)
Infant weight
Compare distributions of weight-for-age (z score)
Difference in means is 0.17 z score
Reduction in average risk of mortality based on risk of death from infectious disease among underweight children estimated to be 8%

12. Institutional Issues
Did district or sub-district management react to intervention?
Check that treatment & control communities have comparable:
Monthly supply of drugs
Funding
Construction or infrastructure improvements
Visits from government or Parish staff
Employment (dismissals, transfers, hiring)

13. External Validity
Idiosyncratic process differed from community to community in experiment
In another context, process could play out entirely differently
Cultural factors key

14. Scaling Up What actually caused the observed effects?
How to replicate the intervention?
Process dependent on NGO facilitators
No way to know which components of monitoring were influential

15. An Alternative Explanation
Possible (but unlikely) that intervention directly influenced providers’ behaviors
Outcomes not necessarily result of increased monitoring
Considered additional treatment of staff meetings only but decided against it
Financial reasons
Ethical reasons

16. Conclusion
Impressive effects, but intervention difficult to replicate
Important piece of causal chain undocumented ?