1. Paying Health Care Providers for Performance: Evidence from Rwanda Paul Gertler UC Berkeley January 2009

2. Research Team Collaboration  Agnes Binagwaho, Rwanda Rwanda MOH  Paulin Basinga, National University of Rwanda  Jeanine Condo, National University of Rwanda  Damien de Walque, World Bank  Paul Gertler, UC Berkeley  Agnes Soucat, World Bank  Jennifer Sturdy, World Bank  Christel Vermeersch, World Bank

3. Overview  Background/Motivation  Rwanda  Program Description  Evaluation Design and Methodology  Baseline Descriptive Statistics  Impact of PBF  Next Steps

4. Context: Developing World  Africa Very poor health status Weak health care systems Brain drain – doctors & nurses leaving Massive AID could be wasted  World Wide (WDR 2004) Low Quality of Care Training/technology have had small effect on Quality Provider absenteeism high & effort low

5. Pay For Performance  Pay Providers a bonus based on performance measurement Improve quality of care and outcomes Improve job satisfaction & retention  Implementation Challenges Measuring performance Cheating/Misreporting

6. Rwanda: Central African Country  9 million people  Genocide in early 1990s  GNP per capita: 250 US$  Weak Health Care Infrastructure 36 Hospitals, 369 health centers Doctors: 1/50,000 inhabitants Nurses: 1/3,900 inhabitants; 17% of nurses in rural areas  Poor health status, but getting better

7. MDG 4: Infant and child mortality

8. Performance-based Financing (PBF)  Local Initiative  Objectives Increase quantity & quality of health services provided Increase health worker motivation  Financial incentives to providers to see more patients and provide higher quality of care Increased resources Financial incentives  Operates through contracts between Government Health facilities providing services

9. Quarterly Payment to Facility i in period t P j = payment per unit of each PBF service j U ijt = number of patients using service j in facility i in period t Q it = facility i’s quality in period t

11. PBF Facility Quality Score Where S kit = facility i ’s Quality index of Service k  Indicator types: Structural: Availability of medical equipment/drugs needed to deliver adequate medical care Process: Clinical content of care (CPGs )

13. PNC Quality Indicators

14. PBF Payment  Facilities had ability to allocate PBF funds  22% increase in budget 77% to salaries 23% to operating costs, equipment  38% increase in compensation

15. Monitoring Facility Reporting  District Comite de Pilotage Approves quarterly payment Based on facility reports & independent audits  Random utilization audit (once quarterly) One focal point per administrative district  Random quality audits (once quarterly) District supervisors based in District Hospital  Interview random sample of patients Identify phantom patients MSH study – less than 3-5% phantom patients

16. Evaluation Questions: Did PBF… Increase the quantity of contracted health services delivered? Improve the quality of contracted health services provided? Improve child health status?

17. Identification Strategy  During decentralization, phased rollout at district level  Identified districts without complete PBF in 2005  Group districts into “similar pairs” based on population density & livelihoods Decentralization reallocated districts Some new districts had PBF in an area of the new district Gov’t rolled PBF to remaining clinics (treatments) Districts matched to these partials controls Others: randomly assign one to treatment and other to control 8 pairs

19. Isolating the incentive effect  PBF Performance incentives Additional resources  Compensate control facilities with equal resources Average of what treatments receive Not linked to performance Money allocated by the health center management

20. Sample: Panel 165 Facilities 2006-08  2145 households in catchment areas Random sample of 14 per clinic

21.  Health Facility Data Financials, Human resources, Equipment, Meds Provider interview for competency (vignette ) 8-10 patient exit Interviews for prenatal process quality HIMS - utilization  General Health Household survey Utilization & Health outcomes HIV testing, sexual behavior  HIV+ Positive Household Survey HIV testing, sexual behavior ART, CD4, adherance Survey Content

22.  Did we isolate incentives effect?  Did we balance T/C groups at baseline?  Is PBF associated with increases in Maternal utilization: Facility Delivery, Prenatal Care Child utilization: Preventive care, immunization Process quality of prenatal care Provider prenatal care competency Child health: height, weight, anemia, morbidity  Did HIV PBF had spillover effects?  Did HIV PBF improve access & outcomes?  Did HIV PBF affect earnings & risk behavior? Research Outline

23. 23

24. Log Expenditures  Randomization balanced baseline  Follow-up balanced, so difference in follow-up outcomes due to incentives not resources

25. Baseline Balance  Utilization (PBF)  Structural Quality Availability of staff, equipment & drugs needed to deliver care (PBF) Little room to improve  Process Quality Competency (Vignettes) Process Quality (Patient exit survey)

26. Baseline Expenditures & Staffing

28. Impact of PBF: Statistical methods  Have balance at baseline on all key outcomes  Use difference in differences analysis Not a pure randomized experiment  Clustered at district-year level  Facility Fixed Effects  Year dummy  Controls: age, parity, education, household size, health insurance, land, value of assets

33. Prenatal Competency & Quality  Standardized vignette presented to provider Unprompted responses for competency Measure of ability/knowledge Based on Rwandan Clinical Practice Guidelines  Process quality Patient exit interview for process quality Clinical content of care Provider effort

34. Prenatal Provider Competency & Quality

35. Baseline Prenatal Provider Competency & Quality

36. Quality Conceptual Framework What They Know (Ability/Technology) What They Do: (Quality) Production Possibility Frontier

37. Goal: Use Pay for Performance to Close Productivity Gap Ability/Technology What They Do PPF Productivity Gap Conditional on Ability Actual Performance

40. Impact of PBF on Prenatal Care Quality

41. Impact of PBF on Provider Knowledge

44. Spillover Effects of HIV/AIDS PBF on Child Preventive Care

45. Impact of PBF on Child Health (z-scores)

46. Results Summary  Balanced at baseline  Expenditures same, so isolate incentives  Impact on utilization Delivery & Child prevention, but not prenatal  Impact on prenatal quality Bigger for better doctors  Reduced child morbidity & Taller children  HIV/AIDS Spillover effects

47. Policy