1. The science of delivery Use of administrative data in Health Result Innovation Trust Fund (HRITF) portfolio Ha Thi Hong Nguyen | Cape Town, 2014

2. What are administrative data? Data on payment to facilities based on verified performance –Can compare with reported data –Typically only available in contracted facilities Data reported in the HMIS system –Can be available for control facilities Individual patient records –Can look at health outcomes and processes of care –Rarely available in many HRITF countries HRITF mostly works with the first 2 categories, and generally calls them operation data

3. The HRITF OP data portfolio CountryStart dateProgram areasCatchment population BeninMar 20128 districts2.2 million (22%) Burkina Faso*Dec 20113 districts813 thousand (5%) BurundiMar 2010Countrywide9.8 million (100%) Cameroon*Littoral: Apr 2011 3 other: Jul 2012 4 regions2.8 million (13%) Kenya*Dec 20111 sub-county200 thousand (0.5%) Nigeria*Dec 20113 LGAs416 thousand (0.2%) ZambiaApr 201211 districts1.5 million (11%) ZimbabweMar 201218 districts4.2 million (30%) AfghanistanApril 200911 provinces9.1 million (33%) LaosMar 20135 provinces2.2 million (33%) Sierra LeoneOct 2010Countrywide5.9 million (100%) Total population is for 2012 (WDI) Note several programs have expanded but OP data are not yet available 3 *Not include recently scaling up areas

4. Why operational data? To monitor programs’ progress as basis for further inquiry and mid-course corrections –Identifying high and low performing indicators –Monitoring where money is spent –Detecting outliers –Comparing with control areas and watching for unintended consequences –Improving implementation design To promote transparency and hold providers accountable for results To evaluate the impact of the program

5. Monitoring program progress to facilitate further inquiries Estimated coverage of institutional/SBA deliveries

6. Identifying high and low performance Zambia: change between Q2 1012 and Q1 2014 in QOC components

7. Monitoring where money is spent on Share of RBF payment for service delivery that went to health center and lower level

8. Monitoring where money is spent on Burundi Zambia Cameroon Zimbabwe Figures reported are averages of all quarters to date 8 Three services absorbing largest share of payment

9. Detecting outliers

10. Assessing relative progress and watching out for negative spillover Afghanistan: number of SBA deliveries in treatment and control facilities Zimbabwe: Diarrhea cases among age 5+ (non-incentivized RBF indicator)

11. Within 5% Difference Improving implementation design Green Category: Verified on a quarterly basis Amber Category Verified bi-monthly - randomly selected 2 months Red Category Verified on a monthly basis Also incorporates new facilities Difference above 5% but below or equal to 10% Difference above 10% Model based on three risk levels Comparison between declared and verified values for 6-month totals Zimbabwe: switching to risk based evaluation based on comparing reported and verified data

12. Promoting transparency and accountability Burundi Benin Nigeria

13. Issues in working with operational data Quality of data Availability of data outside program (catchment population) Capacity to design and manage a database Capacity to analyze data Standardized methods and assumption to calculate coverage Practice of sharing data and using results for decision making Integration with country HMIS