The science of delivery Use of administrative data in Health Result Innovation Trust Fund (HRITF) portfolio Ha Thi Hong Nguyen | Cape Town, 2014

What are administrative data? Data on payment to facilities based on verified performance –Can compare with reported data –Typically only available in contracted facilities Data reported in the HMIS system –Can be available for control facilities Individual patient records –Can look at health outcomes and processes of care –Rarely available in many HRITF countries HRITF mostly works with the first 2 categories, and generally calls them operation data

The HRITF OP data portfolio CountryStart dateProgram areasCatchment population BeninMar districts2.2 million (22%) Burkina Faso*Dec districts813 thousand (5%) BurundiMar 2010Countrywide9.8 million (100%) Cameroon*Littoral: Apr other: Jul regions2.8 million (13%) Kenya*Dec sub-county200 thousand (0.5%) Nigeria*Dec LGAs416 thousand (0.2%) ZambiaApr districts1.5 million (11%) ZimbabweMar districts4.2 million (30%) AfghanistanApril provinces9.1 million (33%) LaosMar provinces2.2 million (33%) Sierra LeoneOct 2010Countrywide5.9 million (100%) Total population is for 2012 (WDI) Note several programs have expanded but OP data are not yet available 3 *Not include recently scaling up areas

Why operational data? To monitor programs’ progress as basis for further inquiry and mid-course corrections –Identifying high and low performing indicators –Monitoring where money is spent –Detecting outliers –Comparing with control areas and watching for unintended consequences –Improving implementation design To promote transparency and hold providers accountable for results To evaluate the impact of the program

Monitoring program progress to facilitate further inquiries Estimated coverage of institutional/SBA deliveries

Identifying high and low performance Zambia: change between Q and Q in QOC components

Monitoring where money is spent on Share of RBF payment for service delivery that went to health center and lower level

Monitoring where money is spent on Burundi Zambia Cameroon Zimbabwe Figures reported are averages of all quarters to date 8 Three services absorbing largest share of payment

Detecting outliers

Assessing relative progress and watching out for negative spillover Afghanistan: number of SBA deliveries in treatment and control facilities Zimbabwe: Diarrhea cases among age 5+ (non-incentivized RBF indicator)

Within 5% Difference Improving implementation design Green Category: Verified on a quarterly basis Amber Category Verified bi-monthly - randomly selected 2 months Red Category Verified on a monthly basis Also incorporates new facilities Difference above 5% but below or equal to 10% Difference above 10% Model based on three risk levels Comparison between declared and verified values for 6-month totals Zimbabwe: switching to risk based evaluation based on comparing reported and verified data

Promoting transparency and accountability Burundi Benin Nigeria

Issues in working with operational data Quality of data Availability of data outside program (catchment population) Capacity to design and manage a database Capacity to analyze data Standardized methods and assumption to calculate coverage Practice of sharing data and using results for decision making Integration with country HMIS