Results-Based Financing & Quality of Care Measuring and Paying for Quality Improvement Dinesh Nair / World Bank

Session Outline: Measuring and Paying for Quality Existing Instruments and Methods Using Data for decision making Verifying Data Accuracy Innovations in Measuring and Paying for Quality

1. Existing Instruments and Methods

Measuring if the right inputs are in place

Measuring processes and results (1) Continuous monitoring (a) Management and Structural Checklist Business/Operation Plan ~60% Min 50% (1) Continuous monitoring PBF Bonus Calculation Tool (b) Process of Care Quality Checklists Health Worker Bonus Allocation ~20% (c) Quantity Checklist Max 50% Liberia: Quality Assessment/ Monitoring Tools ~20% (d) Impact Evaluation

Liberia : Standards for Management Obstructed Labor: Illustrative Checklist Distilling Essential care Items (admission, labor) Chart review elements (see chart review guide for specific criteria) ; each element if recorded = 1 point Charts Admission  1  2  3  4  5 1. Cervical dilation recorded at admission (# of cm)   2. Contraction frequency and duration charted at admission 3. Fetal presentation charted at admission 4. Partograph started when cervical dilation 4 cm or greater Admission Score (x/4) 2. Labor Monitoring (partograph) 1. Cervical dilation recorded at least every 4 hours 2. Frequency and duration contractions recorded at least every 30 minutes 3. Fetal HR recorded at least every 30 minutes Labor Monitoring Score (x/3) Five patient charts reviewed: average score (% adherence best practices) links with bonus Each item has chart review guide that defines criteria

Kyrgyzstan: multiple approaches to measuring quality Record Reviews Simulations of routine labor and delivery, postpartum hemorrhage and eclampsia using Mama Natalie Simulation of newborn resuscitation using Neo Natalie Simulation of surgical safety checklist use Patient interviews by phone include basic quality tracers (access to sanitation facilities; recall health education messages; informal payments and general satisfaction using a Likert scale) https://youtu.be/_miYvoWosS4 Peer to Peer verification and they were learning together

2. Using Data for Decision Making

Nigeria: Institutional Deliveries increased from 20% to 44% during 2015(120% increase) After scall up three state wide in January 2015, after 12 months of PBF, there are clear differences in the increase: Ondo State for instance is flatlining (there are multiple reasons for that, related to health facility autonomy (there was very limited autonomy health facilities had to ask the district for permission on their expenses, also earnings from PBF were cut to 25% instead of the 50%). However, Adamawa and Nasarawa States show very large improvements.

Large variability in Institutional Deliveries across Health Centers Fufore District, Adamawa State Nigeria during 2013 Data include a mix of both quantity data (service productivitiy across service packages) and qualtiy data. This is an example of one specific service, institutional deliveries, that take off after introduction PBF in a pilot district in Adamawa State I December 2013. There is a LARGE VARIABIILTY in the increase. The Mayo Ine HC case is pretty famous: the health center went from 10% coverage for institutional deliveries to 100% coverage in a period of six months.

Burundi. Average total quality score for health centers, by province and time

Quthing District: average quality in health centers is the same after 12 months piloting of PBF due to autonomy problems This is an ICONIC picture of the impact of NO AUTONOMY on Implementation. The same as the previous slides, but averaging the qualities across health centers. The BEFORE and AFTER radar grapsh form a perfect match!!

3. Verifying Data Accuracy

NIGERIA: Quality of Care at PHCs : Raising the Bar Another area where we use data: to adjust the bar by changing the content of the quality measures….here is an example from Nigeria where end 2013, the qualtiy measures were adjusted to weight much heavier process measures of quality. There was a subsequent drop in quality results. Quality of care also improved significantly with emphasis on structural and process of care indicators (higher emphasis on process end 2013 leads to drop) Overall patient perceptions on quality of care is relatively satisfactory Counter-verification of the quality: relative large discrepancies

Concordance in 2015 and 2016 CCSS (quantity counterverification) clearly seems to work. Attempt to introduce mobile phone technology (Android SMART phones) with automatic upload and dashboards to increase frequency and to enhance analysis (in works)

Challenges to Measuring and Rewarding Quality Performance Ex-ante verification by district health team may be too gentle and not accurate: too close for comfort or still old fashioned ‘filling under the banana tree’? Regular counter-verification with credible sanctions are an important requirement Specifying incentives for district supervisors also seems a promising route (share of earnings; accreditation status; carrots and sticks) Introduction of modern ICT such as tablet based checklists, which embed meta data (location; time; interviewer passcode) seem a promising approach too

4. Innovations in Measuring and Paying for Quality

Vignettes Provide a Standard Measure of Practice Virtual Patient presents with symptoms Provider cares for a variety of clinical cases Provider goes through the different clinical domains as when they see a patient Take History Conduct a Physical Exam Order Tests Make a provisional diagnosis Decide on treatment

Technology Aids for Quality Measurements in PBF Tablets for quantified quality checklists (‘balanced score cards’) with automated uploads to a cloud based database and public dashboard. Offline data entry possible (as above) Tablet based solution for Vignettes (under development) Smart phone for community client interviews. Off line data entry possible. Automated uploads to a cloud based database and public dashboard. Results impact on performance payments Web-based public dashboard for performance benchmarking

In Summary Quality is poor and varied Much improvements in access and structural elements of care Improving clinical processes remains the big immediate challenge Innovations are happening in the space of measuring clinical processes Data from measurement needs to translate to decisions

THANK YOU