Research & Evaluation Improving measurement for improvement Thursday, 6 April 2016 Primary Health Care Improvement Global Stakeholders Meeting Geneva, Switzerland Rhea Bright, MPH Quality Improvement & HRH Advisor Office of Health Systems Bureau for Global Health USAID

At what scale are we working in Research and Evaluation? Multi-country: 4 Ukraine: 1 Ecuador: 1 Uganda: 8 Burundi: 2 Kenya: 1 Tanzania: 3 Malawi: 2 South Africa: 1 Swaziland: 3 20 India: 3 Mali: 3 Cote d’Ivoire: 1 Dem. Rep. Congo: 2 6

R&E: What are we trying to accomplish? Activity Scale 1. Validation of 25% of improvement indicators No less than 25% of total number of country reported indicators with completed validity assessment 2. Collecting data from control groups for 10% of indicators Comparison reports on no less than 10% of country-reported indicators 3. Evaluating design of improvement activities for lower- and middle-income countries Every ASSIST country program 4. Evaluation of methods and approaches for effective design and implementation of scale-up Selected ASSIST country programs with scale up activities

Teams independently monitor indicators

Indicator Validation: Tanzania Methods Retrospective, cross-sectional, cohort assessment was conducted in August 2015 3 facilities in Morogoro Urban District Data collected from patient record and facility registers to determine consistency, accuracy, and completeness of patient- and facility-level data Conducted interviews with staff who manage data

Findings: Facility data likely to be inaccurate Indicator Validation: Tanzania Findings: Facility data likely to be inaccurate Inconsistencies transferring patient data from care and treatment cards to facility registers High proportion of incomplete care and treatment cards Not enough data in patient files and records to report on some indicators

Recommendations to improve clinical effectiveness, data validity, and quality improvement processes at health facilities: Expand use of Open Medical Records System (OpenMRS) software (used at Regional Hospital) Train health workers to consistently and regularly update care and treatment cards and medical records Standardize symbols and abbreviations used by clinicians when entering patient data into the medical records, and denoting missing data

Recommendations Improve training for data clerks on national databases, search criteria and queries Expand health workers’ understanding of capabilities of database to perform searches and determine medication effectiveness Improve storage and security of files/records to facilitate rapid retrieval