Assessing technical efficiency of HIV prevention interventions in three sub-Saharan countries S Bautista-Arredondo, SG Sosa-Rubí, D Contreras-Loya, M Opuni, A Kwan, C Chaumont, J Condo, N Martinson, J Coetzee, F Masiye, S Nsanzimana, J Wang'ombe, K Dzekedzeke, O Galarraga, and R Wamai on behalf of the ORPHEA study team July · 2014

Motivation -Need for implementing HIV programs with higher efficiency -Maximizing value for money -Lack of data on updated performance in the region -Previously published evidence suggested enormous heterogeneity in HIV prevention costs and potential waste (PANCEA, 2002) -Need to understand: -Current levels of efficiency -Determinants of more efficient performance

Information needs for optimizing HIV programs -Allocation among interventions -Effectiveness data -Allocation among populations/groups -Epidemiological and behavioral data -Allocation among health inputs -Performance data (M&E) -Determinants of efficiency -Interventions to improve efficiency -Which incentives work better and are more cost-effective? -How can M&E systems and changing in management practices can facilitate more efficient results

THE ORPHEA PROJECT

Aims Research question –Which characteristics predict the most efficient performance in the delivery of HIV services? Objectives –Measure and explain efficiency: -To estimate the total costs and the average cost per output, at the facility level -To estimate levels and determinants of efficiency –Provide recommendations HTC HIV testing and counseling HTC HIV testing and counseling PMTCT Prevention of Mother-to-child Transmission PMTCT Prevention of Mother-to-child Transmission

Key hypotheses -Heterogeneity of unit costs -High variability on average cost per service across facilities -Possible to identify the role of determinants and constraints -Modifiable characteristics that predict higher efficiency -Environment in which facilities operate and make decisions - Not possible to modify through interventions -Overlap between economics and management -Looking at performance at the facility level: potential for improving efficiency

METHODS

Measuring Efficiency Four HIV prevention interventions: HTC, PMTC, MC, FSW Four African Countries: Kenya, Zambia, South Africa, Rwanda Outputs: all services produced in the previous fiscal year Inputs: staff, essential recurrent inputs and services, capital, training and supervision Managerial and environmental characteristics: describing the environment and constraints in which production decisions are made -Identify constraints and determinants 8

Constraints from the firm’s perspective in the short term  Country/Location  Urban vs. rural setting  Funding sources  Facility type / Ownership  HIV/AIDS prevalence  Size of demand  Supply of services (utilities) Constraints from the firm’s perspective in the short term  Country/Location  Urban vs. rural setting  Funding sources  Facility type / Ownership  HIV/AIDS prevalence  Size of demand  Supply of services (utilities) Determinants, can be adjusted at the facility-level  Structure and governance  Training and staff composition  Management  Accountability  Incentives  Sanctions Determinants, can be adjusted at the facility-level  Structure and governance  Training and staff composition  Management  Accountability  Incentives  Sanctions Determinants of efficiency and constraints to more efficient performance

Microeconomic approach -Micro-costing -One-year retrospective data collection -Effort to measure staff’s time allocation (Time-motion) -Measurement of quality using exit interviews, clinical vignettes and the cascade approach -Data collection at different levels: Facility-level information  Staff roster  Drugs and supplies  Utilities  Equipment and buildings Facility-level information  Staff roster  Drugs and supplies  Utilities  Equipment and buildings District-level information  Training  Supervision District-level information  Training  Supervision National-level information  Salaries  Prices of supplies (HIV test kits, ART) National-level information  Salaries  Prices of supplies (HIV test kits, ART)

Measuring quality -Process quality using clinical vignettes and exit interviews -Try to capture quality of the program through the outcome measures using a “cascade” approach -Reflect definition of “comprehensive” prevention packages -Reflect hierarchy or sense of “effective” coverage -Assumption: higher quality of services can be captured by higher success of programs in achieving effective coverage -Example: PMTCT Pregnant women tested for HIV Pregnant women tested and positive for HIV Pregnant, HIV-positive women linked to ART

Estimating efficiency -Estimation of total annual input costs – at the facility level -Estimation of unit average cost per services along the HIV prevention services cascade -Correlation of unit average cost vs. scale of production, controlling for quality -Estimation of cost functions using a translog specification -Include determinants and constraints of efficiency in a joint equations system -Technical efficiency analysis using DEA or other methodology

RESULTS Kenya, Rwanda and Zambia

Unit cost breakdown

HTC PMTCT HTC Staff Composition PMTCT Staff Composition

Cost per client across the service cascade HTCPMTCT

Average cost vs scale for two stages in the cascade

ORPHEA: Policy Implications -Assessing the determinants of efficiency -Weak evidence of economies of scale in the first stage, much stronger in the second stage -Supervision seems to have an important role increasing efficiency -Incentives and complex governing structures increase costs -Our results suggest that quality of services is not the most important predictor of efficiency -Three promising approaches -Measuring performance at the clinical level and revealing disparities -Fairly simple management training and interventions -Looking into the production function of services: staff compositions

Acknowledgements -The ORPHEA study is supported by the Bill and Melinda Gates Foundation. -We gratefully acknowledge the collaboration of our academic partners in Kenya, Rwanda, South Africa, Zambia and the United States.