Lessons learned from the Global Fund’s Price and Quality Reporting (PQR) database T Patrick Aylward, Sophie Logez, Thuy Huong Ha, Monika Zweygarth, Carmen Perez Casas, Luca Li Bassi

Major Topics Key Challenge: Data Quality Overview of the dataset Price Benchmarking at the Global Fund

Why is this important? Pharmaceutical and Health Products ~ 40% of expenditure. Increasing emphasis on value for money

What is Price and Quality Reporting? A web-based system for tracking the purchases of key pharmaceutical and health products Transactional data entered by Principal Recipients upon receipt of goods and verified by Local Fund Agents. PQR contains US$ 2.09 billion 1 of procurement data from 119 countries from Value of goods reported to the PQR as of 08 November 2011

What is Price and Quality Reporting? Reporting is mandatory for six product categories: 1.antiretrovirals, 2.antimalarial medicines, 3.anti-tuberculosis medicines, 4.bednets, 5.condoms, 6.rapid diagnostics tests for HIV and malaria All data is publicly available & updated every 24 hrs: Data will be fed into the WHO Global Price Reporting Mechanism (~25% of data)

Key Challenge: Data Quality 700+ users in 119 countries reporting data High turnover and little training User interface wasn’t acceptable to our customers No dedicated resources to monitoring and governing data Low data quality that prevented system from being used as an operational tool In 2010, the Global Fund overhauled the system to improve data quality and usability of the system

Key Challenge: Data Quality Extreme outliers decreased substantially The new PQR meets or exceeds the expectations of 83% of in-country users 81% of PR respondents agreed with the statement “The PQR is a useful tool that helps me in my job.”

The Dataset US$ 2.09 billion 1 of procurement data from 119 countries covering procurement transactions 1 Value of goods reported to the PQR as of 08 November 2011 CategoryCost (USD)%Total Bednet$889 M42.5% Anti-Retroviral$776 M37.1% Anti-malaria medicine$142 M6.8% Anti-TB medicine$132 M6.3% Diagnostic test$117 M5.6% Condom$37 M1.8% Grand Total$2,093 M100%

The Dataset Used to track trends, facilitate analysis and provide insight into market dynamics

The Dataset Used to track trends, facilitate analysis and provide insight into market dynamics Adult ARVs delivered to recipients in 2009 and 2010

The Dataset Encouraging recipients to compare prices against references and other countries

Price Benchmarking Global Fund Grants 2010 ARV prices generally at or below international reference sources

Price Benchmarking Global Fund Grants Source: PQR Downloaded 27 Sept South Africa data removed from analysis. -Médecins Sans Frontières. Untangling the Web of Antiretroviral Price Reductions. 13th ed. Campaign for Access to Essential Medicines; July Available from:

Price Benchmarking Global Fund Grants Transactions benchmarked against range of historical international reference prices at the time of purchase order Two Key measures: Price Ratio & Cost Above/Below Reference Indicators can be aggregated to the product, recipient, grant, country level Price benchmarking a routine part of grant reviews ConditionPrice RatioCost Above/Below Reference Actual price < reference range Actual price / lower bound of range At least: US$: (Actual price – lower bound)*qty Cost below reference Actual price falls within reference range 1.0US$ 0 Actual price > reference range Actual price / upper bound of range At least: US$: (Actual price – upper bound)*qty Cost above reference

Price Benchmarking Global Fund Grants Example: Country A Product Name Total Product Cost (USD) Reference Price Ratio Cost above reference Efavirenz (EFV) $ 6,510, $ 1,951,134 Efavirenz + Lamivudine + Tenofovir - FDC $ 700, $ 58,124 Lamivudine (3TC) $ 289, $ 44,827 Lamivudine + Nevirapine + Stavudine - FDC $ 13,755, $ 2,947,680 Lamivudine + Nevirapine + Zidovudine - FDC $ 22,551, $ 1,185,731 Lamivudine + Zidovudine - FDC $ 14,906, $ 779,353 Nevirapine (NVP) $ 1,675, $ - Stavudine (d4T) $ 423, $ 40,513 Grand Total $ 60,812, $ 6,891,115

Next Steps

Better reports for PRs

Next Steps

Conclusions The PQR is a public good and is becoming a useful tool to procurement practitioners as well as researchers The PQR helps the Global Fund ensure value for money and has helped to bring transparency to the market The vast majority of ARVs procured in 2010 were purchased at prices either at or below international reference ranges or in line with originator tiered schemes Data challenges remain and users need to be cautious about context and comparing like to like

QUESTIONS

Key Challenge: Data Quality Extreme outliers decreased from 15-20% of data points data points to 4-6% of data points, The new PQR meets or exceeds the expectations of 83% of PR & LFA respondents (up from 40%), 81% of PR respondents agreed with the statement “The PQR is a useful tool that helps me in my job.” PQR Version 3.6 PQR Version 4.0

Price Benchmarking Global Fund Grants Source: PQR Downloaded 27 Sept South Africa data removed from analysis. -Management Sciences for Health. International Drug Price Indicator Guide (accessed 1 September 2011); July Available from: