IAS Satellite Session, 25 July 2017 Tracking manufacturer performance to ensure the uninterrupted provision of timely, high quality early infant HIV diagnosis test results Esther Turunga IAS Satellite Session, 25 July 2017

Overview The point-of-care (POC) early infant diagnosis (EID) and viral load procurement consortium Defining indicators to monitor manufacturer performance Preliminary results of key performance indicator (KPI) monitoring Conclusions and recommendations

Convener and facilitator The global procurement consortium Step 1: Information sharing and joint negotiations with manufacturers Convener and facilitator Step 2: Procurement Supporting more than 100 countries Goal: Uninterrupted provision of timely, high-quality test results. Objective: Catalyze demand for and uptake of fit for purpose point-of-care technologies through affordable, transparent and sustainable pricing arrangements. Expected outcome: Improved and aligned prices, terms and conditions. CHAI UNICEF CHAI UNICEF EGPAF EGPAF MSF MSF The Global Fund USAID Consortium Negotiations Individual Negotiations based on orders The Global Fund Manufacturers USAID Orders placed by individual organizations, but aligned with consortium negotiations Product selection and procurement based on country choice and country operating plans.

Defining KPIs Purpose: Two types of KPIs: Procurement indicators Inform how manufacturer is performing with regards to agreed service levels, quality and cost; Identify room for improvement; and Ensure immediate problem-solving and corrective action. Two types of KPIs: Procurement indicators On-Time, In-Full principle Guaranteed shelf life Quality of delivered devices and consumables Service and maintenance costs In-country device performance indicators Rate of errors, invalids and undetermined Reported device and module breakdowns Response time from receipt of complaint and provision of replacement device Repair time

Early results of KPI monitoring Procurement indicators From October 2016 to June 2017, EGPAF monitored procurement indicators for POC EID devices and tests kits ordered for eight project countries: Cameroon, Côte d’Ivoire, Kenya, Lesotho, Mozambique, Rwanda, Swaziland, and Zimbabwe. Number of devices and cartridges delivered to countries (Pilot phase: November 2016 – June 2017) Country Number of devices Number of cartridges Cameroon 11 2,510 Cote D’Ivoire 5 700 Lesotho 2,520 Mozambique 13 2,500 Rwanda 2 200 Swaziland 6 950 Zimbabwe 10 2,250 TOTAL 52 11,630

Early results: Procurement indicators Number of orders and time taken until manufacturer acknowledge orders 27 orders Mean: 7 days Median: 4 days (range 0 – 50 days) Number of orders and rate of correct shipping documents provided (documents without errors) 23 orders; Rate: 35% Number of orders and rate of on-time delivery against agreed delivery dates N = 23 Rate: 17% Number of orders and rate of agreed guaranteed shelf life delivered Rate: 13% Number and rate of defective platforms and/or missing and accessories (e.g. modules, power cables, etc.) delivered 6 incidences reported; Rate - 13% Are repair services available through the manufacturer or a designated distributor and are replacement parts and/or devices available in-country Out of eight project countries currently implementing POC EID, four do not have designated distributors in-country Costs of annual service contract and/or increment price per test for service and maintenance Range – USD 2,250 - USD 2,280 per device per year if purchased with device in single transaction  or USD 2,500 – USD 2,898 per device per year if purchased separately

Early results: In-country device performance From December 2016 to June 2017, EGPAF monitored device performance indicators in six project countries: Cameroon, Cote d’Ivoire, Lesotho, Rwanda, Swaziland, and Zimbabwe. In-country device performance indicators Results Rate of errors, invalid and undetermined tests 8% (231 out of 2,922 tests performed) Number and rate of platform and module breakdowns reported per year N=7 breakdowns; Rate: 15% Median, range, and IQR time from receipt of complaint to response by manufacturer Median: 0.5 days; Range: 0 to 9 days; IQR: 2 days Median, range, and IQR time taken by manufacturer to provide replacement platform Median: 7 days; Range: 0 to 17 days; IQR: 13 days Median, range and IQR time taken by manufacturer to repair defective platform Median: 17 days; Range: 7-22 days; IQR: 10.5 days Number and proportion of breakdowns covered by the service warranty 7 breakdowns (proportion covered: 100%) Number and rate of intervention reports provided by manufacturer after repair of platform 1; Rate – 14%

Conclusions and recommendations Monitoring manufacturer performance is necessary to ensure the provision of uninterrupted timely, high-quality EID and viral load test results There is a need to standardize KPIs across procurement partners so that quality, price, delivery and service levels can be evaluated uniformly Members of the procurement consortium and other procurement partners should regularly share results of KPI monitoring, and provide regular joint feedback to manufacturers to continuously improve performance Manufacturers should develop and use service and maintenance tracking tools Performance indicators should be an explicit part of the manufacturer’s contract Is there a need to consider incentives and/or penalties for manufacturers who fail to meet expectations?

EGPAF gratefully acknowledges the valuable contributions of CHAI, UNICEF and USAID in defining the KPIs.

Thank you!