The role of site monitoring in quality assurance for point-of-care nucleic acid testing Dr. Raiva Simbi Deputy Director Laboratory Services Ministry of Health and Child Care, Zimbabwe IAS Satelite Session on Integrating POC NAT into Existing QA Programs in the AFRO Region July 2019, Mexico

Outline Background Approaches to external quality assessment (EQA) for point-of-care testing Site monitoring and mentoring in Zimbabwe: Process and tools Results and lessons learned Conclusions

Background MOHCC has been supported by EGPAF and CHAI/UNICEF, with funding from Unitaid, to implement point-of-care early infant diagnosis (POC EID) in Zimbabwe since July 2015 POC EID seeks to address: Poor access to and delays in EID testing Delays or no return of test result Poor ART initiation of HIV-positive infants on treatment MOHCC has rolled out POC EID to 77 testing sites and 230 spoke sites While coverage of conventional, laboratory-based EID screening has increased in recent years, effective testing could be greatly expanded through the integration of point-of-care testing into national EID laboratory networks.

Background Quality Assurance (QA) in point-of-care nucleic acid testing (POC NAT) is a major concern for MOHCC A deliberate effort is needed to monitor, evaluate, assess, and ultimately improve the quality of POC testing QA covers the entire testing cascade: pre-analytical phase (sample collection, handling, packaging, transport, staff competency, etc.) analytical phase (testing procedure, quality control) post-analytical phases (reporting, recording, communication of results, waste disposal)

Competency assessments of platform operators Different approaches used in Zimbabwe for external quality assessment (EQA) of POC NAT Site assessments and improvement plans/upgrades prior to enrolling a POC testing site Competency assessments of platform operators Frequent site monitoring and mentorship visits Proficiency panels administered through the Zimbabwe National Quality Assurance Program (ZINQAP) - Site assessment tool: Ensure the site conditions for quality testing (e.g. air conditioning, cold storage for specimens, thermometers for temperature monitoring) - Competency assessments for platform operators: Are platform operators proficient in operating the testing platform? Are staff competent with processes involved in sample draw, packaging, storage and transportation?

Site monitoring and mentoring process Joint teams conduct site monitoring and mentorship visits EGPAF, CHAI and MOHCC district supervisors Intervals for site monitoring post installation 2 weeks, 6 weeks, 12 weeks, and quarterly thereafter Two standardized checklists administered during the visits testing site monitoring checklist spoke site monitoring checklist Inclusion of district MOHCC staff in monitoring team To build capacity in anticipation for transition Issues picked during monitoring visit addressed through mentoring during same visit

Focus of site monitoring and mentoring visits What is assessed during a visit? Adherence to testing algorithms Patient flow Availability and use of SOPs, job aids, registers, tracking logs, and testing forms: Training and competency of instrument operators/end-users If possible, observation of operator(s)/end user(s) as they complete a test; Instrument placement and performance; Reagents and supply management; Waste management Linkage to care Based on the results of the assessment, immediate mentoring and training are provided – and follow up actions are defined

Experience and common findings of site monitoring and mentoring visits How findings were addressed 1. Clinical Integration On review of registers for infants eligible for HIV testing: majority of infants who qualified had sample drawn same day and analysed within 24 hours For the minority that had not been tested, sites were mentored on how to use their registers to identify eligible infants for HIV testing and make appointments Results were communicated to caregiver same day for the testing sites while spoke sites within 2 days Testing sites prioritized EID samples from certain locations where the risk of non-return for results collection was high Tracking and tracing was instituted where clients did not return for results Most infants found living with HIV were immediately initiated on ART according to national guidelines and had a confirmatory blood sample drawn Reasons for not initiating infants immediately were discussed with the site and corrective action put in place (deferred to treat OIs, refusal, death) Two discordant results confirmed using conventional testing

Experience and common findings of site monitoring and mentoring visits How findings were addressed 1. Clinical Integration Sites were identifying and testing infants from multiple entry points (e.g. pediatric wards, maternity wards) In sites where testing from multiple entry points was not happening, information was shared on the possibility of high yield from those entry points and sites were encouraged to do so

Experience and common findings of site monitoring and mentoring visits How findings were addressed 2. SOPs, Job Aids and Documentation Most sites had SOPs and guidelines displayed and these were used and adhered to by staff Sites without SOPs displayed were advised to display the SOPs and guidelines where they were visible to all staff Some sites had incomplete records in POC EID registers and forms Data on testing forms and registers were verified and corrected Testers and focal persons were advised to regularly review documentation for completeness Mentorship was given on how to complete the various forms and registers

Experience and common findings of site monitoring and mentoring visits How findings were addressed 3. Health Worker or Machine Operator Performance In all sites, tests were mainly conducted by trained staff but attrition due to rotations, leave, resignations made some sites use staff trained on-job. On-job training was conducted to fill gaps before training could be organized District management engaged so that training could be done before rotations, resignations or leave Staff observed testing procedures from sample collection to interpretation of results (blood draw, cartridge filling, closing and insertion, entry of sample ID) Mentorship was conducted on cartridge insertion and closing of the door as several errors were as a result of failure to do this correctly Staff observed universal safety precautions

Experience and common findings of site monitoring and mentoring visits How findings were addressed 4. Instrument Placement and Performance Most instruments were placed in secure areas. A few sites experienced delays in completion of site upgrades as government arm responsible not responsive in some districts Continued engagement of the government arm responsible for all infrastructure All sites conducted temperature monitoring to ensure conditions for optimal machine operation and cartridge storage were maintained In some sites, air conditioning was installed and refrigerators procured as part of minor upgrades Some sites delayed in notifying service provider of machine breakdowns Developed site specific communication plan together with the site staff Reminders by focal persons during routine site support

Experience and common findings of site monitoring and mentoring visits How findings were addressed 4. Instrument Placement and Performance Sites reported high error rates early on in the implementation of POC EID Causes of error were identified. On job mentorship was conducted and in some instances retraining was organized. A QI approach was implemented

Experience and common findings of site monitoring and mentoring visits How findings were addressed 5. Inventory and Waste Management Stock management was not optimal in some sites resulting in last minute orders and artificial stock-outs Sites mentored on stock management and how to use the data for future orders Redistribution of commodities where facilities were found overstocked or with near expiry commodities Potential for improper waste management at Cepheid Xpert sites Designed and procured a lockable waste safety cabinet for sites to prevent accidental collection and improper disposal of Xpert waste All waste was collected and disposed at approved facilities

Experience and common findings of site monitoring and mentoring visits How findings were addressed 6. Linkage to Care All sites are ART initiating sites and therefore do not refer infants for ART initiation elsewhere Most infants diagnosed HIV infected were successfully linked to ART services Reasons for delaying or not initiating infants on ART were discussed with the site Sites initiated tracking and tracing initiatives to bring the infants into care

Experience and common findings of site monitoring and mentoring visits How findings were addressed 7. Sample Transport from Spoke Sites and Receipt at Hub Testing Sites SOPs for sample collection, packaging, labelling and transportation were observed Spoke sites encouraged to continue Most specimens reached the hub testing within 24 hours and in good condition for analysis Specimens that could not be examined within 24hrs were refrigerated for testing within 72hours Most of the sample transporters received training on handling and transportation of specimens Sites had challenges with delayed reimbursement of sample transportation costs Sites mentored on completion of sample transportation log and encouraged to submit on time

Experience and common findings of site monitoring and mentoring visits How findings were addressed 7. Sample Transport from Spoke Sites and Receipt at Hub Testing Sites The use of all available sample transportation methods by most sites was observed This was encouraged as all samples were received at testing sites within 24 hours of collection Log book for recording of samples from spoke sites was available at most sites and updated Sites were encouraged to complete the log book real time to avoid missing some specimens

Conclusions Site monitoring: Is feasible and could be conducted with existing resources Focuses on the right things including the three stages of the testing cascade Pre-analytical, analytical and post analytical Allows sites to overcome and resolve challenges efficiently and quickly Supports the goal of quality assurance i.e. monitor, assess and ultimately improve the quality of POC testing

Acknowledgements MOHCC gratefully acknowledges: Funding and support from Unitaid Unitaid accelerates access to innovation so that critical health products can reach the people who most need them. The Elizabeth Glaser Pediatric AIDS Foundation (EGPAF) EGPAF is the global leader in the fight against pediatric HIV/AIDS, bringing dramatic change to the lives of millions of women, children, and families worldwide. CHAI/UNICEF The health care workers, mothers, children and families of Zimbabwe

Thank You