UNITAID PSI HIV SELF-TESTING AFRICA Usability and Validity of Oral Fluid Self-Tests Among Intended Users: Experiences from Malawi, Zambia And Zimbabwe Dr Euphemia Sibanda – on behalf of STAR researchers

Presentation outline Need for optimisation of user instructions How optimisation was done in STAR Accuracy of oral fluid tests among intended users Planned work with blood- based tests

Instructions for use need optimisation Understandable instructions for use (IFU) critical for accuracy We conducted research to explore participant understanding of IFU optimise IFU Methods included Cognitive interviewing Video - recording of unassisted self- testing Accuracy studies comparing user results vs professional result Findings informed toolkit development

Cognitive interviews Population Urban & rural “Tell me what you think it means.” …. “Now go ahead and do what you think it says.” Zimbabwe Malawi Zambia Population Urban & rural Source of initial IFU version Adapted by Researcher Manufacturer* Information provided to ppt IFU only Demonstration + IFU # Recruited 14 29 17 IFU changes Several One None * Incorporated Zimbabwe findings

Findings from cognitive interviews Challenges interpreting symbols Layout critical for understanding Organisation of packaging Spatial instructions inadequate Importance of good translation Purpose and use of equipment “Looks like a plate on a stove” Man aged 27, Zimbabwe “…but the picture doesn’t make sense. What does the cutlery mean?” Man aged 20, Malawi Intended Observed

Accuracy studies Self-test results compared with national algorithm Zimbabwe (n=303) Malawi (n=291) Zambia (2552) Population and recruitment sites Urban & rural HTC static and mobile facilities Rural household survey household & health facility Information provided IFU, several iterations Demonstration + one IFU Observation during testing Video (n=262) None Video, n>100 Recruitment dates Jul 2014 – Jul 2015 May 2016 – Aug 2016 June 2016 – June 2017

Accuracy Results Self-test results compared with finger-prick RDTs (SOC) Zimbabwe (n=40) 2 HIV+ve 37 HIV-ve Malawi (n=291) 13 HIV+ve, 277 HIV-ve Zambia (n=2552) 240 HIV+ve 2312 HIV-ve % Agreement 92.3% 99.3% 99.1% Sensitivity 95% CI 100% 15.5% -100% 92.9% 66.1% - 99.8% 94.2% 90.4%-96.8% Specificity 91.9% 78.1%–98.3% 99.6% 98%-100% 99.7% 99.3% - 99.9% Accuracy improved with demonstration Lower accuracy among rural participants **Results exclude invalid self-tests**

How about clinical performance vs 4th Generation Laboratory gold standard? See poster TUPEC0842 2nd generation (IgG antibody only) tests have inherent limitations in sensitivity… E.g. Early HIV infection Important as we get closer to the first 90 Zambian study used 4th Gen Abbott & BioRad as alternative reference standard Mention PopART publication on misclassification (H.Ayles) OraQuick versus 4th Generation lab algorithm Agreement 98.5%, Cohen's kappa 0.9125 p<0.001 Sensitivity 87.6* (95% CI 83.0 - 91.4) Specificity 99.7 (95% CI 99.4 - 99.9) * Sensitivity of standard of care finger-prick RDT vs 4th Gen Lab: 93.5% (89.9-96.2)

Conclusions and recommendations Optimisation of IFUs is important for each context Can be marked urban:rural differences Iterative cognitive interviews and accuracy studies – recommended in STAR toolkit under development In Malawi, Zambia and Zimbabwe IFU alone insufficient – need for demonstration With demonstration accuracy was good against Standard of Care reference (finger-prick RDTs) Short video clip was adequate for this purpose However, against 4th Generation Reference standard Poor sensitivity for both Standard of Care and Oral Fluid HIVST Mainly reflecting limitations of 2th Gen RDTs

Planned work in Malawi and Zimbabwe IFU optimisation for two blood based tests Including one 3rd Gen test Accuracy among general population and female sex workers Comparison of user preference and accuracy of the two tests

Acknowledgements Study participants Ministries of Health in Malawi, Zambia and Zimbabwe