Results from the LA rapid testing study: What can they tell us about proposed point of care strategies? Kevin Delaney, MPH Division of HIV/AIDS Prevention Behavioral and Clinical Surveillance Branch Diagnostics Applications Team

Thank You (So Far…) LA County STD Peter Kerndt Apurva Uniyal Staff at LAGLC/Altamed DHAP/OD Bernie Branson BCSB Steve Etheridge Dawn Gnesda Duncan Mackellar Patrick Sullivan QSDM Debra Hansen Serum Bank Dollene Hemmerlein Lab Branch Debra Candal Trudy Dobbs Tom Folks Tim Granade Susan Kennedy Steve McDougal Michelle Owen

Outline Original study methods Problematic specimens Comparing strategies Why prevalence matters

Original Study Objectives Describe the performance of individual rapid tests –Sensitivity, specificity and predictive value Evaluate the performance of combinations of rapid tests –Particular focus on tests designed for use at point of care Foster discussion of a point of care diagnostic algorithm

Testing –Specimens from participants were tested with all 6 FDA approved rapid tests –Serum also tested with EIA and Western blot: Vironostika HIV-1 Microelisa system Genetic Systems HIV-1 Western Blot –Samples of both serum and plasma retained for future testing Tested at CDC with both Genprobe Aptima and Biorad HIV1/2 + O EIA Original Study Methods

Analysis –Sensitivity, specificity and predictive value for individual tests and combinations used in POC strategies: Performed in Sequence First two tests performed in Parallel Second and third must agree to be considered positive Proposed strategies 1, 2, 3 and 4 Original Study Methods

Line List of “Multiply Discordant” Specimens (n=8) – Including CDC results IDLA res AptimaBioradOQ-oOQ-bSp-bSp-plUG-bUg-plCo-bMS-plRe-pl 1 NPP X 3NNNNPNPN 4 NNNPPNNNNPNN 6 NNNNNNPNNNPN 9 NNNNNNNNNNPP 10 PNP X 3NNNNNPPN 11 PPP X 3NPPPNNPP 13 PPP X 3PPPNPNPN 15 PPP X 3PPPNPNPP

Line List of Western blot IND Specimens (n=6) – Including CDC results IDAptimaBioradOQ-oOQ-bSp-bSp-plUG-bUg-plCo-bMS-plRe-pl 16NNNNNNNNPN 17NNPPNNPPPPP 18PP X 3NNNNNNNN 19PP X 3NNNNNNNN 20NP X 3PPPPPPPP 21PP X 3PPPPPPPP

Comparing Strategies Bootstrap resampling –Random selection of individuals with replacement –Random selection of a test combination –1100 theoretical combinations of test and specimen type –Excluded: Combinations of the same test used on the same (n=290) Or different (n=200) specimen types Combinations that used the two Clearview tests together (n=152) Combinations with a CLIA moderate complexity test followed by a CLIA-waived test (n=232) Combinations in which Oraquick performed on oral fluid was used as the second or third test (n=21) –51 two-test and 154 three-test potential combinations

Comparing Strategies Bootstrap resampling –Includes variability introduced by individual test combinations e.g. starting with OMT vs. starting with blood “bad batch” of tests with low specificity –and problematic specimens e.g. likely specimen mix “specific non-specific” reactions with antigens –Medians and bootstrap confidence intervals reported

Comparing Strategies Diagnostic likelihood –The “relative risk” of being infected given that you test: Positive (PLHR)= sensitivity/(1-specificity) Negative (NLHR)= (1-sensitivity)/specificity Good test performance PLHR > 10 and NLHR <.1 Combining with prevalence will give Positive (PLHR) or Negative (NLHR) predictive value

Compare algorithms – Diagnostic Likelihoods = Remember these tests are GOOD! scenario Nlhr 2.5% Nlhr 50% Nlhr 97.5% Plhr 2.5% Plhr 50% Plhr 97.5% AUC 2.5% AUC 50% AUC 97.5% one test (Strategy 1) Two different Rapids (Strategy 2) Inf Two Oraquicks (Strategy 3) Inf Strategy 4+ (allow +,-,- = not infected) Inf Parallel screening Inf Tests must agree Inf

Sensitivity 1-Specificity ROC Curves for various POC Algorithms

Compare algorithms – Predictive Value (prevalence ≈ 5%) scenario Ppv 2.5% Ppv 50% Ppv 97.5% Npv 2.5% Npv 50% Npv 97.5% one test (Strategy 1) Two different Rapids (Strategy 2) Two Oraquicks (Strategy 3) Strategy 4+ (allow +,-,- = not infected) Parallel screening Tests must agree

Compare algorithms – Further testing; Incorrect results scenario Ind 2.5% Ind 50% Ind 97.5% Fn 2.5% Fn 50% Fn 97.5% Fp 2.5% Fp 50% Fp 97.5% one test (Strategy 1) Two different Rapids (Strategy 2) Two Oraquicks (Strategy 3) Strategy 4+ (allow +,-,- = not infected) Parallel screening Tests must agree Remember, these numbers won’t change for a given number of tests

Prevalence affects predictive value, if you have false positives… Predictive value positive – Single test Prevalence

The percentage of specimens going to the lab (red line) will increase as prevalence goes down Predictive value positive – Single test Percentage of tests that will require lab follow-up in strategies 2 and 4 Prevalence

In the worst case (Positive diagnostic likelihood ~ prevalence 70% of specimens would still require laboratory follow-up Predictive value positive – Single test Percentage of tests that will require lab follow-up in strategies 2 and 4 Prevalence

Compare algorithms – Further testing; Incorrect results scenario Ind 2.5% Ind 50% Ind 97.5% Fn 2.5% Fn 50% Fn 97.5% Fp 2.5% Fp 50% Fp 97.5% one test (Strategy 1) Two different Rapids (Strategy 2) Two Oraquicks (Strategy 3) Strategy 4+ (allow +,-,- = not infected) Parallel screening Tests must agree But in that case that’s still less than 20 specimens per 4000 tested…

Specificity of a single rapid test >99% –At low prevalence, the proportion of positive tests that are false positive increases. –Adding a second test resolved nearly all false positives correctly –There will likely always be some problematic specimens no testing strategy can fix Summary

All testing strategies perform exceptionally well –In low prevalence settings a POC strategy that resolves all false positives without lab intervention may be necessary –In high prevalence settings the sensitivity of the first test and the option to test for acute infection should be considered Summary

Sensitivity of all rapid tests was high –Comparable to current EIAs –Negative predictive value suggests < 2 false negatives/10,000 negatives 1% prevalence –Important to consider “realized sensitivity” Summary

Results – Sensitivity of a single test TestSpecimen typeTrue Pos False neg Sensitivity (95% CI) OraquickOral fluid ( ) Whole blood ( ) UnigoldWhole blood ( ) Plasma ( ) CompleteWhole blood ( ) Stat-pakWhole blood ( ) Plasma ( ) MultispotPlasma ( ) RevealPlasma ( )

Results – Specificity of a single test TestSpecimen typeTrue Neg False Pos Specificity (95% CI) OraquickOral fluid ( ) Whole blood ( ) UnigoldWhole blood ( ) Plasma ( ) CompleteWhole blood ( ) Stat-pakWhole blood ( ) Plasma ( ) MultispotPlasma ( ) RevealPlasma ( )

PV+ Results – Predictive value positive, single rapid test Prevalence

Initial Rapid Test Screen 2 nd Rapid test Screen Third Rapid Test “Tie- Breaker” Confirmed Positive Confirmed Negative Negative Positive Negative Positive Sequential Antibody Screening Algorithm

Initial two tests 3 rd Rapid test Confirmed Positive Confirmed Negative Negative/ Negative Either positive Positive Negative Parallel Antibody Screening Algorithm

Initial Rapid Test Screen Rapid Tests 2 and 3 Indeterminate Confirmed Positive Confirmed Negative Negative PositiveBoth Positive 1 of 2 Negative WHO Sequential Antibody Screening Algorithm (2 nd and 3 rd tests must agree) Both Negative

Results – 3 tests Sequential TestsAlgorithm Results 1 st test 2 nd test 3 rd test True Positive False Negative True Negative False Positive Clia-Waived OQ-bSP-bUG-b OQ-oUG-bSP-b SP-bOQ-bUG-b SP-bUG-bOQ-b CompleteOQ-bUG-b Clia- Moderate Complexity OQ-bUG-plMS-PL SP-bMS-PLRe-PL MS-PLRe-PLUG-PL

Results – 3 tests 1 st two in parallel TestsAlgorithm Results 1 st test 2 nd test 3 rd test True Positive False Negative True Negative False Positive Clia-Waived OQ-bSP-bUG-b OQ-oUG-bSP-b SP-bOQ-bUG-b SP-bUG-bOQ-b CompleteOQ-bUG-b Clia- Moderate Complexity OQ-bUG-plMS-PL SP-bMS-PLRe-PL MS-PLRe-PLUG-PL

Results – 3 tests Sequential All must agree TestsAlgorithm Results 1 st test 2 nd test 3 rd test Further Testing True Positive False Negative True Negative False Positive Clia-Waived OQ-bSP-bUG-b OQ-oUG-bSP-b SP-bOQ-bUG-b SP-bUG-bOQ-b CompleteOQ-bUG-b Clia- Moderate Complexity OQ-bUG-plMS-PL SP-bMS-PLRe-PL MS-PLRe-PLUG-PL

Results – 2 tests Sequential TestsAlgorithm Results 1 st test 2 nd test Further testing True Positive False Negative True Negative False Positive Clia-Waived OQ-bSP-b OQ-oUG-b SP-bOQ-b SP-bUG-b CompleteOQ-b Clia- Moderate Complexity OQ-bUG-pl SP-bMS-PL MS-PLRe-PL

Results – 2 tests - A1 Oral / Blood TestsAlgorithm results 1 st test 2 nd test 3 rd test Further testing True Positive False Negative True Negative False Positive Clia-Waived OQ-oSP-bOQ-b OQ-oUG-bOQ-b OQ-oCompleteOQ-b

Results – 3 tests Sequential, +-- not conclusive? TestsAlgorithm Results 1 st test 2 nd test 3 rd test True Positive False Negative True Negative False Positive Clia-Waived OQ-bSP-bUG-b OQ-oUG-bSP-b SP-bOQ-bUG-b SP-bUG-bOQ-b CompleteOQ-bUG-b Clia- Moderate Complexity OQ-bUG-plMS-PL SP-bMS-PLRe-PL MS-PLRe-PLUG-PL

Results – 3 tests - A1 Oral / Blood TestsAlgorithm results 1 st test 2 nd test 3 rd /4 th test Further testing True Positive False Negative True Negative False Positive Clia-Waived OQ-oSP-bOQ-b / UG-b OQ-oUG-bOQ-b / SP-b OQ-oCompleteOQ-b / UG-b OQ-oCompleteOQ-b / SP-b

Location –Two clinics in Los Angeles, CA Participants –Persons of unknown serostatus seeking HIV testing –Persons known to be HIV positive Original Study Methods