1. 1 Risks and Benefits of Home-Use HIV Test Kits Richard Forshee, Ph.D. U.S. Food and Drug Administration Center for Biologics Evaluation and Research Office of Biostatistics and Epidemiology BLOOD PRODUCTS ADVISORY COMMITTEE 96th Meeting, November 16-17, 2009 Bethesda Marriott Hotel 5151 Pooks Hill Rd., Bethesda, MD 20814

2. 2 Acknowledgments Mark Walderhaug, Hong Yang, and Arianna Simonetti Office of Biostatistics and Epidemiology Mark Walderhaug, Hong Yang, and Arianna Simonetti Office of Biostatistics and Epidemiology Elliot Cowan and Hilary Hoffman Office Of Blood Research & Review Elliot Cowan and Hilary Hoffman Office Of Blood Research & Review Bernard Branson, Arielle Lasry, and Stephanie Sansom Centers for Disease Control and Prevention Bernard Branson, Arielle Lasry, and Stephanie Sansom Centers for Disease Control and Prevention

3. 3 Purpose of Model 1 Design a tool to estimate the public health benefits and risks of a home-use HIV test kit under different test characteristics Design a tool to estimate the public health benefits and risks of a home-use HIV test kit under different test characteristics If benefits and risks could be quantified, it would be possible to define a region of sensitivity and specificity where the benefits exceed the risks If benefits and risks could be quantified, it would be possible to define a region of sensitivity and specificity where the benefits exceed the risks

4. 4 Purpose of the Model 2 Benefits and risks of different test outcomes are not comparable, so professional judgment is required Benefits and risks of different test outcomes are not comparable, so professional judgment is required The tool will help decision-makers make informed judgments about the likely tradeoffs The tool will help decision-makers make informed judgments about the likely tradeoffs We seek the committee’s input on these issues We seek the committee’s input on these issues

5. 5 Public Health Benefits and Risks Considered by Model Benefits Benefits –True Positive results –True Negative results Risks Risks –False Negative results –False Positive results –Failed Tests (invalid results or user error) Benefits and risks are in the context of persons who would not otherwise be tested Benefits and risks are in the context of persons who would not otherwise be tested

6. 6 Qualitative Summary of Public Health Benefits and Risks of Different Test Outcomes for a Home-Use HIV Test Kit

7. 7 True Positive Allows earlier medical intervention and entry into care Allows earlier medical intervention and entry into care Knowledge of individual HIV status allows for behavior modification to prevent HIV transmission, and allows for partner notification Knowledge of individual HIV status allows for behavior modification to prevent HIV transmission, and allows for partner notification Knowledge of HIV prevalence can allow for better targeting of public health resources as cases come to medical attention Knowledge of HIV prevalence can allow for better targeting of public health resources as cases come to medical attention

8. 8 True Negative Peace of mind Peace of mind Assistance in appropriate targeting of public health resources Assistance in appropriate targeting of public health resources

9. 9 False Positive Unnecessary personal anxiety Unnecessary personal anxiety Additional testing required Additional testing required Will people seek additional testing? Will people seek additional testing?

10. 10 False Negative False reassurance False reassurance Unsuspected transmission of virus and continued high risk behavior Unsuspected transmission of virus and continued high risk behavior Delayed medical intervention Delayed medical intervention False negative results may include tests taken during window period False negative results may include tests taken during window period

11. 11 Failed Test, Person is HIV+ Another test performed Another test performed –Delay in diagnosis –Additional testing No further testing No further testing –False reassurance –Unsuspected transmission of virus and continued high risk behavior –Delayed medical intervention

12. 12 Failed Test, Person is HIV- Another test performed Another test performed –Possibility for personal anxiety –Delay in determining status –Additional testing No additional testing No additional testing –Personal anxiety –Status remains unknown

13. 13 Subpopulations Modeled 1.Low Risk Heterosexuals (LRH): 0 or 1 partner in previous year 2.High Risk Heterosexuals (HRH): previous year 3.Men Who Have Had Sex With Men (MSM): previous year 4.Injectable Drug Users (IDU): previous year

14. 14 Time Frame of the Model Model estimates annual rates for 2nd and succeeding years after introduction by considering currently active at-risk group (with risky behaviors in past 12 months) as regular users of the test kit Model estimates annual rates for 2nd and succeeding years after introduction by considering currently active at-risk group (with risky behaviors in past 12 months) as regular users of the test kit Use of test kit might be higher or lower in the 1st year after introduction Use of test kit might be higher or lower in the 1st year after introduction

15. 15 Global Inputs 1.Percent of test kits that fail (5%) 2.Sensitivity of test kits that did NOT fail 3.Specificity of test kits that did NOT fail These are treated as hypothetical and will be estimated across a range of plausible values. Sensitivity does not include window period cases.

16. 16 Sensitivity Values Modeled Mean 99.5% 97.0% 95.0% 92.0% 90.0%

17. 17 Specificity Values Modeled Mean 99.5% 99.0% 98.5% 98.0% 97.5%

18. 18 Sub-population Specific Inputs 1.Size of sub-population 2.Percent of sub-population that is untested (previous year) 3.Percent of untested persons who are HIV+ 4.Percent of untested persons who would use a home-use HIV test (highly uncertain)

19. 19 Sub-population Inputs Presenting means only Uncertainty was included in model

20. 20 Sub-population Size

21. 21 Percent Untested

22. 22 Percent Untested HIV+

23. 23 Percent Untested Who Would Use a Home-Use Test Kit

24. 24 Structure of the Model 1 Size of the Sub-population Number Untested HIV +HIV - % Untested % HIV +

25. 25 Structure of the Model 2 HIV + Number Using Test False NegativeTrue Positive Failed Test % Failed Sensitivity % Using Test

26. 26 Model Results

27. 27 Absolute Numbers: Model Results Totals for all sub-populations Persons who would not otherwise be tested

28. 28 Test Sensitivity

29. 29 Test Specificity

30. 30 Using Ratios to Explore Tradeoffs As a surrogate for quantitative estimates of benefit and risk, ratios of benefits to risks can provide some insight into public health tradeoffs As a surrogate for quantitative estimates of benefit and risk, ratios of benefits to risks can provide some insight into public health tradeoffs We will explore the number of beneficial test outcomes to the number of adverse test outcomes We will explore the number of beneficial test outcomes to the number of adverse test outcomes There is no accepted threshold value for any of these ratios There is no accepted threshold value for any of these ratios Professional judgment is required to determine what ratio is acceptable Professional judgment is required to determine what ratio is acceptable

31. 31 True Positive to False Negative The ratio indicates how many newly identified True Positive results are obtained for each False Negative result The ratio indicates how many newly identified True Positive results are obtained for each False Negative result True Positive and False Negative results are related solely to the sensitivity of the test and the number of HIV+ persons taking the test True Positive and False Negative results are related solely to the sensitivity of the test and the number of HIV+ persons taking the test Failed tests for HIV+ persons are counted as False Negative results in this calculation Failed tests for HIV+ persons are counted as False Negative results in this calculation

32. 32 Tradeoff between True Positive Test Results and False Negative Test Results SensitivityRatio 99.520 9713 9510 927 906

33. 33 True Negative to False Positive The ratio indicates how many True Negative results are obtained for each False Positive The ratio indicates how many True Negative results are obtained for each False Positive True Negative and False Positive results are related solely to the specificity of the test and the number of HIV- persons taking the test True Negative and False Positive results are related solely to the specificity of the test and the number of HIV- persons taking the test Failed tests for HIV- persons are counted as False Positive results in this calculation Failed tests for HIV- persons are counted as False Positive results in this calculation

34. 34 Tradeoff between True Negative Test Results and False Positive Test Results SpecificityRatio 99.520 99.018 98.516 98.015 97.513

35. 35 True Positive to False Positive True Positive and False Positive results are related to both the sensitivity and the specificity of the test as well as the number of HIV+ and HIV- persons taking the test True Positive and False Positive results are related to both the sensitivity and the specificity of the test as well as the number of HIV+ and HIV- persons taking the test The ratio indicates how many True Positive results are obtained for each False Positive The ratio indicates how many True Positive results are obtained for each False Positive

36. 36 True Positive to False Positive Ratio 99.597.095.092.090.0 99.53.103.022.962.862.80 99.01.551.511.481.431.40 98.51.031.010.990.950.93 98.00.770.750.740.720.70 97.50.620.600.590.570.56 Specificity Sensitivity Under current assumptions, the ratio of true positive to false positive test results is more strongly related to specificity than to sensitivity.

37. 37 Current BPAC Recommended Performance The lower bound of the 95% confidence interval for sensitivity and specificity must be greater than or equal to 95% The lower bound of the 95% confidence interval for sensitivity and specificity must be greater than or equal to 95% What is the mean sensitivity and specificity needed to meet this? What is the mean sensitivity and specificity needed to meet this? Assume a 2% prevalence and a study population consisting of 100 positive subjects and 4,900 negative subjects Assume a 2% prevalence and a study population consisting of 100 positive subjects and 4,900 negative subjects

38. 38 Test Sensitivity to Meet BPAC Recommended Lower Bound of the 95% CI SuccessFailureMean Lower Bound 99199%97.0% 98298%95.2% 97397%93.6%

39. 39 Test Sensitivity to Meet BPAC Recommended Lower Bound of the 95% CI SuccessFailureMean Lower Bound 465524595.094.4 468022095.594.9 468421695.695.0 470419696.095.4

40. 40 Estimated ratio of true + to false + test results as a function of sensitivity and specificity of tests. Colors represent a temperature map. Assumes 111,000 HIV+ test users and 7,200,000 HIV- test users (mean values from the simulation). Specificity has a strong relationship with the ratio. Ratio of True+ to False+ by Characteristics Typical Currently Approved Rapid Tests BPAC Minimum (see assumptions) 7.6 0.34 Low Sensitivity, Low Specificity High Sensitivity, High Specificity

41. 41 Ratio of True+ and False+ by Test Characteristics. Estimated ratio of true + to false + test results as a function of sensitivity and specificity of tests. Colors represent a temperature map. Assumes 126,000 HIV+ test users and 6,000,000 HIV- test users (75 and 25 percentile values from the simulation). Specificity has a strong relationship with the ratio. Typical Currently Approved Rapid Tests BPAC Minimum (see assumptions) More HIV+ and Fewer HIV- Users 10.4 0.47

42. 42 Summary We have presented a tool to explore the public health benefits and risks of a home- use HIV test kit with different test characteristics We have presented a tool to explore the public health benefits and risks of a home- use HIV test kit with different test characteristics Tool provides estimates of the absolute numbers of each type of test result and three ratios to assist in making informed judgments about public health tradeoffs Tool provides estimates of the absolute numbers of each type of test result and three ratios to assist in making informed judgments about public health tradeoffs Each model result is important for determining the overall public health impact Each model result is important for determining the overall public health impact

43. 43 Discussion 1 Several of the input parameters have significant uncertainty because of a lack of data Several of the input parameters have significant uncertainty because of a lack of data Our estimates that a large number of HIV- persons will use the test implies that high specificity will be required to minimize False+ results Our estimates that a large number of HIV- persons will use the test implies that high specificity will be required to minimize False+ results Ratio of True + to False + is strongly related to specificity of the test under current assumptions Ratio of True + to False + is strongly related to specificity of the test under current assumptions Ratio of True+ to False- is related to sensitivity of the test Ratio of True+ to False- is related to sensitivity of the test

44. 44 Discussion 2 Assessing the public health impact of each test outcome requires a value judgment Assessing the public health impact of each test outcome requires a value judgment We seek the committee’s input regarding the appropriate balance between different test outcomes, e.g. how many False- test results would be acceptable for each newly identified True+ result? We seek the committee’s input regarding the appropriate balance between different test outcomes, e.g. how many False- test results would be acceptable for each newly identified True+ result?

45. 45 Thank you! Richard Forshee, Ph.D. FDA/CBERRichard.Forshee@fda.hhs.gov

46. 46 Reserve Slides

47. 47 Sub-population Size Anderson et al. 2009 Brady 2008 National Survey of Family Growth, various criteria National Survey of Family Growth

48. 48 Percent Untested Assumption Heimer 2007 National Health Interview Survey Jenness 2009

49. 49 Percent Untested HIV+ Entering drug treatment, CDC website Anderson et al. 2009 STD clinics, CDC website Testing of military applicants, CDC website

50. 50 Percent Untested Who Would Use a Home-Use Test Kit Assumption