for the Veterans Aging Cohort Study MEASURES OF UNHEALTHY ALCOHOL USE AMONG HIV INFECTED AND UNINFECTED PATIENTS IN THE VETERANS AGING COHORT STUDY (VACS) Kathleen McGinnis Amy Justice Richard Saitz Kevin Kraemer Kendall Bryant David Fiellin for the Veterans Aging Cohort Study

Objectives Summarize and compare alcohol use in HIV+ and HIV- patients enrolled in VACS Evaluate the AUDIT and AUDIT-C for identifying alcohol dependence and unhealthy use among HIV-infected and uninfected patients in care

Background As persons with HIV live longer due to effective antiretroviral therapies, it has become more important to understand the effects of alcohol on health HIV researchers and clinicians need valid and feasible measures for assessing alcohol use This is the first study to evaluate the AUDIT and AUDIT-C for identifying alcohol dependence and unhealthy use among HIV-infected patients in care

VACS Ongoing 8 Site Study 3,410 HIV-infected veterans 3,409 HIV-uninfected veterans Enrolled between 2002-2008 Self-complete surveys – Baseline and yearly follow-ups Telephone interviews – ~1/2 completed at baseline ~1/4 completed at follow-up

Methods

VACS Alcohol Instruments Self-complete survey – 10 item Alcohol Use Disorders Identification Test (AUDIT) 3 item Alcohol Use Disorders Identification Test (AUDIT-C) 3rd item of AUDIT asks about having 6 or more drinks on one occasion (heavy drinking episode) Telephone interview FU1 – approx ¼ of VACS Composite International Diagnostic Interview (CIDI)–Substance Abuse Module (SAM) 30 day time line follow-back (TLFB)

Analytic Sample 970 men completed the Composite International Diagnostic Interview – Substance Abuse Module (CIDI-SAM) 837 with alcohol data from all sources at Year 1 follow-up and with no more than 1 year in between self-completed survey and telephone interview Because the majority (over 95%) of the sample is male and cutoff’s for unhealthy drinking vary for males and females, we limited this analysis to males only

Reference (“Gold”) Standard Measures Alcohol dependence - based on a CIDI diagnosis of current (past year) alcohol dependence Unhealthy alcohol use CIDI diagnosis of current abuse or dependence Risky drinking based on 30 day TLFB >65 years: >7 drinks over 7 consecutive days or >3 drinks on one day <65 years: >14 drinks over 7 consecutive days or >4 drinks on one day

Comparison Measures AUDIT & AUDIT-C cut points with and without heavy drinking episode in past year

Analysis Various cutoffs of AUDIT and AUDIT-C were compared to the reference standards using Measures of performance Sensitivity Specificity Positive predictive value (PPV) Negative predictive value (NPV) Kappa statistics to measure agreement

Considering Measures of Performance Sensitivity – % with positive score of those with alcohol problem Specificity– % without positive score of those without alcohol problem PPV – % with alcohol problem of those with positive score NPV – % without alcohol problem of those with without positive score It is important to consider which measure is most important for the purpose of the study. For an intervention study, sensitivity (identifying a high percent of those with the problem) would be very important, although one may also want to use a second screen so that those w/out the problem aren’t receiving the intervention. Whereas to evaluate the proportion of people who were treated for a problem, PPV would be very important because one would want to make sure that those identified as having the problem actually have the problem.

Evaluating Kappa Statistics Poor 0.0-0.2 Fair 0.2-0.4 Moderate 0.4-0.6 Good 0.6-0.8 Excellent 0.8-1.0 This is our criteria for evaluating kappa statistics Landis, J.R. and Koch, G. G. (1977) "The measurement of observer agreement for categorical data" in Biometrics. Vol. 33, pp. 159—174

Results & Interpretation

Analytic Sample by HIV Status Characteristic HIV-infected (n=444) HIV-uninfected (n=393) P-value Age in years, mean (SD) 50 (8.4) 54 (9.8) <0.001 Race/ethnicity NS African American 56% 51% White 31% 36% Hispanic/other 13% Alcohol Measures Dependence (CIDI) 8% 7% Unhealthy Use (CIDI/TLFB) 22% 20% The HIV-infected are younger than the uninfected in this sample The race/ethnicty distribution is similar between groups Prevalence of dependence and unhealthy use in the past year is similar between the HIV infected and uninfected

Alcohol Dependence Agreement “Gold Standard” based on CIDI Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 16+ 32 98 57 95 .38 AUDIT 8+/4+* 58 89 30 96 .33 AUDIT 8+/4+ or heavy drinking episode 82 77 22 .26 AUDIT-C 6+ 38 93 29 .27 AUDIT-C 6+ or heavy drinking episode 74 79 97 Sensitivity is highest using AUDIT 8+/4+ or heavy drinking episode in past year; or using AUDIT-C 6+ or heavy drinking episode in past year. However, for those measures/cutpoints, PPV (% w/ alcohol dependence of those with a positive score) is low indicating a high percent of false positives. AUDIT-16+ provides the highest/best PPV, but is still not very good. There is a clear trade off between sensitivity and PPV depending on the measure/cutpoint used. Tradeoff between sensitivity and PPV

Alcohol Dependence Agreement “Gold Standard” based on CIDI Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 16+ 32 98 57 95 .38 AUDIT 8+/4+* 58 89 30 96 .33 AUDIT 8+/4+ or heavy drinking episode 82 77 22 .26 AUDIT-C 6+ 38 93 29 .27 AUDIT-C 6+ or heavy drinking episode 74 79 97 Kappa statistics represent “fair” agreement. All Kappa Statistics fall into the fair category Tradeoff between sensitivity and PPV

Alcohol Dependence Agreement “Gold Standard” based on CIDI Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 16+ 32 98 57 95 .38 AUDIT 8+/4+* 58 89 30 96 .33 AUDIT 8+/4+ or heavy drinking episode 82 77 22 .26 AUDIT-C 6+ or heavy drinking episode 74 79 97 HIV+ 31 94 .37 66 90 36 .41 78 76 19 .22 81 25 .29 HIV- 33 56 .39 48 88 23 .24 86 .30 20 .23 The pattern is similar when we examine alcohol dependence measures of agreement and performance by HIV status.

Alcohol Dependence Agreement “Gold Standard” based on CIDI Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 16+ 32 98 57 95 .38 AUDIT 8+/4+* 58 89 30 96 .33 AUDIT 8+/4+ or heavy drinking episode 82 77 22 .26 AUDIT-C 6+ or heavy drinking episode 74 79 97 HIV+ 31 94 .37 66 90 36 .41 78 76 19 .22 81 25 .29 HIV- 33 56 .39 48 88 23 .24 86 .30 20 .23 All Kappa Statistics fall into the fair (.2-.4) range Kappa statistics fall into the “fair” range.

Unhealthy Drinking Agreement “Gold Standard” based on CIDI and TLFB Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 8+/4+* 46 94 69 87 .46 AUDIT 8+/4+ or heavy drinking episode* 72 85 56 92 .52 AUDIT-C 5+ 52 95 74 88 .53 AUDIT-C 5+ or heavy drinking episode 86 58 91 Sensitivity is using AUDIT 8+/4+ or heavy drinking episode in past year; or using AUDIT-C 5+ or heavy drinking episode in past year, although these measures would still miss picking up around 30% with unhealthy drinking. Additionally, for those measures/cutpoints, PPV (% w/ alcohol dependence of those with a positive score) is low indicating a high percent (42%-44%) of false positives. Again, choosing measures with higher PPV will result in lower sensitivity. Again, there is a clear trade off between sensitivity and PPV depending on the measure/cutpoint used. Clear tradeoff between sensitivity and PPV

Unhealthy Drinking Agreement “Gold Standard” based on CIDI and TLFB Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 8+/4+* 46 94 69 87 .46 AUDIT 8+/4+ or heavy drinking episode* 72 85 56 92 .52 AUDIT-C 5+ 52 95 74 88 .53 AUDIT-C 5+ or heavy drinking episode 86 58 91 All Kappa Statistics fall into the moderate agreement range. Clear tradeoff between sensitivity and PPV All Kappa Statistics fall into the moderate agreement category

Unhealthy Drinking Agreement “Gold Standard” based on CIDI and TLFB Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 8+/4+* 46 94 69 87 .46 AUDIT 8+/4+ or heavy drinking episode* 72 85 56 92 .52 AUDIT-C 5+ 52 95 74 88 .53 AUDIT-C 5+ or heavy drinking episode 86 58 91 HIV+ 42 65 .41 68 90 .50 49 .49 64 59 HIV- 51 73 78 .55 55 96 77 89 .57 57 93 The pattern is similar when we examine unhealthy alcohol use measures of agreement and performance by HIV status.

Unhealthy Drinking Agreement “Gold Standard” based on CIDI and TLFB Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 8+/4+* 46 94 69 87 .46 AUDIT 8+/4+ or heavy drinking episode* 72 85 56 92 .52 AUDIT-C 5+ 52 65 74 88 .53 AUDIT-C 5+ or heavy drinking episode 86 58 91 HIV+ 42 .41 68 90 .50 49 .49 64 59 HIV- 51 95 73 78 .55 55 96 77 89 .57 57 93 All Kappa Statistics fall into the moderate (.4-.6) range Kappa statistics fall into the “moderate” range.

Summary Prevalence of alcohol dependence and unhealthy alcohol use is similar for HIV-infected and HIV-uninfected male patients (8% vs. 7%; and 22% vs. 20%) Of unhealthy drinking, the percent who are also alcohol dependent is similar between HIV+ and HIV- (36% vs. 35%)

Summary Performance measures and Kappa statistics were similar between HIV-infected and uninfected male patients for both alcohol dependence and unhealthy use for most measures Specificity & NPV was 85%+ for most measures Tradeoff between sensitivity and PPV Overall measure of agreement may not be as important as prioritizing sensitivity vs. PPV for the purpose of the study

Limitations Data for the self-completed survey and the telephone interview (TLFB and CIDI-SAM) were not collected at exactly the same time Analysis limited those with no more than 1 year in between their self-completed survey and telephone interview For 75% of these, the survey and interview were no more than 6 months apart Women were not included in the analysis because the % of women in VACS is small and there were no women in the HIV group with unhealthy drinking

Strengths Large sample of both HIV-infected and HIV-uninfected First study to evaluate alcohol measures among HIV-infected Ability to compare the measures between HIV-infected and HIV-uninfected Population is racially/ethnically diverse

Conclusions The SAM-CIDI and 30-day TLFB were collected with extensive effort (and therefore, cost), but for many studies a briefer measure is necessary Measures weren’t substantially different between HIV-infected and uninfected patients indicating that AUDIT is valid to use for persons with HIV When choosing between AUDIT vs. AUDIT-C and cut points, it’s important to consider the purpose of the study/identification Which measure of performance is more important

Note We do not show all AUDIT and AUDIT-C cut points evaluated due to space limitations We did not find substantial differences between the various recommended cut points shown earlier in this presentation Stay tuned for corresponding manuscript that will include comparing additional cut points for AUDIT and AUDIT-C not shown here

Acknowledgements PI and Co-PI: AC Justice, DA Fiellin Scientific Officer (NIAAA): K Bryant Participating VA Medical Centers: Atlanta (D. Rimland), Baltimore (KA Oursler, R Titanji), Bronx (S Brown, S Garrison), Houston (M Rodriguez-Barradas, N Masozera), Los Angeles (M Goetz, D Leaf), Manhattan-Brooklyn (M Simberkoff, D Blumenthal, H Leaf, J Leung), Pittsburgh (A Butt, E Hoffman), and Washington DC (C Gibert, R Peck) Core Faculty: K Mattocks (Deputy Director), K Akgun, S Braithwaite, C Brandt, K Bryant, R Cook, K Crothers, J Chang, S Crystal, N Day, R Dubrow, M Duggal, J Erdos, M Freiberg, M Gaziano, M Gerschenson, A Gordon, J Goulet, N Kim, M Kozal, K Kraemer, V LoRe, S Maisto, P Miller, P O’Connor, C Parikh, C Rinaldo, J Samet Staff: H Bathulapalli, T Bohan, D Cohen, A Consorte, P Cunningham, A Dinh, C Frank, K Gordon, J Huston, F Kidwai, F Levin, K McGinnis, C Rogina, J Rogers, L Sacchetti, M Skanderson, J Tate, E Williams Major Collaborators: VA Public Health Strategic Healthcare Group, VA Pharmacy Benefits Management, Massachusetts Veterans Epidemiology Research and Information Center (MAVERIC), Yale Center for Interdisciplinary Research on AIDS (CIRA), Center for Health Equity Research and Promotion (CHERP), ART-CC, NA-ACCORD, HIV-Causal Major Funding by: National Institutes of Health: NIAAA (U10-AA13566), NIA (R01-AG029154), NHLBI (R01-HL095136; R01-HL090342; RCI-HL100347) , NIAID (U01-A1069918), NIMH (P30-MH062294), and the Veterans Health Administration Office of Research and Development (VA REA 08-266) and Office of Academic Affiliations (Medical Informatics Fellowship).

Appendix (Extra Slides)

VACS Alcohol Use Compared to Other Samples Characteristic VACS HIV+ (n=444) HIV- (n=393) Saitz Rubinsky et al (n=392 outpatients at family medicine clinic) Alcohol Measures Dependence (CIDI) 8% 7% 4% 12% Unhealthy Use (CIDI/TLFB) 22% 20% 7-20% 33% Percent Dependent of Unhealthy Users 36% 35% 1/3 risky users are at risk for dependence 38%

Summary of Some Recommended AUDIT Cut Points to Use Risky drinking (Gual, et al, Alcohol and Alcoholism, 2002) 7+ for men 5+ Positive screen NIAAA Clinical Support Materials, (http://pubs.niaaa.nih.gov) 8+ for men up to 60 4+ for women, adolescents, and men over 60 Can also include binging in past year (5+ drinks for men; 4+ drinks for women) 6+ Hazardous Drinking (Gordon et al, J Fam Prac, 2001) 8+ 3+ Heavy drinking and/or active alcohol abuse or dependence (Bush et al, Arch Intern Med, 1998) See Table 3 (9) 4+ Five Categories (Bryson, Ann Intern Med, 2008) 0=nondrinker 1-2=low-level use 4-5=mild misuse 6-7=moderate misuse 8-12=severe misuse Zones of Risk (WHO, http://whqlibdoc.who.int) 0-7 I alc educ 8-15 II simple advice 16-19 III simple advice, brief counseling, and continued monitoring 20+ IV Ref to specialist for dx eval and trt Alcohol Misuse (Bradley et al, ACER, 2007)

Considering Measures of Performance Sensitivity – % with positive score of those with alcohol problem Specificity– % without positive score of those without alcohol problem PPV – % with alcohol problem of those with positive score NPV – % without alcohol problem of those with without positive score To provide an intervention, need ability to identify those with the problem!

Considering Measures of Performance Sensitivity – % with positive score of those with alcohol problem Specificity– % without positive score of those without alcohol problem PPV – % with alcohol problem of those with positive score NPV – % without alcohol problem of those with without positive score To evaluate treatment utilization, important that those with positive screen actually have an alcohol problem!

Considering Measures of Performance Sensitivity – % with positive score of those with alcohol problem Specificity– % without positive score of those without alcohol problem PPV – % with alcohol problem of those with positive score NPV – % without alcohol problem of those with without positive score We don’t want too many false positives!

Considering Measures of Performance Sensitivity – % with positive score of those with alcohol problem Specificity– % without positive score of those without alcohol problem PPV – % with alcohol problem of those with positive score NPV – % without alcohol problem of those with without positive score We don’t want to mis-identify those with an alcohol problem!

Considering Measures of Performance Sensitivity – % with positive score of those with alcohol problem Specificity– % without positive score of those without alcohol problem PPV – % with alcohol problem of those with positive score NPV – % without alcohol problem of those with without positive score To provide an intervention, need ability to identify those with the problem! We are going to focus on sensitivity and PPV To evaluate treatment utilization, important that those with positive screen actually have an alcohol problem!

Alcohol Dependence Agreement “Gold Standard” based on CIDI Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 16+ 32 98 57 95 .38 AUDIT 8+/4+* 58 89 30 96 .33 AUDIT 8+/4+ or heavy drinking episode 82 77 22 .26 AUDIT-C 6+ 38 93 29 .27 AUDIT-C 6+ or heavy drinking episode 74 79 97

Unhealthy Drinking Agreement “Gold Standard” based on CIDI and TLFB Comparison Measure Sensitivity Specificity PPV NPV Kappa Overall AUDIT 8+/4+* 46 94 69 87 .46 AUDIT 8+/4+ or heavy drinking episode* 72 85 56 92 .52 AUDIT-C 5+ 52 95 74 88 .53 AUDIT-C 5+ or heavy drinking episode 86 58 91 Sensitivity is using AUDIT 8+/4+ or heavy drinking episode in past year; or using AUDIT-C 5+ or heavy drinking episode in past year, although these measures would still miss picking up around 30% with unhealthy drinking. Additionally, for those measures/cutpoints, PPV (% w/ alcohol dependence of those with a positive score) is low indicating a high percent (42%-44%) of false positives. Again, choosing measures with higher PPV will result in lower sensitivity. Again, there is a clear trade off between sensitivity and PPV depending on the measure/cutpoint used.