1. The Measurement of Antiretroviral Adherence in HIV
Sharon Mannheimer, MD
Harlem Hospital / Columbia University
Treatment Adherence Services Quality Learning Network meeting
May 3, 2007

2. Overview Background on Adherence in HIV Adherence Measurement
CASE Adherence Index
Other Self-Report Measures

3. Importance of Adherence
Nonadherence associated with:
Virologic failure
Worse immunologic (CD4) outcomes
Higher Hospitalization rates
OIs / HIV disease progression
Increased Mortality
Resistance (at some adherence levels)

4. Survival vs. Adherence
Progression to death among 847 initially ART-naïve HIV+ subjects with >12 mos. follow-up; adherence > 75% (circles) vs. adherence <75% (squares). Hogg, et al. AIDS 2002,16:

5. How much adherence is enough?

6. Adherence to Antiretroviral Therapy and Virologic Failure Paterson, Annals of Internal Medicine, 2000
82.1
71.4
66.7
54.6
21.7
mems
>95
<70
Adherence, %
Adherence (by MEMS) significantly associated with virologic failure
(P<0.001)

7. Adherence Measurement
No “gold standard”
Many methods are impractical in clinical settings
Simple measures predictive of HIV outcomes would be valuable

8. Classification of Adherence Measures:
Direct or
Indirect

9. Direct Measures of Adherence:
direct observation
measuring levels of the drug in body fluids (“Therapeutic Drug Monitoring”)
biologic markers
monitoring clinic attendance

10. Indirect Measures of Adherence:
self-report
provider assessment
electronic monitoring devices (MEMS)
pill count
medication refill rate (pharmacy)
monitoring for an expected therapeutic outcome

11. Direct vs. Indirect Measures:
In general, direct measures are more objective and yield more reliable assessments of adherence
each method has limitations

12. Problems with direct measures:
Direct observation: usually not practical
Therapeutic drug monitoring: costly, inconvenient, not widely available, reflects recent adherence only
Biologic marker (e.g. HIV viral load): may not correlate 100% with adherence, factors other than adherence could effect marker
Clinic attendance: does not necessarily correlate with medication adherence

13. Problems with indirect measures:
Self report: can overestimate adherence
Provider assessment: physicians poor at predicting adherent behavior
Electronic monitoring devices (e.g. MEMS caps): costly, bulky, for only 1 drug, measures only opening, interferes with pillbox use, inaccuracies can occur with improper use (e.g., pocketing doses)
Pill count: ”pill dumping,” patient can forget to bring bottles, does not assess timing
Refill rate: only practical if patients use same pharmacy, not 100% correlation

14. Benefits of Self-Report
Easy to administer
Inexpensive
May reveal reasons for missed doses
Self-report of nonadherence very reliable
Adherence measured by self-report correlates with HIV laboratory and clinical outcomes

15. Self-Report Methods No gold standard
AACTG 3- or 4-day self-report format widely used
Day-by-day, dose-by-dose recall of each ART med. Over prior 3 or 4 days
Other simpler formats available:
CPCRA 7-day self-report
Visual Analog Scale
CASE Adherence Index

16. The Case Adherence Index Questionnaire
Please ask each question and circle the corresponding number next to the answer, then add up the numbers circled to calculate Index score.
1. How often do you feel that you have difficulty taking your HIV medications
on time? By “on time” we mean no more than two hours before or two hours
after the time your doctor told you to take it.
4      Never
3      Rarely
2      Most of the time
1 All of the time
2. On average, how many days PER WEEK would you say that you missed
at least one dose of your HIV medications?
1      Everyday
2      4-6 days/week
3      2-3 days/week
4      Once a week
5      Less than once a week
6      Never
When was the last time you missed at least one dose of you HIV medications?
1      Within the past week
2      1-2 weeks ago
3      3-4 weeks ago
4      Between 1 and 3 months ago
5      More than 3 months ago
INDEX SCORE: ______ (> 10 = good adherence, < 10 = poor adherence)

17. Development of CASE Index
Developed during a large Health Resources and Services Administration (HRSA)-funded evaluation study ( ) of 12 US adherence support programs
Special Projects of National Significance (SPNS)
Cross-site evaluation coordinated by the New York Academy of Medicine’s (NYAM) Center for Adherence Support Evaluation (CASE)
Geographically diverse sites ; sites representative of the AIDS epidemic in the United States

18. CASE Cross-Site Evaluation
CASE insured uniform data collection:
Standardized core data elements
Uniform Instrument
Central interviewer and chart abstractor training
Uniform measurement periods
Adherence questions:
Individual questions about adherence behavior
AACTG 3-day self-report

19. Adherence Intervention & Evaluation Sites
Health Services Center, Inc., Hobson City, AL
Chase Brexton Health Services, Inc., Baltimore, MD
Dimock Community Health Center, Roxbury, MA
Harlem Hospital Center, New York, NY
Helena Hatch Special Care Center, Washington University, St. Louis, MO
Johns Hopkins University School of Medicine, Baltimore, MD
Mission Neighborhood Health Center, San Francisco, CA
Multnomah County Health Department, Portland, OR
SUNY Downstate Medical Center, Brooklyn, NY
St. Luke's Roosevelt Hospital Center, New York, NY
North Broward Hospital District, Ft. Lauderdale, FL
Urban Health Study, San Francisco, CA
Geographically diverse sites
St. Luke’s was on the original list of study participants. I’m not sure if they contributed any cases to Rajat’s analysis.

20. Participants in Adherence Analysis
1,154 participants in HRSA/SPNS cross-site study:
enrolled between July 1, 2000 and July 1, 2003
524 cases included in adherence analyses:
Had at least 1 follow-up
On ART at baseline and follow-up
had corresponding CD4 and HIV RNA data at the first 3-month follow-up
I would note that this rate of sample attrition was not unusual given the treatment population.

21. Participant Characteristics%
Gender
Male 65
Female 34
Transgender 1
Race/Ethnicity
African American 66
Latino 5
White 26
Education
Not high school graduate 41
High school graduate or GED 36
Some college/technical school 23
Mean age, years
(SD = 8.6)
N=524
34% women
High % nonwhites

22. Participant Characteristics - 2%
Self-reported HIV risk behavior
Men reporting sex with men (MSM) 29
Injecting drug use 16
MSM and injected drugs 2
Heterosexual contact 45
Heterosexual contact and injected drugs 3
Blood transfusion, blood components, or tissue
Other
Mean CD4 count, cells/mm3(SD)
256 (251)
Median CD4 count, cells/mm3
193
Mean log10 HIV RNA level (SD)
3.99 (1.35)
Heterosex most common, 16% idu, advanced hiv w/ low cd4 at entry

23. Analysis of HRSA/SPNS Cross-Site Adherence Data: Development of the CASE Index
Principal component analysis performed:
Responses to 3 adherence questions explained 69% of total variation in adherence, higher than any other combination of questions
Responses to each of the 3 CASE questions carried approximately equal importance
Mannheimer, et al. AIDS Care 2006;18:

24. The Case Adherence Index
3 adherence questions:
1. Frequency of “difficulty taking HIV medication on time (no more than two hours before or two hours after the time your doctor told you to take it)” –
Response options: Never, Rarely, Most of the time, or All of the time
2. frequency of “average number of days per week at least one dose of HIV medications was missed”
Response options: Everyday, 4-6 days per week, 2-3 days per week, Once a week, Less than once a week, or Never
3. “Last time missed at least one dose of HIV medications”
Response options: Within the past week, 1-2 weeks ago, 3-4 weeks ago, 1 to 3 months ago, More than 3 months ago, or Never
3 of the adh items in cross-site questionnaire were used to develop index:

25. The Case Adherence Index – Statistics / Scoring
Responses coded:
For #1 (reverse coded) - possible range of 1 to 4 points
For #2 and #3 - possible range of 1 to 6 points
Composite score obtained by adding responses:
Range 3 to 16
Higher scores indicate better adherence

26. The Case Adherence Index Questionnaire
Please ask each question and circle the corresponding number next to the answer, then add up the numbers circled to calculate Index score.
1. How often do you feel that you have difficulty taking your HIV medications
on time? By “on time” we mean no more than two hours before or two hours
after the time your doctor told you to take it.
4      Never
3      Rarely
2      Most of the time
1 All of the time
2. On average, how many days PER WEEK would you say that you missed
at least one dose of your HIV medications?
1      Everyday
2      4-6 days/week
3      2-3 days/week
4      Once a week
5      Less than once a week
6      Never
When was the last time you missed at least one dose of you HIV medications?
1      Within the past week
2      1-2 weeks ago
3      3-4 weeks ago
4      Between 1 and 3 months ago
5      More than 3 months ago
INDEX SCORE: ______ (> 10 = good adherence, < 10 = poor adherence)

27. CASE Index Compared to AACTG 3-day Self Report
CASE Index’s sensitivity and specificity relative to 3-day self-report at cut-off of ≥ 95% was calculated
Based on the analysis, CASE Index was recoded as a dichotomy where:
CASE Index scores > 10 indicated high adherence
CASE Index scores ≤ 10 indicated low adherence

28. Sensitivity and Specificity of CASE Adherence Index vs
Sensitivity and Specificity of CASE Adherence Index vs. 3-day recall adherence self-report
CASE Index Score
Sensitivity (%)
Specificity (%) 5
99.28
8.00 6
98.32
18.00 7
96.64
30.00 8
92.57
52.00 9
84.65
77.00 10
74.10
99.00 11
62.83 12
55.64
100.00 13
48.20 14
38.61 15
25.42

29. CASE Index Concurrent Validity with 3-day Self Report
Logistic regression showed strong correlation:
Odds of 3-day self-report > 95% was at least 60 times more for patients with a CASE Index score > 10 compared to those with a CASE Index score ≤ 10 (p < 0.001) across four serial cross-section follow-up periods (3, 6, 9 and 12 months after enrollment)
Receiver Operating Characteristic Curves (ROC) showed a very strong association between 3-day self-report at 95% and CASE Adherence Index Scores (>10 vs. < 10) across the four measurement quarters
Mannheimer, et al. AIDS Care 2006;18:

30. Relationships between Self-reported Adherence Measures and HIV RNA
CASE Adherence Index was strongly associated with:
a 1 log decrease in HIV RNA levels (p ≤ 0.05)
achieving HIV RNA < 400 copies/ml (p ≤ 0.05)
Association between 3-day self-report with HIV RNA was not as strong:
significance for a 1-log decrease from baseline HIV only at 6-month follow-up
significance for HIV < 400 copies/ml also only at 6-month follow-up
Changing gears- show how each measure correlates w/ hivrna (then CD4) outcomes

31. Relationship between Self-reported Adherence and HIV RNA
Adherence Measure
Comparison
HIV RNA
Measure
Odds ratio estimates
3 Months
6 Months
9 Months
12 Months
3-Day Self-Report
> 95%
vs. ≤ 95%
1-log Decrease
1.23
2.26**
1.53
1.16
< 400
0.97
2.30**
1.66*
CASE
Adherence
Index
> 10
vs. ≤ 10
1.52*
1.90**
1.76**
2.13**
1.60**
1.68**
1.87**
1.60*
Note: * p < ** p < 0.05
Serial cross-sections – assessing odds ratio of 1 log decr in hiv rna & hivrna < 400 w/
3d sr >95
CASE > 10 (CASE> fold higher odds of 1log decr and undetect rna vs index < 10)
3d sr signif for both only at 6 months

32. Relationships between Self-reported Adherence Measures and CD4 Lymphocyte Counts
A significant relationship between CASE Adherence Index and changes in CD4 lymphocyte counts from baseline only at 12 months
There were no observed relationships between changes in CD4 and 3-day self-report

33. Limitations of HRSA/SPNS data
Only 524 of 1,154 individuals in local sites’ adherence programs were included in adherence analyses
ART Naive and ART experienced but not currently receiving ART were excluded from the analyses
High attrition rates
Social desirability of self-report
Adherence instruments administered in same interview
Only self-reported adherence
Lapsed ART cases were those who were ever on ART but were not on ART at Baseline Intervention Entry.

34. CASE Index Summary a new measure of self-reported ART adherence
easy to administer and score
high degree of sensitivity and specificity with the 3-day self-report (concurrent validity)
a better predictor of HIV RNA changes over time than 3-day self-report
Should we say that neither method good at predicting CD4 outcome?

35. Other Self-Report Methods
AACTG 3- or 4- day recall
CPCRA 7-day recall
Visual Analog Scale

36. AACTG 3-day recall

37. CPCRA 7-day recall

38. Virologic Outcome by Adherence
Virologic Outcome by Adherence* in two CPCRA Antiretroviral Trials Mannheimer, et al. CID 2002
% HIV RNA
<50 copies
/ml
Self report
Month 1 (n=1074)
Month 4
Month 8
Month 12 )
(n=922)
(n=699)
(n=531)
P < for difference between categories at months 4,8,12
*by adherence self-report C•P•C•R•A

39. Immunologic Outcome by Adherence
Immunologic Outcome by Adherence* in two CPCRA Antiretroviral Trials Mannheimer, et al. CID 2002
Change in CD4
(cells/mm3) from baseline
Month 1
Month 4
Month 8
Month 12
(n=1074)
(n=922)
(n=699)
(n=531)
P < 0.05 for difference between categories at months 4,8,12
*by adherence self-report C•P•C•R•A

40. Visual Analog Scale
“Put a cross on the line below at the point showing your best guess about how much medication you have taken in the last month.
We would be surprised if this was 100% for most people, e.g. 0% means you have taken no medication; 50% means you have taken half your medication; 100% means you have taken every single dose of medication.”
_______________________________________________
0 10% 20% 30% 40% 50% 60% 70% 80% 90% %
Walsh AIDS 2002, 16: ; Oyugi JAIDS 2004;36:1100–1102

41. Summary Adherence critical for successful HIV treatment
Many methods available for measuring adherence
CASE Index
easy to administer and score
correlates with HIV RNA outcomes