1. Measuring Adherence in Microbicide Trials Robert Pool Centre For International Health Research WHO-ICMR-CONRAD-IPM Meeting on Regulatory Issues in Microbicide Research New Delhi October 28-31 2007

2. THE ISSUE Testing the efficacy of microbicides requires accurate measurement of product use – adherence This also contributes to our understanding of acceptability

3. The problem Direct observation is impossible Self-reporting can be unreliable The topic is sensitive There are different kinds of non-adherence There are different reasons for over/under-reporting This has consequences for the accessibility of data and the methods used to collect it deliberate accidental

4. How can we collect information on behaviour and adherence? Self-reporting questionnaires open in-depth interviews self-assessment –diaries –ballot boxes –ACASI ( audio computer assisted self interview ) “Respondent-independent” Applicator counts Biological markers –Chemical traces in the body or on the applicator “Smart” applicators & rings

5. Advantages and disadvantages of self-reporting Standardised Easy to do and to analyse Large numbers & generalisation Avoids embarrassment Avoids desirability bias Immediate (diaries) Difficult to analyse Small numbers & limited generalisation Embarrassment & desirability Time lapse, memory OPEN INTERVIEWS Inflexible Hidden interpretation Embarrassment & desirability bias Time lapse, memory QUESTIONNAIRES Inflexible Impersonal SELF-ASSESSMENT Complementary Main criticism: self-reporting is unreliable per se. Evidence of respondents not being truthful Flexible Open interpretation Rapport, personal

6. Advantages and disadvantages of “respondent-independent” methods Easy for researchers More objective Objective Accurate Objective Accurate: “the chemical in the body” Microbicide chemicals not absorbed (?) Whose vagina? Trust Acceptability BIOMARKERS Inconvenient for participants Participants may not return applicators Doesn’t record other uses of product APPLICATOR COUNTS Whose vagina Trust Acceptability SMART APPLICATORS More objective, though still not perfect Problematic assumptions

7. Using different methods in microbicide research - some examples Pop Council Phase II study in S. Africa: compared interviews and ACASI Carraguard trial assessed product use by using a dye that stains vaginal mucous on used applicators Pop Council/HPRU are comparing ACASI & interviews, and comparing these to the dye-based technique and a technique for identifying semen in the vagina IPM is developing a “smart” applicator that could measure insertion CAPRISA trial: ancillary study using a biomarker (tenofovir disoproxil fumarate (TDF) levels in the genital tract) and comparing to in-depth interviews MDP: Triangulation between applicator returns and self-reporting through questionnaires, coital diaries and in-depth interviews

8. The best methods THREE POSSITIONS: 1.We already know what the best method is and should use/develop 2.We don’t know but should find the best method by comparison 3.Different methods will always be needed to cover different facets and we should try to develop the best combination This combination must be rigorous, practical and acceptable And should combine : –QUAL and QUANT –Self-report and respondent-independent –Self-assessment and face-to-face Self-report Independent QUAL QUANT - +

9. MDP 301 Objective: To determine the efficacy and safety of two concentrations of “ PRO 2000/5 Gel ” compared to placebo in preventing vaginally acquired HIV infection Six sites 9,673 HIV negative women randomised to 3 arms, followed for 12 months (24 in Uganda) The Microbicides Development Programme

10. Measuring adherence in MDP 301 Regular clinic interview, including questions on sexual behaviour and gel use Applicator returns, including used/unused at home or lost, compared to number given Memory aid Social science & triangulation with randomly selected sub- sample of 600 women

11. Applicator returns THE MDP TRIANGULATION PROCESS Women Clinic interview Coital Diary 4 weeks before CRF Comparison Form 1.Interview 2.Comparison & probing inconsistencies IDI T&T Nvivo Summaries Nvivo T&T IDI 1.Interview 2.Comparison with women Male partners Women, community Summaries T&T Nvivo FGDs FEEDBACK TO TRIAL

12. Inconsistency between questionnaire and triangulated data Tendency toward under-reporting of numbers of sex acts, gel and condom use in the clinical interviews Though there is also some over-reporting of gel and condom use Reasons for discrepancies: –desirability effect (over-reporting) –participant forgetting –participant making a mistake –participant misunderstanding –interviewer mistake  Inaccurate reporting is largely unintentional

13. # gel used reported in last week - # returned applicators/week Week 4: for woman who reported having sex in last week (n=4110) More returned than reported Less returned than reported 59% agree to ± 1 applicator 79% agree to ± 2 applicators

14. Consistency between reported gel use in clinic questionnaire and triangulated dataset compared to used applicator returns

15. Reasons for over/under-reporting gel use Confusion about when the week starts/ends Problems matching dates of interviews and applicator returns OVER-REPORTING Desirability bias UNDER-REPORTING Forgetting, mistakes, misunderstandings Double use (a couple of women) Inserting but no sex (reported at about 8% of visits) Use of product to “practice” (common at one site) Other uses of gel – “drying” and cleansing vagina (some evidence at some sites) Other uses of gel – hair gel, skin cream, etc. (anecdotal, rumour) Sharing (limited evidence) Squeezing out but not using (some evidence, particularly at some sites, but not widespread). Problematic because deliberate

16. Reasons for non-adherence ( 475 women in 852 data sets) Didn’t like gel/difficult to insert 3% Forgot5% Partner opposition10% Didn’t have it with her14% No opportunity to insert14% Ran out54%

17. Conclusion: reasons for continuing to use mixed methods, including qualitative methods, to study adherence Biological or electronic techniques are probably most accurate for reporting actual product use, but they are unlikely to be entirely foolproof And even if we have an effective biological or electronic technique, we still need to know and understand the reasons for non-adherence so that we can take measures to reduce it Adherence is also closely linked to acceptability, and understanding non-adherence is a necessary part of understanding acceptability So some form of mixed method design, including qualitative methods, will always be needed