1. Overview Trial Design George W. Rutherford, M.D.

2. Methodological Challenges in Biomedical HIV Prevention Trials Committee Members: Stephen Lagakos Harvey Alter Ronald Bayer Solomon Benatar Ronald Brookmeyer Carlos del Rio David Feigal Els Goetghebeur Laura Guay Sally Hodder Shabbar Jaffar Edward Kirumira George Rutherford Olive Shisana Gina Wingood IOM Staff:  Patrick Kelley  Alicia Gable  Sarah Scheening  Rachel Passman

3. Gates Foundation Charge to IOM Committee Examine methodological challenges in HIV prevention trials, meaning trials of biomedical interventions other than vaccines –to improve methodology, design, and conduct of HIV prevention trials –focus on microbicide and pre-exposure prophylaxis (PrEP) trials –increase likelihood of success and to enable donors to optimally invest resources

4. Charge (continued) “ This study will not address broader ethical issues such as adequacy of informed consent, compensation for trial-related adverse events, access to HIV treatment for seroconverters, and best practices for engaging community members.”

5. Specific Tasks 1. Review select phase 2 and 3 HIV prevention trials; provide an assessment of best practices for site preparedness and estimation of incidence. 2. Make recommendations for methodological best practices, including: loss of study power through lower-than-expected incidence and high pregnancy rates

6. Specific Tasks (continued) other design considerations, including choice of endpoints and control groups methods for monitoring the interim results of trials, including pooling of data from trials testing the same product methods for improving adherence to study regimens and the quality of self-reported behavioral data optimizing retention of trial participants.

7. Background/Premise Highly efficacious vaccine will not be available in near future New non-vaccine biomedical interventions not likely to have very high effectiveness (e.g, greater than 50- 60%). Yet modest/moderate advances would be very important in preventing new HIV infections. Effectiveness of most non-vaccine biomedical interventions will be heavily influenced by adherence and risk-taking behavior (high efficacy might translate into moderate effectiveness) Therefore, future late-stage trials need to be funded and designed to be able to –detect and quantify modest intervention effects and connection with behavior –adequately evaluate safety

8. Report Chapters 1.Basic design features 2.Behavioral risk reduction counseling 3.Pregnancy 4.Adherence 5.Recruitment and retention 6.Site preparedness 7.Estimating HIV incidence 8.Monitoring and Analysis 9.Alternative designs

9. Some Design Issues 1.Impact of incidence, adherence, effect size on sample size/power 2.Surrogate endpoints 3.Blinded and unblinded control arms 4.Randomized comparisons of concurrent behavioral interventions

10. 1. Impact of incidence, adherence, effect size on sample size/power Product adherence Biological efficacy Effectiveness 90% 81% 90%70%63% 90%50%45% 60%90%56% 60%70%42% 60%50%30% Non-adherence attenuates the observed treatment effect

11. Reduced effect size has substantial impact on power. If non- adherence reduces efficacy from 50% to 30% (RR=0.7), actual power drops from 90% to 40% 1.Impact of incidence, adherence, effect size on sample size/power

12. Recap: Sample Size, Power, Adherence Non-adherence can substantially reduce effectiveness, and lead to actual study power that’s much lower than planned Overestimating HIV incidence rate, or underestimating pregnancy or lost-to-follow-up rates can have similar detrimental effects on actual power Yet important to detect modest/moderate effect sizes Implications (Chapters 2, 4,5,6,7): Realistically estimate placebo HIV incidence, adherence, pregnancy and LTFU rates when planning sample size/duration; plan for realistic effect sizes Minimize LTFU rates, maximize adherence Continue to follow women who become pregnant for HIV outcomes

13. 2. Surrogate endpoints Ideally, (large and expensive) late-stage effectiveness trials would be undertaken after a successful proof-of-concept or efficacy trial With a reliable and more frequent surrogate endpoint, you can conduct proof-of-concept and efficacy trials with relative small numbers of subjects and in short time frame Without a reliable surrogate endpoint, efficacy trials must rely on incident HIV infection as an endpoint –Forces them to be larger and relatively expensive

14. 2. Surrogate endpoints (Chapter 2) Prioritize identification of reliable surrogate endpoint Consider hybrid designs: –‘Efficacy’ designs designed to detect short term effects but with some longer-term follow-up to assess ‘effectiveness’ –‘Effectiveness’ designs with interim-analysis stopping criteria based on the lack of short term efficacy

15. 3. Blinded and unblinded control arms Traditional: double-blind trial –Microbicide and Placebo arms: M versus P Alternatives: –M versus unblinded control: M versus C e.g., all subjects receive condom counseling; randomized ½ to also get M –M versus P versus C (e.g., HPTN 035) Issues: –Possible biological effects of placebo –Blinding: effects of knowledge of treatment arm on HIV infection risk (via effects on adherence and risk taking behavior) and on retention –Applicability of trial results to real-world use: makes them 50% bigger

16. 3. Blinded and unblinded control arms Trial Design: Microbicide vs Control vs Placebo (M vs P vs C) + Maintains advantages of previous designs and avoids disadvantages, except possibly differential retention between C and other arms + Comparison of P vs C could shed light on biological effects of P, and on how knowledge of not receiving M could affect behavior + Comparisons of M and P would be expected to reflect direct effects of M (relative to placebo) on HIV infection susceptibility -3-arm study more expensive than 2-arm study; but long-term ‘costs’ may be less HPTN 035 includes both control groups More such studies are encouraged

17. 4. Randomized comparisons of concurrent behavioral interventions in late-stage biomedical intervention trials Traditional design of biomedical HIV prevention trial (say, using a microbicide): –All subjects receive counseling for risk-taking behavior and offered condoms –Subjects randomized to microbicide (M) versus placebo (P) Designed to provide information about effect of biomedical intervention No direct assessment possible for effects of counseling Could be addressed by integrating randomized comparisons of behavioral intervention strategies into biomedical trials

18. 4. Randomized comparisons of concurrent behavioral interventions One way: partially blinded factorial design: Example: –Suppose 2 behavioral interventions (standard (S) versus intensive (I) counseling) –Microbicide (M) versus placebo (P) –4 arms: M+I, M+S, P+I, P+S

19. Partially blinded factorial design: assessing microbicide effects MicrobicidePlacebo Intensive Counseling MIMIPIPI Standard Counseling MSMSPSPS

20. Partially blinded factorial design: assessing behavioral intervention effects MicrobicidePlacebo Intensive Counseling MIMIPIPI Standard Counseling MSMSPSPS

21. Partially-blinded factorial designs: features No increase in sample size (if no qualitative interaction) Partial blinding to get similar risk taking behavior within the standard counseling and within the intensive counseling arms –Unlikely to lead to qualitative interaction Implementation requires early and close participation of behavioral researchers

22. Summary Without improving the sensitivity and efficiency of the evaluation process, modest yet important effects may be missed, and our ability to reduce risk-taking behavior—which impacts effectiveness of biomedical interventions--will not adequately advance. This argues for doing trials carefully, and not being overly optimistic in their planning and design We believe that this is doable, yet it will require that sponsors carefully consider budgets and time constraints so as to permit thi Is it worth the effort? Important to consider what’s at stake

23. Projected numbers of new HIV infections before a vaccine becomes available, assuming a 2.5% annual decrease in the number of infections in 2007 (NEJM, 4/10/08)