1. Emerging Issues with Adaptation of Clinical Trial Design in Drug Development* H.M. James Hung Division of Biometrics I, Office of Biostatistics, OPaSS, CDER, FDA Presented in FDA/Industry Workshop, Bethesda, Maryland, September 16, 2005 *The view expressed here is not necessarily of the U.S. Food and Drug Administration.

2. J.Hung, 2005 FDA/Industry Workshop 2 Contributors Sue-Jane Wang (OB/OPaSS/CDER/FDA) Robert ONeill (OB/OPaSS/CDER/FDA) John Lawrence (DB1/OB/OPaSS/CDER/FDA) Acknowledgment Thanks are due to Charles Anello for valuable comments. This presentation is partly based on the research, supported by FDA/CDER RSR funds, #05-02, #05-14, 04-06, 02-06, 99/00-008, 96-010A.

3. J.Hung, 2005 FDA/Industry Workshop 3 Outline Design consideration Logistics issues Statistical design/analysis issues Regulatory experiences Utility and pitfalls of design adaptation Summary

4. J.Hung, 2005 FDA/Industry Workshop 4 Design Consideration As a guiding principle, for Phase III trial, fixed information design without any possibility of early stopping is preferred. minimize operational bias valid statistical test available unbiased estimator of treatment effect available valid confidence interval available These are the keys to the quality of statistical inference vital to ability of drawing conclusion from trial data.

5. J.Hung, 2005 FDA/Industry Workshop 5 Design Consideration Flexible designs Group-sequential design adaptive design Phase 2/3 combination design ………….. Utility of flexible designs in Phase II trials is probably obvious, at least for exploratory purpose, without subjecting to too many false positives.

6. J.Hung, 2005 FDA/Industry Workshop 6 Design Consideration Example: Surrogate markers are used in 1st phase of a clinical trial for predicting clinical effect or changing design in 2nd phase, e.g., - changing clinical endpoints - dropping or adding treatment arms - re-estimating sample size - enriching the patient population Clinical benefits are assessed at the end (second phase) of the trial where design specifications may have been modified.

7. J.Hung, 2005 FDA/Industry Workshop 7 Design Consideration When flexible design is considered, we need to ask: How to proceed logistically? - standard operation procedure (SOP) - ability to comply with SOP - responsibility of every participant Are trial conducts reviewable by regulatory agency? Goal: minimize operational bias

8. J.Hung, 2005 FDA/Industry Workshop 8 Design Consideration When flexible design is considered, we need to ask: Is valid statistical test available? Is unbiased estimator of treatment effect available? Is valid confidence interval available?

9. J.Hung, 2005 FDA/Industry Workshop 9 Logistics Issues Standard Operation Procedure charters of all parties including independent thirty party (e.g., DSMB, CRO) who see what? how to ensure compliance with SOP? how to minimize possible influence of adaptation on investigator/patient behavior? will knowledge of external trials have adverse influence on the current trial?

10. J.Hung, 2005 FDA/Industry Workshop 10 Statistical Design/Analysis Issues Much research has provided valid statistical methods for analysis of primary endpoint in flexible design trials - what point estimate and interval estimate to report in drug label needs more attention (have concern about using these estimates for future superiority/NI trials)

11. J.Hung, 2005 FDA/Industry Workshop 11 Statistical Design/Analysis Issues Unclear how to best analyze important secondary endpoints after primary endpoint is positive Unclear whether validity of multiplicity adjustment still holds Unclear how to incorporate design efficiency consideration into entire drug development program - involve prediction of benefit/risk ratio - per-trial statistical efficiency is insufficient

12. J.Hung, 2005 FDA/Industry Workshop 12 Regulatory Experience #1 In an oncology trial, time to progression (TTP) was a primary surrogate endpoint and overall survival (OS) was the ultimate primary endpoint. Group-sequential design was employed with an alpha-spending function for TTP. After TTP was highly significant at an interim time and the trial was stopped, how to test OS given that only 20% of the planned total deaths is available? - May use the spending function of TTP for OS

13. J.Hung, 2005 FDA/Industry Workshop 13 Regulatory Experience #2 In a heart failure trial, the primary endpoint is a composite endpoint of clinical outcomes (death, MI, stroke, etc.). Sample size and total # of events are increased, based on a new projected effect size from the data of an interim analysis. How to test the component endpoints and other secondary endpoints? - relationship in location parameters of and correlation between the endpoints need to be considered

14. J.Hung, 2005 FDA/Industry Workshop 14 Regulatory Experience #3 A trial is designed to study mortality endpoint. It is then divided into two subtrials to study a non-mortal endpoint (e.g., quality of life). - alpha allocation - replication concept - complication with interim analysis - p-value adjustment for the non-mortal endpoint - how to handle competing-risk problem

15. J.Hung, 2005 FDA/Industry Workshop 15 Regulatory Experience #4 Surrogate markers or biomarkers are used in early stages for predicting drug effect or changing design elements in later stages, e.g., - changing clinical endpoints - dropping or adding treatment arms - re-estimating sample size - enriching the patient population Clinical benefits are assessed at the end of the trial whose design specifications may have been modified.

16. J.Hung, 2005 FDA/Industry Workshop 16 Regulatory Experience #4 Q: Can clinical endpoint data from exploratory stages be combined with Phase III data in analysis for confirmatory purpose? If yes, how to proceed in analysis? - Weighted combination Z statistic and the corresponding point estimate and CI can be useful tools. Need to prespecify the weight ! - Concept of overall type I error can be unclear - Combining two stages at best results in one study

17. J.Hung, 2005 FDA/Industry Workshop 17 Regulatory Experience #4 - Issue with # of hypotheses to explore in Phase II (pertaining to statistical control of total type I errors)? No (or ignored) if Phases II and III are not combined Yes (needing attention) in Phases II and III are combined This makes little sense. [Hung, Wang, ONeill (2004)] How to proceed logistically?

18. J.Hung, 2005 FDA/Industry Workshop 18 Utility and Pitfalls of Design Adaptation Major pitfalls concern compromise of integrity of trial conduct. Major utility is potential saving of a trial if the planning assumptions fall short. Flexibility can enhance the probability of trial success with integrity protected if properly planned and complied.

19. J.Hung, 2005 FDA/Industry Workshop 19 Utility and Pitfalls of Design Adaptation For statistical analysis, with proper determination of weighting, a weighted combination of Z statistics can be attractive and powerful. For any fixed weight, Z is valid under the null hypothesis. A consistent estimator and valid confidence interval for the parameter Z tests are available to draw conclusion consistent with that from Z. How to find a good w?

20. J.Hung, 2005 FDA/Industry Workshop 20 Utility and Pitfalls of Design Adaptation If the weight w is random, then with properly adjusted critical value, Z can be valid. Q: Are point estimator and CI available to enable us to draw conclusion consistent with that from Z?

21. J.Hung, 2005 FDA/Industry Workshop 21 Utility and Pitfalls of Design Adaptation Dont rely too much on the conditional power evaluated at the interim observed estimate or any data observed at too early interim looks - planning a trial conservatively is still needed.

22. J.Hung, 2005 FDA/Industry Workshop 22 Summary More careful planning is necessary for flexible design (group-sequential or adaptive) in regulatory applications - operational bias is a critical issue Conventional statistical methods need tune up in use of flexible designs - adjust critical value of statistical test - find valid point estimate and CI

23. J.Hung, 2005 FDA/Industry Workshop 23 Summary Analysis methods suiting flexible designs may be preferable to properly adjusted conventional methods - need to explain the differences to our clients (can be very difficult) - still need valid point estimate and CI

24. J.Hung, 2005 FDA/Industry Workshop 24 Summary Too much adaptation should be discouraged - what hypothesis to test becomes unclear - protecting integrity of trial is difficult - what infrastructures are needed for logistics is still quite unknown - reviewability by regulatory agency is unknown and very difficult

25. J.Hung, 2005 FDA/Industry Workshop 25 Selected References Bauer, Köhne (1994, Biometrics) Bauer, Röhmel (1995, Stat. In Med.) Lan, Trost (1997, ASA Proceedings) Proschan, Hunsberger (1995, Biometrics) Fisher (1998, Stat. In Med.) Shen, Fisher (1999, Biometrics) Cui, Hung, Wang (1999, Biometrics) Bauer, Kieser (1999, Statistics in Medicine) Posch, Bauer (1999, Biometrical J.) Kieser, Bauer, Lehmacher (1999, Biometrical J.) Lehmacher, Wassmer (1999, Biometrics) Müller & Schäfer (2001, Biometrics) Hellmich (2001, Biometrics) Liu & Chi (2001, Biometrics)

26. J.Hung, 2005 FDA/Industry Workshop 26 Bauer, Brannath, Posch (2002, Method Inform Med) Brannath, Posch, Bauer (2002, JASA) Cui (2002, Encyclopedia of Biopharmaceutical Statistics) Li, Shih, Xie, Lu (2002, Biostatistics) Lawrence (2002, J. of Biopharmaceutical Statistics) Lawrence (2002, J. of Pharmaceutical Statistics) Lawrence & Hung (2002, Biometrical J.) Jennison & Turnbull (2003, Statistics in Medicine) Tsiatis & Mehta (2003, Biometrika) Liu, Proschan, Pledger (2003, JASA) Proschan, Liu, Hunsberger (2003, Statistics in Medicine) Wang, Hung, ONeill (2004, ASA Proceedings) Hung, Wang, ONeill (2004, ASA talk) Liu (2002, 2003, 2004, ASA talks, other talks) Lan (2000-2005 ASA, ICSA, FDA talks) Mehta (2004, talk on FDA-MIT workshop)

27. J.Hung, 2005 FDA/Industry Workshop 27 Hung, Wang, ONeill (2004, BASS talk) Lan, Soo, Siu, Wang (2005, J. of Biopharmaceutical Statistics) Hung, Cui, Wang, Lawrence (2005, J. of Biopharmaceutical Statistics) Hung (2005, WNAR talk) Wang, Hung (2005, ASA talk) Liu (2005, WNAR talk)