1. BAYESIAN ADAPTIVE DESIGN & INTERIM ANALYSIS Donald A. Berry dberry@mdanderson.org Donald A. Berry dberry@mdanderson.org

2. 2 2 Some references l Berry DA (2003). Statistical Innovations in Cancer Research. In Cancer Medicine e6. Ch 33. BC Decker. (Ed: Holland J, Frei T et al.) l Berry DA (2004). Bayesian statistics and the efficiency and ethics of clinical trials. Statistical Science. l Berry DA (2003). Statistical Innovations in Cancer Research. In Cancer Medicine e6. Ch 33. BC Decker. (Ed: Holland J, Frei T et al.) l Berry DA (2004). Bayesian statistics and the efficiency and ethics of clinical trials. Statistical Science.

3. 3 3 Benefits l Adapting; examples n Stop early (or late!) n Change doses n Add arms n Drop arms l Final analysis n Greater precision (even full follow-up) n Earlier conclusions l Adapting; examples n Stop early (or late!) n Change doses n Add arms n Drop arms l Final analysis n Greater precision (even full follow-up) n Earlier conclusions

4. 4 4 Goals l Learn faster: More efficient trials l More efficient drug/device development l Better treatment of patients in clinical trials l Learn faster: More efficient trials l More efficient drug/device development l Better treatment of patients in clinical trials

5. 5 5 OUTLINE: EXAMPLES l Extraim analysis l Modeling early endpoints l Seamless Phase II/III trial l Adaptive randomization n Phase II trial in AML n Phase II drug screening process n Phase III trial l Extraim analysis l Modeling early endpoints l Seamless Phase II/III trial l Adaptive randomization n Phase II trial in AML n Phase II drug screening process n Phase III trial

6. 6 6 EXTRAIM ANALYSES* l Endpoint: CR (detect 0.42 vs 0.32) l 80% power: N = 800 l Two extraim analyses, one at 800 l Another after up to 300 added pts l Maximum n = 1400 (only rarely) l Accrual: 70/month l Delay in assessing response l Endpoint: CR (detect 0.42 vs 0.32) l 80% power: N = 800 l Two extraim analyses, one at 800 l Another after up to 300 added pts l Maximum n = 1400 (only rarely) l Accrual: 70/month l Delay in assessing response *Modeling due to Scott Berry *Modeling due to Scott Berry

7. 7 7 l After 800 pts accrued, have response info on 450 pts l Find pred prob of stat sig when full info on 800 pts available l Also when full info on 1400 l Continue if... l Stop if... l If continue, n via pred prob l Repeat at 2 nd extraim analysis l After 800 pts accrued, have response info on 450 pts l Find pred prob of stat sig when full info on 800 pts available l Also when full info on 1400 l Continue if... l Stop if... l If continue, n via pred prob l Repeat at 2 nd extraim analysis

8. vs 0.80

9. 9 9 MODELING EARLY ENDPOINTS: LONGITUDINAL MARKERS l Example CA125 in ovarian cancer l Use available data from trial (& outside of trial) to model relationship over time with survival, depending on Rx l Predictive distributions l Use covariates l Seamless phases II & III l Example CA125 in ovarian cancer l Use available data from trial (& outside of trial) to model relationship over time with survival, depending on Rx l Predictive distributions l Use covariates l Seamless phases II & III

10. 10 CA125 data & predictive distributions of survival for two of many patients* > *Modeling due to Scott Berry *Modeling due to Scott Berry

11. Days Patient #1 Treatment

12. Patient #1

13. Days Patient #2

15. 15 Methods l Analytical l Multiple imputation l Analytical l Multiple imputation

16. 16 SEAMLESS PHASES II/III* l Early endpoint (tumor response, biomarker) may predict survival? l May depend on treatment l Should model the possibilities l Primary endpoint: survival l But observe relationships l Early endpoint (tumor response, biomarker) may predict survival? l May depend on treatment l Should model the possibilities l Primary endpoint: survival l But observe relationships *Inoue, et al (2002 Biometrics)

17. 17 Good resp Good resp No resp Survival advantage No survival advantage Phase 2 Phase 3 Conventional drug development 6 mos 9-12 mos > 2 yrs Stop Seamless phase 2/3 < 2 yrs (usually) Not Market

18. 18 Seamless phases l Phase 2: 1 or 2 centers; 10 pts/mo, randomize E vs C l If pred probs look good, expand to Phase 3: Many centers; 50 pts/mo (Initial centers continue accrual) l Max n = 900 [Single trial: survival data combined in final analysis] l Phase 2: 1 or 2 centers; 10 pts/mo, randomize E vs C l If pred probs look good, expand to Phase 3: Many centers; 50 pts/mo (Initial centers continue accrual) l Max n = 900 [Single trial: survival data combined in final analysis]

19. 19 Early stopping l Use pred probs of stat sig l Frequent analyses (total of 18) using pred probs to: n Switch to Phase 3 n Stop accrual for s Futility s Efficacy n Submit NDA l Use pred probs of stat sig l Frequent analyses (total of 18) using pred probs to: n Switch to Phase 3 n Stop accrual for s Futility s Efficacy n Submit NDA

20. 20 Conventional Phase 3 designs: Conv4 & Conv18, max N = 900 (same power as adaptive design) Conventional Phase 3 designs: Conv4 & Conv18, max N = 900 (same power as adaptive design) Comparisons

21. 21 Expected N under H 0

22. 22 Expected N under H 1

23. 23 Benefits l Duration of drug development is greatly shortened under adaptive design: n Fewer patients in trial n No hiatus for setting up phase 3 n All patients used for s Phase 3 endpoint s Relation between response & survival l Duration of drug development is greatly shortened under adaptive design: n Fewer patients in trial n No hiatus for setting up phase 3 n All patients used for s Phase 3 endpoint s Relation between response & survival

24. 24 Possibility of large N l N seldom near 900 l When it is, its necessary! l This possibility gives Bayesian design its edge [Other reason for edge is modeling response/survival] l N seldom near 900 l When it is, its necessary! l This possibility gives Bayesian design its edge [Other reason for edge is modeling response/survival]

25. 25 l Troxacitabine (T) in acute myeloid leukemia (AML) combined with cytarabine (A) or idarubicin (I) l Adaptive randomization to: IA vs TA vs TI l Max n = 75 l End point: Time to CR (< 50 days) l Troxacitabine (T) in acute myeloid leukemia (AML) combined with cytarabine (A) or idarubicin (I) l Adaptive randomization to: IA vs TA vs TI l Max n = 75 l End point: Time to CR (< 50 days) ADAPTIVE RANDOMIZATION Giles, et al JCO (2003)

26. 26 Adaptive Randomization l Assign 1/3 to IA (standard) throughout (until only 2 arms) l Adaptive to TA and TI based on current results l Results l Assign 1/3 to IA (standard) throughout (until only 2 arms) l Adaptive to TA and TI based on current results l Results

27. 27

28. 28 Compare n = 75 Drop TI

29. 29 Summary of results CR < 50 days: n IA:10/18 = 56% n TA: 3/11 = 27% n TI: 0/5 = 0% Criticisms... CR < 50 days: n IA:10/18 = 56% n TA: 3/11 = 27% n TI: 0/5 = 0% Criticisms...

30. 30 SCREENING PHASE II DRUGS l Many drugs l Tumor response l Goals: n Treat effectively n Learn quickly l Many drugs l Tumor response l Goals: n Treat effectively n Learn quickly

31. 31 Standard designs l One drug (or dose) at a time; no drug/dose comparisons l Typical comparison by null hypothesis: RR = 20% l Progress hopelessly slow! l One drug (or dose) at a time; no drug/dose comparisons l Typical comparison by null hypothesis: RR = 20% l Progress hopelessly slow!

32. 32 Standard 2-stage design First stage 20 patients: l Stop if 4 or 9 responses l Else second set of 20 First stage 20 patients: l Stop if 4 or 9 responses l Else second set of 20

33. 33 An adaptive allocation l When assigning next patient, find r = P(rate 20% | data) for each drug l Assign drugs in proportion to r l Add drugs as become available l Drop drugs that have small r l Drugs with large r phase 3 l When assigning next patient, find r = P(rate 20% | data) for each drug l Assign drugs in proportion to r l Add drugs as become available l Drop drugs that have small r l Drugs with large r phase 3

34. 34 Suppose 10 drugs, 200 patients 9 drugs have mix of RRs 20% & 40%, 1 has 60% (nugget) 9 drugs have mix of RRs 20% & 40%, 1 has 60% (nugget) <70% >99% Identify nugget … With probability: In average n: Identify nugget … With probability: In average n: 110 50 Adaptive also better at finding 40%, & sooner Standard Adaptive Standard

35. 35 Suppose 100 drugs, 2000 patients 99 drugs have mix of RRs 20% & 40%, 1 has 60% (nugget) 99 drugs have mix of RRs 20% & 40%, 1 has 60% (nugget) Adaptive also better at finding 40%, & sooner <70% >99% Identify nugget … With probability: In average n: Identify nugget … With probability: In average n: 1100 500 Standard Adaptive Standard

36. 36 Consequences l Treat pts in trial effectively l Learn quickly l Attractive to patients, in and out of the trial l Better drugs identified sooner; move through faster l Treat pts in trial effectively l Learn quickly l Attractive to patients, in and out of the trial l Better drugs identified sooner; move through faster

37. 37 PHASE III TRIAL l Dichotomous endpoint l Q = P(p E > p S |data) l Min n = 150; Max n = 600 l After n = 50, assign to arm E with probability Q n Except that 0.2 P(assign E) 0.8 l (Not optimal, but …) l Dichotomous endpoint l Q = P(p E > p S |data) l Min n = 150; Max n = 600 l After n = 50, assign to arm E with probability Q n Except that 0.2 P(assign E) 0.8 l (Not optimal, but …)

38. 38 Recommendation to DSMB to l Stop for superiority if Q 0.99 l Stop accrual for futility if P(p E – p S PF n PF depends on current n... l Stop for superiority if Q 0.99 l Stop accrual for futility if P(p E – p S PF n PF depends on current n...

39. 39 PF

40. 40 Common prior density for p E & p S l Independent l Reasonably non-informative l Mean = 0.30 l SD = 0.20 l Independent l Reasonably non-informative l Mean = 0.30 l SD = 0.20

41. 41

42. 42 Updating After 20 patients on each arm n 8/20 responses on arm 1 n 12/20 responses on arm 2 After 20 patients on each arm n 8/20 responses on arm 1 n 12/20 responses on arm 2

43. 43

44. 44 Assumptions l Accrual: 10/month l 50-day delay to assess response l Accrual: 10/month l 50-day delay to assess response

45. 45 Need to stratify. But how? Suppose probability assign to experimental arm is 30%, with these data...

46. 46

47. 47 One simulation; p S = 0.30, p E = 0.45 Final Std12/38 19/60 20/65 Exp38/83 82/16787/178 Superiority boundary

48. 48 9 mos. End Final Std 8/39 15/57 18/68 Exp 11/42 32/81 22/87 One simulation; p E = p S = 0.30 Futility boundary

49. 49 Operating characteristics

50. 50 FDA: Why do this? Whats the advantage? l Enthusiasm of PIs l Comparison with standard design... l Enthusiasm of PIs l Comparison with standard design...

51. 51 Adaptive vs tailored balanced design w/same false-positive rate & power (Mean number patients by arm) ORR Arm p S = 0.20 p E = 0.35 p S = 0.30 p E = 0.45 p S = 0.40 p E = 0.55 StdExpStdExpStdExp Adaptive681687917874180 Balanced171 203 216 Savings10331242514236

52. 52 Consequences of Bayesian Adaptive Approach l Fundamental change in way we do medical research l More rapid progress l Well get the dose right! l Better treatment of patients l... at less cost l Fundamental change in way we do medical research l More rapid progress l Well get the dose right! l Better treatment of patients l... at less cost

53. 53 OUTLINE: EXAMPLES l Extraim analysis l Modeling early endpoints l Seamless Phase II/III trial l Adaptive randomization n Phase II trial in AML n Phase II drug screening process n Phase III trial l Extraim analysis l Modeling early endpoints l Seamless Phase II/III trial l Adaptive randomization n Phase II trial in AML n Phase II drug screening process n Phase III trial