1. Optimal cost-effective Go-No Go decisions Cong Chen*, Ph.D. Robert A. Beckman, M.D. *Director, Merck & Co., Inc. EFSPI, Basel, June 2010

2. Sorry for not being able to attend in person…

3. 33 Outline Introduction Introduction Benefit-cost ratio analysis of POC design strategies Benefit-cost ratio analysis of POC design strategies Discussion Discussion –POC strategy and risk mitigation –Phase III futility analysis

4. 44 How to fish smartly? Low success rate and predictability Constraint on societal cost Numerous POC possibilities Biology and tech revolution

5. 55 Proof-of-concept trial A randomized double-blinded phase II trial with type I/II error rate (α, β) for detection of Δ based on a surrogate marker A randomized double-blinded phase II trial with type I/II error rate (α, β) for detection of Δ based on a surrogate marker –Go to Phase III if p-value <α Choice of (α, β, Δ) is based on a heuristic argument in practice and is under-explored in statistical literature Choice of (α, β, Δ) is based on a heuristic argument in practice and is under-explored in statistical literature

6. 66 Issues to be addressed What is a more cost-effective sample size for a POC trial? What is a more cost-effective sample size for a POC trial? What is the optimal bar for a Go decision to Phase III? What is the optimal bar for a Go decision to Phase III? How to re-allocate resource when there are more POC trials of similar interest? How to re-allocate resource when there are more POC trials of similar interest?

7. 77 Benefit-cost ratio analysis Probability of Go if probability of drug truly active in the setting is POS Probability of Go if probability of drug truly active in the setting is POS –(1-POS)*α+POS*(1-β) Expected total sample size (SS) Expected total sample size (SS) –Phase II SS + Prob(Go)*Phase III SS Benefit cost ratio Benefit cost ratio –Power of carrying active drug (1-β) to Phase III divided by expected total SS

8. 88 Two designs Assumptions Assumptions –Same Δ of interest, e.g., 50% improvement in median progression-free-survival –Sample size for Phase III is fixed at 800 once a Go decision is made after POC Two choices of (α, β) Two choices of (α, β) –(10%, 20%) or a ~160 patient/~110 events trial –(10%, 40%) or a ~80 patient trial but higher empirical bar (~0.8Δ vs 0.6Δ) for a Go decision

9. 99 Results for comparison POSSizePr(Go)Power Expected total SS Power/ total SS 10%16017%80%3000.27 8015%60%2000.30 20%16024%80%3500.23 8020%60%2400.25 30%16031%80%4000.20 8025%60%2800.21 Smaller trial is more cost-effective. More gains (15- 30% improvement) can be realized after optimization.

10. 10 Optimal designs under fixed Phase II resource POS (α, β) Empirical GNG bar 0.1(6.7%, 26.7%) Δ 0.71Δ 0.2(7.2%, 25.3%) Δ 0.69Δ 0.3(8.0%, 23.7%) Δ 0.66Δ Assumptions: α, β Δ 1) Phase II is resourced for (α, β)=(0.1,0.2), which has an implicit Go bar of 0.6 Δ 2) Relative sample size of Phase II to Phase III is 20% (e.g., 160 pts vs 800 pts)

11. 1111 Resource optimization Budgeted for conducting one 160 patient POC trial, but has two POC trials of similar interest Budgeted for conducting one 160 patient POC trial, but has two POC trials of similar interest –Consensus is that one has higher POS (P1=30%) than the other (P2=20%) –Phase III trial uses same design once Go Two scenarios for comparison under varying ratio of POC budget (C2)/Phase III cost (C3) assuming sample size is proportional to cost Two scenarios for comparison under varying ratio of POC budget (C2)/Phase III cost (C3) assuming sample size is proportional to cost –Two drugs have same value –The one with lower POS has 50% higher value

12. 1212 Optimal resource split under same value

13. 1313 Optimal resource split and Go bar under same value

14. 1414 Optimal resource split and Go bar under different value

15. 1515 Conclusions Optimal (α, β) can be easily optimized from benefit-cost ratio analysis Optimal (α, β) can be easily optimized from benefit-cost ratio analysis Number of POC trials and respective Go bars depend on Phase II resource, Phase III cost, perceived POS and projected value Number of POC trials and respective Go bars depend on Phase II resource, Phase III cost, perceived POS and projected value Similar analysis reveals that a greater Δ has to be considered when relationship between surrogate marker and OS is less certain Similar analysis reveals that a greater Δ has to be considered when relationship between surrogate marker and OS is less certain –Uncertainty is highest in non-randomized trials!

16. 16 POC strategy More smaller trials, each with a higher Go bar, are generally preferred More smaller trials, each with a higher Go bar, are generally preferred –Adequately powered for larger Δ of true interest Similar analysis shows that simultaneous investigation is more cost-effective than sequential investigation Similar analysis shows that simultaneous investigation is more cost-effective than sequential investigation

17. 1717 Avastin POC strategy Indication#pts/armSource Colon33-36 JCO 2003; 21: 60-65 RCC37-40 NEJM 2003; 349: 427-434 NSCLC32-34 JCO 2004; 22: 2184-91 Breast10-18 Semin Oncol 2003; 5(suppl 16):117 All trials have 3 arms (low/high dose and placebo) with 80% power for doubling of PFS

18. 1818 PFS effect of recently approved innovative drugs Trial HR (central review) HR (local review) RCC/Sorafenib0.440.51 RCC/sunitinib0.420.42 CRC/panitumumab0.540.39 BC/lapatinib0.490.59 BC/Bev+pac vs pac 0.420.48

19. 1919 Pros and cons Smaller trials Smaller trials –Easier to accrue, faster to complete, and have better quality control –Empirical findings of large treatment effect are more exciting, and help with Phase III accrual –More vulnerable to baseline imbalance More trials More trials –Reduces missed opportunities (type III error) and increases overall probability of success –May inflate program level type I error rate

20. 2020 Risk mitigation Apply minimization or other randomization techniques for better baseline balance Apply minimization or other randomization techniques for better baseline balance Follow-up patients for survival after primary objective for Phase II is achieved Follow-up patients for survival after primary objective for Phase II is achieved –Initiation of Phase III may be delayed while waiting for Phase II OS data to mature –May revisit a Go or No-Go decision as necessary after OS data become available –Strength of OS data may be used for setting futility bar of Phase III trial as appropriate Revisit those less promising ones from Phase II after leading indications of same drug achieve major milestones in Phase III Revisit those less promising ones from Phase II after leading indications of same drug achieve major milestones in Phase III

21. 21 Futility bar at interim for an ongoing Phase III trial A hypothetical Phase III trial – –Designed to have 90% power for detection of Δ in OS before accounting for any futility analysis – –Trial stops for futility at interim if one-sided p- value > α based on survival info of fraction r after 50% of the cost is spent Benefit = overall power adjusted for futility – –May be further adjusted with value as needed Expected cost = 0.5+0.5*Prob(Go) – –where Prob(Go)=(1-POS)*α+POS*(1-β) and β satisfies Z α +Z β =r 1/2 (Z 0.025 +Z 0.1 )

22. 22 Benefit-cost ratio analysis at 25% info for 30% POS α (cut- off) Empirical bar Overall power Expected cost Power/ cost 1-∞90.0%1.000.90 0.6-0.16Δ88.3%0.861.03 0.5086.8%0.821.06 0.309*0.31Δ80.8%0.741.09 0.20.53Δ73.6%0.691.07

23. 23 Optimal futility bars POSInfo (r)α (cut-off p) Empirical bar Overall power 30%15%45.0%0.10Δ80.2% 20%36.8%0.23Δ80.3% 25%30.9%0.31Δ80.8% 50%15%51.6%-0.03Δ82.8% 20%42.5%0.13Δ82.6% 25%35.5%0.23Δ82.8% Optimal bar decreases with POS and increases with information. Positive trend is generally required.