Tobias Mielke QS Consulting Janssen Pharmaceuticals Considerations on Model-Based Approaches for Proof of Concept in Multi-Armed Studies JSM – 1st of August 2018 Tobias Mielke QS Consulting Janssen Pharmaceuticals

Problem statement Phase 2 of drug development: Show that the drug works Model the dose-response relation and pick dose for confirmatory testing Typical approach: Study 1: Top dose vs. Control to establish proof-of-concept Study 2: Multiple doses vs. Control to model dose-response Problem: Time, ressources and use of information How to use data better by combining PoC and Dose-Finding?

Dose Finding vs. PoC Good PoC designs: Study only most promsing (top) dose vs. control Good dose-finding designs: Evaluate the dose-response where the „action“ takes place. Problem: This place is not known in advance PoC only with (top) dose vs. control -> no information on dose-response

Adaptive PoC-DF example in OA (Miller et al. 2014) Endpoint: WOMAC painscore Assumption: Effect on top dose: 8mm, Standard deviation 22mm Design options evaluated One study: 3 doses vs. Placebo - 440 patients for 90% power at α=10% (Tukey). Too risky, as not enough knowledge on efficacy of compound Two studies: PoC (N=140) followed by separate DF (N=440) Too expensive, if DF initiated. No data available guiding selection of DF-doses. One combined study: PoC (N=140) + 2 new arms and all arms filled up to N=440. No data available guiding selection of additional doses One combined study: PoC (N=175 on 3 arms) with dose selection based on interim.

Adaptive PoC-DF example in OA (Miller et al. 2014) One combined study: PoC (N=175 on 3 arms) with dose selection based on interim: Design idea: medium study dose shall guide dose selection: ADDPLAN DF 4.0: Simulating & Analyzing adaptive dose-finding studies

Testing PoC: Test for any difference using MCPMod MCPMod for dose-finding (Bretz et al. (2005)) Test 𝐻 0 : 𝜇 0 = 𝜇 1 =…= 𝜇 𝐺 using optimized contrast test (e.g. Tukey test in Miller) Use „Optimized contrasts coefficients“ to test against flat dose-response: Coefficients 𝑐 0 ,…, 𝑐 𝐺 with 𝑖=0 𝐺 𝑐 𝑖 =0 Test 𝐻 0;𝑐 : 𝑐 𝑇 𝜇=0 to reject 𝐻 0 Given assumption 𝜇 ∗ on 𝜇: 𝑃 𝜇 ∗ =𝜇 𝑐 𝑇 𝑋 𝑐 𝑇 𝛴𝑐 > 𝑐 1−𝛼 =1−𝛷 𝑐 1−𝛼 − 𝑐 𝑇 𝜇 ∗ 𝑐 𝑇 𝛴𝑐 → 𝑚𝑎𝑥 𝑐 Benefits: Data is shared between doses Multiplicity doesn‘t depend on number of doses, but number of contrast vectors.

Testing PoC: Test for any difference using MCPMod Example: 𝑋~𝑁(𝑑,1), 𝑑∈ 0,1 Sample size required for 90% power at one sided α=2.5%: Optimal PoC design: Look only at „best dose“ vs. Control Interimediate dose possible, but will cost power/patients. Ways to retrieve some of the lost power: Unequal allocation: 40%, 20%, 40%: 55 patients required for linear trend Dose selection: e.g., „0.0, 0.1, 1.0“: 54 patients required for linear trend Study arms Pairwise comparisons Linear trend 2 (0,1) 46 3 (0, 0.5, 1) 75 66 4 (0, 0.33, 0.67, 1) 104 80 5 (0, 0.25, 0.5, 0.75, 1) 130 90

Testing PoC: Test for any difference Some problems with the conventional PoC: Each extra dose costs „power“ Proof of concept stage needs to be appropriately powered for „Go“ Uncertainty on true effect -> potentially under- or overpowered study Result only based on missing significance – no information on magnitude of effects A potential alternative: Lalonde et al.(2007): Set „minimum acceptable effect“ and „target effect“ Two error levels: α1 and α2: Stop, if effect significantly (α2) below target effect Go, if effect significantly (α1) above minimum acceptable effect and no „Stop“ Pause, else. Controls risk of dropping promising development at level α2

Testing PoC: Test for a target difference Generalization of Lalonde framework to multi-armed studies: Stop, if effect significantly below target (at α2) – you may not reach the target. 𝐻 0;𝑇𝑉 : 𝑖=1 𝐺 𝜇 𝑖 ≥ 𝜇 0 +𝑇𝑉 at α2 All hypothesis to be rejected at α2 -> no multiplicity correction required. Go, if some sign of efficacy (at α1): 𝐻 0;𝑀𝐴𝑉 : 𝑖=1 𝐺 𝜇 𝑖 ≤ 𝜇 0 +𝑀𝐴𝑉 at α1 At least one hypothesis to be rejected -> multiplicity correction required. Could use for this part MCPMod to test for any difference … But where is now the problem?

Testing PoC: Test for a target difference Testing: 𝐻 0;𝑇𝑉 : 𝑖=1 𝐺 𝜇 𝑖 ≥ 𝜇 0 +𝑇𝑉 at α2 If just one arm on the „Null“: Good. If multiple arms: Loss in power. 1, 2, 3, 4 active arms vs. Placebo Linear dose response 1, 2, 3, 4 active arms vs. Placebo All active doses on plateau (constant max. effect)

Testing PoC: Model based test for a target difference Testing: 𝐻 0;𝑇𝑉 : 𝑖=1 𝐺 𝜇 𝑖 ≥ 𝜇 0 +𝑇𝑉 at α2 Alternative approach: Use MCPMod models Dose response function: η 𝑑,𝜃 = 𝜃 0 + 𝜃 1 𝑓(𝑑, 𝜃 ∗ ) with 𝜃 ∗ defined in MCPMod BLUE: 𝐹 𝑇 𝛴 −1 𝐹 −1 𝐹 𝑇 𝛴 −1 𝑌 with 𝐹 𝑇 ≔ 1 … 1 𝑓( 𝑑 0 , 𝜃 ∗ ) … 𝑓( 𝑑 𝐺 , 𝜃 ∗ ) Estimator for difference at maximum dose: η 𝑑 𝐺 ,𝜃 −η 𝑑 0 ,𝜃 = 0,𝑓 𝑑 𝐺 , 𝜃 ∗ −𝑓 𝑑 0 , 𝜃 ∗ 𝐹 𝑇 𝛴 −1 𝐹 −1 𝐹 𝑇 𝛴 −1 𝑌= 𝛾 𝑇 𝜃 Distribution of estimator of difference: 𝛾 𝑇 𝜃 ~𝑁( 𝛾 𝑇 𝜃, 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾) Test for target effect: 𝑍= ∆−𝛾 𝑇 𝜃 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾 > 𝑧 1−𝛼 2 -> Maximum effect significantly below Δ. Model-based effect estimates and confidence intervals instead of pairwise comparisons. Shortcut:

Testing PoC: Model based test for a target difference Dashed line: Power for Linear Dotted line: Power for EMax Test for target effect: 𝑍= ∆−𝛾 𝑇 𝜃 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾 > 𝑧 1−𝛼 2 Model based test: Improves power but doesn‘t control error (biased estimator!) 1, 2, 3, 4 active arms vs. Placebo Linear dose response 1, 2, 3, 4 active arms vs. Placebo All active doses on plateau (constant max. effect)

Testing PoC: Model based test under model uncertainty … including model uncertainty Dose response functions: η 𝑑,𝜃 = 𝜃 0 + 𝜃 1 𝑓(𝑑, 𝜃 ∗ ) with 𝜃 ∗ from MCPMod BLUE: 𝐹 𝑇 𝛴 −1 𝐹 −1 𝐹 𝑇 𝛴 −1 𝑌 with 𝐹 𝑇 ≔ 1 … 1 𝑓( 𝑑 0 , 𝜃 ∗ ) … 𝑓( 𝑑 𝐺 , 𝜃 ∗ ) Estimator for difference at maximum dose: η 𝑑 𝐺 ,𝜃 −η 𝑑 0 ,𝜃 = 0,𝑓 𝑑 𝐺 , 𝜃 ∗ −𝑓 𝑑 0 , 𝜃 ∗ 𝐹 𝑇 𝛴 −1 𝐹 −1 𝐹 𝑇 𝛴 −1 𝑌= 𝛾 𝑇 𝜃 Distribution of estimator of difference: 𝛾 𝑇 𝜃 ~𝑁( 𝛾 𝑇 𝜃, 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾) Test for target effect: 𝑍= ∆−𝛾 𝑇 𝜃 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾 > 𝑧 1−𝛼 2 -> Maximum effect significantly below Δ. Reject only, if all model based estimates from MCPMod exclude Δ. Model-based effect estimates and confidence intervals instead of pairwise comparisons. Shortcut:

Testing PoC: Model based test under model uncertainty Dashed line: Power for Linear Dotted line: Power for EMax Dot-dash: Power for “both” Test for target effect: 𝑍= ∆−𝛾 𝑇 𝜃 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾 > 𝑧 1−𝛼 2 Test slightly overshoots target – but stopping probability controlled… 1, 2, 3, 4 active arms vs. Placebo Linear dose response 1, 2, 3, 4 active arms vs. Placebo All active doses on plateau (constant max. effect)

Testing PoC: Model based test under model uncertainty Dashed line: Power for Linear Dotted line: Power for EMax Dot-dash: Power for “both” … as long as true model is in the candidate set: … so what to do? 1, 2, 3, 4 active arms vs. Placebo Exponential dose response 1, 2, 3, 4 active arms vs. Placebo Sigmoidal dose response

Testing PoC: Model based test under model uncertainty … so what to do? Include more models into the candidate set of models: Higher chance for at least one model with bias in correct direction to control stopping probability Less likely to stop, as too many models need simultaneously „Stop“ Make proper dose-response modelling: Also fit the nonlinear parameters to the data Bias could be reduced, but effect estimates are now only asymptotically normally distributed Use Bayesian modelling instead: Assign prior model probability and prior distribution on all model parameters Given data, calculate credible intervals on maximum effect … will also generally not control false stopping probability

Thank you for your attention! References: Bretz, F., Pinheiro, J.C., Branson, M. (2005), “Combining Multiple Comparisons and Modeling Techniques in Dose-Response Studies” Biometrics, 61: 738-748 Lalonde, R.L., Kowalski, K.G., Hutmacher, M.M., Ewy, W., Nichols, D.J., Milligan, P.A., Corrigan, B.W., Lockwood, P.A., Marshall, S.A., Benincose, L.J., Tensfeldt, T.G., Parivar, K., Amantea, M., Glue, P., Koide, H. And Miller, R., (2007), “Model-based Drug Development”. Clinical Pharmacology & Therapeutics, 82: 21-32 Miller, F., Björnsson, M., Svensson, O. and Karlsten, R. (2014) “Experiences with an adaptive design for a dose-finding study in patients with osteoarthritis.” Contemporary Clinical Trials, 37: 189-199 Thank you for your attention!