1. Tobias Mielke QS Consulting Janssen Pharmaceuticals
Considerations on Model-Based Approaches for Proof of Concept in Multi-Armed Studies JSM – 1st of August 2018
Tobias Mielke
QS Consulting
Janssen Pharmaceuticals

2. Problem statement Phase 2 of drug development:
Show that the drug works
Model the dose-response relation and pick dose for confirmatory testing
Typical approach:
Study 1: Top dose vs. Control to establish proof-of-concept
Study 2: Multiple doses vs. Control to model dose-response
Problem:
Time, ressources and use of information
How to use data better by combining PoC and Dose-Finding?

3. Dose Finding vs. PoC Good PoC designs:
Study only most promsing (top) dose vs. control
Good dose-finding designs:
Evaluate the dose-response where the „action“ takes place.
Problem:
This place is not known in advance
PoC only with (top) dose vs. control -> no information on dose-response

4. Adaptive PoC-DF example in OA (Miller et al. 2014)
Endpoint: WOMAC painscore
Assumption: Effect on top dose: 8mm, Standard deviation 22mm
Design options evaluated
One study: 3 doses vs. Placebo patients for 90% power at α=10% (Tukey).
Too risky, as not enough knowledge on efficacy of compound
Two studies: PoC (N=140) followed by separate DF (N=440)
Too expensive, if DF initiated. No data available guiding selection of DF-doses.
One combined study: PoC (N=140) + 2 new arms and all arms filled up to N=440.
No data available guiding selection of additional doses
One combined study: PoC (N=175 on 3 arms) with dose selection based on interim.

5. Adaptive PoC-DF example in OA (Miller et al. 2014)
One combined study: PoC (N=175 on 3 arms) with dose selection based on interim:
Design idea: medium study dose shall guide dose selection:
ADDPLAN DF 4.0: Simulating & Analyzing adaptive dose-finding studies

6. Testing PoC: Test for any difference using MCPMod
MCPMod for dose-finding (Bretz et al. (2005))
Test 𝐻 0 : 𝜇 0 = 𝜇 1 =…= 𝜇 𝐺 using optimized contrast test (e.g. Tukey test in Miller)
Use „Optimized contrasts coefficients“ to test against flat dose-response:
Coefficients 𝑐 0 ,…, 𝑐 𝐺 with 𝑖=0 𝐺 𝑐 𝑖 =0
Test 𝐻 0;𝑐 : 𝑐 𝑇 𝜇=0 to reject 𝐻 0
Given assumption 𝜇 ∗ on 𝜇:
𝑃 𝜇 ∗ =𝜇 𝑐 𝑇 𝑋 𝑐 𝑇 𝛴𝑐 > 𝑐 1−𝛼 =1−𝛷 𝑐 1−𝛼 − 𝑐 𝑇 𝜇 ∗ 𝑐 𝑇 𝛴𝑐 → 𝑚𝑎𝑥 𝑐
Benefits:
Data is shared between doses
Multiplicity doesn‘t depend on number of doses, but number of contrast vectors.

7. Testing PoC: Test for any difference using MCPMod
Example: 𝑋~𝑁(𝑑,1), 𝑑∈ 0,1
Sample size required for 90% power at one sided α=2.5%:
Optimal PoC design: Look only at „best dose“ vs. Control
Interimediate dose possible, but will cost power/patients.
Ways to retrieve some of the lost power:
Unequal allocation: 40%, 20%, 40%: 55 patients required for linear trend
Dose selection: e.g., „0.0, 0.1, 1.0“: 54 patients required for linear trend
Study arms
Pairwise comparisons
Linear trend
2 (0,1) 46
3 (0, 0.5, 1) 75 66
4 (0, 0.33, 0.67, 1)
104 80
5 (0, 0.25, 0.5, 0.75, 1)
130 90

8. Testing PoC: Test for any difference
Some problems with the conventional PoC:
Each extra dose costs „power“
Proof of concept stage needs to be appropriately powered for „Go“
Uncertainty on true effect -> potentially under- or overpowered study
Result only based on missing significance – no information on magnitude of effects
A potential alternative: Lalonde et al.(2007):
Set „minimum acceptable effect“ and „target effect“
Two error levels: α1 and α2:
Stop, if effect significantly (α2) below target effect
Go, if effect significantly (α1) above minimum acceptable effect and no „Stop“
Pause, else.
Controls risk of dropping promising development at level α2

9. Testing PoC: Test for a target difference
Generalization of Lalonde framework to multi-armed studies:
Stop, if effect significantly below target (at α2) – you may not reach the target.
𝐻 0;𝑇𝑉 : 𝑖=1 𝐺 𝜇 𝑖 ≥ 𝜇 0 +𝑇𝑉 at α2
All hypothesis to be rejected at α2 -> no multiplicity correction required.
Go, if some sign of efficacy (at α1):
𝐻 0;𝑀𝐴𝑉 : 𝑖=1 𝐺 𝜇 𝑖 ≤ 𝜇 0 +𝑀𝐴𝑉 at α1
At least one hypothesis to be rejected -> multiplicity correction required.
Could use for this part MCPMod to test for any difference
… But where is now the problem?

10. Testing PoC: Test for a target difference
Testing: 𝐻 0;𝑇𝑉 : 𝑖=1 𝐺 𝜇 𝑖 ≥ 𝜇 0 +𝑇𝑉 at α2
If just one arm on the „Null“: Good. If multiple arms: Loss in power.
1, 2, 3, 4 active arms vs. Placebo
Linear dose response
1, 2, 3, 4 active arms vs. Placebo
All active doses on plateau (constant max. effect)

11. Testing PoC: Model based test for a target difference
Testing: 𝐻 0;𝑇𝑉 : 𝑖=1 𝐺 𝜇 𝑖 ≥ 𝜇 0 +𝑇𝑉 at α2
Alternative approach: Use MCPMod models
Dose response function: η 𝑑,𝜃 = 𝜃 0 + 𝜃 1 𝑓(𝑑, 𝜃 ∗ ) with 𝜃 ∗ defined in MCPMod
BLUE: 𝐹 𝑇 𝛴 −1 𝐹 −1 𝐹 𝑇 𝛴 −1 𝑌 with 𝐹 𝑇 ≔ 1 … 1 𝑓( 𝑑 0 , 𝜃 ∗ ) … 𝑓( 𝑑 𝐺 , 𝜃 ∗ )
Estimator for difference at maximum dose:
η 𝑑 𝐺 ,𝜃 −η 𝑑 0 ,𝜃 = 0,𝑓 𝑑 𝐺 , 𝜃 ∗ −𝑓 𝑑 0 , 𝜃 ∗ 𝐹 𝑇 𝛴 −1 𝐹 −1 𝐹 𝑇 𝛴 −1 𝑌= 𝛾 𝑇 𝜃
Distribution of estimator of difference: 𝛾 𝑇 𝜃 ~𝑁( 𝛾 𝑇 𝜃, 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾)
Test for target effect: 𝑍= ∆−𝛾 𝑇 𝜃 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾 > 𝑧 1−𝛼 2
-> Maximum effect significantly below Δ.
Model-based effect estimates and confidence intervals instead of pairwise comparisons.
Shortcut:

12. Testing PoC: Model based test for a target difference
Dashed line: Power for Linear
Dotted line: Power for EMax
Test for target effect: 𝑍= ∆−𝛾 𝑇 𝜃 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾 > 𝑧 1−𝛼 2
Model based test: Improves power but doesn‘t control error (biased estimator!)
1, 2, 3, 4 active arms vs. Placebo
Linear dose response
1, 2, 3, 4 active arms vs. Placebo
All active doses on plateau (constant max. effect)

13. Testing PoC: Model based test under model uncertainty
… including model uncertainty
Dose response functions: η 𝑑,𝜃 = 𝜃 0 + 𝜃 1 𝑓(𝑑, 𝜃 ∗ ) with 𝜃 ∗ from MCPMod
BLUE: 𝐹 𝑇 𝛴 −1 𝐹 −1 𝐹 𝑇 𝛴 −1 𝑌 with 𝐹 𝑇 ≔ 1 … 1 𝑓( 𝑑 0 , 𝜃 ∗ ) … 𝑓( 𝑑 𝐺 , 𝜃 ∗ )
Estimator for difference at maximum dose:
η 𝑑 𝐺 ,𝜃 −η 𝑑 0 ,𝜃 = 0,𝑓 𝑑 𝐺 , 𝜃 ∗ −𝑓 𝑑 0 , 𝜃 ∗ 𝐹 𝑇 𝛴 −1 𝐹 −1 𝐹 𝑇 𝛴 −1 𝑌= 𝛾 𝑇 𝜃
Distribution of estimator of difference: 𝛾 𝑇 𝜃 ~𝑁( 𝛾 𝑇 𝜃, 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾)
Test for target effect: 𝑍= ∆−𝛾 𝑇 𝜃 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾 > 𝑧 1−𝛼 > Maximum effect significantly below Δ.
Reject only, if all model based estimates from MCPMod exclude Δ.
Model-based effect estimates and confidence intervals instead of pairwise comparisons.
Shortcut:

14. Testing PoC: Model based test under model uncertainty
Dashed line: Power for Linear
Dotted line: Power for EMax
Dot-dash: Power for “both”
Test for target effect: 𝑍= ∆−𝛾 𝑇 𝜃 𝛾 𝑇 𝐹 𝑇 𝛴 −1 𝐹 −1 𝛾 > 𝑧 1−𝛼 2
Test slightly overshoots target – but stopping probability controlled…
1, 2, 3, 4 active arms vs. Placebo
Linear dose response
1, 2, 3, 4 active arms vs. Placebo
All active doses on plateau (constant max. effect)

15. Testing PoC: Model based test under model uncertainty
Dashed line: Power for Linear
Dotted line: Power for EMax
Dot-dash: Power for “both”
… as long as true model is in the candidate set:
… so what to do?
1, 2, 3, 4 active arms vs. Placebo
Exponential dose response
1, 2, 3, 4 active arms vs. Placebo
Sigmoidal dose response

16. Testing PoC: Model based test under model uncertainty
… so what to do?
Include more models into the candidate set of models:
Higher chance for at least one model with bias in correct direction to control stopping probability
Less likely to stop, as too many models need simultaneously „Stop“
Make proper dose-response modelling:
Also fit the nonlinear parameters to the data
Bias could be reduced, but effect estimates are now only asymptotically normally distributed
Use Bayesian modelling instead:
Assign prior model probability and prior distribution on all model parameters
Given data, calculate credible intervals on maximum effect
… will also generally not control false stopping probability

17. Thank you for your attention!
References: Bretz, F., Pinheiro, J.C., Branson, M. (2005), “Combining Multiple Comparisons and Modeling Techniques in Dose-Response Studies” Biometrics, 61: Lalonde, R.L., Kowalski, K.G., Hutmacher, M.M., Ewy, W., Nichols, D.J., Milligan, P.A., Corrigan, B.W., Lockwood, P.A., Marshall, S.A., Benincose, L.J., Tensfeldt, T.G., Parivar, K., Amantea, M., Glue, P., Koide, H. And Miller, R., (2007), “Model-based Drug Development”. Clinical Pharmacology & Therapeutics, 82: Miller, F., Björnsson, M., Svensson, O. and Karlsten, R. (2014) “Experiences with an adaptive design for a dose-finding study in patients with osteoarthritis.” Contemporary Clinical Trials, 37:
Thank you for your attention!