1. EFPSI-BBS Meeting on M&S, Basel 13 September 2012
Using M&S to develop a novel longitudinal model-based approach for efficacy assessments, with an application to biosimilars in rheumatoid arthritis
D Renard1, B Bieth1, F Mentré2, G Heimann1, I Demin1,
B Hamrén1, S Balser3
1Modeling and Simulation, Novartis, Basel (Switzerland), 2 UMR 738, INSERM and Universite Paris Diderot, Paris (France), 3 Sandoz Biopharmaceuticals, Holzkirchen (Germany)

2. Outline Background M&S approach Simulation models
Statistical methodology
Some results
Next steps
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

3. Problem that we attempt to address
Question
Can the size of a Phase III study be reduced by optimizing the choice of analysis methodology, as compared to using the standard approach (end-point analysis) ?
An innovative model-based approach was developed
Aim to maintain strict regulatory standards for phase III
Sound statistical properties
Fully pre-specified analysis
Note: this is NOT how pharmacometric (M&S) analyses are routinely applied in drug development!
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

4. Application: Biosimilars in rheumatoid arthritis (RA)
In RA studies, standard efficacy assessments rely on the ACR20 after 24 weeks of treatment (ACR20=American College of Rheumatology 20% response criterion)
Objective is to demonstrate similar efficacy between the reference and biosimilar products
Formally achieved through statistical equivalence testing
Standard method:
Based on response rates (#ACR20 responders/#patients)
Only uses data collected at Week 24
Equivalence is inferred when the 95% confidence interval (CI) for the treatment effect is included within the equivalence margins
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

5. M&S approach Step 1. Establish simulation models
Requirement: simulate realistic study outcomes for ACR20 (including compound specific characteristics)
Simulation models were built using a mix of literature (summary level) and internal (patient level) data
Models components: time course of response, different sources of variability, and dropout characteristics
Step 2: Development of statistical methodology
Longitudinal model-based approach for equivalence testing
Rooted to nonlinear mixed effect (NLME) modeling, entails pre-specifying different candidate models and relies on model averaging techniques
Step 3. Simulations and assessment of performance
An extensive simulation program was conducted to assess performance of the longitudinal model-based approach
More is needed!
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

6. Longitudinal model-based meta-analysis as basis for setting up realistic simulation models
9 anti-TNF agents, 37 studies
ACR20 responses
to methotrexate (MTX)
Symbols = study results (size proportional to # patients)
Solid lines connect points from each study
Broken lines = smooth curves for each patient population
Demin et al, Clin Pharmacol Ther (2012)
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

7. Simulation models
Different simulation models were considered to ensure robustness of the conclusions
Models were set up to mimick different compounds based on meta-analytic characterization
Internal data were used to inform variability parameters in the models
An example of model is given on slide 10
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

8. Literature database was used to build assumptions about dropout
Dropout increases proportionally over time to reach 15% at week 24
Patients not responding at the previous visit were twice more likely to drop out compared to patients who were responders
Dropout for non responders assumed to be due to lack of efficacy
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

9. Longitudinal model-based testing (1)
NLME models are pre-specified to describe transition probabilities based on the Markov assumption linking the entire longitudinal data.
Pre-specified
candidate models
Model averaging
Equivalence testing
Lacroix BD et al, Clin Pharmacol Ther 2009, 86:
ACR20
Visits (week) 2 4 20 24
Pr10 1
Pr01
The response rate at week 24 is a function of transition probabilities:
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

10. Longitudinal model-based test (2)
Markov models are specified for two independent transition probabilities, e.g.
10 candidate models are pre-specified, which differ through constraints in parameters and function of time
Range of simple up to more complex models
Estimating ACR20 response rates for a given model
Individual response probabilities are derived from the Markov property (previous slide) using modeled transition probabilities
Population response rates are derived by integrating out the random effects (η10, η11)
Pre-specified
candidate models
Model averaging
Equivalence testing
i= subject
k=study visit
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

11. Longitudinal model-based test (3)
Key concept: model averaging
Used to estimate the response rates at week 24 by combining results from the different candidate models
Point estimate = weighted average of the individual model estimates
Weights are functions of a statistical criterion (BIC)
Larger weights are assigned to models that fit the data better.
Pre-specified
candidate models
Model averaging
Equivalence testing
k = treatments
m = models
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

12. Longitudinal model-based test (4)
Pre-specified
candidate models
Model averaging
Equivalence testing
Example: candidate models
10 candidate models
3 models shown with corresponding weights (w)
W=.18
W=.75
W=.00
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

13. Longitudinal model-based test (5)
Pre-specified
candidate models
Model averaging
Equivalence testing
Example: model averaging
10 candidate models
3 models shown with corresponding weights (w)
Model average estimate (thick black)
W=.18
W=.75
W=.00
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

14. Longitudinal model-based test (6)
Pre-specified candidate models
Model averaging
Equivalence testing
Bootstrap is used to derive a confidence interval for the treatment difference at week 24.
Bootstrap datasets are built by resampling over subjects.
The 95% CI is compared with the equivalence margins for equivalence testing.
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

15. Model-based analysis versus classical test
Model-based approach does not change the nature of the comparability testing and makes it more efficient!
Difference in response rates (%)
Simulation #
Symbols correspond to point estimates and bars to 95% CI
10 simulation runs assuming strict equivalence (n=180/arm)
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

16. Simulation results: Power
40% reduction in sample size compared to the classical test at power levels of 80 and 90%
Power assessed based on 1000 simulations
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

17. Simulation results: Type 1 error
Type I error rate is close to the 2.5% nominal level
Assessment based on 1000 simulations (bars represent Monte-Carlo error)
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

18. Summary of methodology
The model-based analysis uses all data collected to derive an estimate and its confidence interval of the treatment effect at the end of the study (week 24)
Model averaging is used to prevent against model misspecification
Number of patients was reduced up to 40% with the longitudinal model-based analysis – confirmed by additional simulation scenarios and sensitivity analyses
Extensions: the principles could be applied to other types of endpoints in RA (e.g. DAS28), other therapeutic areas for biosimilarity assessments, or for efficacy assessments in late stage clinical development of new drugs
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments

19. Summary of health authority interactions
Initial project feedback from EMA was negative
Overall encouraging feedback obtained at EMA/EFPIA workshop (Dec 2011)
Absence of theoretical results to justify type I error control appears to be a critical concern deserving careful consideration
How can regulatory acceptance be gained ?
Planned interaction with EMA through Innovation Task Force process
Perform large simulation study to evaluate type 1 error control (ongoing)
Use model-based approach as supportive analysis in future studies
Present and discuss the method with the scientific community
| EFPSI-BBS meeting | D Renard | 13 Sep 2012 | Longitudinal model-based approach for efficacy assessments