Oxford Inflammatory Bowel Disease MasterClass Understanding non-inferiority trial designs Dr Vipul Jairath Bsc DPhil (Oxon) MRCP NIHR Clinical Lecturer Translational Gastroenterology Unit Nuffield Department of Medicine University of Oxford

Aim of this talk  To provide a practical guide to the clinician:  What they are  Why they are conducted  Determining an acceptable margin of non-inferiority  Design considerations  Perils and pitfalls – of which there many!  Interpreting the results  An IBD trial example

 Conventional Designs  Parallel group  Non-inferiority  Equivalence  Cluster  Factorial  Cross-over  Multi-arm  Adaptive Designs  Sample size re-estimation  Dropping treatment arms  Change allocation ratio  Change primary endpoint  Sequential  Alternative Designs  Stepped-Wedge  Complex Interventions  Patient Preference  Zelen  N of 1  Post marketing surveillance Conventional and novel trial designs

Biosimilar development pathway  EMA  similar clinical efficacy between the similar and reference product should be demonstrated in adequately powered RCTs, preferably double blind equivalence trials  FDA ....Non-inferiority designs are acceptable which should demonstrate no clinically meaningful difference in efficacy and shows that the biosimilar poses no more risk in safety or immunogenicity

What are non-inferiority or equivalence trials?  Superiority trial:  Designed to prove that E is better than C  Lack of difference ≠ equivalence  Non-inferiority trial:  Designed to show that E is only marginally inferior to C (-∆)  A one sided comparison (interested in degree of detriment)  Equivalence trial:  Designed to show that E is not appreciably inferior or superior to C (a two-sided comparison; -∆, +∆)

Null and alternative hypotheses  Parallel group trial:  Designed to show new intervention is superior  Null hypothesis = no difference between treatments  We reject the null hypothesis, decide one is superior to the other if there is sufficient evidence  Non-inferiority/equivalence trials:  Null hypothesis is reversed  Null hypothesis is that there is a difference  Aim to show one intervention is not inferior (or is equivalent) to the other

Null and alternative hypotheses Non-inferiority trialEquivalence trial

Superiority, non-inferiority or equivalence? CI does not contain zero SUPERIORITY TRIAL EQUIVALENCE TRIAL CI is in the window of equivalence NON-INFERIORITY TRIAL CI is within Delta (margin of non-inferiority)

Why conduct a non-inferiority or equivalence trial?  To test new form of an existing drug  Not ethical to do a placebo response trial  Even in E is non-inferior to C on the primary efficacy endpoint, it would still need ancillary benefits:  Better secondary endpoints  Better safety profile  Easier route of administration  Simpler dosing regimen  Cheaper to produce  Compliance expected to be better outside of RCT  Equivalence trials largely used for bioequivalence studies

Garattini, Lancet, 2007

Designing the non-inferiority/equivalence trial  Select the non-inferiority margin (∆)  Run the trial comparing E to C  Calculate the confidence interval around the difference between treatments  Look at lower bound of CI  If the lower bound of the CI is within the margin -∆, the new treatment is deemed non- inferior and trial a success  Select ∆, “minimally clinically important difference in the primary endpoint”  Run the trial comparing E to C  Calculate the confidence interval around the difference between treatments  Look at upper and lower bound of CI  If these are within -∆ to +∆, the new treatment is deemed equivalent and trial a success

How is the control arm event rate calculated?  Overestimate this may lead to an underpowered trial  Look at historical event rates in the control arm ideally based on meta-analysis  Beware that even these event rates could be outdated  Feasibility data, own experiences  Do a prespecified interim analysis during the trial and adjust the sample size accordingly  Extending trial duration to meet the number of events

How is the NI/equivalence margin chosen?  This is the crux of the trial and not entirely scientific:  Overly conservative = inability to detect non-inferiority  Overly liberal = risk of claiming non-inferiority  Clinical judgement:  Ask experts (e.g. Delphic procedure) or patient groups  This is likely to be insufficient for regulators  Choice of NI margin: absolute versus relative risk reduction  Reality: Clinical judgement, statistical (budget)  If inappropriate thresholds set and uncontested this could lead to the uptake of treatments detrimental to patients Mulla, S et al. JAMA 2012

Preserving 50% of the “minimal treatment effect”

More effective treatments need to preserve > 50%

How is the sample size calculated?  In principle similar to other trials  Proportion of patients expected to experience outcome  Significance level (α, Type 1 error = “false positive”)  Power (1-β, 1- Type 2 error = “false negative”)  ∆  90% power, since less than this biases towards non- inferiority/equivalence  The sample size increases with:  Greater power  Smaller ∆

Sample size examples for an NI trial EC ∆ N A 88% B 88% C 85% D 90%88% % power; 2.5% once sided alpha, 10% dropout rate

What are the important design aspects?  Rigorous methods: poor rigour rewards in NI/equivalence trials; penalises in superiority trials  Eligibility: patients in the trial should be similar to trials which established effectiveness of the standard intervention  Dose: of the standard intervention should be similar to those found to be effective in previous trial  Low doses: erroneous equivalence, where both interventions have no clinical response  High doses: can claim equivalence, but excess AEs  Concomitant medications: high response rates may be due to the effect of concomitant medications

Design aspects: Dose selection

How can “Biocreep” be avoided?  A concern particularly with non-inferiority trials  A slightly inferior drug becomes the comparator for the next generation of compounds and so on.  Over time new drugs may only have efficacy close to that of placebo  Will occur if a new drug with lower efficacy than the comparator is approved with a wide ∆  Avoid this by:  Using gold standard as the active comparator  Incorporate a placebo arm into the trial (?Ethical)

How should the results be analysed and presented?  Intention to treat analysis  Preserves randomisation  Produced a conservative estimate in superiority trial  For a NI/equivalence trial this in not conservative as will make the groups more similar!  Per-protocol analysis  Compromises randomisation  Non-adherent often prognostically worse  Both should be presented  If consistent = reassuring  In not consistent = inference of non-inferiority/equivalence is weakened

What can the results be from a non-inferiority trial?

Reporting non-inferiority and equivalence trials Topic item number Additional statement required Design1that the trial was designed to show non-inferiority or equivalence Background2rationale for using a non-inferiority or equivalence design Participants, interventions and outcomes 3, 4 & 6 whether the eligibility criteria, interventions (e.g. dose), and outcomes are similar to those of any trial(s) which established the efficacy of the standard intervention Objectives, sample size and analysis method 5, 7 & 12 whether the hypothesis is of non-inferiority or equivalence (one- or two- sided), and the magnitude of difference used to define this hypothesis Numbers analysed16whether intention to treat or alternative analyses were done Outcomes17 for each outcome, 'a figure showing confidence intervals and margins of equivalence may be useful' Interpretation20should take into account the non-inferiority or equivalence hypothesis

An example from an IBD trial

Some take home messages  NI trials are acceptable when a new therapy has a sufficiently favourable property that clinicians/patients are willing to sacrifice some degree of benefit relative to an approved therapy  Goals for the three trials are different  Superiority: E is better than C  Equivalence: E is not too different from C  Non-inferiority: E is not much worse than C

Some take home messages  The margin of ∆ should be prespecified and justified clinically  How was event rate in the control arm estimated  Sample size and power  Check the conduct of the trial  Eligibility, dosing, concomitant medication  Poor rigour rewards in these designs!  Look at the analysis of the trial  PP and ITT analysis and if they concur  Check that only trials planned as non-inferiority/equivalence are reported as such and not claimed retrospectively!

Some take home messages  Equivalence or non-inferiority trials have specific challenges and both designs acceptable dependent on regulator  Methodological rigour to prevent erroneous conclusions and unacceptably inferior products entering the market  Large sample sizes  Less incentive to conduct them well  Emerging biosimilars will employ these designs and we must appreciate the key principles to inform treatment choices

Suggested reading Readings from text books  Wang D, Bakhai A. Clinical Trials: A Practical Guide to Design, Analysis and Reporting. REMEDICA (1 Nov 2005). Chapters 12 & 14  Senn S. Statistical Issues in Drug Development (2nd edition). Wiley, Chichester, 2007, ISBN Chapter 15 Papers  Jones B, Lewis J, Ebbutt E. Trials to assess equivalence: the importance of rigorous methods. BMJ. 1996; 313: 36-9  Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340:c869.  Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA Mar 8; 295(10):