What is a non-inferiority trial, and what particular challenges do such trials present? Andrew Nunn MRC Clinical Trials Unit 20th February 2012

Objective of an NI trial In contrast to a superiority trial, in which our aim is to show the new treatment to be more effective than the control in a non-inferiority trial the objective is to show that the new treatment is as good as the control. How good is good enough? It is impossible to show that two treatments are equally effective, only that they differ by no more than a specified amount – this is the margin of non-inferiority.

Non-inferiority Equivalence requires that the new treatment must be neither worse nor better than the control by a fixed amount the difference control - new intervention effect is both > -δ and < δ. In contrast to equivalence with non-inferiority we are only interested in determining whether new treatment is no worse by an amount δ.

The null hypothesis The situation is the reverse of what is required in a superiority design. For superiority H0 is there is no difference. For non-inferiority H0 is there is a difference of at least δ. The alternative hypotheses are also reversed

Choosing δ Three important assumptions made when planning a Phase III non-inferiority trial are: response rate expected in the control arm response rate expected in the intervention arm the acceptable margin of non-inferiority (δ) Most studies assume an equality of response between the two study arms.

Constraints on δ If the intervention is expected to be inferior to the control regimen, as in BLISTER, this will put a limit on how small the margin of inferiority, δ, can be in order to preserve a ‘reasonable’ amount of the effect size of the control arm. Keeping δ small will mean increasing sample size.

BLISTER primary endpoint Differences in the two treatment arms in the: a) the proportion of participants classed as treatment success (3 or less significant blisters present on examination) at 6 weeks. b) proportion of participants reporting grade 3, 4 and 5 (mortality) adverse events which are possibly, probably or definitely related to BP medication during the 52 week follow-up.

BLISTER 6 week efficacy assumptions Prednisolone arm = 95% effective Doxycycline arm = 70% effective, i.e. 25% inferior δ = 12% This means the lower limit of the 90% CI for the difference in efficacy is -37%. Weighed against this disadvantage is the expectation that the number of severe AEs will be reduced. The required sample size is 256

What the protocol says For the purpose of this study a lower level of response of 58% (95% - 37%) has been assumed; this is substantially greater than the complete lack of response that would be expected if participants remained untreated. A survey of the UK DCTN membership showed that a point estimate of 25% inferiority in effectiveness would be acceptable assuming a gain in the safety profile of at least 10%

The analysis Superiority trials are analysed by ITT because it is the most conservative and least likely to be biased. ITT (intention to treat) principle ignores the fact that patients may not always receive their all their allocated treatment.

ITT analysis in an NI trial ITT analysis of non-inferiority trials is not conservative since the inclusion of patients who violate the protocol will tend to minimise differences between study arms thereby increasing the possibility of declaring non-inferiority.

Per-protocol is also biased! Per-protocol analyses, in which only those adherent to the protocol are included, are also biased since not all randomised patients are included and although one might expect such an analysis to remove unwanted noise it also has the potential to wrongly conclude there is no difference when a difference exists.

Which analysis? The CPMP state that ‘similar conclusions from both an ITT and per protocol analysis are required’ to declare non-inferiority. CPMP also states that the primary analysis should be per protocol, ‘since it is most sensitive for the detection of any real difference’. CPMP = Committee on Proprietary Medical Products

Analysis scenarios As well as performing a per protocol and ITT analysis Sensitivity analyses also enable alternative scenarios to be evaluated. This should reduce the possibility of falsely declaring a regimen to be non-inferior.

Not forgetting …… In most NI trials there is a gain to be had in the experimental group. The experimental arm may be shorter, less toxic, cheaper, etc. If less adverse events is not one of the benefits - these should also be non-inferior by an agreed criteria. In BLISTER an agreed amount of inferiority in efficacy is acceptable provided there is a significant reduction in severe (grade 3 or 4) adverse events.

A different approach Whereas in a superiority trial we must beware over analysing the data in an attempt to find a beneficial effect, In an NI study we should go out of our way to find differences in order not to wrongly ascribe ‘no difference’. Our conclusions will be most convincing when all the evidence points in the same direction.

Some possible outcomes (difference in efficacy from control) -40% -25% Non-inferiority not shown margin of non-inferiority (δ)

Does non-significant=non-inferior? Non-significant does not mean non-inferior, more often it means lack of power. Can a superiority trial be used to demonstrate non-inferiority? Only if the acceptable non-inferiority margin has been determined before the study began. Non-inferiority is not incompatible with a significant difference in either direction but is usually unlikely – except in BLISTER where we could demonstrate non-inferiority and show the doxycycline arm to be significantly inferior! .

Conduct of NI trials NI trials should be conducted with a high degree of rigour to avoid the possibility of falsely declaring non-inferiority. Only patients likely to comply should be enrolled. Losses should be kept to a minimum As far as possible the same protocol and laboratory methods should be used as in the studies that demonstrated the effectiveness of the control regimen If possible the trial should be conducted double blind

Conclusions A major concern among regulators is that the efficacy of the control is not well established in many NI trials. Unless NI trials are conducted with rigour there is a serious risk of declaring non-inferiority inappropriately. The margin of non-inferiority needs to be determined before the start of the trial and should take into account both clinical and statistical considerations. Non-inferiority needs to be demonstrated not only for efficacy but needs also consider safety.