1. Understanding the Concept of Equivalence and Non-Inferiority Trials CM Gibson, 2000

2. TerminologyTerminology The term active-control trial refers to all studies in which the control treatment is an active one.The term active-control trial refers to all studies in which the control treatment is an active one. –If the intent is to show that the differences between control and study treatments are not large in either direction the study is called an equivalence trial. –If the intent of a study is to demonstrate that the study treatment is not substantially worse than than the control treatment the study is called a non-inferiority trial. Both types of trials seek to reject the possibility that differences in treatment effects equal or exceed a preset limit or margin.Both types of trials seek to reject the possibility that differences in treatment effects equal or exceed a preset limit or margin. The term active-control trial refers to all studies in which the control treatment is an active one.The term active-control trial refers to all studies in which the control treatment is an active one. –If the intent is to show that the differences between control and study treatments are not large in either direction the study is called an equivalence trial. –If the intent of a study is to demonstrate that the study treatment is not substantially worse than than the control treatment the study is called a non-inferiority trial. Both types of trials seek to reject the possibility that differences in treatment effects equal or exceed a preset limit or margin.Both types of trials seek to reject the possibility that differences in treatment effects equal or exceed a preset limit or margin. Jay P. Siegel, American Heart Journal April 2000:S166-S170

3. Equivalency & Non Inferiority in Plain English Goal of an Equivalency Study: “With 95% certainty can I say that the mortality rates lie within 1% of each other for these two drugs”?Goal of an Equivalency Study: “With 95% certainty can I say that the mortality rates lie within 1% of each other for these two drugs”? Goal of a Non-Inferiority Study: “With 95% certainty can I say that drug A is no worse than 1% than drug B”?Goal of a Non-Inferiority Study: “With 95% certainty can I say that drug A is no worse than 1% than drug B”? Goal of an Equivalency Study: “With 95% certainty can I say that the mortality rates lie within 1% of each other for these two drugs”?Goal of an Equivalency Study: “With 95% certainty can I say that the mortality rates lie within 1% of each other for these two drugs”? Goal of a Non-Inferiority Study: “With 95% certainty can I say that drug A is no worse than 1% than drug B”?Goal of a Non-Inferiority Study: “With 95% certainty can I say that drug A is no worse than 1% than drug B”? CM Gibson, 2000

4. Superiority: Does the 95% CI contain zero?  Equivalence: Does the 95% CI lie between  1% and +1%?  1% 0%0% +1%+1% 0%0%+1%+1% To left of  1% is clinically meaningful To left of  1% is clinically meaningful Between  1% and +1% is not clinically meaningful Between  1% and +1% is not clinically meaningful To right of  1% is clinically meaningful To right of  1% is clinically meaningful Understanding Equivalence CM Gibson, 2000

5. TerminologyTerminology An alternative term for a non-inferiority trial is sufficiency trial.An alternative term for a non-inferiority trial is sufficiency trial. –This designation implies simply that the trial is designed to demonstrate that the activity of the study drug is, compared with the active control, sufficient to meet the trial’s purpose (eg, to support an expert recommendation or a regulatory approval). An alternative term for a non-inferiority trial is sufficiency trial.An alternative term for a non-inferiority trial is sufficiency trial. –This designation implies simply that the trial is designed to demonstrate that the activity of the study drug is, compared with the active control, sufficient to meet the trial’s purpose (eg, to support an expert recommendation or a regulatory approval). Jay P. Siegel, American Heart Journal April 2000:S166-S170

6. BackgroundBackground Prior to undertaking the trial, the degree of inferiority (or difference) that is clinically relevant must first be establishedPrior to undertaking the trial, the degree of inferiority (or difference) that is clinically relevant must first be established A trial is then designed and to reject the hypothesis that a difference of that size or larger exists 1.A trial is then designed and to reject the hypothesis that a difference of that size or larger exists 1. The efficacy of a new therapeutic agent would be established if its treatment effects are proven to be at least equivalent to those observed from a standard-of-care regimen, which itself has a well established and substantial level of efficacy 2.The efficacy of a new therapeutic agent would be established if its treatment effects are proven to be at least equivalent to those observed from a standard-of-care regimen, which itself has a well established and substantial level of efficacy 2. If a new agent not only demonstrates its equivalence with standard therapy, but also offers advantages in terms of either toxicity, ease of administration, or cost, the new agent may provide an important advance in clinical care 2.If a new agent not only demonstrates its equivalence with standard therapy, but also offers advantages in terms of either toxicity, ease of administration, or cost, the new agent may provide an important advance in clinical care 2. Prior to undertaking the trial, the degree of inferiority (or difference) that is clinically relevant must first be establishedPrior to undertaking the trial, the degree of inferiority (or difference) that is clinically relevant must first be established A trial is then designed and to reject the hypothesis that a difference of that size or larger exists 1.A trial is then designed and to reject the hypothesis that a difference of that size or larger exists 1. The efficacy of a new therapeutic agent would be established if its treatment effects are proven to be at least equivalent to those observed from a standard-of-care regimen, which itself has a well established and substantial level of efficacy 2.The efficacy of a new therapeutic agent would be established if its treatment effects are proven to be at least equivalent to those observed from a standard-of-care regimen, which itself has a well established and substantial level of efficacy 2. If a new agent not only demonstrates its equivalence with standard therapy, but also offers advantages in terms of either toxicity, ease of administration, or cost, the new agent may provide an important advance in clinical care 2.If a new agent not only demonstrates its equivalence with standard therapy, but also offers advantages in terms of either toxicity, ease of administration, or cost, the new agent may provide an important advance in clinical care 2. 2. Thomas R. Fleming, American Heart Journal April 2000:S171-S176 1. Jay P. Siegel, American Heart Journal April 2000:S166-S170

7. The Choice of Active Control If efficacy is to be established through comparison with an active control, it is essential that the active control have a well established, predictable, quantifiable effect at a dose and regimen that has been well studied.If efficacy is to be established through comparison with an active control, it is essential that the active control have a well established, predictable, quantifiable effect at a dose and regimen that has been well studied. This effect is best established by multiple randomized controlled trials, comparing the active control with placebo.This effect is best established by multiple randomized controlled trials, comparing the active control with placebo. If efficacy is to be established through comparison with an active control, it is essential that the active control have a well established, predictable, quantifiable effect at a dose and regimen that has been well studied.If efficacy is to be established through comparison with an active control, it is essential that the active control have a well established, predictable, quantifiable effect at a dose and regimen that has been well studied. This effect is best established by multiple randomized controlled trials, comparing the active control with placebo.This effect is best established by multiple randomized controlled trials, comparing the active control with placebo. Jay P. Siegel, American Heart Journal April 2000:S166-S170

8. Setting the Value of the Margin Establishing a margin - the smallest unacceptable degree of clinical inferiority (and of superiority in an equivalence trial) of the new treatment must be prospectively definedEstablishing a margin - the smallest unacceptable degree of clinical inferiority (and of superiority in an equivalence trial) of the new treatment must be prospectively defined In acute MI studies, this has traditionally been a 1% difference in mortality, a difference which resulted in changes in practice patterns following GUSTO 1. This is an approximate 15% relative reduction in mortality.In acute MI studies, this has traditionally been a 1% difference in mortality, a difference which resulted in changes in practice patterns following GUSTO 1. This is an approximate 15% relative reduction in mortality. An estimate of the efficacy of the active control group is then made based upon event rates prior trials.An estimate of the efficacy of the active control group is then made based upon event rates prior trials. Based upon the projected efficacy of the active control, the trial is then appropriately powered to determine if the two drugs lie within this margin.Based upon the projected efficacy of the active control, the trial is then appropriately powered to determine if the two drugs lie within this margin. Establishing a margin - the smallest unacceptable degree of clinical inferiority (and of superiority in an equivalence trial) of the new treatment must be prospectively definedEstablishing a margin - the smallest unacceptable degree of clinical inferiority (and of superiority in an equivalence trial) of the new treatment must be prospectively defined In acute MI studies, this has traditionally been a 1% difference in mortality, a difference which resulted in changes in practice patterns following GUSTO 1. This is an approximate 15% relative reduction in mortality.In acute MI studies, this has traditionally been a 1% difference in mortality, a difference which resulted in changes in practice patterns following GUSTO 1. This is an approximate 15% relative reduction in mortality. An estimate of the efficacy of the active control group is then made based upon event rates prior trials.An estimate of the efficacy of the active control group is then made based upon event rates prior trials. Based upon the projected efficacy of the active control, the trial is then appropriately powered to determine if the two drugs lie within this margin.Based upon the projected efficacy of the active control, the trial is then appropriately powered to determine if the two drugs lie within this margin. Jay P. Siegel, American Heart Journal April 2000:S166-S170

9. Parameters of Comparison Patient population, concomitant therapies and endpoints are important both for estimating active- control effect size and for ensuring a fair comparison between study drug and active control.Patient population, concomitant therapies and endpoints are important both for estimating active- control effect size and for ensuring a fair comparison between study drug and active control. Jay P. Siegel, American Heart Journal April 2000:S166-S170

10. Non-Inferiority Trial Example 5.15.15.55.56.06.06.56.56.96.9 Window of Non-Inferiority Margin InferiorInferiorSuperiorSuperior Hypothetical Control Treatment Event Rate Hypothetical Control Treatment Event Rate If the experimental treatment event rate is < 5.1, then the experimental treatment would be superior to the active control.If the experimental treatment event rate is < 5.1, then the experimental treatment would be superior to the active control. If the experimental treatment event rate falls between the 5.1 and 6.9 range, then the experimental treatment is non-inferior to the active control.If the experimental treatment event rate falls between the 5.1 and 6.9 range, then the experimental treatment is non-inferior to the active control. If the experimental treatment event rate is > 6.9, then the experimental treatment is inferior to the active control.If the experimental treatment event rate is > 6.9, then the experimental treatment is inferior to the active control. If the experimental treatment event rate is < 5.1, then the experimental treatment would be superior to the active control.If the experimental treatment event rate is < 5.1, then the experimental treatment would be superior to the active control. If the experimental treatment event rate falls between the 5.1 and 6.9 range, then the experimental treatment is non-inferior to the active control.If the experimental treatment event rate falls between the 5.1 and 6.9 range, then the experimental treatment is non-inferior to the active control. If the experimental treatment event rate is > 6.9, then the experimental treatment is inferior to the active control.If the experimental treatment event rate is > 6.9, then the experimental treatment is inferior to the active control.

11. Equivalence Trial Example 5.15.16.06.06.96.9 Equivalence Established Equivalence not established Hypothetical Control Treatment Event Rate Hypothetical Control Treatment Event Rate In an equivalence trial, if the experimental treatment event rate falls between 5.1 and 6.9, the study would establish equivalence between the experimental therapy and the active control.In an equivalence trial, if the experimental treatment event rate falls between 5.1 and 6.9, the study would establish equivalence between the experimental therapy and the active control. If the experimental treatment event rate falls outside the 5.1 to 6.9 range, the study would fail to establish equivalence between the experimental treatment and the active control.If the experimental treatment event rate falls outside the 5.1 to 6.9 range, the study would fail to establish equivalence between the experimental treatment and the active control. In an equivalence trial, if the experimental treatment event rate falls between 5.1 and 6.9, the study would establish equivalence between the experimental therapy and the active control.In an equivalence trial, if the experimental treatment event rate falls between 5.1 and 6.9, the study would establish equivalence between the experimental therapy and the active control. If the experimental treatment event rate falls outside the 5.1 to 6.9 range, the study would fail to establish equivalence between the experimental treatment and the active control.If the experimental treatment event rate falls outside the 5.1 to 6.9 range, the study would fail to establish equivalence between the experimental treatment and the active control.

12. Differences Between Equivalency and Non-Inferiority Trials What if there is superiority in an equivalency trial? If a strategy proves be to superior, (I.e. the event rate is below 5.1% in the previous trials), this would be viewed as an equivalent result in an equivalency study.What if there is superiority in an equivalency trial? If a strategy proves be to superior, (I.e. the event rate is below 5.1% in the previous trials), this would be viewed as an equivalent result in an equivalency study. What if there is superiority in a non-inferiority trial? Superiority can be claimed.What if there is superiority in a non-inferiority trial? Superiority can be claimed. Conclusion: Non-inferiority trial design retains the ability to show superiorityConclusion: Non-inferiority trial design retains the ability to show superiority What if there is superiority in an equivalency trial? If a strategy proves be to superior, (I.e. the event rate is below 5.1% in the previous trials), this would be viewed as an equivalent result in an equivalency study.What if there is superiority in an equivalency trial? If a strategy proves be to superior, (I.e. the event rate is below 5.1% in the previous trials), this would be viewed as an equivalent result in an equivalency study. What if there is superiority in a non-inferiority trial? Superiority can be claimed.What if there is superiority in a non-inferiority trial? Superiority can be claimed. Conclusion: Non-inferiority trial design retains the ability to show superiorityConclusion: Non-inferiority trial design retains the ability to show superiority CM Gibson 2000

13. 0 +1 Mortality (%) Absolute Difference (95% CI) Other BetterBetter P Value for Equivalence InTIME-2 6.776.60   0.17 (  1.0, 0.68) 0.047 ASSENT-2 6.166.18 0.02 (  0.59, 0.62) 0.006 GUSTO-III 7.477.24  0.23 (  1.11, 0.66) NS Examples of Equivalency Studies in Thrombolysis Trials ASSENT-2 Investigators. Lancet. 1999;354:716-722; Adapted from GUSTO-III Investigators. N Engl J Med. 1997;337:1118-1123. Adapted from Giugliano RP, et al. Circulation. 1999;100:I-651. n-PA TNK-tPA r-PA t-PA t-PA t-PA t-PA