Non-Inferiority Exposed: Uses and Abuses Ajay J. Kirtane, MD, SM Center for Interventional Vascular Therapy Columbia University Medical Center / New York Presbyterian Hospital

Ajay J. Kirtane, MD Off-Label: Data-driven I/we have no real or apparent conflicts of interest to report. Off-Label: Data-driven

Why perform a non-inferiority trial? New therapy may offer important advantages over currently available effective therapies Improved safety Better tolerability/fewer side effects Ease of use (2nd generation DES, QD drug, etc.) Less expensive Increased market competition (?) Placebo control trial unethical but still want to demonstrate that the new treatment is better than nothing (“putative placebo”) approach Adapted from D. Cohen

Basic Clinical Trial Designs Non-Inferiority Increasingly common as it gets harder and harder to demonstrate incremental gains over successful therapies Particularly in EBM-rich fields (Cardiology) where we have many effective therapies already Non-inferiority designs are especially common with device and drug iterations This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 5

Basic Clinical Trial Designs Non-Inferiority: Endpoints Endpoint selection for non-inferiority trials is similar to that of trials assessing a traditional superiority design Surrogate endpoint (e.g. late loss) Clinical endpoint (e.g. rate of target vessel failure) The choice of endpoint will affect the clinical applicability of the trial This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 6

Basic Clinical Trial Designs A Non-Inferiority Trial 1000 patients randomized to DES A or DES B 50 clinical restenosis events with DES A; 50 with DES B Basic non-inferiority question: What is the probability that DES B could be worse than DES A? This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 7

Basic Clinical Trial Designs Non-Inferiority Explained In order to define “worse”, we first need to figure out how much “worse” we are willing to accept This is defined as the non-inferiority “delta” or “margin” and is almost always ARBITRARY Let’s set it at 20% for this example Thus, we want to be sure (or at least 95% confident) that DES B is not 20% worse than DES A This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 8

Basic Clinical Trial Designs Non-Inferiority: Choice of a Delta The choice of “delta” is the most critical aspect of a non-inferiority trial This affects not only whether the trial is “positive” or “negative” but more importantly relates to the clinical relevance of the trial findings The best-chosen deltas reflect a clinical (not statistical) focus This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 9

Possible Non-Inferiority Results Superior Non-inferior Non-inferior Non-inferior? Inconclusive Inconclusive Inconclusive? Inferior ∆ Relative Risk (or Treatment Difference) with 95% CI ← New Treatment Better New Treatment Worse →

Selecting a non-inferiority margin Critical to pre-specify the non-inferiority margin to avoid Type I error (false positive results) Potential approaches Clinical rationale expert opinion (“what is the maximum difference you would tolerate?”) Regulatory rationale based on previous trials Statistical + Clinical rationale designed to preserve some minimum proportion of benefit vs. placebo (“putative placebo” approach) Rule of thumb: Margin cannot be greater than the smallest effect size that the active comparator would be expected to have vs. placebo Key Point When reading about a non-inferiority trial, always check the non-inferiority margin If the margin is “too wide”, the trial results are questionable no matter how strong the p-value c/o D. Cohen

Basic Clinical Trial Designs Non-Inferiority: DES A vs. DES B Back to the example (1000 patient study of two DES): DES A is the gold standard with a TLR rate of 10% (50/500) DES B is being tested, and has a 10% TLR rate as well (50/500) The delta is set at 20% (relative) We will accept DES B as non-inferior to DES A if the upper limit of the confidence interval of the TLR rate of DES B is not 20% worse than the TLR rate of DES A This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 16

Basic Clinical Trial Designs Non-Inferiority: DES A vs. DES B In a prospective randomized trial this is calculated using relative or absolute risk: The upper limit of the confidence interval of the relative risk of TLR with DES B (compared to DES A) should not be greater than 1.2 This is essentially the same as demanding that the upper limit of the confidence interval of the difference in TLR rates with DES B (compared to DES A) should not be greater than 2% This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 17

DES B vs. DES A: Risk Difference but upper CI is 3.7% Inconclusive Results (technically negative trial) 2% Risk Difference with 95% CI ← DES A Better DES B Worse →

Basic Clinical Trial Designs Non-Inferiority: DES A vs. DES B Even though the rates of TLR were identical with both stents, we cannot claim non-inferiority at this sample size! How could we have designed the trial to allow DES B to be non-inferior? Larger delta (40% relative delta) Greater sample size This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 19

Basic Clinical Trial Designs Non-Inferiority: DES A vs. DES B With a 10% rate of TLR in both arms: Total N DES A DES B Risk Difference Upper 95% CI of Risk Difference 500 10% 0.1 - 0.1 = 0 5.2% 1000 3.7% 1500 3.0% 2000 2.6% 4000 1.8% This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 20

Basic Clinical Trial Designs Non-Inferiority: DES A vs. DES B With a 10% rate of TLR with DES A and a 9% rate with DES B, the sample size would have gone down With a 10% rate of TLR with DES A and a 11% rate with DES B, the sample size would have gone up astronomically This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 21

Basic Clinical Trial Designs Non-Inferiority: Basic Principles Apply Tighter deltas prevent “inconclusive results” but make it harder to be declared non-inferior Wider deltas and larger sample sizes make it easier to be declared non-inferior, but larger sample sizes with a wide delta may also result in inconclusive results This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 23

Comparator Creep BMS DES 1 DES 2 DES 3 DES 4 Restenosis Rate c/o D. Cohen

Data Quality/Trial Conduct In a superiority trial, the sponsor has a strong incentive to minimize errors in trial conduct and measurement, since these errors would bias the trial results toward the null In a non-inferiority trial, the incentives are reversed Key issues in trial conduct that can bias toward a finding of “non-inferiority” Enrolling a low risk population unlikely to benefit from either treatment Non-compliance and crossover Insensitive outcome measure Loss to follow-up Most important practical and theoretical issue with non-inferiority trials c/o D. Cohen

Basic Clinical Trial Designs Non-Inferiority: Summary Increasingly common Always look at the raw event rates and confidence intervals, and ask whether these rates make sense In some cases, the p-values are less relevant than the actual rates Watch for deltas that do not make clinical sense, and beware of “creep” This is the Bulleted List slide. To create this particular slide, click the NEW SLIDE button on your toolbar and choose the BULLETED LIST format. (Top row, second from left) The Sub-Heading and footnote will not appear when you insert a new slide. If you need either one, copy and paste it from the sample slide. If you choose not to use a Sub-Heading, let us know when you hand in your presentation for clean-up and we’ll adjust where the bullets begin on your master page. Also, be sure to insert the presentation title onto the BULLETED LIST MASTER as follows: Choose View / Master / Slide Master from your menu. Select the text at the bottom of the slide and type in a short version of your presentation title. Click the SLIDE VIEW button in the lower left hand part of your screen to return to the slide show. (Small white rectangle) 27