1. Biostatistics Case Studies 2016 Youngju Pak, PhD. Biostatistician ypak@labiomed.org Session 2 Understanding Equivalence and Noninferiority testing

2. Testing Inequality v.s. Equality Testing Inequality Ha: | mean(treatment ) - mean (control ) | ≠ 0 H0: | mean(treatment ) - mean (control ) | = 0 Testing equivalence H a : δ 1 < mean(trt 1) – mean (trt2) < δ 2 H 0 : mean(trt 1) – mean (trt2) ≤ δ 1 or mean(trt 1) – mean (trt2) ≥ δ 2, δ 1 & δ 2 is called “Equivalence Margin Testing Noninferority, noninfiority ity margin

3. Case Study Ophthalmology 2006; 113:70-76.

4. Abstract

5. Primary Outcome and Study Size Study Size - Page 72 bottom of column 1: Primary Outcome - Page 72 middle of column 1: Needs Consensus PI’s Gamble

6. Non-Inferiority Study Usually a new treatment or regimen is compared with an accepted treatment or regimen or standard of care. The new treatment is assumed inferior to the standard and the study is designed to show overwhelming evidence that it is at least nearly as good, i.e., non- inferior. It usually has other advantages, e.g., oral vs. inj. A negative inferiority study fails to detect inferiority, but does not necessarily give evidence for non-inferiority. The accepted treatment is usually known to be efficacious already, but an added placebo group may also be used.

7. How to determine Sample Size? For IOP study, we have –H a : mean IOP change uf – mean IOP change f < 1.5 –H 0 : mean IOP change uf – mean IOP change f ≥ 1.5 thus, we are only interested in the upper limit of the difference  Non-inferiority  one-sided T-test Thus we reject the H 0 if Signal/ Noise < some clinical value. But N for a non-inferiority test require more complicated parameters such as the non- centrality parameter of the t-distribution (a Two One Sided T-test is usually used for the equivalence test ).

8. Let’s run a software from www.stat.uiowa.edu/~rlenth/Power Information you will need –Equivalence Margin Non-Inferiority Margin(NIM) =1.5 for the IOP study –Assumed mean difference in change of IOP between two groups -> usually zero difference assumed but it is assumed 0.5 for the IOP study –SD of changes of IOP = 3.5 –α (usually set to 2.5%) since the confidence level of the confidence interval is (100-2 x α) %

9. Sample size for IOP study

10. Three dimensional power curve for a non-inferiority test

11. How do we determine if the fixed method is non-inferior to the unfixed method? Regardless of study aim – to prove treatments equivalent or to prove them different - inference can be based on: Primary Outcome: IOP reduction D= D uf – D f, where D f = mean IOP reduction with fixed therapy Typical superiority/inferiority study: Compare to 0. Non-inferiority study: Compare to δ 2, a pre-specified margin of equivalence (1.5 here). = 95% CI for D(= D uf – D f ) = “true (population) values for D”

12. Typical Analysis: Inferiority or Superiority H 0 : D uf – D f = 0 H 1 : D uf – D f ≠ 0 Aim: H 1 → therapies differ α = 0.05 & N=2194 Power = 80% when Δ=1, SD=3.5 Fixed is inferior = 95% CI for D = “true (population) values for D” Fixed is superior 0 0 D u – D f [Not used in this paper] 0 No difference detected D u – D f

13. Typical Analysis: Inferiority Only H 0 : D u – D f ≤ 0 H 1 : D u – D f > 0 Aim: H 1 → fixed is inferior α = 0.025 & N=2194 Power = 80% for when Δ=1, SD=3.5 Fixed is inferior = 95% CI for D u – D f = “true (population) values for D” 0 0 D uf – Df [Not used in this paper] 0 Inferiority not detected D uf – D f ( α = 0.05 → N=2153 )

14. Non-Inferiority H 0 : D u – D f ≥ 1.5 H 1 : D u – D f < 1.5 Aim: H 1 → fixed is non-inferior α = 0.025 & N=2194 Power = 80% for When Δ= 0.5, NIM=1.5 Fixed is non-inferior = 95% CI for D u – D f = “true (population) values for D” 0 0 D uf – D f [As in this paper] 0 Non-Inferiority not detected D uf – D f 1.5 Fixed is inferior 1.5

15. Inferiority and Non-Inferiority Fixed is non-inferior = 95% CI for D u – D f = “true (population) values for D” 0 0 0 Neither is detected D uf – D f 1.5 Fixed is inferior 01.5 Fixed is “non-clinically” inferior D^ uf = 9.0 D^ f = 8.7 D^ = 0.3 95% CI = -0.1 to 0.7 Observed Results: Fixed is non-inferior 01.5

16. Conclusions: General “Negligibly inferior” would be a better term than non- inferior. All inference can be based on confidence intervals. Pre-specify the comparisons to be made. Cannot test for both non-inferiority and superiority. Power for only one or for multiple comparisons, e.g., non-inferiority and inferiority. Power can be different for different comparisons. Very careful consideration must be given to choice of margin of equivalence (1.5 here). You can be risky and gamble on what expected differences will be (0.5 here), but the study is worthless if others in the field would find your margin too large.

17. FDA Guidelines : http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformati on/guidances/ucm202140.pdf http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformati on/guidances/ucm202140.pdf Where, M 1 = Full effect of the active control compare with the test drug M 2 = NI Margin

18. Self-Quiz 1.Give an example in your specialty area for a superiority /inferiority study. Now modify it to an equivalence study. Now modify it to a non-inferiority study. 2.T or F: The main point about non-inferiority studies is that we are asking whether a treatment is as good or better vs. worse than another treatment, so it uses a one-sided test. 3.Power for a typical superiority test is the likelihood that you will declare treatment differences (p<0.05) if treatments really differ by some magnitude Δ. Explain what power means for a non-inferiority study. 4.T or F: Last-value-carried-forward is a good way to handle drop-outs in a non-inferiority study. Explain. continued

19. Self-Quiz 5.T or F: In a non-inferiority study, you should first test for non- inferiority with a confidence interval, and then use a t-test to test for superiority, but only if non-inferiority was established at the first step. 6.What is the meaning of the equivalence margin, and how do you determine it? continued

20. Self-Quiz 7.Suppose the primary outcome for a study is a serum inflammatory marker. If it’s assay is poor (low reproducibility), then it is more difficult to find treatment differences in a typical superiority/inferiority study than for a better assay, due to this noise. Would it be easier or more difficult to find non-inferiority with this assay, compared to a better assay? 8.Does the assumed treatment difference (0.5 here) for power calculations have the same meaning as the difference used for power calculations in a typical superiority/inferiority study?