Issues of Simultaneous Tests for Non-Inferiority and Superiority Tie-Hua Ng*, Ph. D. U.S. Food and Drug Administration Presented at MCP.

Slides:



Advertisements
Similar presentations
1/2/2014 (c) 2001, Ron S. Kenett, Ph.D.1 Parametric Statistical Inference Instructor: Ron S. Kenett Course Website:
Advertisements

1
Feichter_DPG-SYKL03_Bild-01. Feichter_DPG-SYKL03_Bild-02.
© 2008 Pearson Addison Wesley. All rights reserved Chapter Seven Costs.
Copyright © 2003 Pearson Education, Inc. Slide 1 Computer Systems Organization & Architecture Chapters 8-12 John D. Carpinelli.
Chapter 1 The Study of Body Function Image PowerPoint
Copyright © 2011, Elsevier Inc. All rights reserved. Chapter 6 Author: Julia Richards and R. Scott Hawley.
Author: Julia Richards and R. Scott Hawley
1 Copyright © 2013 Elsevier Inc. All rights reserved. Appendix 01.
STATISTICS HYPOTHESES TEST (I)
Properties Use, share, or modify this drill on mathematic properties. There is too much material for a single class, so you’ll have to select for your.
UNITED NATIONS Shipment Details Report – January 2006.
RXQ Customer Enrollment Using a Registration Agent (RA) Process Flow Diagram (Move-In) Customer Supplier Customer authorizes Enrollment ( )
1 RA I Sub-Regional Training Seminar on CLIMAT&CLIMAT TEMP Reporting Casablanca, Morocco, 20 – 22 December 2005 Status of observing programmes in RA I.
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
Measurements and Their Uncertainty 3.1
Properties of Real Numbers CommutativeAssociativeDistributive Identity + × Inverse + ×
FACTORING ax2 + bx + c Think “unfoil” Work down, Show all steps.
Year 6 mental test 5 second questions
Year 6 mental test 10 second questions
Overview of Lecture Partitioning Evaluating the Null Hypothesis ANOVA
Lecture 2 ANALYSIS OF VARIANCE: AN INTRODUCTION
1 Discreteness and the Welfare Cost of Labour Supply Tax Distortions Keshab Bhattarai University of Hull and John Whalley Universities of Warwick and Western.
REVIEW: Arthropod ID. 1. Name the subphylum. 2. Name the subphylum. 3. Name the order.
On Comparing Classifiers : Pitfalls to Avoid and Recommended Approach
PP Test Review Sections 6-1 to 6-6
EU market situation for eggs and poultry Management Committee 20 October 2011.
EU Market Situation for Eggs and Poultry Management Committee 21 June 2012.
Chi-Square and Analysis of Variance (ANOVA)
Bellwork Do the following problem on a ½ sheet of paper and turn in.
2 |SharePoint Saturday New York City
VOORBLAD.
Hypothesis Tests: Two Independent Samples
Chapter 4 Inference About Process Quality
Copyright © 2012, Elsevier Inc. All rights Reserved. 1 Chapter 7 Modeling Structure with Blocks.
1 RA III - Regional Training Seminar on CLIMAT&CLIMAT TEMP Reporting Buenos Aires, Argentina, 25 – 27 October 2006 Status of observing programmes in RA.
Factor P 16 8(8-5ab) 4(d² + 4) 3rs(2r – s) 15cd(1 + 2cd) 8(4a² + 3b²)
Basel-ICU-Journal Challenge18/20/ Basel-ICU-Journal Challenge8/20/2014.
1..
CONTROL VISION Set-up. Step 1 Step 2 Step 3 Step 5 Step 4.
© 2012 National Heart Foundation of Australia. Slide 2.
Understanding Generalist Practice, 5e, Kirst-Ashman/Hull
Model and Relationships 6 M 1 M M M M M M M M M M M M M M M M
25 seconds left…...
Subtraction: Adding UP
Rational Functions and Models
Januar MDMDFSSMDMDFSSS
Statistical Inferences Based on Two Samples
© The McGraw-Hill Companies, Inc., Chapter 10 Testing the Difference between Means and Variances.
Analyzing Genes and Genomes
1 Let’s Recapitulate. 2 Regular Languages DFAs NFAs Regular Expressions Regular Grammars.
We will resume in: 25 Minutes.
©Brooks/Cole, 2001 Chapter 12 Derived Types-- Enumerated, Structure and Union.
Essential Cell Biology
Chapter Thirteen The One-Way Analysis of Variance.
Intracellular Compartments and Transport
PSSA Preparation.
Experimental Design and Analysis of Variance
Essential Cell Biology
Testing Hypotheses About Proportions
Immunobiology: The Immune System in Health & Disease Sixth Edition
Simple Linear Regression Analysis
1 Chapter 13 Nuclear Magnetic Resonance Spectroscopy.
Energy Generation in Mitochondria and Chlorplasts
January Structure of the book Section 1 (Ch 1 – 10) Basic concepts and techniques Section 2 (Ch 11 – 15): Inference for quantitative outcomes Section.
Murach’s OS/390 and z/OS JCLChapter 16, Slide 1 © 2002, Mike Murach & Associates, Inc.
Presentation transcript:

Issues of Simultaneous Tests for Non-Inferiority and Superiority Tie-Hua Ng*, Ph. D. U.S. Food and Drug Administration Presented at MCP 2002 August 5-7, 2002 Bethesda, Maryland _______ * The views expressed in this presentation are not necessarily of the U.S. Food and Drug Administration.

2 Simultaneous Tests for Non-Inferiority and Superiority Multiplicity adjustment is not necessary –Intersection-union principle (IU) Dunnett and Gent (1996) –Closed testing procedure (CTP) Morikawa and Yoshida (1995) Indisputable

3 A Big Question Is Multiplicity Adjustment Necessary?

4 IsMultiplicityAdjustmentNecessary?

5 Outline Assumptions and Notations Switching between Superiority and Non-Inferiority Is Simultaneous Testing Acceptable? Use of Confidence Interval in Hypothesis Testing --- Pitfall Problems of Simultaneous Testing Conclusion

6 Assumptions/Notations Normality and larger is better T: Test/Experimental treatment (  t ) S: Standard therapy/Active control (  s )  : Non-Inferiority Margin (> 0) For a given d (real number), define –Null: H 0 (d): T  S - d –Alternative: H 1 (d): T > S - d Non-Inferiority: d =  Superiority: d = 0

7 Non-Inferiority (d =  ) H 0 (  ): T  S -  against H 1 (  ): T > S -  H0()H0() H1()H1() ° T Boundary WorseBetter Mean Response S 

8 Superiority (d = 0) H 0 (0): T  S against H 1 (0): T > S H 0 (0) H 1 (0) ° T Boundary WorseBetter Mean Response S

9 Switching between Superiority and Non-Inferiority CPMP (Committee for Proprietary Medicinal Products), European Agency for the Evaluation of Medicinal Products Points to Consider on Switching Between Superiority and Non-Inferiority,

10 Switching between Superiority and Non-Inferiority (2) Non-Inferiority Trial –If H 0 (  ) is rejected, proceed to test H 0 (0) –No multiplicity issue, closed testing procedure Superiority Trial –Fail to reject H 0 (0), proceed to test H 0 (  ) –No multiplicity issue –Post hoc specification of 

11 Switching between Superiority and Non-Inferiority (3) Non-inferiority Trial –Intention-to-treat (ITT) –Per protocol (PP) Superiority Trial –Primary: Intention-to-treat (ITT) –Supportive: Per protocol (PP) Assume ITT = PP

12 Simultaneous Testing One-sided 100(1 -  )% lower Confidence Interval for T - S Test is worse Test is better Mean Difference (T – S) 0 -- Superiority Non-inferiority Neither

13 Simultaneous Testing (2) Multiplicity adjustment is not necessary –Dunnett and Gent (1996) Intersection-Union (IU): Superiority: Both H 0 (  ) and H 0 (0) are rejected –Morikawa and Yoshida (1995) Closed Testing Procedure (CTP): Test H 0 (0) when H 0 (  )  H 0 (0) is rejected

14 Simultaneous Testing (3) Discussion Forum (October 1998) –London –PSI (Statisticians in Pharmaceutical Industry) Is Simultaneous Testing of Equivalence [Non- Inferiority] and Superiority Acceptable? –Superiority trial: Fail to reject H 0 (0) No equivalence/non-inferiority claim –Ok: Morikawa and Yoshida (1995) Ref: Phillips et al (2000), DIJ

15 Is Simultaneous Testing Acceptable?

16 Use of Confidence Interval in Hypothesis Testing H 0 (d): T  S - d (at significance level  ) One-sided 100(1-  )% lower CI for T-S Reject H 0 (d) if and only if the CI excludes -d Test is worseTest is better Mean Difference (T – S) -d Reject H 0 (d) Do not reject H 0 (d)

17 Use of Confidence Interval in Hypothesis Testing (2) If CI = (L,  ), then H 0 (d) will be rejected for all -d < L. A Tricky Question –Suppose CI = (-1.999,  ), L = H 0 (2): T  S - 2 is rejected (d=2) since -d < L Can we conclude that T > S - 2? Yes, if H 0 (2) is prespecified. No, otherwise.

18 Use of Confidence Interval in Hypothesis Testing (3) Post hoc specification of H 0 (d) is a No

19 Simultaneous Testing: Problems H 0 (d 1 ) and H 0 (d 2 ), for d 1 > d 2 One-sided (1 -  )100% lower CI for T - S Test is worse Test is better Mean Difference (T – S) -d 2 -d 1 Reject H 0 (d 2 ) Reject H 0 (d 1 ) Neither

20 Simultaneous Testing: Problems (2) H 0 (d 1 ), H 0 (d 2 ) and H 0 (d 3 ), for d 1 > d 2 > d 3 One-sided (1 -  )100% lower CI for T - S Test is worse Test is better Mean Difference (T – S) -d 3 -d 1 Reject H 0 (d 3 ) Reject H 0 (d 2 ) None -d 2 Reject H 0 (d 1 )

21 Simultaneous Testing: Problems (3) H 0 (d 1 ), H 0 (d 2 ),…, H 0 (d k ), for d 1 > d 2 > … > d k One-sided (1 -  )100% lower CI for T - S Test is worse Test is better Mean Difference (T – S) -d k -d 1 Reject H 0 (d k ) Reject H 0 (d 2 ) None -d 2 Reject H 0 (d 1 ) … … d 3

22 Simultaneous Testing: Problems (4) Choose k large enough  Pr[-d 1 < Lower limit < -d k ] close to 1  Max |d k - d k-1 | < a given small number  Simultaneous testing of H 0 (d i ), i = 1,…, k  Post hoc specification of H 0 (d)

23 Simultaneous Testing: Problems (5) Number of Nested hypotheses Exploratory (many H 0 (d)) Confirmatory (one H 0 (d)) …………. k ………… Simultaneous H 0 (  ) and H 0 (0)

24 Simultaneous Testing: Problems (6) What is wrong with IU and CTP? Nothing Pr[Rejecting at least one true null]   What kind of problems?

25 Simultaneous Testing: Problems (7) Post hoc specification of H 0 (d)  Let -d 0 = 100(1 -  )% lower limit -   Reject H 0 (d 0 ), since -d 0 < lower limit  Repeat the same trial independently  Pr[Rejecting H 0 (d 0 )] = 0.5 +

26 Simultaneous Testing: Problems (8) Simultaneous testing of many H 0 (d) –Repeat the same trial independently –Low probability of confirming the finding 1 st trial: Reject H 0 (d j ) but not H 0 (d j+1 ) 2 nd trial: Pr[Rejecting H 0 (d j )] is low (e.g., 0.5+)

27 Simultaneous Testing: Problems (9) Simultaneous testing of H 0 (  ) and H 0 (0)? Confirm the finding   = 2  Known variance  Let   T - S  Significance level  =  80% power for H 0 (  ) (at  = 0)

28 Simultaneous Testing: Problems (10) f  (  ) = Pr[Rejecting H 0 (  ) |  ] f 0 (  ) = Pr[Rejecting H 0 (0) |  ]

29 Simultaneous Testing: Problems (11) Test one null hypothesis H 0 (  ) Suppose that H 0 (  ) is rejected Repeat the same trial independently Pr[Rejecting H 0 (  ) again] = f  (  )

30 Simultaneous Testing: Problems (12) Test H 0 (  ) and H 0 (0) simultaneously Suppose that H 0 (  ) or H 0 (0) is rejected Repeat the same trial independently Pr[Rejecting the same null hypothesis again] = [1 - w(  )] · f  (  ) + w(  ) · f 0 (  ) = f  (  ) - f 0 (  ) [1 – f 0 (  )/f  (  )], where w(  ) = f 0 (  )/f  (  )

31 Simultaneous Testing: Problems (13) [1 - w(  )] · f  (  ) + w(  ) · f 0 (  ) where w(  ) = f 0 (  )/f  (  ) Simultaneous tests in the 2 nd trial

32 Simultaneous Testing: Problems (14) Ratio: 1 – [f 0 (  )/f  (  )] [1 – f 0 (  )/f  (  )] Ratio may be as low as 0.75

33 Conclusion Many H 0 (d): Problematic Not type I error rate H 0 (  ) and H 0 (0): Acceptable? If “zero tolerance policy”: No If 25% reduction cannot be tolerated: No If 25% reduction can be tolerated: Yes

34 Is Simultaneous Testing of H 0 (  ) and H 0 (0) Acceptable?

35 You be the judge

36 References Dunnett and Gent (1976), Statistics in Medicine, 15, Committee for Proprietary Medicinal Products (CPMP; 2002). Points to Consider on Switching Between Superiority and Non-Inferiority. Morikawa T, Yoshida M. (1995), Journal of Biopharmaceutical Statistics, 5: Phillips et al., (2000), Drug Information Journal, 34:

37