1. Closed Testing and the Partitioning Principle Jason C. Hsu The Ohio State University MCP 2002 August 2002 Bethesda, Maryland

2. Principles of Test-Construction Union-Intersection Testing UIT Union-Intersection Testing UIT S. N. Roy Intersection-Union Testing IUT Intersection-Union Testing IUT Roger Berger (1982) Technometrics Closed testing Closed testing Marcus, Peritz, Gabriel (1976) Biometrika Partitioning Partitioning Stefansson, Kim, and Hsu (1984) Statistical Decision Theory and Related Topics, Berger & Gupta eds., Springer-Verlag. Finner and Strassberger (2002) Annals of Statistics Equivariant confidence set Equivariant confidence set Tukey (1953) Scheffe (195?) Dunnett (1955)

3. Partitioning confidence sets Multiple Comparison with the Best Multiple Comparison with the Best Gunnar Stefansson & Hsu 1-sided stepdown method (sample-determined steps) = Naik/Marcus-Peritz-Gabriel closed test 1-sided stepdown method (sample-determined steps) = Naik/Marcus-Peritz-Gabriel closed testHsu Multiple Comparison with the Sample Best Multiple Comparison with the Sample Best Woochul Kim & Hsu & Stefansson Bioequivalence Bioequivalence Ruberg & Hsu & G. Hwang & Liu & Casella & Brown 1-sided stepdown method (pre-determined steps) 1-sided stepdown method (pre-determined steps) Roger Berger & Hsu

4. Partitioning 1. Formulate hypotheses H 0i :    i * for i  I  i  I  i * = entire parameter space  i  I  i * = entire parameter space {  i * : i  I } partitions the parameter space {  i * : i  I } partitions the parameter space 2. Test each H 0i * :    i *, i  I, at  3. Infer    i if H 0i * is rejected 4. Pivot in each  i a confidence set C i for  5.  i  I C i is a 100(1  )% confidence set for 

5. Partitioning 1. Formulate hypotheses H 0i :    i for i  I  i  J  I = entire parameter space  i  J  I = entire parameter space 2. For each J  I, let  J * =  i  J  i  (  j  J  j ) c 3. Test each H 0J * :    J *, J  I, at  {  J * : J  I} partitions the parameter space {  J * : J  I} partitions the parameter space 4. Infer    J if H 0J * is rejected 5. Pivot in each  J a confidence set C J for  6.  J  J C J is a 100(1  )% confidence set for 

6. MCB confidence intervals  i  max j  i  j  [  (Y i  max j  i Y j  W) , (Y i  max j  i Y j + W) + ], i = 1, 2, …, k Upper bounds imply subset selection Upper bounds imply subset selection Lower bounds imply indifference zone selection Lower bounds imply indifference zone selection

7. Multiple Comparison with the Best 1. H 01 : Treatment 1 is the best 2. H 02 : Treatment 2 is the best 3. H 03 : Treatment 3 is the best 4. … Test each at  using 1-sided Dunnett’s Test each at  using 1-sided Dunnett’s Collate the results Collate the results

8. Partitioning picture

9. Union-Intersection Testing UIT 1. Form H a :  H ai (an “or” thing) 2. Test H 0 :  H 0i, the complement of H a 1. If reject, infer at least one H 0i false 2. Else, infer nothing

10. Closed Testing 1. Formulate hypotheses H 0i :    i for i  I 2. For each J  I, let  J =  i  J  i 3. Form closed family of null hypotheses {H 0J :    J : J  I} 4. Test each H 0J at  5. Infer    i  J  i if all H 0J’ with J  J’ rejected 6. Infer    i if all H 0J’ with i  J’ rejected

11. Oneway model Y ir =  i +  ir, i = 0, 1, 2, …, k, r = 1, …, n i  ir are i.i.d. Normal(0,  2 ) Dose i “efficacious” if  i >  1 +  ICH E10 (2000) Superiority if   0 Superiority if   0 Non-inferiority if  < 0 Non-inferiority if  < 0 Equivalence is 2-sided Equivalence is 2-sided Non-inferiority is 1-sided Non-inferiority is 1-sided

12. Closed testing null hypotheses (sample-determined steps) 1. H 02 : Dose 2 not efficacious 2. H 03 : Dose 3 not efficacious 3. H 01 : Doses 2 and 3 not efficacious Test each at  Test each at  Collate the results Collate the results

13. Partitioning null hypotheses (sample-determined steps) 1. H 01 : Doses 2 and 3 not efficacious 2. H 02 : Dose 2 not efficacious but dose 3 is 3. H 03 : Dose 3 not efficacious but dose 2 is Test each at  Test each at  Collate the results Collate the results

14. Partitioning implies closed testing Partitioning implies closed testing because A size  test for H 0i is a size  test for H 0i A size  test for H 0i is a size  test for H 0i Reject H 01 : Doses 2 and 3 not efficacious implies either dose 2 or dose 3 efficacious Reject H 01 : Doses 2 and 3 not efficacious implies either dose 2 or dose 3 efficacious Reject H 02 : Dose 2 not efficacious but dose 3 efficacious implies it is not the case dose 3 is efficacious but not dose 2 Reject H 02 : Dose 2 not efficacious but dose 3 efficacious implies it is not the case dose 3 is efficacious but not dose 2 Reject H 01 and H 02 thus implies dose 2 efficacious Reject H 01 and H 02 thus implies dose 2 efficacious

15. Intersection-Union Testing IUT 1. Form H a :  H ai (an “and” thing) 2. Test H 0 :  H 0i, the complement of H a 1. If reject, infer all H 0i false 2. Else, infer nothing

16. PK concentration in blood plasma curve

17. Bioequivalence defined Bioequivalence: clinical equivalence between 1. Brand name drug 2. Generic drug Bioequivalence parameters AUC = Area Under the Curve AUC = Area Under the Curve C max = maximum Concentration C max = maximum Concentration T max = Time to maximum concentratin T max = Time to maximum concentratin

18. Average bioequivalence Notation   = expected value of brand name drug  2 = expected value of generic drug Average bioequivalence means.8 <   /  2 < 1.25 for AUC and.8 <   /  2 < 1.25 for C max

19. Bioequivalence in practice If log of observations are normal with means   and  2 and equal variances, then average bioequivalence becomes log(.8) <    2 < log(1.25) for AUC and log(.8) <    2 < log(1.25) for C max

20. Partitioning Partition the parameter space as 1. H 0< :    2 <  log(0.8) 2. H 0> :    2 > log(1.25) 3. H a : log(.8) <    2 < log(1.25) Test H 0 each at . Infer log(.8) rejected. Controls P{incorrect decision} at .

21. Dose-Response (Phase II)

22. Anti-psychotic drug efficacy trial Dose of Seroquel (mg) Dose of Seroquel (mg) 075150300600750 n515248515153 iiii4.784.223.743.563.583.93 SE0.230.220.230.230.230.22 Arvanitis et al. (1997 Biological Psychiatry) CGI = Clinical Global Impression

23. Minimum Effective Dose (MED) Minimu Effective Dose =MED =smallest i so that  i >  1 +  for all j, i  j  k Want an upper confidence bound MED + so that P{MED < MED + }  100(1  )%

24. Closed testing inference Infer nothing if H 01 is accepted Infer nothing if H 01 is accepted Infer at least one of doses 2 and 3 effective if H 01 is rejected Infer at least one of doses 2 and 3 effective if H 01 is rejected Infer dose 2 effective if, in addition to H 01, H 02 is rejected Infer dose 2 effective if, in addition to H 01, H 02 is rejected Infer dose 3 effective if, in addition to H 01, H 03 is rejected Infer dose 3 effective if, in addition to H 01, H 03 is rejected

25. Closed testing method (sample-determined steps) Start from H 01 to H 02 and H 03 Start from H 01 to H 02 and H 03 Stepdown from smallest p-value to largest p-value Stepdown from smallest p-value to largest p-value Stop as soon as one fails to reject Stop as soon as one fails to reject Multiplicity adjustment decreases from k to k  1 to k  2  to 2 from step 1 to 2 to 3 … to step k  1 Multiplicity adjustment decreases from k to k  1 to k  2  to 2 from step 1 to 2 to 3 … to step k  1

26. Partitioning picture

27. Tests of equalities (pre-determined steps) 1. H 0k :  1 =  2 =  =  k H ak :  1 =  2 =  <  k 2. H 0(k  1) :  1 =  2 =  =  k  1 H a(k  1) :  1 =  2 =  <  k  1 3. H 0(k  2) :  1 =  2 =  =  k  2 H a(k  2) :  1 =  2 =  <  k  2 4.  5. H 02 :  1 =  2 H a2 :  1 <  2

28. Closed testing of equalities Null hypotheses are nested Null hypotheses are nested 1. Closure of family remains H 0k  H 02 2. Test each H 0i at  3. Stepdown from dose k to dose k  1 to  to dose 2 4. Stop as soon as one fails to reject 5. Multiplicity adjustment not needed

29. Testing equalities is easy 1. H 0k :  1 =  =  k 2.  3. H 02 :  1 =  2 H 0i  H 0i H 0i  H 0i 1. H 0k :  1   k 2.  3. H 02 :  1   2 H 0i  H 0i

30. Partitioning null hypotheses (for pre-determined steps) 1. H 0k :Dose k not efficacious 2. H 0(k-1) :Dose k efficacious but dose k  1 not efficacious 3. H 0(k-1) :Doses k and k  1 efficacious but dose k  2 not efficacious 4.  5. H 02 :Doses k  3 efficacious but dose 2 not efficacious Test each at  Test each at  Collate the results Collate the results

31. Partitioning inference 1. Infer nothing if H 0k is accepted 2. Infer dose k effective if H 0k is rejected 3. Infer dose k  1 effective if, in addition to H 0k, H 0(k-1) is rejected 4. Infer dose k  2 effective if, in addition to H 0k and H 0(k-1), H 03 is rejected 5. 

32. Partitioning method (for pre-determined steps) Stepdown from dose k to dose k  1 to  to dose 2 Stepdown from dose k to dose k  1 to  to dose 2 Stop as soon as one fails to reject Stop as soon as one fails to reject Multiplicity adjustment not needed Multiplicity adjustment not needed Any pre-determined sequence of doses can be used Any pre-determined sequence of doses can be used Confidence set given in Hsu and Berger (1999 JASA) Confidence set given in Hsu and Berger (1999 JASA)

33. Pairwise t tests for partitioning Size  tests for H 0k  H 02 are also size  test for H 0k  H 02 1. H 0k :Dose k not efficacious 2. H 0(k-1) :Dose k  1 not efficacious 3. H 0(k-2) :Dose k  2 not efficacious 4.  5. H 02 :Dose 2 not efficacious Test each with a size-  2-sample 1-sided t-test Test each with a size-  2-sample 1-sided t-test

34. Testing equalities is easy 1. H 0k :  1 =  =  k 2.  3. H 02 :  1 =  2 H 0i  H 0i H 0i  H 0i 1. H 0k :  1   k 2.  3. H 02 :  1   2 H 0i  H 0i

35. Could reject for the wrong reason H0H0H0H0 HaHaHaHaneither