Closed Testing and the Partitioning Principle Jason C. Hsu The Ohio State University MCP 2002 August 2002 Bethesda, Maryland

Principles of Test-Construction Union-Intersection Testing UIT Union-Intersection Testing UIT S. N. Roy Intersection-Union Testing IUT Intersection-Union Testing IUT Roger Berger (1982) Technometrics Closed testing Closed testing Marcus, Peritz, Gabriel (1976) Biometrika Partitioning Partitioning Stefansson, Kim, and Hsu (1984) Statistical Decision Theory and Related Topics, Berger & Gupta eds., Springer-Verlag. Finner and Strassberger (2002) Annals of Statistics Equivariant confidence set Equivariant confidence set Tukey (1953) Scheffe (195?) Dunnett (1955)

Partitioning confidence sets Multiple Comparison with the Best Multiple Comparison with the Best Gunnar Stefansson & Hsu 1-sided stepdown method (sample-determined steps) = Naik/Marcus-Peritz-Gabriel closed test 1-sided stepdown method (sample-determined steps) = Naik/Marcus-Peritz-Gabriel closed testHsu Multiple Comparison with the Sample Best Multiple Comparison with the Sample Best Woochul Kim & Hsu & Stefansson Bioequivalence Bioequivalence Ruberg & Hsu & G. Hwang & Liu & Casella & Brown 1-sided stepdown method (pre-determined steps) 1-sided stepdown method (pre-determined steps) Roger Berger & Hsu

Partitioning 1. Formulate hypotheses H 0i :    i * for i  I  i  I  i * = entire parameter space  i  I  i * = entire parameter space {  i * : i  I } partitions the parameter space {  i * : i  I } partitions the parameter space 2. Test each H 0i * :    i *, i  I, at  3. Infer    i if H 0i * is rejected 4. Pivot in each  i a confidence set C i for  5.  i  I C i is a 100(1  )% confidence set for 

Partitioning 1. Formulate hypotheses H 0i :    i for i  I  i  J  I = entire parameter space  i  J  I = entire parameter space 2. For each J  I, let  J * =  i  J  i  (  j  J  j ) c 3. Test each H 0J * :    J *, J  I, at  {  J * : J  I} partitions the parameter space {  J * : J  I} partitions the parameter space 4. Infer    J if H 0J * is rejected 5. Pivot in each  J a confidence set C J for  6.  J  J C J is a 100(1  )% confidence set for 

MCB confidence intervals  i  max j  i  j  [  (Y i  max j  i Y j  W) , (Y i  max j  i Y j + W) + ], i = 1, 2, …, k Upper bounds imply subset selection Upper bounds imply subset selection Lower bounds imply indifference zone selection Lower bounds imply indifference zone selection

Multiple Comparison with the Best 1. H 01 : Treatment 1 is the best 2. H 02 : Treatment 2 is the best 3. H 03 : Treatment 3 is the best 4. … Test each at  using 1-sided Dunnett’s Test each at  using 1-sided Dunnett’s Collate the results Collate the results

Partitioning picture

Union-Intersection Testing UIT 1. Form H a :  H ai (an “or” thing) 2. Test H 0 :  H 0i, the complement of H a 1. If reject, infer at least one H 0i false 2. Else, infer nothing

Closed Testing 1. Formulate hypotheses H 0i :    i for i  I 2. For each J  I, let  J =  i  J  i 3. Form closed family of null hypotheses {H 0J :    J : J  I} 4. Test each H 0J at  5. Infer    i  J  i if all H 0J’ with J  J’ rejected 6. Infer    i if all H 0J’ with i  J’ rejected

Oneway model Y ir =  i +  ir, i = 0, 1, 2, …, k, r = 1, …, n i  ir are i.i.d. Normal(0,  2 ) Dose i “efficacious” if  i >  1 +  ICH E10 (2000) Superiority if   0 Superiority if   0 Non-inferiority if  < 0 Non-inferiority if  < 0 Equivalence is 2-sided Equivalence is 2-sided Non-inferiority is 1-sided Non-inferiority is 1-sided

Closed testing null hypotheses (sample-determined steps) 1. H 02 : Dose 2 not efficacious 2. H 03 : Dose 3 not efficacious 3. H 01 : Doses 2 and 3 not efficacious Test each at  Test each at  Collate the results Collate the results

Partitioning null hypotheses (sample-determined steps) 1. H 01 : Doses 2 and 3 not efficacious 2. H 02 : Dose 2 not efficacious but dose 3 is 3. H 03 : Dose 3 not efficacious but dose 2 is Test each at  Test each at  Collate the results Collate the results

Partitioning implies closed testing Partitioning implies closed testing because A size  test for H 0i is a size  test for H 0i A size  test for H 0i is a size  test for H 0i Reject H 01 : Doses 2 and 3 not efficacious implies either dose 2 or dose 3 efficacious Reject H 01 : Doses 2 and 3 not efficacious implies either dose 2 or dose 3 efficacious Reject H 02 : Dose 2 not efficacious but dose 3 efficacious implies it is not the case dose 3 is efficacious but not dose 2 Reject H 02 : Dose 2 not efficacious but dose 3 efficacious implies it is not the case dose 3 is efficacious but not dose 2 Reject H 01 and H 02 thus implies dose 2 efficacious Reject H 01 and H 02 thus implies dose 2 efficacious

Intersection-Union Testing IUT 1. Form H a :  H ai (an “and” thing) 2. Test H 0 :  H 0i, the complement of H a 1. If reject, infer all H 0i false 2. Else, infer nothing

PK concentration in blood plasma curve

Bioequivalence defined Bioequivalence: clinical equivalence between 1. Brand name drug 2. Generic drug Bioequivalence parameters AUC = Area Under the Curve AUC = Area Under the Curve C max = maximum Concentration C max = maximum Concentration T max = Time to maximum concentratin T max = Time to maximum concentratin

Average bioequivalence Notation   = expected value of brand name drug  2 = expected value of generic drug Average bioequivalence means.8 <   /  2 < 1.25 for AUC and.8 <   /  2 < 1.25 for C max

Bioequivalence in practice If log of observations are normal with means   and  2 and equal variances, then average bioequivalence becomes log(.8) <    2 < log(1.25) for AUC and log(.8) <    2 < log(1.25) for C max

Partitioning Partition the parameter space as 1. H 0< :    2 <  log(0.8) 2. H 0> :    2 > log(1.25) 3. H a : log(.8) <    2 < log(1.25) Test H 0 each at . Infer log(.8) rejected. Controls P{incorrect decision} at .

Dose-Response (Phase II)

Anti-psychotic drug efficacy trial Dose of Seroquel (mg) Dose of Seroquel (mg) n iiii SE Arvanitis et al. (1997 Biological Psychiatry) CGI = Clinical Global Impression

Minimum Effective Dose (MED) Minimu Effective Dose =MED =smallest i so that  i >  1 +  for all j, i  j  k Want an upper confidence bound MED + so that P{MED < MED + }  100(1  )%

Closed testing inference Infer nothing if H 01 is accepted Infer nothing if H 01 is accepted Infer at least one of doses 2 and 3 effective if H 01 is rejected Infer at least one of doses 2 and 3 effective if H 01 is rejected Infer dose 2 effective if, in addition to H 01, H 02 is rejected Infer dose 2 effective if, in addition to H 01, H 02 is rejected Infer dose 3 effective if, in addition to H 01, H 03 is rejected Infer dose 3 effective if, in addition to H 01, H 03 is rejected

Closed testing method (sample-determined steps) Start from H 01 to H 02 and H 03 Start from H 01 to H 02 and H 03 Stepdown from smallest p-value to largest p-value Stepdown from smallest p-value to largest p-value Stop as soon as one fails to reject Stop as soon as one fails to reject Multiplicity adjustment decreases from k to k  1 to k  2  to 2 from step 1 to 2 to 3 … to step k  1 Multiplicity adjustment decreases from k to k  1 to k  2  to 2 from step 1 to 2 to 3 … to step k  1

Partitioning picture

Tests of equalities (pre-determined steps) 1. H 0k :  1 =  2 =  =  k H ak :  1 =  2 =  <  k 2. H 0(k  1) :  1 =  2 =  =  k  1 H a(k  1) :  1 =  2 =  <  k  1 3. H 0(k  2) :  1 =  2 =  =  k  2 H a(k  2) :  1 =  2 =  <  k  2 4.  5. H 02 :  1 =  2 H a2 :  1 <  2

Closed testing of equalities Null hypotheses are nested Null hypotheses are nested 1. Closure of family remains H 0k  H Test each H 0i at  3. Stepdown from dose k to dose k  1 to  to dose 2 4. Stop as soon as one fails to reject 5. Multiplicity adjustment not needed

Testing equalities is easy 1. H 0k :  1 =  =  k 2.  3. H 02 :  1 =  2 H 0i  H 0i H 0i  H 0i 1. H 0k :  1   k 2.  3. H 02 :  1   2 H 0i  H 0i

Partitioning null hypotheses (for pre-determined steps) 1. H 0k :Dose k not efficacious 2. H 0(k-1) :Dose k efficacious but dose k  1 not efficacious 3. H 0(k-1) :Doses k and k  1 efficacious but dose k  2 not efficacious 4.  5. H 02 :Doses k  3 efficacious but dose 2 not efficacious Test each at  Test each at  Collate the results Collate the results

Partitioning inference 1. Infer nothing if H 0k is accepted 2. Infer dose k effective if H 0k is rejected 3. Infer dose k  1 effective if, in addition to H 0k, H 0(k-1) is rejected 4. Infer dose k  2 effective if, in addition to H 0k and H 0(k-1), H 03 is rejected 5. 

Partitioning method (for pre-determined steps) Stepdown from dose k to dose k  1 to  to dose 2 Stepdown from dose k to dose k  1 to  to dose 2 Stop as soon as one fails to reject Stop as soon as one fails to reject Multiplicity adjustment not needed Multiplicity adjustment not needed Any pre-determined sequence of doses can be used Any pre-determined sequence of doses can be used Confidence set given in Hsu and Berger (1999 JASA) Confidence set given in Hsu and Berger (1999 JASA)

Pairwise t tests for partitioning Size  tests for H 0k  H 02 are also size  test for H 0k  H H 0k :Dose k not efficacious 2. H 0(k-1) :Dose k  1 not efficacious 3. H 0(k-2) :Dose k  2 not efficacious 4.  5. H 02 :Dose 2 not efficacious Test each with a size-  2-sample 1-sided t-test Test each with a size-  2-sample 1-sided t-test

Testing equalities is easy 1. H 0k :  1 =  =  k 2.  3. H 02 :  1 =  2 H 0i  H 0i H 0i  H 0i 1. H 0k :  1   k 2.  3. H 02 :  1   2 H 0i  H 0i

Could reject for the wrong reason H0H0H0H0 HaHaHaHaneither