Multiple comparisons 郭士逢 輔大生科系 2008 Advanced Biostatistics

Comparing means Using a single-factor ANOVA, we test –H 0 : μ 1 = μ 2 =…= μ k Rejection of H 0 ≠ all k means are different, and we don’t know where differences are located. It is invalid to employ multiple t tests. => multiple comparison tests

Tukey test The q value is calculated by dividing a difference between by SE. –The error mean square from ANOVA and n is the number of data in each of groups. –The critical value q α,ν,k, α is the significance level, ν is the error degrees of freedom for ANOVA, and k is the total number of means being tested.

What is the hypothesis? A two-tailed hypothesis for the pairwise difference is: –H 0 : μ B = μ A, H A : μ B ≠ μ A –H 0 : μ B - μ A = 0, H A : μ B - μ A ≠ 0 –H 0 : μ B - μ A = μ 0, H A : μ B - μ A ≠ μ 0 For k groups, k(k-1)/2 different pairwise comparisons can be made.

Sample size Equal sample sizes are desirable for maximum power and robustness. If the k group sizes are unequal,

Ambiguous results A multiple comparison test yield ambiguous results, in the form of overlapping sets of similarities. –Ex. An experiment consisting of 4 groups of data, conclusion might arrive at the following

Newman-Keuls test Test statistic, q, is calculated as for Tukey’s test, but difference lies in the determination of critical value. –The critical value q α,ν,p, where p is the number of means in the range of means being tested. –Employ different critical values for different range of means – multiple range test

Confidence intervals If one mean is different from all other If two or more means are concluded to be the same, pooled sample mean is the best estimated, and the CI is:

CI for the difference If μ A and μ B are concluded to be different, the 1- α confidence interval for μ A - μ B may be computed by Tukey’s procedure as

Sample size Estimate the sample size to obtain a confidence interval of a specified width for a difference in population means. –Similar to two-sample test, but employ q rather than t statistics. –Where d is the half-width of the CI, s2 is the estimate of the error variance, and k is the total number of means. DF (=k(n-1)) is the error degrees of freedom.

Control group Comparison of a control mean to each other group mean, investigator is only interested in the k-1 comparisons. The test statistic of Dunnett’s test –Where or

Sheffe’s multiple contrasts Test null hypothesis H 0 : μ B - μ A =0 The S test –where –critical value

Agreement among procedures Scheffe’s test may not yield the same conclusion as the Tukey test. –Detect fewer differences –More type II error –Lack of power

Contrast Ex. Testing if the mean level of treatment 2, 3, and 4 is no different from the mean level of treatment 5. The hypotheses could be stated as, H 0 : (μ 2 + μ 3 + μ 4 ) / 3 - μ 5 = 0 H 0 : μ 2 / 3 + μ 3 / 3 + μ 4 / 3 - μ 5 = 0 –Coefficients of μ i : c 2 = 1/3, c 3 = 1/3, c 4 = 1/3, c 5 = 1 –Sum of μ i is always zero.

Test statistic The test statistic, S, is

Confidence interval For the difference between two: For a contrast,