Analysis of Variance and Multiple Comparisons Comparing more than two means and figuring out which are different

Analysis of Variance (ANOVA) Despite the name, the procedures compares the means of two or more groups Null hypothesis is that the group means are all equal Widely used in experiments, it is less common in anthropology

ANOVA in Rcmdr Statistics | Means | One-way ANOVA –Accept or change the model name –Select a group (only factors are listed here) –Select a response variable (only numeric variables are listed here) –Check Pairwise comparison of means

> AnovaModel.1 <- aov(Area ~ Segment, data=Snodgrass) > summary(AnovaModel.1) Df Sum Sq Mean Sq F value Pr(>F) Segment e-15 *** Residuals Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 > numSummary(Snodgrass$Area, groups=Snodgrass$Segment, + statistics=c("mean", "sd")) mean sd n

Results Since the ANOVA statistic is less than our critical value (.05), we reject the null hypothesis that the mean Areas of Segments 1 = 2 = 3 But we usually want to know more Since we did not make predictions in advance our comparisons are post hoc

Multiple Comparisons To find out which means are different from each other we have to compare the various combinations: 1 with 2, 1 with 3, and 2 with 3 (we could also perform other comparisons such as 1 and 2 with 3 but they are rare in anthropology

More Kinds of Errors Our statistical tests have focused on setting the Type I error rate at.05 – the comparisonwise error rate But this error rate holds for a single test. If we do many tests, the chance that we will commit at least one Type 1 error will be higher – the experimentwise error rate

Calculating Errors If the probability of a Type I error is.05, the probability of not making a Type I error is (1 -.05) =.95 The probability of not making a Type I error twice is.95 2 =.9025, three times =.8574, four times =.8145

Calculating Errors The probability of making at least one Type I error is –Twice – ( ) =.0975 –Thrice – ( ) =.1426 –Four times – ( ) =.1855 The probability of making at least one Type I error increases with each additional test

curve((1-(1-.05)^x), 1, 50, 50, yaxp=c(0,.9, 9), xaxp=c(0, 50, 10), xlab="Number of Comparisons", ylab="Type I Error Rate", las=1, main="Experimentwise Error Rate") curve((1-(1-.01)^x), 1, 50, 50, lty=2, add=TRUE) text(30,.92, expression(p == 1-(1-.05)^x), pos=4) text(30,.37, expression(p == 1-(1-.01)^x), pos=4) abline(h=seq(.1,.9,.1), v=seq(0, 50, 5), lty=3, col="gray") legend("topleft", c("Comparisonwise p =.05", "Comparisonwise p =.01"), lty=c(1, 2), bg="white")

Multiple Comparisons Multiple Comparisons procedures take experimentwise error into account when comparing the group means There are a number of methods available, but we’ll stick with Tukey’s Honestly Significant Differences (aka Tukey’s range test)

Tukey’s HSD One of the few multiple comparison tests that can adjust for different sample sizes among the groups You requested this test in Rcmdr when you checked “Pairwise comparison of the means”

>.Pairs <- glht(AnovaModel.1, linfct = mcp(Segment = "Tukey")) > summary(.Pairs) # pairwise tests Simultaneous Tests for General Linear Hypotheses Multiple Comparisons of Means: Tukey Contrasts Fit: aov(formula = Area ~ Segment, data = Snodgrass) Linear Hypotheses: Estimate Std. Error t value Pr(>|t|) == <1e-04 *** == <1e-04 *** == Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 (Adjusted p values reported -- single-step method)

> confint(.Pairs) # confidence intervals Simultaneous Confidence Intervals Multiple Comparisons of Means: Tukey Contrasts Fit: aov(formula = Area ~ Segment, data = Snodgrass) Quantile = % family-wise confidence level Linear Hypotheses: Estimate lwr upr == == ==

NonParametric ANOVA The non-parametric alternative to ANOVA is the Kruskal-Wallis Rank Sum Test The null hypothesis is that the medians of the groups are equal If the test is significant, a multiple comparison method is available to identify which groups are different

Kruskal-Wallis in Rcmdr Statistics | Nonparametric tests | Kruskal-Wallis test –Select a group (only factors are listed here) –Select a response variable (only numeric variables are listed here)

Multiple Comparisons If there are significant differences the function kruskalmc() in package pgirmess will tell you what groups are different

> kruskal.test(Area ~ Segment, data=Snodgrass) Kruskal-Wallis rank sum test data: Area by Segment Kruskal-Wallis chi-squared = , df = 2, p- value = 1.113e-11 library(pgirmess) > kruskalmc(Area ~ Segment, data=Snodgrass) Multiple comparison test after Kruskal-Wallis p.value: 0.05 Comparisons obs.dif critical.dif difference TRUE TRUE FALSE