Posthoc Comparisons finding the differences

Statistical Significance What does a statistically significant F statistic, in a Oneway ANOVA, tell us? What does a statistically significant F statistic, in a Oneway ANOVA, tell us?

Statistical Significance A small p value tells us that there is a low probability that the variability in the means is due to sampling error alone. A small p value tells us that there is a low probability that the variability in the means is due to sampling error alone. We conclude there is more variability between the group means than would be expected by sampling error alone. We conclude there is more variability between the group means than would be expected by sampling error alone.

Overall Purpose A statistically significant F statistic does not tell us where the difference lies. A statistically significant F statistic does not tell us where the difference lies. It only tells us that somewhere among the means there is a difference. It only tells us that somewhere among the means there is a difference. Posthoc comparisons help us find where the difference lies. Posthoc comparisons help us find where the difference lies.

Hypotheses Hypotheses for the Oneway ANOVA: Hypotheses for the Oneway ANOVA: Null Hypothesis:  1 =  2  3...  k  1 =  2  3...  k Alternative Hypothesis:  i =/=  j for at least one pair.  i =/=  j for at least one pair. At least two of the population means are different. At least two of the population means are different.Where: k = the number of population means k = the number of population means

Hypotheses When posthoc comparison procedures are used, there are separate null and alternative hypotheses for each comparison that is made. When posthoc comparison procedures are used, there are separate null and alternative hypotheses for each comparison that is made.

Type I Error Rate Should we simply use as many t-tests as we need to understand a Oneway ANOVA? Should we simply use as many t-tests as we need to understand a Oneway ANOVA? Such a strategy would result in an inflated Type I error rate. Such a strategy would result in an inflated Type I error rate. The Bonferroni procedure allows the researcher to use multiple t-tests with an adjusted alpha level. The Bonferroni procedure allows the researcher to use multiple t-tests with an adjusted alpha level.

Type I Error Rate When using the Bonferroni procedure, the desired familywise alpha is divided by the number of comparisons that will be made. When using the Bonferroni procedure, the desired familywise alpha is divided by the number of comparisons that will be made. The resulting alpha level is used for each comparison. The resulting alpha level is used for each comparison. For the case of five comparisons and a familywise alpha=.05, alpha=.01 is used for each comparison. For the case of five comparisons and a familywise alpha=.05, alpha=.01 is used for each comparison.

The Tukey Procedures The Tukey HSD test (honestly significant difference) is a very helpful procedure. The Tukey HSD test (honestly significant difference) is a very helpful procedure. This procedure performs all possible pairwise comparisons between cell means. This procedure performs all possible pairwise comparisons between cell means. It assumes equal cell sizes and variances. It assumes equal cell sizes and variances.

The Tukey Procedures The Tukey-Kramer test also performs all possible pairwise comparisons between cell means. The Tukey-Kramer test also performs all possible pairwise comparisons between cell means. It does not assume equal cell sizes but does assume equal variances. It does not assume equal cell sizes but does assume equal variances. The Tukey procedures are considered liberal (good power, less control over Type I error rate). The Tukey procedures are considered liberal (good power, less control over Type I error rate).

The Scheffé Procedure The Scheffé procedure can test all possible contrasts, including pairwise comparisons. The Scheffé procedure can test all possible contrasts, including pairwise comparisons. It can be used to compare combinations of cells to each other. It can be used to compare combinations of cells to each other. It is considered conservative (lower power, more control over Type I rate). It is considered conservative (lower power, more control over Type I rate).

Planned Comparisons Planned comparisons are used when there are specific a priori contrasts of interest. Planned comparisons are used when there are specific a priori contrasts of interest. They are based on predictions from theory. They are based on predictions from theory. They must be orthogonal. No information can be used twice. They must be orthogonal. No information can be used twice. You assign weights to cells to create two new cells to be compared. You assign weights to cells to create two new cells to be compared.

Writing About Results It is often helpful to give your audience information about how the cells means rank relative to each other. It is often helpful to give your audience information about how the cells means rank relative to each other. This can be done with tables, graphically, and in text. This can be done with tables, graphically, and in text. See Huck for some useful samples. See Huck for some useful samples.

Our Research Design

The Research Question Are classroom structural characteristic (class size, number of ELL children, etc.) different across the three stress groups? Are classroom structural characteristic (class size, number of ELL children, etc.) different across the three stress groups?