Independent Samples ANOVA

Outline of Today’s Discussion 1.Independent Samples ANOVA: A Conceptual Introduction 2.The Equal Variance Assumption 3.Cumulative Type I Error & Post Hoc Tests

The Research Cycle Real World Research Representation Research Results Research Conclusions Abstraction Data Analysis MethodologyGeneralization ***

Part 1 Independent Samples ANOVA: Conceptual Introduction

Independent Samples ANOVA 1.Potential Pop Quiz Question: In your own words, explain why a researcher would choose to use an ANOVA rather than a t-test? 2.Ratios, Ratios, Ratios (that is, one number “over” another number) 3.Let’s consider some concepts that you already know…but probably don’t think of as ratios…

Independent Samples ANOVA A z-score is a ratio! We consider the difference (numerator) within the “context” of the variability (denominator).

Independent Samples ANOVA A t statistic is a ratio! We consider the difference (numerator) within the “context” of the variability (denominator).

Independent Samples ANOVA 1.Like the Z-score and the t statistic, the ANOVA (analysis of variance) is also a ratio. 2.Like the t-test, the ANOVA is used to evaluate the difference between means, within the context of the variability. 3.The ANOVA is distinct from the t-test, however, in that the ANOVA can compare multiple means to each other (the t-test can only do two at a time). 4.Let’s compare the t-test & ANOVA a little further…

Independent Samples ANOVA The ANOVA is also called the F Statistic. Cool Factoid: t 2 = F

Independent Samples ANOVA 1.Scores vary! It’s a fact of life. :-) 2.Let’s consider three sources of variability, and then return to how ANOVA deals with these….

Independent Samples ANOVA 1.One source of variability is the treatment effect - the different levels of the independent variable may cause variations in scores. 2.Another source of variability is individual differences - participants enter an experiment with different abilities, motivation levels, experiences, etc.. People are unique! This causes scores to vary. 3.A third source of variability is experimental error - whenever we make a measurement, there is some amount of error (e.g., unreliable instruments, chance variations in the sample). The error in our measurements causes scores to vary.

Independent Samples ANOVA 1.Researchers try to minimize those last two sources of variability (individual differences, and experimental error). 2.They do this by “control” and by “balancing”, as we saw in the last section. 3.Individual differences and experimental error cause variability, both within and between treatment groups. 4.By contrast, treatment effects only cause variability between treatment groups…

Independent Samples ANOVA Notice that the treatment effect pertains only to the between group variability.

Independent Samples ANOVA The ANOVA is also called the F Statistic. Remember: It’s just another ratio. …no big wup! ;) Let’s unpack this thing… F = Variability Between Groups Variability Within Groups

Independent Samples ANOVA Here’s the ANOVA, “unpacked”. The ANOVA is also called the F Statistic. F = Treatment Effects + Individual Differences + Experimental Error Individual Differences + Experimental Error

Independent Samples ANOVA The ANOVA is also called the F Statistic. Here’s the ANOVA, “unpacked” in a different way. Error variation includes both individual differences & experimental error. F = Systematic Variation + Error Variation Error Variation

Independent Samples ANOVA The ANOVA is also called the F Statistic. Here’s the ANOVA, “unpacked” in a different way. Systematic variation includes only the treatment effects.. F = Error Variation + Systematic Variation Error Variation

Independent Samples ANOVA This is what the null hypothesis predicts. It states that treatment effects are zero. The ANOVA is also called the F Statistic. F = 0 + Individual Differences + Experimental Error Individual Differences + Experimental Error

Independent Samples ANOVA The null hypothesis for the ANOVA. In the population, all means are equal. H 0 :  1 =  2 =  3 =  4

Independent Samples ANOVA The alternate hypothesis for the ANOVA. In the population, NOT all means are equal. H 1 : Not H 0

Independent Samples ANOVA 1.Let’s consider a set of pictures, to further develop some intuitions about the F statistic (i.e., the ANOVA). 2.Remember that the F statistic is this ratio: 3.Between Grp Variance / Within Grp Variance

Independent Samples ANOVA Total Variance Here, the variances are equal F ratio = 1 Retain Ho.

Independent Samples ANOVA Here, the variances aren’t equal F ratio < 1 Retain Ho. Total Variance

Independent Samples ANOVA Here, the variances aren’t equal F ratio > 1 Reject Ho! Total Variance

Independent Samples ANOVA 1.You remember our old friend, the variance?… 2.To get the variance we need to determine the sum of squares, then divide by the degrees of freedom (n for a population, n-1 for a sample). 3.Let’s see the total variance, and how ANOVA breaks it down into smaller portions…

Independent Samples ANOVA The sums of squares for the ANOVA, which is also called the F Statistic.

Independent Samples ANOVA The degrees of freedom for the ANOVA, which is also called the F Statistic.

Independent Samples ANOVA The summary table for the ANOVA, which is also called the F Statistic.

Independent Samples ANOVA Would someone explain this, and relate it to ANOVA?

Part 2 The Equal Variance Assumption

1.The ANOVA is based on a few assumptions, one of which is most important… 2.The Equal Variance Assumption - For the ANOVA to be appropriate, the variance (dispersion) must be comparable across the groups or conditions. 3.SPSS can help us to understand whether to retain the assumption, or reject the assumption….

The Equal Variance Assumption In our SPSS output, we can view the “Test” of Homogeneity of Variances. SPSS computes a special statistic called the Levene Statistic.

The Equal Variance Assumption As always, when the the observed alpha level, “Sig.” < 0.05, we reject something! When the “Sig.” > 0.05, we retain something! Here, we RETAIN, the equal variance assumption.

The Equal Variance Assumption So, there are TWO STEPS!!!!! Step 1: We decide whether to retain or reject the equal variance assumption. If we reject, we can’t use ANOVA (perhaps a non-parametric test c/b used). If we retain, we go on to step 2. Step 2: We decide whether to retain or reject the null hypothesis for our study, namely that all  ’s are equal. For this we need to look at the ANOVA table…

The Equal Variance Assumption If the observed alpha level, ‘Sig.’ < 0.05, We can reject, the study’s null hypothesis. Otherwise we retain the null hypothesis. In this example, would we retain or reject?

Part 3 Cumulative Type I Error & Post Hoc Tests

Cumulative Type 1 Error & Post Hoc Tests 1.The ANOVA is very flexible in that it allows us to compare more than 2 groups (or conditions) simultaneously. 2.The overall ANOVA is called the “omnibus ANOVA” or “omnibus F”: This is the test in which all means of interest are compared simultaneously. 3.An omnibus F merely indicates that at least one of the means is different another mean. 4.The omnibus F does NOT indicate which one is different from which!

Cumulative Type 1 Error & Post Hoc Tests 1.If we conduct an experiment a sufficiently large number of times…we are bound to find a “significant” F-value…just by chance! 2.In other words, as we run more and more statistical comparisons, the probability of finding a “significant” result accumulates… 3.Cumulative Type 1 error - (also called “familywise” type 1 error) the increase in the likelihood of erroneously rejecting the null hypothesis when multiple statistical comparisons are made.

Cumulative Type 1 Error & Post Hoc Tests 1.To guard against the cumulative type 1 error, there are various procedures for “correction”, i.e., controlling type 1 error all called ‘Post Hoc Tests’. 2.Each procedure for correcting cumulative type 1 error involves a slight modification to the critical value (i.e., the number to beat). 3.Specifically, the critical value is increased so that it becomes “harder to beat, the number to beat”. :-) 4.Three of the more common “correction” procedures (i.e., Post Hoc Tests) are the Scheffe, the Tukey, and the Dunnet.

Cumulative Type 1 Error & Post Hoc Tests 1.If your F statistic is still significant after the critical value has been “corrected” by one of these post hoc tests tests, you have made a strong case. And remember, the burden of proof is on you. 2.We will not go into the differences among these three post hoc tests here…but the Scheffe is considered the most conservative “check” on cumulative type 1 error.

Cumulative Type 1 Error & Post Hoc Tests 1.To summarize, the post hoc tests allow us to do two things. 2.First, post hoc tests allow us to see exactly which pairs of means differ from each other (the omnibus F can’t do that when there are more than 2 means). 3.Second, post hoc tests control for cumulative type 1 error.

Cumulative Type 1 Error & Post Hoc Tests Post Hoc Tests in SPSS