ANOVA Analysis of Variance: Why do these Sample Means differ as much as they do (Variance)? Standard Error of the Mean (“variance” of means) depends upon Population Variance (  /  n) Why do subjects differ as much as they do from one another?  Many Random causes (“Error Variance”) or  Many Random causes plus a Specific Cause (“Treatment”) Making Sample Means More Different than SEM

Why Not the t-Test If 15 samples are ALL drawn from the Same Populations: 105 possible comparisons Expect 5 Alpha errors (if using p<0.05 criterion) If you make your criterion 105 X more conservative (p<0.0005) you will lose Power

The F-Test ANOVA tests the Null hypothesis that ALL Samples came from The Same Population Maintains Experiment Wide Alpha at p<0.05 Without losing Power A significant F-test indicates that At Least One Sample Came from a different population (At least one X-Bar is estimating a Different Mu)

The Structure of the F-Ratio F = The Differences (among the sample means) you got The Differences you could expect to find (If H 0 True) Expectation (If this doesn’t sound familiar, Bite Me!) Evaluation Estimation (of SEM)

The Structure of the F-Ratio F = Average Error of Estimation of Mu by the X-Bars Variability of Subjects within each Sample If H 0 True: Size of Denominator determines size of Numerator If a treatment effect (H 0 False): Numerator will be larger than predicted by denominator

The Structure of the F-Ratio F = Between Group Variance Within Group Variance If a treatment effect (H 0 False): If H 0 True: F = Error Variance Error Variance Approximately Equal With random variation F = Error plus Treatment Variance Error Variance Numerator is Larger

Probability of F  as F Exceeds 1 F = Between Group Variance Within Group Variance If a treatment effect (H 0 False): If H 0 True: F = Error Variance Error Variance Approximately Equal With random variation F = Error plus Treatment Variance Error Variance Numerator is Larger

For U Visual Learners Reflects SEM (Error) H 0 True: Error Plus Treatment H 0 False: Sampling Distributions

Keep the Data, Burn the Formulas

Do These Measures Depend on What Drug You Took? Drug A & B don’t look different, but Drug C looks different From Drug A & B

Partitioning the Variance Each Subject’s deviation score can be decomposed into 2 parts: How much his Group Mean differs from the Grand Mean How he differs from his Group Mean If Grand Mean = 100: Score-1 in Group A =117; Group A mean =115 ( ) = ( ) + ( ) 17 = Score-2 in Group A = 113; Group A mean = 115 (113 – 100) = ( (113 – 115) 13 =

Partitioning the Variance in the Data Set Total Variance (Total Sum of Squared Deviations from Grand Mean) Sum (Xi-Grand Mean)^2 Variance among Subjects Within each group (sample) Sum ( Xi – Group mean)^2 for All subjects in all Groups Variance among Samples Sum (X-Bar – Grand Mean)^2 For all Sample Means SS-Total SS-Within SS-Between

Step 1: Calculate SS-Total

Step 2: Calculate SS-Between Multiply by n (sample size) because: Each subject’s raw score is composed of: A deviation of his sample mean from the grand mean (and a deviation of his raw score from his sample mean)

Step 3: Calculate SS-Within SS-Total – SSb = SSw – = Should Agree with Direct Calculation

Direct Calculation of SSw

Step 4: Use SS to Compute Mean Squares & F-ratio The differences among the sample means are over 11 x greater than if: All three samples came from the Same population None of the drugs had a different effect Look up the Probability of F with 2 & 9 dfs Critical F 2,9 for p<0.01 = 8.02 Reject H 0 Not ALL of the drugs have the same effect

The F-Table

The ANOVA Summary Table

What Do You Do Now? A Significant F-ratio means at least one Sample came from a Different Population. What Samples are different from what other Samples? Use Tukey’s Honestly Significant Difference (HSD) Test

Tukey’s HSD Test Can only be used if overall ANOVA is Significant A “Post Hoc” Test Used to make “Pair-Wise” comparisons Structure: Analogous to t-test But uses estimated Standard Error of the Mean in the Denominator Hence a different critical value (HSD) table

Tukey’s HSD Test Equal N Unequal N

Assumptions of ANOVA 1.All Populations Normally distributed 2.Homogeneity of Variance 3.Random Assignment ANOVA is robust to all but gross violations of these theoretical assumptions

Effect Size MS treatment is really MS b Which is T + E S = 0.10 M = 0.25 L = 0.40 What’s the Question?