Statistics for the Social Sciences Psychology 340 Fall 2013 Tuesday, October 15, 2013 Analysis of Variance (ANOVA)

Homework Assignment Due 10/22 Chapter 12: 14, 15, 21, 22 (Use SPSS for #21 & 22. Print out your output, identify the relevant statistics and probabilities on the output, and write out responses to all parts of each question) Chapter 13: 1, 4, 7, 8

Statistical analysis follows design The 1 factor between groups ANOVA: –More than two –Independent & One score per subject –1 independent variable

Last Time Basics of ANOVA Why Computations (Definitional & Computational Formulas) Questions about any of the above before we move on?

Today Brief review of last time ANOVA table Assumptions in ANOVA Post-hoc and planned comparisons Effect sizes in ANOVA ANOVA in SPSS Writing up ANOVA results in research reports The structural model in ANOVA

Example Effect of knowledge of prior behavior on jury decisions – Dependent variable: rate how innocent/guilty – Independent variable: 3 levels Criminal record Clean record No information (no mention of a record)

Analysis of Variance ◦ Need a measure that describes several difference scores ◦ Variance  Variance is essentially an average squared difference MBMB MAMA MCMC Criminal recordClean recordNo information Test statistic Observed variance Variance from chance F-ratio = More than two groups

ANOVA Tables SourceSSdfMSF BetweenSS Between df Between MS Between F WithinSS Within df Within MS Within TotalSS Total df Total SourceSSdfMSF Between Within Total Results from criminal record study displayed as ANOVA table: Generic ANOVA table:

Assumptions in ANOVA Populations follow a normal curve Populations have equal variances

Why do the ANOVA? What’s the big deal? Why not just run a bunch of t- tests instead of doing an ANOVA? – Experiment-wise error (see pg 391) – The type I error rate of the family (the entire set) of comparisons » α EW = 1 - (1 - α) c where c = # of comparisons » e.g., If you conduct two t-tests, each with an alpha level of 0.05, the combined chance of making a type I error is nearly 10 in 100 (rather than 5 in 100) – Planned comparisons and post hoc tests are procedures designed to reduce experiment-wise error

Testing Hypotheses with ANOVA ◦ Step 2: Set decision criteria ◦ Step 3: Compute your test statistics  Compute your estimated variances  Compute your F-ratio ◦ Step 4: Make a decision about your null hypothesis Hypothesis testing: a four step program ◦ Step 1: State your hypotheses ◦ Additional tests: Planned comparisons & Post hoc tests  Reconciling our multiple alternative hypotheses

Testing Hypotheses with ANOVA Null hypothesis: H 0 : all the groups are equal MBMB MAMA MCMC ◦ Step 1: State your hypotheses Hypothesis testing: a five step program  Alternative hypotheses (H A )  Not all of the populations all have same mean The ANOVA tests this one!! The ANOVA tests this one!! Choosing between these requires additional test

1 factor ANOVA XBXB XAXA XCXC  Alternative hypotheses (H A )  Not all of the populations all have same mean Planned contrasts and Post-hoc tests: ◦ Further tests used to rule out the different alternative hypotheses ◦ reject ◦ fail to reject

Which follow-up test? Planned comparisons – A set of specific comparisons that you “planned” to do in advance of conducting the overall ANOVA – General rule of thumb, don’t exceed the number of conditions that you have (or even stick with one fewer) Post-hoc tests – A set of comparisons that you decided to examine only after you find a significant (reject H 0 ) ANOVA – Often end up looking at all possible pair-wise comparisons

Planned Comparisons Different types – Simple comparisons - testing two groups – Complex comparisons - testing combined groups – Bonferroni procedure Use more stringent significance level for each comparison – Divide your desired α-level by the number of planned contrasts Basic procedure: – Within-groups population variance estimate (denominator) – Between-groups population variance estimate of the two groups of interest (numerator) – Figure F in usual way

Planned Comparisons Example: compare criminal record & no info grps XBXB XAXA XCXC Criminal recordClean recordNo information ) Within-groups population variance estimate (denominator) 2) Between-groups population variance estimate of the two groups of interest (numerator)

Planned Comparisons Example: compare criminal record & no info grps Criminal recordClean recordNo information ) Within-groups population variance estimate (denominator) 2) Between-groups population variance estimate of the two groups of interest (numerator) 3) Figure F in usual way F crit (1,12) = 4.75 α = 0.05 Fail to reject H 0 : Criminal record and no info are not statistically different XBXB XAXA XCXC

Post-hoc tests Generally, you are testing all of the possible comparisons (rather than just a specific few) – Different types Tukey’s HSD test Scheffe test Others (Fisher’s LSD, Neuman-Keuls test, Duncan test) – Generally they differ with respect to how conservative they are.

Effect sizes in ANOVA The effect size for ANOVA is r 2 – Sometimes called η 2 (“eta squared”) – The percent of the variance in the dependent variable that is accounted for by the independent variable Recall:

Effect sizes in ANOVA The effect size for ANOVA is r 2 – Sometimes called η 2 (“eta squared”) – The percent of the variance in the dependent variable that is accounted for by the independent variable

One-Way ANOVA in SPSS Enter the data: similar to independent samples t-test, observations in one column, a second column for group assignment Analyze: compare means, 1-way ANOVA Your grouping variable is the “factor” and your continuous (outcome) variable goes in the “dependent list” box Specify any comparisons or post hocs at this time too – Planned Comparisons (contrasts): are entered with 1, 0, & -1 – Post-hoc tests: make sure that you enter your α-level Under “options,” you can request descriptive statistics (e.g., to see group means)

ANOVA in Research Articles F(3, 67) = 5.81, p <.01 Means given in a table or in the text Follow-up analyses – Planned comparisons Using t tests

1 factor ANOVA Reporting your results – The observed difference – Kind of test – Computed F-ratio – Degrees of freedom for the test – The “p-value” of the test – Any post-hoc or planned comparison results “The mean score of Group A was 12, Group B was 25, and Group C was 27. A one-way ANOVA was conducted and the results yielded a significant difference, F(2,25) = 5.67, p < Post hoc tests revealed that the differences between groups A and B and A and C were statistically reliable (respectively t(13) = 5.67, p < 0.05 & t(13) = 6.02, p <0.05). Groups B and C did not differ significantly from one another”

The structural model and ANOVA The structural model is all about deviations Score (X) Group mean (M) Grand mean (GM) Score’s deviation from group mean (X-M) Group’s mean’s deviation from grand mean (M-GM) Score’s deviation from grand mean (X-GM)