Research Methods and Data Analysis in Psychology Spring 2015 Kyle Stephenson

Overview – Day 14 Review Repeated Measures Analysis ▫What happens to the error variance?

Review What do you remember from last class?

Research Methods Basic Concepts Scientific Method Variance Effect Size Measurement Selecting Participants Research Design Descriptive Research Correlational Research Cross-sectional Designs Longitudinal Designs Experimental Studies Simple Experiments Advanced Experimental Design Statistics Descriptive Central Tendency Variation Distributions Outliers Graphing Inferential Correlation Pearson’s R Regression Means Differences T-tests ANOVA Interactions Presentation of Findings Scientific Writing Oral Presentation Ethics

Research Methods Basic Concepts Scientific Method Variance Effect Size Measurement Selecting Participants Research Design Descriptive Research Correlational Research Cross-sectional Designs Longitudinal Designs Experimental Studies Simple Experiments Advanced Experimental Design Statistics Descriptive Central Tendency Variation Distributions Outliers Graphing Inferential Correlation Pearson’s R Regression Means Differences T-tests ANOVA Interactions Presentation of Findings Scientific Writing Oral Presentation Ethics

Problem with Multiple Tests When an experiment has more than two conditions, it is no longer appropriate to use a t-test to analyze the differences between condition means. When one t-test is conducted, the probability of making a Type I error is only 5%. However, when several t-tests are conducted the overall probability of making a Type I is much higher.

What Does an ANOVA Do? Compares “within group variance” to “between group variance” If group doesn’t matter, these should be the same (group means should bounce around about as much as individual scores)

F-test F = MS bg / MS wg Compare this calculated value of F to the critical value of F based on the alpha-level and the degrees of freedom. If the calculated value of F exceeds the critical value of F, then we conclude that at least one of the means differs significantly from one or more of the others.

Follow-up to a Main Effect Post hoc tests or multiple comparisons are used to determine which means differ significantly Examples include: Tukey’s test, Scheffe’s test and Newman-Keuls test If the F-test is not significant, follow-up tests are not conducted because the independent variable has no effect.

ANOVA for Factorial Designs Sums of squares between-groups (SS bg ) can be broken down to test for different main effects and interactions. In a two-way factorial design (A X B), the total variance is composed of four parts 1.Main effect of A 2.Main effect of B 3.A x B interaction 4.Error variance

Follow-up to an Interaction If the F-test shows that an interaction is significant, tests of simple main effects are conducted. A simple main effect is an effect of one independent variable at a particular level of another independent variable. For a two-way interaction of A X B: 1.Simple main effect of A at B1 2.Simple main effect of A at B2 3.Simple main effect of B at A1 4.Simple main effect of B at A2

Why Are Repeated Measures Designs Helpful? Much of the error variance in experiments is related to individual differences – some people just score higher/lower overall because of who they are (nothing to do with IV) This variability is factored into the error variance (denominator of both t and f values), meaning we need bigger mean differences to obtain a significant result

But, if we use the same person for all levels of the IV, a person who scores high in one condition should also score relatively high in the other condition(s) ▫Scores from one condition to the next will be correlated. Use this correlation to estimate how much of the error variance is due to individual differences, then cut that amount out of the final error variance estimate

What’s The Result? Smaller denominator for t and f tests = larger t and f values = more likely to obtain significant results. You can get the same, or stronger, effects using fewer participants But, remember to watch out for order effects

Take-Home Repeated measures designs are usually more powerful because they cut individual differences out of the error variance