Experiment Basics: Designs Psych 231: Research Methods in Psychology

Experimental designs Some specific experimental designs. Some bad (but not uncommon) designs (and potential fixes) Some good designs 1 Factor, two levels 1 Factor, multi-levels Factorial (more than 1 factor) Between & within factors Experimental designs

Factorial experiments Two or more factors Some vocabulary Factors - independent variables Levels - the levels of your independent variables 2 x 4 design means two independent variables, one with 2 levels and one with 4 levels “Conditions” or “groups” is calculated by multiplying the levels, so a 2x4 design has 8 different conditions B1 B2 B3 B4 A1 A2 Factorial experiments

Factorial experiments Two or more factors Main effects - the effects of your independent variables ignoring (collapsed across) the other independent variables Interaction effects - how your independent variables affect each other Example: 2x2 design, factors A and B Main effect of A: A1 vs. A2 A A1 A2 Dependent Variable A A1 A2 Dependent Variable B1 B2 Factorial experiments

Factorial experiments Two or more factors Main effects - the effects of your independent variables ignoring (collapsed across) the other independent variables Interaction effects - how your independent variables affect each other Example: 2x2 design, factors A and B Main effect of A: A1 vs. A2 Main effect of B: B1 vs. B2 A A1 A2 Dependent Variable B1 B2 A A1 A2 Dependent Variable B1 B2 Factorial experiments

Factorial experiments Two or more factors Main effects - the effects of your independent variables ignoring (collapsed across) the other independent variables Interaction effects - how your independent variables affect each other Example: 2x2 design, factors A and B Main effect of A: A1 vs. A2 Main effect of B: B1 vs. B2 Interaction: At A1, B1 is bigger than B2 At A2, B1 and B2 don’t differ A A1 A2 Dependent Variable B1 B2 Everyday interaction = “it depends on …” Factorial experiments

Rate how much you would want to see a movie (1 no interest, 5 high interest): Hail, Caesar! – new Cohen Brothers movie in 2016 Ask men and women – looking for an effect of gender Not much of a difference: no effect of gender Interaction effects

Maybe the gender effect depends on whether you know who is in the movie. So you add another factor: Suppose that George Clooney or Scarlett Johansson might star. You rate the preference if he were to star and if he were not to star. Effect of gender depends on whether George or Scarlett stars in the movie or not This is an interaction Interaction effects A video lecture from ThePsychFiles.com podcast

Results of a 2x2 factorial design The complexity & number of outcomes increases: A = main effect of factor A B = main effect of factor B AB = interaction of A and B With 2 factors there are 8 basic possible patterns of results: 1) No effects at all 2) A only 3) B only 4) AB only 5) A & B 6) A & AB 7) B & AB 8) A & B & AB Results of a 2x2 factorial design

2 x 2 factorial design Interaction of AB A1 A2 B2 B1 Marginal means What’s the effect of A at B1? What’s the effect of A at B2? Condition mean A1B1 Condition mean A2B1 Marginal means B1 mean B2 mean A1 mean A2 mean Main effect of B Condition mean A1B2 Condition mean A2B2 Main effect of A 2 x 2 factorial design

Examples of outcomes Main effect of A ✓ Main effect of B Dependent Variable B1 B2 30 60 45 60 45 30 30 60 Main Effect of A At A1: B1 = B2 At A2: B1 = B2 The effect of A doesn’t depend on level of B Main effect of A ✓ Main effect of B X Interaction of A x B X Examples of outcomes

Examples of outcomes Main effect of A Main effect of B ✓ Dependent Variable B1 B2 60 60 60 30 30 30 45 45 Main Effect of A At A1: B1 - B2 = 30 At A2: B1 - B2 = 30 The effect of A doesn’t depend on level of B Main effect of A X Main effect of B ✓ Interaction of A x B X Examples of outcomes

Examples of outcomes Main effect of A Main effect of B Dependent Variable B1 B2 60 30 45 60 45 30 45 45 Main Effect of A At A1: B1 - B2 = +30 At A2: B1 - B2 = -30 The effect of A does depend on level of B Main effect of A X Main effect of B X Interaction of A x B ✓ Examples of outcomes

Examples of outcomes Main effect of A ✓ Main effect of B ✓ Dependent Variable B1 B2 30 60 45 30 30 30 30 45 Main Effect of A At A1: B1 - B2 = 0 At A2: B1 - B2 = 30 The effect of A does depend on level of B Main effect of A ✓ Main effect of B ✓ Interaction of A x B ✓ Examples of outcomes

Anxiety and Test Performance Let’s add another variable: test difficulty. anxiety low mod high 80 35 50 70 80 main effect of difficulty test performance high low mod anxiety easy easy medium hard Test difficulty 80 80 80 medium 65 80 hard 65 80 60 main effect of anxiety Yes: effect of anxiety depends on level of test difficulty Interaction ? Anxiety and Test Performance

Factorial designs Consider the results of our class experiment ✓ ✓ X Main effect of cell phone ✓ 1.19 0.73 X Main effect of site type An Interaction between cell phone and site type Factorial designs Report for each main effect and the interaction Resource: Dr. Kahn's reporting stats page Means (& SDs) from the table ANOVA, alpha level 0.05 E.g., “F(1,126) = 26.8, p < .05”

Factorial Designs Advantages Interaction effects Always consider the interaction effects before trying to interpret the main effects Adding factors decreases the variability Because you’re controlling more of the variables that influence the dependent variable This increases the statistical Power of the statistical tests Increases generalizability of the results Because you have a situation closer to the real world (where all sorts of variables are interacting) Factorial Designs

Factorial Designs Disadvantages Experiments become very large, and unwieldy The statistical analyses get much more complex Interpretation of the results can get hard In particular for higher-order interactions Higher-order interactions (when you have more than two interactions, e.g., ABC). Factorial Designs