1. Experiment Basics: Designs
Psych 231: Research Methods in Psychology

2. Experimental designs Some specific experimental designs.
Some bad (but not uncommon) designs (and potential fixes)
Some good designs
1 Factor, two levels
1 Factor, multi-levels
Factorial (more than 1 factor)
Between & within factors
Experimental designs

3. Factorial experiments
Two or more factors
Some vocabulary
Factors - independent variables
Levels - the levels of your independent variables
2 x 4 design means two independent variables, one with 2 levels and one with 4 levels
“Conditions” or “groups” is calculated by multiplying the levels, so a 2x4 design has 8 different conditions B1 B2 B3 B4 A1 A2
Factorial experiments

4. Factorial experiments
Two or more factors
Main effects - the effects of your independent variables ignoring (collapsed across) the other independent variables
Interaction effects - how your independent variables affect each other
Example: 2x2 design, factors A and B
Main effect of A: A1 vs. A2 A A1 A2
Dependent Variable A A1 A2
Dependent Variable B1 B2
Factorial experiments

5. Factorial experiments
Two or more factors
Main effects - the effects of your independent variables ignoring (collapsed across) the other independent variables
Interaction effects - how your independent variables affect each other
Example: 2x2 design, factors A and B
Main effect of A: A1 vs. A2
Main effect of B: B1 vs. B2 A A1 A2
Dependent Variable B1 B2 A A1 A2
Dependent Variable B1 B2
Factorial experiments

6. Factorial experiments
Two or more factors
Main effects - the effects of your independent variables ignoring (collapsed across) the other independent variables
Interaction effects - how your independent variables affect each other
Example: 2x2 design, factors A and B
Main effect of A: A1 vs. A2
Main effect of B: B1 vs. B2
Interaction:
At A1, B1 is bigger than B2
At A2, B1 and B2 don’t differ A A1 A2
Dependent Variable B1 B2
Everyday interaction = “it depends on …”
Factorial experiments

7. Rate how much you would want to see a movie (1 no interest, 5 high interest):
Hail, Caesar! – new Cohen Brothers movie in 2016
Ask men and women – looking for an effect of gender
Not much of a difference: no effect of gender
Interaction effects

8. Maybe the gender effect depends on whether you know who is in the movie. So you add another factor:
Suppose that George Clooney or Scarlett Johansson might star. You rate the preference if he were to star and if he were not to star.
Effect of gender depends on whether George or Scarlett stars in the movie or not
This is an interaction
Interaction effects
A video lecture from ThePsychFiles.com podcast

9. Results of a 2x2 factorial design
The complexity & number of outcomes increases:
A = main effect of factor A
B = main effect of factor B
AB = interaction of A and B
With 2 factors there are 8 basic possible patterns of results:
1) No effects at all
2) A only
3) B only
4) AB only
5) A & B
6) A & AB
7) B & AB
8) A & B & AB
Results of a 2x2 factorial design

10. 2 x 2 factorial design Interaction of AB A1 A2 B2 B1 Marginal means
What’s the effect of A at B1?
What’s the effect of A at B2?
Condition
mean
A1B1
Condition
mean
A2B1
Marginal means
B1 mean
B2 mean
A1 mean
A2 mean
Main effect of B
Condition
mean
A1B2
Condition
mean
A2B2
Main effect of A
2 x 2 factorial design

11. Examples of outcomes Main effect of A ✓ Main effect of B
Dependent Variable B1 B2 30 60 45 60 45 30 30 60
Main Effect
of A
At A1: B1 = B2
At A2: B1 = B2
The effect of A doesn’t depend on level of B
Main effect of A ✓
Main effect of B X
Interaction of A x B X
Examples of outcomes

12. Examples of outcomes Main effect of A Main effect of B ✓
Dependent Variable B1 B2 60 60 60 30 30 30 45 45
Main Effect
of A
At A1: B1 - B2 = 30
At A2: B1 - B2 = 30
The effect of A doesn’t depend on level of B
Main effect of A X
Main effect of B ✓
Interaction of A x B X
Examples of outcomes

13. Examples of outcomes Main effect of A Main effect of B
Dependent Variable B1 B2 60 30 45 60 45 30 45 45
Main Effect
of A
At A1: B1 - B2 = +30
At A2: B1 - B2 = -30
The effect of A does depend on level of B
Main effect of A X
Main effect of B X
Interaction of A x B ✓
Examples of outcomes

14. Examples of outcomes Main effect of A ✓ Main effect of B ✓
Dependent Variable B1 B2 30 60 45 30 30 30 30 45
Main Effect
of A
At A1: B1 - B2 = 0
At A2: B1 - B2 = 30
The effect of A does depend on level of B
Main effect of A ✓
Main effect of B ✓
Interaction of A x B ✓
Examples of outcomes

15. Anxiety and Test Performance
Let’s add another variable: test difficulty.
anxiety
low
mod
high 80 35 50 70 80
main effect
of difficulty
test performance
high
low
mod
anxiety
easy
easy
medium
hard
Test difficulty 80 80 80
medium 65 80
hard 65 80 60
main effect
of anxiety
Yes: effect of anxiety depends on level of test difficulty
Interaction ?
Anxiety and Test Performance

16. Factorial designs Consider the results of our class experiment ✓ ✓ X
Main effect of cell phone ✓
1.19
0.73 X
Main effect of site type
An Interaction between cell phone and site type
Factorial designs
Report for each main effect and the interaction
Resource: Dr. Kahn's reporting stats page
Means (& SDs) from the table
ANOVA, alpha level 0.05
E.g., “F(1,126) = 26.8, p < .05”

17. Factorial Designs Advantages Interaction effects
Always consider the interaction effects before trying to interpret the main effects
Adding factors decreases the variability
Because you’re controlling more of the variables that influence the dependent variable
This increases the statistical Power of the statistical tests
Increases generalizability of the results
Because you have a situation closer to the real world (where all sorts of variables are interacting)
Factorial Designs

18. Factorial Designs Disadvantages
Experiments become very large, and unwieldy
The statistical analyses get much more complex
Interpretation of the results can get hard
In particular for higher-order interactions
Higher-order interactions (when you have more than two interactions, e.g., ABC).
Factorial Designs