1. Experiment Basics: Designs
Psych 231: Research Methods in Psychology

2. Announcements Quiz 7 due Friday
Don’t forget that Exam 2 is coming up (Mon. Oct 22)
For Fun: 538’s article Debate Interruptions
Read it and think about: how to measure your DVs, inter-rater reliability, and operational definitions
Announcements

3. So far we’ve covered a lot of the general details of experiments
Now let’s consider some specific experimental designs.
Some bad (but not uncommon) designs (and potential fixes)
Some good designs
1 Factor, two levels
1 Factor, multi-levels
Factorial (more than 1 factor)
Between & within factors
Experimental designs

4. Poorly designed experiments
Bad design example 1: Does standing close to somebody cause them to move? (theory of personal space)
“hmm… that’s an empirical question. Let’s see what happens if …”
Design: you stand closely to people and see how long before they move
Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”)
Fix: introduce a (or some) comparison group(s)
Very Close (.1 m)
Close (.5 m)
Not Close (1.0 m)
Poorly designed experiments

5. Poorly designed experiments
Bad design example 2:
Does a relaxation program decrease the urge to smoke?
2 groups
relaxation training group
no relaxation training group
The participants choose which group to be in
Training
group
No training
(Control) group
Poorly designed experiments

6. Poorly designed experiments
Bad design example 2:
Non-equivalent control groups
Self
Assignment
Independent Variable
Dependent Variable
Training
group
Measure
participants
No training
(Control) group
Measure
Random
Assignment
Problem: selection bias for the two groups
Fix: need to do random assignment to groups
Poorly designed experiments

7. Poorly designed experiments
Bad design example 3:
Does a relaxation program decrease the urge to smoke?
Pre-test desire to smoke
Give relaxation training program
Post-test desire to smoke
Poorly designed experiments

8. Poorly designed experiments
Bad design example 3:
One group pretest-posttest
design
Dependent Variable
Independent Variable Pre vs. Post
Dependent Variable
Pre-test
Training
group
Post-test
Measure
participants
Pre-test
No Training
group
Post-test
Measure
Fix: Add another factor
Problems include: history, maturation, testing, and more
Poorly designed experiments

9. So far we’ve covered a lot of the general details of experiments
Now let’s consider some specific experimental designs.
Some bad (but not uncommon) designs
Some good designs
1 Factor, two levels
1 Factor, multi-levels
Factorial (more than 1 factor)
Between & within factors
Experimental designs

10. 1 factor - 2 levels Good design example
How does anxiety level affect test performance?
Two groups take the same test
Grp1(low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough
Grp2 (moderate anxiety group): 5 min lecture on the importance of good grades for success
What are our IV and DV?
1 Factor (Independent variable), two levels
Basically you want to compare two treatments (conditions)
The statistics are pretty easy, a t-test
1 factor - 2 levels

11. 1 factor - 2 levels Good design example
How does anxiety level affect test performance?
participants
Low
Moderate
Test
Random
Assignment
IV: Anxiety
Dependent Variable
1 factor - 2 levels

12. 1 factor - 2 levels Good design example
How does anxiety level affect test performance?
One factor
Use a t-test to see if these points are statistically different
low
moderate
test performance
anxiety
anxiety
Two levels
low
moderate
= 20 60 80
Observed difference between conditions
T-test =
Difference expected by chance
Based on estimate of error R
1 factor - 2 levels

13. 1 factor - 2 levels Advantages:
Simple, relatively easy to interpret the results
Is the independent variable worth studying?
If no effect, then may don’t bother with a more complex design
Sometimes two levels is all you need
One theory predicts one pattern and another predicts a different pattern
1 factor - 2 levels

14. 1 factor - 2 levels Interpolation Disadvantages:
“True” shape of the function is hard to see
Interpolation and Extrapolation are not a good idea
low
moderate
test performance
anxiety
What happens within of the ranges that you test?
Interpolation
1 factor - 2 levels

15. 1 factor - 2 levels Extrapolation Disadvantages:
“True” shape of the function is hard to see
Interpolation and Extrapolation are not a good idea
Extrapolation
low
moderate
test performance
anxiety
What happens outside of the ranges that you test?
high
1 factor - 2 levels

16. So far we’ve covered a lot of the general details of experiments
Now let’s consider some specific experimental designs.
Some bad (but not uncommon) designs
Some good designs
1 Factor, two levels
1 Factor, multi-levels
Factorial (more than 1 factor)
Between & within factors
Experimental designs

17. 1 Factor - multilevel experiments
For more complex theories you will typically need more complex designs (more than two levels of one IV)
1 factor - more than two levels
Basically you want to compare more than two conditions
The statistics are a little more difficult, an ANOVA (Analysis of Variance)
1 Factor - multilevel experiments

18. 1 Factor - multilevel experiments
Good design example (similar to earlier ex.)
How does anxiety level affect test performance?
Groups take the same test
Grp1(low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough
Grp2 (moderate anxiety group): 5 min lecture on the importance of good grades for success
Grp3 (high anxiety group): 5 min lecture on how the students must pass this test to pass the course
1 Factor - multilevel experiments

19. 1 factor - 3 levels participants Low Moderate Test Random Assignment
IV: Anxiety
Dependent Variable
High
1 factor - 3 levels

20. 1 Factor - multilevel experiments
low
mod
test performance
anxiety
anxiety
low
mod
high
high 80 60 60
1 Factor - multilevel experiments

21. 1 Factor - multilevel experiments
Advantages
Gives a better picture of the relationship (functions other than just straight lines)
Generally, the more levels you have, the less you have to worry about your range of the independent variable
low
moderate
test performance
anxiety
2 levels
high
low
mod
test performance
anxiety
3 levels
1 Factor - multilevel experiments

22. 1 Factor - multilevel experiments
Disadvantages
Needs more resources (participants and/or stimuli)
Requires more complex statistical analysis (ANOVA [Analysis of Variance] & follow-up pair-wise comparisons)
1 Factor - multilevel experiments

23. Pair-wise comparisons
The ANOVA just tells you that not all of the groups are equal.
Tests: High = Moderate = Low
If your conclusion is that not all groups are equal (you get a “significant ANOVA”), then you should do further tests to see where the differences are:
Pair-wise comparisons (think of these as doing what a t-test would do):
High vs. Low
High vs. Moderate
Low vs. Moderate
Pair-wise comparisons