1. Psychology 3450W: Experimental Psychology
Spring, 2018
Professor Delamater

2. Experimental Research: Basic Experimental Designs and Control Issues
1. Basic Types of Experimental Designs
Control Issues
Bias in Experimental Research
Lets look at each of these in more detail…

3. Between & Within Subject Designs
Between Subject (Group) Design – Different subjects participate in the different groups of the study.
Within Subject (Group) Design – The same subjects participate in each condition of the study.
Between Subject Design Within Subject Design
Experimental Group Control Group Experimental Control
S1 S5 S1 S1
S2 S6 S2 S2
S3 S7 S3 S3
S4 S8 S4 S4

4. Between & Within Subject Designs
Between Subject (Group) Design – Different subjects participate in the different groups of the study.
Between Subject Design
Experimental Group Control Group
S1 S5
S2 S6
S3 S7
S4 S8
2 Disadvantages:
1. Large N required (compared to within group designs)
2. Potential for nonequivalence

5. Between & Within Subject Designs
Methods of controlling for nonequivalence
1. Random Assignment – works for large numbers of subjects because subject variables will be equally distributed across the different groups of the study.
- block randomization ensures = n/group
2. Matching (on particular subject variables)
- used when sample size is low
- matching variable is known about
- matching variables can be measured easily
Find Matched Pairs (trios, quartets, etc) and then randomly assign to the different groups.
e.g., Effects of caffeine on color perception. Match gender, vision, art major or not, etc.

6. Between & Within Subject Designs
Problems with matching
1. We may not have identified the most critical variable to match.
2. It can be difficult to find enough subjects that are matched on all variables of interest. In this case, we do the best we can and make sure that the different variables we’ve identified as important ones are equally distributed across the groups.

7. Between & Within Subject Designs
Within Subject (Group) Design – The same subjects participate in each condition of the study.
Within Subject Design
Experimental Control
S1 S1
S2 S2
S3 S3
S4 S4
2 Advantages:
1. Small N required (compared to between group designs)
2. Statistically more sensitive at detecting effects
3. No nonequivalence problem

8. Between & Within Subject Designs
Within Subject (Group) Design – The same subjects participate in each condition of the study.
Within Subject Design
Experimental Control
S1 S1
S2 S2
S3 S3
S4 S4
Main Disadvantage:
Sequence (or Order) Effects – Performance in one condition may influence performance in other conditions.
Types of Sequence Effects:
a. Progressive Effect
b. Carry-over Effect

9. Between & Within Subject Designs
Progressive Effect – This refers to a general effect of experiencing multiple treatment conditions (e.g., fatigue, practice).
Carry-over Effect – This refers to a specific effect of one treatment condition upon another. Different orders of exposure to the different treatment conditions could lead to different results.

10. Between & Within Subject Designs
Progressive Effect – This refers to a general effect of experiencing multiple treatment conditions (e.g., fatigue, practice).
Carry-over Effect – This refers to a specific effect of one treatment condition upon another. Different orders of exposure to the different treatment conditions could lead to different results.
e.g., Expose subjects to an Easy and a Hard problem and measure the time it takes for them to complete each task.
The order in which they could be exposed to these problems is either:
1. Easy – Hard, or
2. Hard – Easy
Suppose it takes 6 s to solve the easy problem and 10 to solve the difficult, and that this does not depend upon order. No carry over effect.

11. Between & Within Subject Designs
e.g., Expose subjects to an Easy and a Hard problem and measure the time it takes for them to complete each task.
The order in which they could be exposed to these problems is either:
1. Easy – Hard, or
2. Hard – Easy
Suppose it takes 6 s to solve the easy problem and 10 to solve the difficult, and that this does not depend upon order. No carry over effect.
But, suppose solving the easy problem makes it easier to solve the difficult problem, whereas doing the difficult problem first makes it harder to solve the easy problem.
Easy (6s) – Hard (8s)
Hard (10s) – Easy (12s)
In this situation the easy task is performed more rapidly than the hard task when they are the first task performed. But when they are the 2nd task performed, the hard problem is performed more rapidly than the easy one. This is an example of a Carry-Over Effect (specific orders matter!).

12. Between & Within Subject Designs
Counterbalancing – A method used to control for Sequence Effects
Types of counterbalancing procedures:
1. Complete counterbalancing (N!)
2. Partial counterbalancing (Latin Square)
3. Reverse counterbalancing
4. Block randomization
If subjects are exposed to each condition of the study only once and the number of conditions is not too large, then a complete counterbalancing procedure should be used.
e.g., suppose we wish to have subjects rate 3 different types of wine (A, B, & C). We would need a total of 3 * 2 * 1 = 6 subjects, or multiples of 6.

13. Between & Within Subject Designs
Complete Counterbalancing – N! will determine the number of orders
Order of exposure to the wines
1st, 2nd, 3rd
A B C
A C B
B A C
B C A
C A B
C B A
We could run 6 people, 12, 18, etc
See how this controls for progressive effects
To assess a difference between the wines we average over all the As and compare to Bs and also Cs.
Each wine is tasted in each sequential position of the sequence an equal number of times.
Therefore, if there was a general progressive effect, this would apply equally to each wine. Any difference in wine rating could not be explained in this way.
Subject 1:
Subject 2:
Subject 3:
Subject 4:
Subject 5:
Subject 6:

14. Between & Within Subject Designs
Partial Counterbalancing – Latin Square Design
Order of exposure to the wines
1st, 2nd, 3rd, 4th
A B D C
B C A D
C D B A
D A C B
Suppose there are 4 wines to rate. This requires a minimum of 4 * 3 * 2 * 1 = 24 subjects. That may be too many.
So we can use a Latin Square partial counterbalancing procedure which requires a minimum of 4 subjects (or multiples of 4).
Two Criteria for the Latin Square: (1) each condition occupies each position once, and (2) each condition is preceded and followed by every other condition.
Subject 1:
Subject 2:
Subject 3:
Subject 4:
The rule is: A, B, X, C, X-1, D, X-2, etc
where “X” is the final condition

15. Between & Within Subject Designs
Partial Counterbalancing – Reverse Counterbalancing
Suppose each condition is to be assessed twice.
ABCDDCBA will be the sequence.
See how this controls for progressive effects.

16. Between & Within Subject Designs
Partial Counterbalancing – Block Randomization
Suppose each condition is to be assessed many times (not just once or twice).
ABCD | CADB | ACBD | etc
In each “block” of ratings each condition appears once, and the order in that block is randomized.
See how this controls for progressive effects.

17. Between & Within Subject Designs
The main problem with counterbalancing is that carry-over effects are not adequately dealt with.
Recall the Easy vs Hard problem we considered earlier:
Easy (6s) – Hard (8s)
Hard (10s) – Easy (12s)
In this case, the pattern of results differ depending upon the order in which the two conditions are experienced.
Averaging over the two Easy conditions shows no difference compared to the two Hard conditions. Thus, we would mistakenly conclude that there is no difference in these two cases.
The best thing to do here is to SWITCH to a between group design. This design avoids the problem entirely because each subject is only tested once and there is no carry-over problem.
This particular example is sometimes referred to as asymmetric transfer.

18. Between & Within Subject Designs
How do we distinguish between an overall effect, a progressive effect, and carry-over effects?
Answer: Evaluate the data in different ways.
Order of exposure to the wines
1st, 2nd, 3rd
A B C
A C B
B A C
B C A
C A B
C B A
Subject 1:
Subject 2:
Subject 3:
Subject 4:
Subject 5:
Subject 6:
XA XB XC
Looks like wine A is preferred to wine B which is preferred to wine C. Of course, statistics would need to be done to confirm this.

19. Between & Within Subject Designs
How do we distinguish between an overall effect, a progressive effect, and carry-over effects?
Answer: Evaluate the data in different ways.
Order of exposure to the wines
1st, 2nd, 3rd
A B C
A C B
B A C
B C A
C A B
C B A
Subject 1:
Subject 2:
Subject 3:
Subject 4:
Subject 5:
Subject 6:
X1 X2 X3
Ratings are progressively getting smaller, independent of wine. This suggests there is a progressive effect at work. No differences across positions would indicate no progressive effect.

20. Between & Within Subject Designs
How do we distinguish between an overall effect, a progressive effect, and carry-over effects?
Answer: Evaluate the data in different ways.
Order of exposure to the wines
1st, 2nd, 3rd
A B C
A C B
B A C
B C A
C A B
C B A
Subject 1:
Subject 2:
Subject 3:
Subject 4:
Subject 5:
Subject 6:
The same pattern of ratings appear in each position. Thus, there is no carry-over effect.

21. Between & Within Subject Designs
How do we distinguish between an overall effect, a progressive effect, and carry-over effects?
Answer: Evaluate the data in different ways.
Order of exposure to the wines
1st, 2nd, 3rd
A B C
A C B
B A C
B C A
C A B
C B A
Subject 1:
Subject 2:
Subject 3:
Subject 4:
Subject 5:
Subject 6:
Different patterns of ratings appear in each position. Thus, there are carry-over effects.
Time to switch to a between-subjects experimental design to get rid of this contamination.

22. Between & Within Subject Designs
Experimental Designs and Control Issues in Developmental Research
Cross-sectional design – This is a between-group experimental design in which distinct groups of people of different ages are assessed at the same time on some independent variable or set of independent variables.
e.g., examine how IQ is a function of age by comparing people who are at different ages in life.
Control Issue: Cohort effects. There is a potential non-equivalence problem caused by the fact that people cannot possibly be randomly assigned to the different age groups of the study. So, for instance, does IQ really change with age or are there other differences experienced by the distinct age groups that may account for any apparent differences. People of different generations have different life experiences, for instance, and have different culturally accepted ways of living, different attitudes, etc.

23. Between & Within Subject Designs
Experimental Designs and Control Issues in Developmental Research
Longitudinal design – This is a within-subject experimental design in which the same set of people are assessed at different ages.
e.g., examine how IQ is a function of age, but follow the same people and administer the test repeatedly over many years.
Control Issue: Attrition. Many people may not persist in the experiment for different reasons, so it would be difficult to have a uniform population of people at all the desired time points.
Control Issue: Sequence Effects. With multiple measures taken on the same individual over many time points one does not know if repeated testing itself may affect the results. Counterbalancing is not possible in this procedure.
Note: These potential problems don’t invalidate the research, but, at the same time, any potential limitations in the research procedures should be understood by the investigator.

24. Between & Within Subject Designs
Bias in Experimental Research
Experimenter Bias – This is where experimenter expectations may influence the way in which the experiment is conducted or analyzed and this could affect the results either in an unconscious or conscious way.
e.g., Clever Hans – The famous horse who allegedly could count…
Can the horse count and perform simple math operations?
Or, unconscious bias?

25. Between & Within Subject Designs
Bias in Experimental Research
Experimenter Bias – This is where experimenter expectations may influence the way in which the experiment is conducted or analyzed and this could affect the results either in an unconscious or conscious way.
e.g., Clever Hans – The famous horse who allegedly could count…
Controls:
1. Use computerized, automated, methods of administering the task and collecting the data.
2. Double blind technique: Neither the experimenter nor the subject knows which condition of the experiment they are in. Thus, any conscious or unconscious effects the experimenter may exert on the study cannot be applied differentially to the different groups of the study.

26. Between & Within Subject Designs
Bias in Experimental Research
Subject Bias – This is where the subject’s expectations may influence the manner in which they behave that can confound the experiment.
e.g., If the subject is sensitive to the “demand characteristics” of the experiment, then they might infer the true intent of the study. If they understand the true intent of the study, then they may behave in a manner in which they perceive to be acceptable to the experimenter, or they may defy the experimenter and purposely behave in a way to contradict the way they think the experimenter wishes them to behave. In short, if the subject “knows” what the experiment is about, this can cause the subject to behave unnaturally, and this could adversely affect the results.
Controls:
1. Double blind technique: This works here as well, providing that they effectively conceal the demand characteristics of the study.
2. Deception: If the true intent of the study is disguised sufficiently, then subjects will not be able to infer the true intent of the study. For ethical reasons, subjects would have to be informed about the study’s true intent at the end of the study. More later.
3. Field Research: If experimenter is collecting data by observing people in a public situation, people do not realize they are part of a study and will behave naturally.