1. Within subjects designs
Definition
Reasons for using within subjects designs
Stage of Practice effects
Two types of practice effects
Order effects
Sequence effects
Within Subjects

2. Within subjects designs
Stage of practice effects (continued)
Remedies
Complete W.S. design
Incomplete W.S. design
Limitations of within subjects designs
Examples of W.S. designs
Grice & Hunter (1964)
Kahneman et al. (1993)
Lee & Katz (1998)
Within Subjects

3. Within subjects designs
Definition
Within Subjects

4. Within subjects designs – definition
When a variable is manipulated within subjects, all subjects receive all levels of that variable.
A given study can use only between groups variables, only within-subjects variables, or a combination of the two.
Within Subjects

5. Within subjects designs – definition
For example, suppose you want to know which of three kinds of car is most comfortable to drive on a long journey.
You have a Ford, a Chevy, and a Toyota, and 10 drivers (the subjects)
Each driver drives each car on the same length journey and rates each for comfort
Here, each driver serves as their own control – so, if one driver is like the princess in “The Princess & the Pea” then their fussiness has the same impact on ratings for each kind of car (assuming there is no subject by treatment (car) interaction).
Within Subjects

6. Within subjects designs
Definition
Reasons for using within subjects designs
Within Subjects

7. Within subjects designs
Reasons for using within subjects designs
Few subjects are available
Increase efficiency
No acceptable matching procedure
Increase sensitivity
Study differences in subjects over time
Compare to between groups design
Within Subjects

8. Reasons for using the W.S. design
Few subjects are available
E.g., research with patients with particular impairments that are important but uncommon, such as deep dyslexia or prosopagnosia
Within Subjects

9. Reasons for using the W.S. design
Few subjects are available
Increase efficiency
Answer more questions with the same number of subjects
E.g., instead of dividing 40 subjects among two treatment groups for one study, use them in two separate studies.
Within Subjects

10. Reasons for using the W.S. design
Few subjects are available
Increase efficiency
No acceptable matching procedure
For example, if you cannot measure enthusiasm, speed of processing, efficiency of attention, etc.
Within Subjects

11. Reasons for using the W.S. design
Few subjects are available
Increase efficiency
No acceptable matching procedure
Increase sensitivity
Sensitivity refers to the ability to detect differences in performance produced by the treatment
Analogous to turning up the magnification of a microscope
Each subject is their own control, reducing error variability due to individual differences. This reduces denominator of statistical test (otherwise denominator includes individual differences unrelated to treatment effect):
Error + Treatment
Error
Within Subjects

12. Reasons for using the W.S. design
Few subjects are available
Increase efficiency
No acceptable matching procedure
Increase sensitivity
Study differences in subjects over time
Learning
Psychophysics
Whenever you want subjects to compare two or more stimuli relative to one another
E.g., Kahneman et al. (1993)
If you want to study learning, you need before and after tests.
Psychophysics – typically many measurements on few subjects.
Within Subjects

13. Reasons for using the W.S. design
Few subjects are available
Increase efficiency
No acceptable matching procedure
Increase sensitivity
Study differences in subjects over time
Compare to between groups design
Treatment might have different effect in within subjects vs. between groups designs.
E.g., Grice & Hunter (1964)
Grice & Hunter (1964): classical conditioning experiment. Two different intensities of sound as conditioned stimuli. In general, a more intense C.S. leads to stronger classical conditioning – that’s the “effect of intensity”. G & H found stronger effect of intensity in a within subjects version of the experiment than in a between groups version.
Within Subjects

14. Within subjects designs
Stage of practice effects
Definition
Two types of stage of practice effects
Order effects
Sequence effects
Remedies
Complete within subjects design
Incomplete within subjects design
Within Subjects

15. Stage of Practice Effects – Definition
The changes subjects undergo with repeated testing are called stage of practice effects.
With repeated testing, subjects’ performance on a task may get:
better if a skill is being developed;
worse if fatigue or boredom increase.
Within Subjects

16. Within subjects designs
Stage of practice effects
Definition
Two types of stage of practice effects
Order effects
Sequence effects
Remedies
Complete within subjects design
Incomplete within subjects design
Within Subjects

17. Two types of stage of practice effects
Order effects
these result from the position in the sequence of treatments that a particular treatment has.
Within Subjects

18. Order effects
If B and D give different results, is that treatment effect?
Subjects might just be more tired, or more skilled, when they get D
A B C D
Within Subjects

19. Two types of stage of practice effects
Sequence effects
These result from interactions among the treatments (also known as differential transfer effects).
Within Subjects

20. Sequence effects A B C D D C A B
This situation could produce sequence effects – is a B / C difference due to the treatment or due to different effects of following A vs. D?
A B C D
D C A B
Differential transfer arises when performance in a given condition varies depending upon the condition that precedes it. Poulton says that balancing techniques do not eliminate this problem. He recommends doing both within subjects and between groups versions of an experiment to find out if differential transfer is a problem. For a large within subjects design, results from the first ordinal position constitute a between groups design:
A B C D E
B C D E A
C D E A B
D E A B C
E A B C D
Within Subjects

21. Within subjects designs
Stage of practice effects
Definition
Two types of stage of practice effects
Order effects
Sequence effects
Remedies
Complete within subjects design
Incomplete within subjects design
Within Subjects

22. Stage of Practice effects – Remedies
Before considering remedies, we have to distinguish between two types of W.S. design:
Complete within subjects design
Incomplete within subjects design
Within Subjects

23. Stage of Practice effects – Remedies
Complete within subjects design
Subjects get each treatment often enough to balance stage of practice effects for each subject.
Subjects get each treatment only once. Levels of I.V. confounded with order levels are presented in
Within Subjects

24. Stage of Practice effects – Remedies
Incomplete within subjects design
Subjects get each treatment only once.
Levels of I.V. are confounded with order levels are presented in
Levels of independent variable are confounded with order levels are presented in within a subject because each subject only gets each treatment once. Across subjects, we can eliminate this problem by varying the order of presentation of treatments from one subject to another. However, the confound is there within a subject so, unlike with the complete within subjects approach, we cannot analyze data for a single subject.
Within Subjects

25. Within subjects designs
Remedies
Complete within subjects design
Block randomization
ABBA counterbalancing
Incomplete within subjects design
Within Subjects

26. Complete Within Subjects Designs
There are two approaches to arranging the order of treatments in a complete within subjects design.
Block randomization
ABBA counterbalancing
Within Subjects

27. Block randomization
Each block of trials contains one trial for each treatment.
Number of blocks = number of times each treatment is administered.
Order of treatments randomized within a block
Works better with many trials per treatment
Note the distinction between this approach and block randomization used to assign subjects to treatments in the between groups design – here, we are using block randomization to assign treatments to trials within one subject.
Within Subjects

28. ABBA counterbalancing
In general, counterbalancing controls for practice effects by presenting the treatments in multiple sequences
ABBA Counter-balancing presents treatments in a sequence, then presents them in the reverse sequence.
Repeat as often as needed to generate desired amount of data per treatment
Within Subjects

29. ABBA counterbalancing
Can be used with any # of treatments and repeated any # of times within an experiment
For 3 treatments, use ABCCBA, etc.
Must repeat whole sequence, not just a part of it
Sequence can be repeated any number of times within an experiment (as long as whole sequence is repeated, not just a part of it). (E.g. ABCCBAABCCBA, but not ABCCBAABCCB.)
Within Subjects

30. ABBA counterbalancing
Anticipation effects may be a problem, especially if there are many cycles through the sequence.
Works well when practice effects are linear.
Does not work with non-linear practice effects. For non-linear effects, stabilize performance with practice trials before recording data
Anticipation effects: when subjects learn the sequence and prepare for the treatment that will arrive on the next trial. This reduces generalizability of results because you are measuring how people perform in a condition when they know that that condition is about to occur and can prepare themselves optimally for it, not how they would ordinarily do the task in that condition in the world.
Within Subjects

31. Trial RT Practice effect
This shows a linear practice effect – increase in speed is the same every trial. ABBA counter-balancing works in this case.

32. Linear practice effectRT
Trial #
Within Subjects

33. Trial RT Practice effect
This shows a nonlinear practice effect – increase in speed is larger in the early trials. ABBA counterbalancing is no help in this case.

34. Non-linear practice effectRT
Trial #
Within Subjects

35. Within subjects designs
Incomplete within subjects designs
Definition
All possible orders
Selected orders
Latin square
Random starting order with rotation
Within Subjects

36. Incomplete W.S. design – definition
Each subject gets each treatment once.
Practice effects are balanced across subjects rather than within subjects.
Levels of the I.V. are confounded with order of presentation within any subject
Thus data for individual subjects are not interpretable
Within Subjects

37. Incomplete W.S. design – definition
In this design:
Hypothesis is tested within subjects.
Practice effects are controlled between groups of subjects.
Within Subjects

38. Incomplete W.S. design General rule for these designs:
Each treatment condition must appear in each ordinal position of the sequence equally often.
The techniques that follow vary in what additional counter-balancing effects they achieve, but all achieve this effect, so all produce interpretable data.
Within Subjects

39. All possible orders Preferred incomplete W.S. design technique
All treatments appear in each ordinal position equally often.
Each treatment precedes & follows every other one equally often at each ordinal position
Within Subjects

40. For 3 treatments (A, B, and C): treatment order Subj # 1st 2nd 3rd 1 A B C 2 A C B 3 B A C 4 B C A 5 C A B 6 C B A

41. Selected orders When we have many treatments, we use selected orders.
We often have 5 or more treatments in one study.
5 treatments = 120 possible orders.
6 treatments = 720 possible orders.
Too many subjects!
When we have many treatments, we use selected orders.
That is, from the set of all possible orders we use only a subset.
Within Subjects

42. Selected orders – Latin square
Each treatment appears equally often at each ordinal position
Each treatment precedes & follow every other treatment exactly once
Limited to experiments with an even number of treatments
Procedures for creating Latin Squares appear in advanced texts.
Within Subjects

43. Selected orders – random starting order with rotation
Start with any order
With each new subject, rotate each treatment one position to the left in the sequence
each condition appears in each ordinal position an equal number of times
but each condition precedes & follows same conditions throughout
advantages: simplicity, applicability
Within Subjects

44. Random starting order with rotation – example with four treatments
Subj # Treatment order 1 D A C B 2 A C B D 3 C B D A 4 B D A C 5 D A C B
Within Subjects

45. Limitations of W.S. designs
W.S. designs cannot be used:
On subject variables such as age and sex.
With unfolding sequences of successive events (for example, animal in Operation condition cannot also be in Anesthesia-only condition).
If each treatment takes a long time (e.g., 1 year).
Within Subjects

46. Examples of within subjects designs
Grice & Hunter (1964)
Kahneman et al. (1993)
Lee & Katz (1998)
Within Subjects

47. Grice & Hunter (1964) Classical conditioning study
Two different intensities of sound as C.S.s
In general, a more intense C.S. gives stronger classical conditioning
G & H found stronger effect of sound intensity in a within subjects version of the study than in a between groups version
This result suggests that a given intensity of sound is psychologically stronger or more intense if you hear it in the context of a less intense sound stimulus.
Within Subjects

48. Kahneman et al. (1993)
Examined effects of three pain characteristics on the memory for pain.
Duration of pain
Worst moment
Final moment
Within Subjects

49. Kahneman et al. (1993) Condition A
Subject keeps hand in 14° C water for 60 seconds
Condition B
Subject keeps hand in water for 90 seconds
60 seconds at 14° C plus 30 extra seconds during which temperature rises gradually to 15° C
Within Subjects

50. Kahneman et al. (1993) One trial per condition
Half of subjects got A first then B
Half of subjects got B first then A
7 minute distracter task
Subjects asked which condition they preferred to repeat
60% chose B
Within Subjects

51. Kahneman et al. (1993) D.V. was choice of pain.
You can only use this D.V. with a within-subjects design
Subjects must get both conditions if they are to choose between them
Within Subjects

52. Kahneman et al. (1993)
You could do this experiment with a different D.V. – say, pain ratings – which would allow a between groups design
But would groups be comparable?
More ‘sissies’ in one group than the other?
Within Subjects

53. Lee & Katz (1998) Study of figurative language
Distinguished between irony and sarcasm
Both ‘figures’ involve saying something you know is not true
Lee & Katz: sarcasm has a victim; irony does not
Within Subjects

54. Lee & Katz (1998) Example Made on a rainy day – irony
“What a sunny day”
Made on a rainy day – irony
Made on a rainy day to someone who predicted sunshine – sarcasm
Within Subjects

55. Lee & Katz (1998) Manipulation:
Subjects read eight passages and rated each for sarcasm on a 7-point scale (that is the dependent variable).
Two I.V.s manipulated within subjects: prediction and victim identity
We’ll look at victim identity today
Within Subjects

56. Lee & Katz (1998) Prediction Victim identity
A prediction made in the passage was either true or false
E.g., prediction that it will be a sunny day
Victim identity
Either the speaker or the listener
E.g., either the speaker or the listener had predicted sunshine
Within Subjects

57. Lee & Katz (1998) Speaker as victim: Mean rating = 4.90 S.d. = 1.34
Listener as victim: Mean rating = 6.43 S.d. = 0.73
Notice that variability is greater for “speaker as victim” – suggests that these stimuli seemed unfamiliar, subjects were uncertain what to do with them.
Within Subjects

58. Lee & Katz (1998)
Same passage was rated as a better example of sarcasm when listener was the victim
Why? Perhaps because people don’t usually make sarcastic remarks about themselves
Within Subjects

59. Lee & Katz (1998)
Subjects are expressing an opinion – is a remark sarcastic?
They may vary in sensitivity to sarcasm or the probability they would use sarcasm
Comparing rated sarcasm for Speaker and Listener conditions between groups would let group differences on sensitivity or probability of use affect means
Within Subjects