1. MRC Cognition and Brain Sciences Unit Graduate Statistics Course
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

2. 8: Repeated Measures ANOVA
Repeated Measures & Mixed Model ANOVA
Within- & Between- Subject Designs
Ian Nimmo-Smith
28 November 2009
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

3. What we will cover in this talk
Two sample t-Test vs. Paired t-Test
Repeated Measures as an extension of paired measures
Single factor Within-Subject design
Sphericity
Two (or more) factors Within-Subject design
Mixed designs combining Within- and Between-Subject factors
Mixed Models, e.g. both Subjects & Items as Random Effects factors
Testing for Normality
Single degree of freedom approach
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

4. Repeated Measures Subjects tested in more than one condition
Increased sensitivity
but added complications!
Not looking at the Temporal/Time Series aspects of Repeated Measures.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

5. Repeated Measures Independent samples t-test vs. Paired t-test
Two groups tested under each of two conditions
One group tested under both conditions
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

6. Independent samples t-test
2 groups of 6 subjects
Each group tested under different conditions
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

7. Independent samples t-test
Comparison of the two conditions will have to take account of the full between-subject variability
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

8. Independent samples t-test
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

9. Independent samples t-test
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

10. Independent samples t-test
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

11. Paired samples t-test Score2-Score1
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

12. Paired samples t-test MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

13. Two ways of comparing Score 1 and Score 2
One sample t-test of the difference
Paired samples t-test
Identical outcomes!
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

14. Paired samples t-test
The correlation between Score 1 and Score 2 is due to repeated testing on the same subject.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

15. Repeated Measures
Analysing differences between conditions measured ‘within subjects’ can avoid the variability due to ‘between subjects’ differences and lead to more powerful tests.
Repeated Measures ANOVA generalises the Paired Sample t-test approach.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

16. Random Factors and Generalisability
Another way of looking at ANOVA models for these designs is by explicitly representing Subjects as Levels of a Factor.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

17. Random Factors and Generalisability
Do we want to generalise to a population of which the subjects are a sample?
If so we describe Subjects as a Random Factor.
For designs for which there is more than one additional Experimental Factor this has important implications for how the data must be analysed.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

18. Random Factors and Error Terms
There is a separate Error Term for each combination of Within Subject Factors.
SPSS -> GLM -> Repeated Measures … does the hard work for us.
Correlations between Repeated Measures help to make the design more powerful
but inhomogeneities can complicate the situation
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

19. Between Subject Design
40 subjects are allocated to 4 treatment groups.
Control
Semantic
Lexical
Phobic
Each subject is given ten trial of a lexical decision task
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

20. MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

21. Between-Subjects Design
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

22. Between-Subjects Design
Total Sum of Squares (SS)
115.9 on 39 d.f.
Priming
(Between groups)
38.9 on 3 d.f.
Error SS
(Between subjects within groups)
77.0 on 36 d.f.
Within
Group 1
9 d.f
Within
Group 2
9 d.f
Within
Group 3
9 d.f
Within
Group 4
9 d.f
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

23. Within Subject Design
Each of 10 Subjects are tested under all 4 conditions
Repeated Measures design
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

24. Within Subject Design
Each subjects data is linked by being on the same line; the conditions become separate variables.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

25. Between vs. Within Subjects Design
Between group profile
Within Individual and mean Group profiles
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

26. Within Subject Design Each comparison (e.g. Semantic vs. Phobic)
could be done by doing a Paired Sample t-Test
T(9) = 3.767, P=0.004 (two-tailed)
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

27. ANOVA of Paired t-Test Total 60.95 on 19 d.f. Between Ss
Within Ss
29.5 on 10 d.f.
Between
Conditions
18.05 on 1 d.f.
Error (Conditions)
[Ss x Conditions]
11.45 on 9 d.f.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

28. Why use Repeated Measures designs?
Repeated Measures ANOVA combines the evidence from all these Paired Samples T-Tests to get more powerful tests because they have (typically)
Higher degrees of freedom
Smaller Mean Square Error
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

29. Analyze -> General Linear Models -> Repeated Measures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

30. Setting up a Repeated Measure variable
Name the repeated measures factor
and state the number of levels
Add
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

31. Setting up a Repeated Measure variable
We can assign a name to the Repeated Measure
(by default it will unhelpfully be called MEASURE_1)
Measure
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

32. Setting up a Repeated Measure variable
Insert name here
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

33. Setting up a Repeated Measure variable
We call the Repeated Measure ‘Correct’ to reflect
that all four columns are ‘Number Correct’
Add
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

34. Setting up a Repeated Measure variable
We now need to indicate the relationship
between the Factor and the Measure
Define
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

35. Setting up a Repeated Measure variable
Select
Apply
The column called ‘Control’ corresponds to
Level 1 of ‘priming’ for measure ‘Correct’, etc.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

36. Setting up a Repeated Measure variable
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

37. Within-Subjects ANOVA
Ignore this bit!
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

38. Within-Subjects ANOVA
Total
on 39 d.f.
Between Ss
48.4 on 9 d.f.
Within Ss
67.5 on 30 d.f.
Between
Conditions
38.9 on 3 d.f.
Error (Conditions)
[Ss x Conditions]
28.6 on 27 d.f.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

39. Sphericity rears its ugly head
Repeated Measures ANOVA relies on homogeneity assumptions for the distribution of Within-Subject variability.
SPHERICITY is the technical name given to this criterion.
Mauchly’s Test of Sphericity
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

40. Sphericity adjustments in action
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

41. Strategies for handling violation of sphericity
Single Degree of Freedom contrasts (modelled on Paired Sample T-Test) using corresponding error terms.
Epsilon: Correction factor for degrees of freedom in
F Ratios
Greenhouse-Geisser
Huynh-Feldt
Lower bound
Multivariate approach.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

42. Correction Factors Unadjusted: F(3, 27) = 12.241
G-G: F(2.2, 20.1) =
H-F: F(3.0,26.9) =
Lower bound: F(1, 9) =
I.e. the observed F-ratio is assessed against
a distribution with adjusted degrees of freedom.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

43. Choice of Correction Factors
Strategy:
If Mauchly is OK, quote unadjusted.
If not, use Greenhouse-Geisser
G-G is known to be conservative.
H-F attempts to correct for this but can overcorrect!
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

44. More than one Repeated Measures Factor?
2 x 2 Within Subject design
Valence: Positive vs. Negative
Orientation: Up vs. Down
All subjects tested in all 4 combinations.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

45. More than one Repeated Measures Factor?
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

46. Add factors for Valence (Positive vs. Negative)
and Orientation (Up vs. Down)
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

47. MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

48. MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

49. The column called ‘posup’ corresponds to Level 1 of Factor ‘valence’ and Level 1 of Factor ‘orient’ for measure ‘Correct’, etc.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

50. MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

51. Three Strata MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

52. Total 115.9 on 39 d.f. Between Ss Within Ss 48.4 on 9 d.f.
Error (Conditions)
[Ss x Conditions]
28.6 on 27 d.f.
Between
Conditions
38.9 on 3 d.f.
Error V
8.1 on 9 V
32.4 on 1 O
6.4 on 1
Error O
8.1 on 9
VxO
0.1 on 1
Error VxO
12.4 on 9
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

53. When Repeated Measures factors have only two levels ...
… Sphericity is not an issue.
(This extends the case for the Paired Sample T-test and points the way to a Single Degree of Freedom contrast approach.)
But note that there is a separate Sphericity test for each of the three strata, Valence, Orientation, and Valence x Orientation.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

54. Mixed Design with both Within- and Between- Subject factors
One Between Subjects factor:
Sex (2 levels, M vs F)
One Within Subject factor:
Condition (4 levels, Con, Sem, Lex, Pho)
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

55. MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

56. MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

57. MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

58. MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

59. Hierarchical Decomposition in 1W2B RM ANOVA
Total
on 39 d.f.
Within Ss
67.5 on 30 d.f.
Between Ss
48.4 on 9 d.f.
Ss x Conditions
28.6 on 27 d.f.
Sex
14.4/1
Error
(Between)
34/8
Sex x Priming
2.2/3
Error (Priming)
26.4/24
Priming
38.9 /3
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

60. Pros and Cons of Repeated Measures Designs
More sensitive/powerful
More efficient
Con:
Fatigue
Transfer effects
Learning
Issues:
Blocked vs Mixed
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

61. Within Subject Designs
Advantages: reduces background 'noise', eliminates individual differences, need fewer subjects, repeated measures statistical tests more sensitive.
Disadvantages: Order effects: learning/practice ; fatigue; practical problems (e.g. long-acting effects of the independent variable, e.g. teaching methods.)
Solutions: Counterbalance order (can be difficult); Train until asymptote on learning curve is reached (time-consuming). Test on different days.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

62. Between Subject Design
Advantages: no order effects
Disadvantages: more individual differences, less sensitivity
Could use pre-test baseline / difference score
Question: Are Subjects a confound? I.e. could your results simply be due to individual differences between subjects?
Very important: Random sampling and random allocation to conditions.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

63. Mixed ANOVA Designs Some Between Subject Factors
Some Within Subject (Repeated Measures) Factors
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

64. Further issue ‘Language as Fixed Effect’ ‘Normality’
Omitted from the presentation but available in the slides
‘Normality’
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

65. More Random Effects Factors
Clark 1973 Language as Fixed Effect Fallacy
Are the Items a (random) sample from a population to which you want to generalise?
If so, Items should be included as a Random Factor too.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

66. Mixed Model ANOVA
This describes the situation where we have one or more Fixed Effects Factor and one or more Random Effects Factor
A full analysis is (now) possible but complicated.
MIXED MODELS & REML
A controversial compromise has now been adopted in some fields/journals.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

67. ‘By Subject’ and ‘By Item’ ANOVAs
Clark (1973) did a review of published studies where separate ‘By Item’ and ‘By Subject’ analyses had been published.
It was not possible (then, and without full data) to calculate the appropriate Quasi-F (F’) statistic taking simultaneous account of Subject and Item variability.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

68. F1, F2 and Min-F’
He showed that F’ must lie between an upper bound Max-F’ and a lower bound Min-F’ which could the calculated from knowledge of F1 (‘By Subject’) and F2 (‘By Item’) and their degrees of freedom.
High enough Min-F’ could allow simultaneous generalisation to both Subjects and to Items.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

69. Amnesia sets in ...
For a while everyone used and reported Min-F’ and groaned that it presented a much higher hurdle for significance.
Gradually the community forgot and now people tend just to look at F1 and F2 on their own and reject a Null Hypothesis if both are significant.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

70. People have started arguing (again)
Raaijmakers, J. G. W., Schrijnemakers, J. M. C., & Gremmen, F. (1999). How to deal with "The language-as fixed-effect fallacy": Common misconceptions and solutions. Journal of Memory and Language, 41,
Quene, H & van den Bergh, H (submitted to JML in 2001). On Multi-Level Analysis as a remedy against "The language-as fixed-effect fallacy”.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

71. Issues Items not a Random Factor Items are a Random Factor
Can you regard the set of items as a fixed test?
Counterbalancing could help with some designs (Raaijmakers)
Items are a Random Factor
Complex Mixed Model ANOVA is now computationally available.
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

72. Testing for Normality
ANOVA models are based on assumptions of Normality
Tests of normality should be applied to the residuals rather than to the raw data
When assessing normality in a within subject design it is the distribution of the differences between pairs of conditions that matters
This means you have to do the ANOVA before you can decide on (non-)normality issues
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

73. Saving and assessing the residuals4 2 3 1
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

74. Residuals (2) MRC CBU Graduate Statistics Lectures
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

75. Residuals (3) No evidence of non-normality
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

76. Residuals (4)
Using the script at
compute sum = cdf.chisq(zre_1**2 + zre_2**2 + zre_3**2 + zre_4**2,4).
NPAR TESTS
/K-S(UNIFORM)= sum
/MISSING ANALYSIS.
No evidence of non-normality
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

77. Single degree of freedom approach for within-subject designs
Propagates from the feature that repeated measures ANOVA of paired measures are equivalent to simple ANOVA of the differences between the pair of measures
Choose a contrast of interest, e.g.
difference;
slope;
interaction
Perform a simple ANOVA on this derived measure which represents 1 of the available within-subject degrees of freedom
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models

78. Next Week Thursday 3 December at 11:00 Peter Watson will talk on
Latent variable models: factor analysis and all that
26 November 2009
MRC CBU Graduate Statistics Lectures
8: Repeated Measures and Mixed Models