To go further: intra- versus interindividual variability

Up to now: irrespective of models Multilevel data – Definition – Diagram Simple effects and contrasts Correction for the number of tests RM Anova versus mixed linear models – Advantages – Disadvantages – Conclusion: when to use one or the other Interpretation of parameters, regression equation, and drawing an expected mean graph Conditions of applications Mediation, confounding and moderation 2

Up to now: specific for models Specific to RM ANOVA – Equality of covariance and sphericity (and corrections when sphericity not met) Specific to multilevel models – Fixed versus random effects – Selection of random effects in mixed linear models 3

Definition: multilevel data Data can be non-independent for various reasons: – Clustered in groups (children in classes) – Several measures from same individual – Conditions while taking the measures (e.g., half of them taken in the morning, half in the afternoon) 4

Graphically: 2 levels 5

When is an effect not estimable When there is only one observation for a combination of cells… Why? Because the effect is indistinguishable from residual/individual variability 6

Contrasts Allow to test specific patterns in group effects But… They can be significant when the data do not fit your hypothesized pattern – Some data get ignored They encourage researchers to fish (and you will finally get something) – Be careful what you wish for because you may get it 7

Advantages of RM ANOVA Global test over all levels of categorical variables Unbiased (especially important for small sample sizes) Well-accepted in psychology 8

Disadvantages of RM ANOVA Number of observations by cell have to be equal (even one missing data means all observations for this subject are useless) Number of cells have to be equal (for subjects) Compound symmetry or sphericity assumption (is it substantially credible?) 9

When to use multilevel models When you have missing data When by design you expect a different number of observations per level 2 unit (e.g., follow-up of individuals) When you want to include a continuous level- 1 variable When random intercept or random slope is of substantive interest 10

Conditions of applications RM ANOVA and multilevel modeling have 2 conditions of application in common: – Normality of the DV by cell of the IV Few outliers – Homoscedasticity (equality of variance) – (Linearity: trivial in ANOVA since we only estimate mean differences)

12 Confounding X Y X Y 3 levels of Z X and Y are associated within each level of Z 3 levels of Z X and Y are not associated within each level of Z

Mediation: a special case of confounding 13 IVCDV Mediation IVDV Confounding C IVAC Confounding 2 DV

Mediation: Baron and Kenny There must be a significant relationship between the independent variable and the dependent variable, 2.There must be a significant relationship between the independent variable and the mediating variable, and 3.The mediator must be a significant predictor of the outcome variable in an equation including both the mediator and the independent variable The mediator must be caused by the IV and causing the DV 14 MacKinnon, D. P., Krull, J. L., Lockwood, C. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science. 1(4)

Moderation and interaction A variable moderates another when the association between the IV and the DV is different for one level of the moderator variable than for another level. Example: the effect of antibiotics on response time is very small for people not drinking alcohol but is large for people drinking alcohol (alcohol is the moderator variable) The difference in response time due to antibiotics differs across alcohol level – 2 different in the sentence 15

How to proceed Look at interaction plots Estimate the interaction effect If significant: – Estimate simple effects (relationship between IV and DV for a specific level of a third (moderator) variable) 16

RM ANOVA: condition of application If compound symmetry is met, OK If not, then test sphericity ( is an estimate of sphericity, range 0-1 with 1 = sphericity met) If sphericity not OK, – you could do a MANOVA (i.e., treat each condition as a DV and look at all the DVs together) – You could correct for sphericity (this in fact corrects the df). Best practice: >.75, use Huynh-Feldt correction <.75, use Greenhouse-Geisser correction 17

How to determine whether an IV is fixed or random Are the level of the IV chosen in a planned manner? Or were they randomly selected among other possible instances? 18

Comparing model Likelihood ratio test, if estimation is done with maximum likelihood (ML)… But ML produces biased estimates. – 2(Log(L1)-log(L0)) follows a Chi-square(p1-p2) that is only asymptotically true, p value is overestimated AIC/BIC and other information criteria – No absolute meaning – Can only be used to compare models: relative fit: The lower the value, the better the model 19

Interindividual variability Autists who have more stereotypy have less quality of life In other words, the variability in stereotypy between 2 different individuals (interindividual variability) has an impact on quality of life 20

Intraindividual variability An autist who increases in stereotypy will have a lower quality of life In other words, the variability in stereotypy within an individual (intraindividual variability) has an impact on quality of life 22