1. Issues with Mixed Models

2. Model doesn’t converge… OR

3. Convergence

4. Likelihood Landscape

6. Maximum Likelihood Estimation

10. Likelihood = the probability of seeing the data we actually collected given a particular model Maximum Likelihood Estimates = those values that make the observed data most likely to have happened

11. Sources of Convergence Problems You estimate more parameters than data (or, in general, too many parameters Severe collinearity (e.g., two predictors are exactly correlated) Missing cells in your design Predictors of vastly different metrics

12. Failure to converge GENDER ATTITUDEmalefemale polite160 informal1632 … and then trying to test the ATTITUDE*GENDER interaction

13. How can this happen? “Death by Design” (coined by Roger Mundry)

14. design analysis

15. Solutions to Convergence Problems Drop a random slope (not preferred, should be reported) Drop subjects/items for which there is not enough data (not preferred, should be reported) Rescale variables so that they lie range between 0 and 1; or make them on similar metrics overall Center continuous predictors Nonlinear transformations of skewed predictors

16. Solutions to Convergence Problems Change order of variable names in model formula Have a balanced and complete design

17. p-values

18. The p-value conundrum What are the degrees of freedom? How to get p-values out of mixed models is not entirely straightforward… Douglas Bates

19. “There are a number of ways to compute p-values from LMEMs, none of which is uncontroversially the best.” Barr et al. (2013)

20. Ways to get p-values t-test/F-test with normal approximation Likelihood Ratio Test Boostrapping Permutation Markov Chain Monte Carlo (MCMC)

21. Getting p-vals with normal approximation xmdl coefs=data.frame(summary(xmdl)@coefs) coefs$p = 2*(1-pnorm(abs(coefs$t.value))) coefs

22. Function for getting p-vals with normal approximation create.sig.table = function(x){ coefs=data.frame(summary(x)@coefs) coefs$p = 2*(1-pnorm(abs(coefs$t.value))) coefs$sig = character(nrow(coefs)) coefs[which(coefs$p < 0.05),]$sig = "*" coefs[which(coefs$p < 0.01),]$sig = "**" coefs[which(coefs$p < 0.001),]$sig = "***" return(coefs) }

23. Likelihood Ratio Test First model needs to be nested in second

24. Likelihood Ratio The likelihood ratio expresses how many times more likely the data are under one model than the other

25. Likelihood Ratio Test

27. Important when doing likelihood ratio tests lmer(…,REML=FALSE) http://anythingbutrbitrary.blogspot.com/2012/06/r andom-regression-coefficients-using.html

28. Final issue: Random slopes

29. DANGEROUS!!! Random intercept only models are known to be very anti-conservative in many circumstances (cf. Barr et al., 2013, Schielzeth & Forstmeier, 2008)

30. Schielzeth & Forstmeier (2008) Random intercept only

31. Type I error simulation 10 subjects 10 data points each 5 of those in condition A, 5 in B LRTinterceptML0.052 LRTslopeML0.035 LRTinterceptREML0.052 LRTslopeREML0.035 z-testinterceptML0.053 z-testslopeML0.039 z-testinterceptREML0.054 z-testslopeREML0.042

32. Add to this explicit subject slopes for A/B 10 subjects 10 data points each 5 of those in condition A, 5 in B LRTinterceptML0.24 LRTslopeML0.15 LRTinterceptREML0.24 LRTslopeREML0.069 z-testinterceptML0.24 z-testslopeML0.079 z-testinterceptREML0.25 z-testslopeREML0.091

33. Add to this explicit subject slopes for A/B 10 subjects 10 data points each 5 of those in condition A, 5 in B LRTinterceptML0.24 LRTslopeML0.15 LRTinterceptREML0.24 LRTslopeREML0.069 z-testinterceptML0.24 z-testslopeML0.079 z-testinterceptREML0.25 z-testslopeREML0.091

34. Add to this explicit subject slopes for A/B + take item slopes 10 subjects 10 data points each 5 of those in condition A, 5 in B LRTinterceptML0.18 LRTslopeML0.085 LRTinterceptREML0.18 LRTslopeREML0.052 z-testinterceptML0.21 z-testslopeML0.064 z-testinterceptREML0.23 z-testslopeREML0.08

35. LRTinterceptML0.18 LRTslopeML0.085 LRTinterceptREML0.18 LRTslopeREML0.052 z-testinterceptML0.21 z-testslopeML0.064 z-testinterceptREML0.23 z-testslopeREML0.08 Add to this explicit subject slopes for A/B + take item slopes 10 subjects 10 data points each 5 of those in condition A, 5 in B

36. “Keep it maximal”

37. random effects justified by the design vs. random effects justified by the data Barr et al. (2013)

38. “Keep it maximal” “for whatever fixed effects are of critical interest, the corresponding random effects should be in that analysis” Barr et al. (2013)

39. That’s it (for now)