Mixed models and their uses in meta-analysis The difference between analysing individual observational units and parameter estimates What I know, and don’t know Weighted regression Weighted regression in meta-analysis Fixed and random components of statistical models: basics of mixed models Weighted regression with mixed models example

Choose the line (i.e. choose the values of ) that minimize the absolute distances of each point from it. Least squares Maximum likelihood

Precision of our measurement of obs 6 Do we still want to treat e6 and e10 the same way?

Assign a lower weight on those observations that are less precise

Normally, when we conduct our own study, we take physical measurements of (x,y) on each « observational unit » If we use the same method to take these measurements then the precision (k) will be the same for all. Minimizing or minimizing will give the same result. Usually don’t need to worry about weighting observations But you do if different observations are measured with different levels of precision.

Analysis vs. Meta-analysis Single analysis: Define your statistical population (all those « observational units » that could be chosen given your sampling method) Randomly sample n observational units  from this statistical population Measure (x, y) on each of the n observational units Analyse with a statistical model Precision of measurements usually the same for all observational units Meta-analysis: Define your statistical population (all those « publications » that could be chosen given your sampling method) Randomly sample n publications  from this statistical population Obtain measures (x, y) from each of the n publications Analyse with a statistical model Precision of measurements usually different in each publication

With weighting of 1/variance Without weighting: Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 1.6200 1.4102 1.149 0.2803 x 0.5912 0.2079 2.843 0.0193 * --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 2.181 on 9 degrees of freedom Multiple R-squared: 0.4732, Adjusted R-squared: 0.4147 F-statistic: 8.085 on 1 and 9 DF, p-value: 0.0193 x y vars 1 1 1.816497 1 2 2 2.808872 1 3 3 1.977704 1 4 4 3.889419 1 5 5 3.600235 1 6 6 6.149848 1 7 7 7.370940 1 8 8 8.014575 1 9 9 8.583731 1 10 10 9.628222 1 11 11 3.000000 4 With weighting of 1/variance Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 0.7023 0.9212 0.762 0.465306 x 0.8206 0.1447 5.670 0.000306 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 1.372 on 9 degrees of freedom Multiple R-squared: 0.7813, Adjusted R-squared: 0.757

A quick review of some basic concepts in statistical modelling …

A statistical model contains « parameters » that have to be estimated A statistical model contains « parameters » that have to be estimated. What is a parameter estimate? A normal distribution has two « parameters ». The true values of these are unknown, but can be estimated, with some degree of error, from the observations. The parameter sigma is completely determined once we know alpha, beta and x.

Least-squares estimate of slope: The precision (standard error) of the slope estimate: How much the y’s vary from the line Sample size How much the x’s vary

Sample size Vary range of x’s Residual variation simulation Changing sample size Changing range of x Changing residual variation of y

If we repeat our study many times in exactly the same way, our estimates will randomly vary around the true value following approximately a normal distribution (central limit theorem) The standard error of a parameter is a quantification of the precision of estimation of that parameter. It tells us how much our estimate of this parameter will vary if we were to repeat our estimates many times. Larger standard errors mean less precision in our estimate, because our estimate will tend to differ more (randomly, centered on the true value) from the true value.

Q: how does the slope of y~x change with temperature across studies? Problem 1: we know that the precision of the estimates of the slope will differ across studies, depending on The sample sizes used The range of x values used Everything besides temperature that can affect the residual variation of y’s given x. Problem 2: we rarely have available the actual observations on each observational unit. We only have statistical summaries in the form of parameter estimates (here, slopes). But, we have the parameter estimate (slopes, intercepts, means) and their standard errors (usually…).

Example: Does limb length decrease relative to body size as one goes further north (Allen’s rule)? slope temperature We collect slope estimates (b) and their standard errors (se.b) from many different studies (one per study for now), and also get the mean annual temperature of the site in which the study was done. Study Slope se.b temperature 1 0.75 0.1 20 2 0.66 0.05 15 3 0.8 0.21 18 fit<-lm(slopes~temperature,data=dat,weights=1/se.b^2)

The weights argument can be used in a linear model Now, what happens if you have more than one line (« observational unit ») for each study? For instance, if studies report slope estimates for different species, or locations…. The parameter estimates within a given study will be correlated because they were taken by the same people, at the same time, in the same geographical locations, etc.

Calculating weights of composite effect variables. Let’s say that we have chosen to measure our « effect » as the difference between two means (say, before and after): Each of these two means will have its associated standard errors – s(m1) and s(m2) – but we need the standard error of the difference between these two means; i.e. s(E). Here is an approximate formula for combining standard errors (it assumes that the m’s are independent):

Standard errors of common composite functions m1 & m2 are two parameter estimates (means, slopes, intercepts, correlation coefficients… and « k » is a constant

A simulation to show what happens if we ignore different amounts of measurement error (i.e. different levels of precision) between studies. Model 1: Actual data without measurement error: X = 20 random values from N(mu=20,sigma=10) Model 2: We don’t have « y » but, rather, estimates of y (y1) that each have the same amount of measurement error (sigma=5) Model 3: We don’t have « y » but, rather, estimates of y (y2) that each have different amounts of measurement error (sigma=5) y2=y +N(0,5)

Parameter No measurement error Constant measurement error Variable measurement error Ignore it weights Intercept 1.4468 -5.1738 -34.208 3.0710 SE 2.3179 3.2849 36.481 3.3428 P 0.541 0.133 0.361 0.370 Slope 1.1265 1.2334 1.972 0.9946 0.1098 0.1555 1.727 0.2015 p 6e-9 3e-7 0.269 1e-4 SEresiduals 4.569 6.47 71.86 7.049 R2 0.854 0.775 0.068 0.575 True relationship:

Main message so far: IF you are extracting « effect sizes » from different papers that are parameter estimates (means, slopes, correlation coefficients…) and not the actual direct measurements you only have one « effect size » per paper THEN You must also get the standard errors of these effect sizes (which measures their measurement error) And you must use these standard errors to weight each effect size so that papers with more precision (i.e. smaller standard errors) are given more weight.

Now, mixed models… What happens if you are extracting « effect sizes » from different papers that are parameter estimates (means, slopes, correlation coefficients…) and not the actual direct measurements And you only have more than one « effect size » per paper

We measure two variables, x and y, on some soil samples taken in the valley outside the field station.

This part applies to all observation: it is the fixed component This part differs randomly for each observation: it is the random component

We measure two variables, x and y, on some soil samples taken in two locations: the valley and the hillside outside the field station.

Doesn’t seem right! Looks like two groups, each with a different slope and intercept Typical analysis: ANCOVA

ANCOVA. Still has only one source of random variation ANCOVA. Still has only one source of random variation. The differences between the two groups are not modelled as random differences. Reference slope Reference intercept How much intercepts of each group differ from the reference intercept: « Treatment Contrasts » How much slopes of each group differ from the reference slope: « Contrasts »

We randomly choose a series of locations in the park We measure two variables, x and y, on randomly chosen soil samples taken in each location. 𝛼 𝑖 =𝛼+ 𝛿 𝑖 𝛽 𝑖 =𝛽+ 𝜙 𝑖 Relationship location i y Average relationship b a x

… i randomly chosen locations j randomly chosen samples nested within each location Equation for location i Fixed effect random effect

Mixed models in R Function lmer() in the lme4 package Function lme() in the nlme package If you use lmer(), you should also install the package « lmerTest » Relationship location i y Average relationship b a lmer(y~x+(1|group),data=…) lmer(y~x+(-1+x|group),data=…) lmer(y~x+(x|group),data=…) x

… i randomly chosen locations j randomly chosen samples nested within each location Simplest case: Variance components Random variation of the mean values of y between locations Random variation of the individual values of y within each location

Total random variation: Proportion of random variation due to differences between locations Proportion of random variation due to differences between observations within each location

Basic format of the lmer function First example: 76 soils randomly chosen across southern Quebec Each soil randomly divided into 4 pots (2 harvests each pot; harvest dates NOT randomly chosen) 4 individuals per pot randomly planted Go to R code… #to use the lmer function for mixed models library(lme4) #to obtain probabilities for t-tests, F-tests etc. library(lmerTest) Basic format of the lmer function fit<-lmer(Y~fixed effects +(random parameters of fixed part|nesting structure),data= ,…)

76 soils across southern Quebec: 0.8184+0.000+6.4e-15 vc.model.wheat<-lmer(log(masse_en_g)~1+(1|sol)+(1|pot),data=dat) summary(vc.model.wheat) Linear mixed model fit by REML ['lmerMod'] Formula: log(masse_en_g) ~ 1 + (1 | sol) + (1 | pot) Data: dat REML criterion at convergence: 3148.3 Scaled residuals: Min 1Q Median 3Q Max -1.9420 -0.8694 -0.3944 0.9891 1.8929 Random effects: Groups Name Variance Std.Dev. pot (Intercept) 0.000e+00 0.000e+00 sol (Intercept) 6.434e-15 8.021e-08 Residual 8.184e-01 9.046e-01 Number of obs: 1192, groups: pot, 304; sol, 76 Fixed effects: Estimate Std. Error t value (Intercept) -3.1768 0.0262 -121.2 76 soils across southern Quebec: 0.8184+0.000+6.4e-15 4 pots for each soil: 0.8184+0.000 4 individuals per harvest: 0.8184 Source of variation % added variation Soils 0.00% Pots 0.00% Individuals ~100% Mean ln(biomass) per plant Almost all variation in ln(biomass) comes from plants growing in the same pot. What could explain this variation? These different plants within a pot have different ages at harvest.

Step 2: We include the plant ages (harvest dates) Step 2: We include the plant ages (harvest dates). We didn’t randomly choose these harvest dates, rather we chose two dates, and so we include this as a fixed effect. vc.model.wheat<-lmer(log(masse_en_g)~age_jour+(1|sol)+(1|pot),data=dat) REML criterion at convergence: 727.5 Scaled residuals: Min 1Q Median 3Q Max -5.9244 -0.5862 0.1007 0.6642 2.7963 Random effects: Groups Name Variance Std.Dev. pot (Intercept) 0.00000 0.0000 sol (Intercept) 0.03998 0.1999 Residual 0.09345 0.3057 Number of obs: 1192, groups: pot, 304; sol, 76 Fixed effects: Estimate Std. Error df t value Pr(>|t|) (Intercept) -4.704e+00 2.953e-02 1.545e+02 -159.28 <2e-16 *** age_jour 1.181e-01 1.266e-03 1.116e+03 93.29 <2e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Problems: (1) this model doesn’t allow the slopes to randomly vary across soils (i.e. it assumes that all of these soils allow the same growth rate; (2) it does allow random intercepts. The intercept is the mean ln(mass) at time 0 (i.e. when the mass of the seedling when it begins emerging from the seed). This is unlikely to change between soils.

Step 3: There are two fixed harvest dates, so we include this as a fixed effect, fix the intercept (since this is the mean ln(biomass) at time 0, and shouldn’t vary across soils and pots), and allow random slopes vc.model.wheat<-lmer(log(masse_en_g)~age_jour+(-1+age_jour|sol)+(-1+age_jour|pot),data=dat) Random effects: Groups Name Variance Std.Dev. pot age.jour 3.343e-05 0.005782 sol age.jour 2.583e-04 0.016073 Residual 7.011e-02 0.264779 Number of obs: 1192, groups: pot, 304; sol, 76 Fixed effects: Estimate Std. Error df t value Pr(>|t|) (Intercept) -4.704667 0.016117 889.400000 -291.9 <2e-16 *** age.jour 0.118130 0.002172 116.300000 54.4 <2e-16 *** ---

Nesting structures Nested classification Cross-classification (or « non-nested » classification)

Including weighting for differing precisons in meta-analysis: lmer(y~x+(1|group),weights=SE2,…) This is the opposite of the « weights » function in lm(): weights an optional vector of ‘prior weights’ to be used in the fitting process. Should be NULL or a numeric vector. …. In particular, the diagonal of the residual covariance matrix is the squared residual standard deviation parameter sigma times the vector of inverse weights. Therefore, if the weights have relatively large magnitudes, then in order to compensate, the sigma parameter will also need to have a relatively large magnitude.

Exemple: Facultative adjustment of the offspring sex ratio and male attractiveness: a systematic review and meta-analysis. Biol. Rev. (2017), 92, pp. 108–134.