From t-test to multilevel analyses Del-2 Stein Atle Lie, statistician, professor Uni Health, Uni Research http://folk.uib.no/msesl/GLM

Outline Pared t-test (Mean and standard deviation) Two-group t-test (Mean and standard deviations) Linear regression GLM (general linear models) GLMM (general linear mixed model) … PASW (former SPSS), Stata, R, gllamm (Stata)

Pared t-test The straightforward way to analyze two repeated measures is a pared t-test. Measure at time1 or location1 (e.g. Data1) is directly compared to measure at time2 or location2 (e.g. Data2) Is the difference between Data1 and Data2 (Diff = Data1-Data2) unlike 0?

Pared t-test The pared t-test will only be performed for complete (balanced) data. What happens if we delete two observations from data2? (Only 8 complete pairs remain)

Two group t-test If we now consider the data from time1 and time2 (or location1 and location2) to be independent (even if their not) and use a two group t-test on the full dataset, 2*10 observations

ID Grp (or time) Dummy Data 1007 1 6,1 1008 4,6 1009 5 1010 4,8 1011 7,9 1012 13,8 1013 1,4 1014 9,5 1015 12,2 1016 8,5 2 8,3 9,8 15,6 9,2 10,4 3,3 8,4 16,1

Two group t-test (n=20 [10+10]) PASW: T-TEST GROUPS=Grp(1 2) /VARIABLES=Data.

Two group t-test Observe that mean for Grp1 and Grp2 is equal to mean for Data1 and Data2 (from the pared t-test) And that the mean difference is also equal! The difference between pared t-test and two group t-test lies in the Variance - and the number of observations and therefore in the standard deviation and standard error and hence in the p-value and confidence intervals

Two group t-test (s1=s2) s1 s2 m1 m2 D

Two group t-test (s1=s2) s1 s2

ANOVA (Analysis of variance (s1=s2=s3) m1 m2 m3

ANOVA (Analysis of variance (s1=s2=s3)

Linear regression If we now perform an ordinary linear regression with the data as outcome (dependent variable) and the group (time) variable (Grp=1 and 2) as independent variable

Linear regression (n=20) Stata: . regress data grp Source | SS df MS Number of obs = 20 -------------+------------------------------ F( 1, 18) = 1.38 Model | 21.0124998 1 21.0124998 Prob > F = 0.2554 Residual | 274.01701 18 15.2231672 R-squared = 0.0712 -------------+------------------------------ Adj R-squared = 0.0196 Total | 295.02951 19 15.5278689 Root MSE = 3.9017 ------------------------------------------------------------------------------ data | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- grp | 2.05 1.744888 1.17 0.255 -1.615873 5.715873 _cons | 5.33 2.75891 1.93 0.069 -.4662545 11.12625

Linear regression The coefficient for group is identical to the mean difference and the standard error, t-statistic, and p‑value are identical to those found in a two‑group t‑test

Linear regression Now exchange the independent variable for group (Grp=1 and 2) with a dummy variable (dummy=0 for grp=1 and dummy=1 for grp=2) the coefficient for the dummy is equal to the coefficient for grp (the mean difference) and the coefficient for the constant term is equal to the mean for grp1 (the standard error is not!)

Linear regression (n=20) Stata: . regress data dummy Source | SS df MS Number of obs = 20 -------------+------------------------------ F( 1, 18) = 1.38 Model | 21.0124998 1 21.0124998 Prob > F = 0.2554 Residual | 274.01701 18 15.2231672 R-squared = 0.0712 -------------+------------------------------ Adj R-squared = 0.0196 Total | 295.02951 19 15.5278689 Root MSE = 3.9017 ------------------------------------------------------------------------------ data | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- dummy | 2.05 1.744888 1.17 0.255 -1.615873 5.715873 _cons | 7.38 1.233822 5.98 0.000 4.787836 9.972164

Linear models in Stata In ordinary linear models (regress and glm) in Stata one may add an option for clustered data – to obtain robust standard errors adjusted for intragroup correlation. (This is ideal when you want to adjust for clustered data, but are not interested in the correlation within or between groups)

Linear regression (n=20) Stata: . regress data dummy, cluster(id) Linear regression Number of obs = 20 F( 1, 9) = 2.64 Prob > F = 0.1388 R-squared = 0.0712 Root MSE = 3.9017 (Std. Err. adjusted for 10 clusters in id) ------------------------------------------------------------------------------ | Robust data | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- dummy | 2.05 1.262145 1.62 0.139 -.8051699 4.90517 _cons | 7.38 1.224847 6.03 0.000 4.609204 10.1508

Linear models in Stata Thus, we now have an alternative to the pared t‑test. The mean difference is identical to that obtained from the pared t‑test, and the standard errors (and p-values) are adjusted for intragroup correlation As an alternative we may use the program gllamm (Generalized Linear Latent And Mixed Models) in Stata http://www.gllamm.org/

gllamm (n=20) gllamm (Stata): . gllamm data dummy, i(id) number of level 1 units = 20 number of level 2 units = 10 ------------------------------------------------------------------------------ data | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- dummy | 2.05 1.167852 1.76 0.079 -.2389486 4.338949 _cons | 7.379808 1.172819 6.29 0.000 5.081124 9.678492 Variance at level 1 6.8193955 (3.0174853) level 2 (id) var(1): 6.8114516 (4.5613185)

Linear models in Stata If we now delete two of the observations in Grp2 We then have coefficients (“mean differences”) calculated based on all (n=18) data and standard errors corrected for intragroup correlation - using the commands <regress>, <glm> or <gllamm>

Linear regression (n=18) Stata: . regress data dummy, cluster(id) Linear regression Number of obs = 18 F( 1, 9) = 1.63 Prob > F = 0.2332 R-squared = 0.0587 Root MSE = 4.1303 (Std. Err. adjusted for 10 clusters in id) ------------------------------------------------------------------------------ | Robust data | Coef. Std. Err. t P>|t| [95% Conf. Interval] -------------+---------------------------------------------------------------- dummy | 1.9575 1.531486 1.28 0.233 -1.506963 5.421963 _cons | 7.38 1.228869 6.01 0.000 4.600105 10.1599

gllamm (n=18) gllamm (Stata): . gllamm data dummy, i(id) number of level 1 units = 18 number of level 2 units = 10 log likelihood = -48.538837 ------------------------------------------------------------------------------ data | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- dummy | 2.458305 1.253552 1.96 0.050 .0013882 4.915223 _cons | 7.357426 1.232548 5.97 0.000 4.941677 9.773176 Variance at level 1 6.4041537 (3.3485133) level 2 (id) var(1): 8.7561818 (5.1671805)

Intra class correlation (ICC) Variance at level 1 6.4041537 (3.3485133) level 2 (id) var(1): 8.7561818 (5.1671805) The total variance is hence 6.4041 + 8.7561= 15.1603 (and the standard deviation is hence 3.8936) The proportion of variance attributed to level 2 is therefore ICC = 8.7561/15.1603 = 0.578

Linear regression Ordinary linear regression Assumes data is Normal and i.i.d. (identical independent distributed)

Linear regression Y X Regression line: y = b0 + b1·x b1 b0 residual b1 (x1,y1) (xn,yn) (xi,yi) b0 Kortisol * Months Height * Weight Kortisol * Time X

Linear regression Assumptions: 1) y1, y2,…, yn are independent normal distributed 2) The expectation of Yi is: E(Yi) = b0 + b1·xi (linear relation between X and Y) 3) The variance of Yi is: var(Yi) = s2 (equal variance for ALL values of X)

Linear regression Assumptions - Residualer (ei): yi = a + b·xi + ei 1) e1, e2,…, en are independent normal distributed 2) The expectation of ei is: E(ei) = 0 3) The variance of ei is: var(Yi) = s2

Ordinary linear regression The formula for an ordinary regression can thus be expressed as: yi = b0 + b1·xi + ei ei ~N(0, se2)

Random intercept model Y Regression lines: yij = b0 + b1·xij+vij (x11,y11) b1 (xnp,ynp) b0+uj (xij,yij) su se X

Random intercept model For a random intercept model, we can express the regression line(s) - and the variance components as yij = b0 + b1·xij + vij vij = uj + eij eij ~N(0, se2) (individual) uj ~N(0, su2) (group)

Random intercept model Alternatively we may express the formulas, for the simple variance component model, in terms of random intercepts: yij = b0j + b1·xij + eij b0j = b0 + uj eij ~N(0, se2) (individual) uj ~N(0, su2) (group)