Introduction to Linear Mixed Effects Kiran Pedada PhD Student (Marketing) March 26, 2015

Correlated Data Forms of correlated data:  Time Series data  Repeated measurements  Longitudinal data  Spatial data Source:

Linear Mixed Effects Models Mixed model analysis provides a general, flexible approach in the situations of correlated data. Mixed model consists of two components:  Fixed effects – usually the conventional linear regression part  Random effects – associated with individual experimental units produced at random from the data generating process. Source:

Linear Mixed Effects Models The standard form of a linear mixed effects model: Y= β X+Zb+u Source: Fixed effect Random effect Error Y is the n x1 response vector, and n is the number of observations. X is an n x p fixed-effects design matrix. β is a p x 1 fixed-effects vector. Z is an n x q random-effects design matrix. b is a q x 1 random-effects vector. u is the n x 1 observation error vector.

Random Effect and Error Vectors Random effects vector, b, and the error vector, ε are assumed to be independent and distributed as follows : b ~ N (0, σ 2 D(θ)) ε ~ N (0, σ 2 I) Where D is a symmetric and positive semi definite matrix, parameterized by a variance component vector θ, I is an n x n identity matrix, and σ 2 is the error variance. Source:

Bodo Winter Example General Form of Linear Mixed Model: Y= β X+Zb+u Bodo Winter Fixed Effect Model: Pitch ~ politeness + sex + u Based on the general form of Linear Mixed Model, we can write the Bodo Winter Example as follows: Y= β 1 X 1 + β 2 X 2 + u Where, Y is the response variable, i.e., Pitch and X 1 and X 2 are the fixed effects, i.e., politeness and sex. β 1 and β 2 are fixed effect parameters. Source:

Mixed Effect Model If we add one or more random effects to the fixed effect model, then model will become a Mixed Effect Model. Let us add one random effect (for subject). Thus, the Mixed Effect Model will look like the following: Y= β 1 X 1 + β 2 X 2 + Zb + u ε Where, Z is the random effect, i.e., multiple responses per subject. And b is random effect parameter.

Matrix Notation of the Bodo Winter Mixed Model Y = X β + Z b + u 200 X X 2 2 X 1200 X X 1 X = β = 200 X 2 2 X 1 Source: Dr. Westfall Notes To make the example simple, let us consider 1 fixed effect and one random effect. Let us say, there are 40 female subjects with 5 repetitions on each subject. Half of the subjects are observed in formal case (1) and other half in informal case (0)

Matrix Notation of the Bodo Winter Mixed Model X β = 200 X 2 2 X 1 = 200 X 1 Source:Source: Dr. Westfall Notes x

Matrix Notation of the Bodo Winter Mixed Model Z = 200 X 40 Source: Dr. Westfall Notes

Matrix Notation of the Bodo Winter Mixed Model Z b = 40 X X 40 Source: Dr. Westfall Notes

Variance-covariance Matrix Y= β 1 X 1 + β 2 X 2 + Zb + u ε Cov( ε) = Cov (Zb + u) = Cov (Zb)+ Cov( u) = Z Cov (b) Z T + σ 2 I Source: Dr. Westfall Notes Cov( ε) = Z Cov (b) Z T + σ 2 I

Variance-covariance Matrix Z Cov (b) Z T + σ 2 I 200 X 200 Source: Dr. Westfall Notes b b b b b b b b b b

R simulation R Simulation