Introduction to Linear Mixed Effects

Kiran Pedada

PhD Student (Marketing) March 26, 2015

Correlated Data Forms of correlated data:     Time Series data Repeated measurements Longitudinal data Spatial data Source: http://www.stat.missouri.edu/~spinkac/stat8320/LinearMixedModels.pdf

Linear Mixed Effects Models • Mixed model analysis provides a general, flexible approach in the situations of correlated data.

Mixed model consists of two components:   Fixed effects – usually the conventional linear regression part Random effects – associated with individual experimental units produced at random from the data generating process. Source: http://www.stat.cmu.edu/~hseltman/309/Book/chapter15.pdf

http://www.mathworks.com/help/stats/linear-mixed-effects-models.html

Linear Mixed Effects Models The standard form of a linear mixed effects model: Y= β X+Zb+u Fixed effect Error Random effect

Y

is the

n

x1 response vector, and

n

is the number of observations.

X β

is an is a

n x p

fixed-effects design matrix.

p x

1 fixed-effects vector.

Z

is an

n

x

q

random-effects design matrix.

b u

is a

q

is the x 1 random-effects vector.

n

x 1 observation error vector.

Source: http://www.mathworks.com/help/stats/linear-mixed-effects-models.html

Random Effect and Error Vectors Random effects vector, b, and the error vector, ε independent and distributed as follows : are assumed to be b ~ N (0, σ 2 D(θ)) ε ~ N (0, σ 2 I) Where D is a symmetric and positive semi definite matrix, parameterized by a variance component vector θ , I is an n x n identity matrix, and σ

2

is the error variance.

Source: http://www.mathworks.com/help/stats/linear-mixed-effects-models.html

Bodo Winter Example General Form of Linear Mixed Model: Y= β X+ Zb +u Bodo Winter Fixed Effect Model: Pitch ~ politeness + sex + u Based on the general form of Linear Mixed Model, we can write the Bodo Winter Example as follows: Y= β 1 X 1 + β 2 X 2 + u Where, Y is the response variable, i.e., Pitch and X effects, i.e., politeness and sex. β 1 and β 2 1 and X 2 are the fixed are fixed effect parameters.

Source: http://www.bodowinter.com/tutorial/bw_LME_tutorial.pdf

Mixed Effect Model If we add one or more random effects to the fixed effect model, then model will become a Mixed Effect Model.

Let us add one random effect (for subject).

Thus, the Mixed Effect Model will look like the following: Y= β 1 X 1 + β 2 X 2 + Zb + u ε Where, Z is the random effect, i.e., multiple responses per subject. And b is random effect parameter.

Matrix Notation of the Bodo Winter Mixed Model To make the example simple, let us consider 1 fixed effect and one random effect. Let us say, there are 40 female subjects with 5 repetitions on each subject. Half of the subjects are observed in formal case (1) and other half in informal case (0). 200 X 1 200 X 1 200 X 1 200 X 1 Y = X β + Z b + u 200 X 2 2 X 1 200 X 40 40 X 1 X = 200 X 2 1 6 200 Source: Dr. Westfall Notes β = 2 X 1

Matrix Notation of the Bodo Winter Mixed Model X β = 200 X 2 2 X 1 x = 200 X 1

Matrix Notation of the Bodo Winter Mixed Model Z = 200 X 40 1 5 6 200 Source: Dr. Westfall Notes

Matrix Notation of the Bodo Winter Mixed Model Z b = 200 X 40 40 X 1 Source: Dr. Westfall Notes

Variance-covariance Matrix Y= β 1 X 1 + β 2 X 2 + Zb + u ε Cov( ε) = Cov (Zb + u) = Cov (Zb)+ Cov( u) = Z Cov (b) Z T + σ 2 I

Cov(

ε)

= Z Cov (b) Z T +

σ 2 I Source: Dr. Westfall Notes

Variance-covariance Matrix b b b Z Cov (b) Z T + σ 2 I b b b b b b b 200 X 200 Source: Dr. Westfall Notes

R simulation R Simulation