Transcript CFA-SAS - East Carolina University

Multilevel Linear Modeling
aka HLM
The Design
•
•
•
•
We have data at two different levels
In this case, 7,185 students (Level 1)
Nested within 160 Schools (Level 2)
We want to investigate effects at both
levels.
Get the Data
•
•
•
•
•
Download data file in XLS format
Print the Cheat-Sheet
Boot up SAS and import the data
name the imported member “HLM”
The first analysis will be have no
predictors, only means.
Level 1 Equation
Y ij   0 j  e ij
• The math achievement of the ith student at
the jth school =
• the intercept for the jth school (the mean at
that school)
• + error.
Note: We are using “0” instead of “a” to stand for
intercept.
Level 2 Equation
 0 j   00   0 j
• The intercept (mean match achievement)
at the jth school =
• The average intercept (mean) across
schools
• + (group j intercept) – (overall intercept)
– the effect of being in the jth school.
Combine the Two Equations
• Substitute (00 + 0j), from the Level 2
equation, for 0j in the Level 1 equation
Y ij   00   0 j  e ij
•
•
•
•
A student’s score =
The average intercept across schools
+ the effect of being at the jth school
+ other stuff
SAS
title 'Model 1: Unconditional Means
Model, Intercepts Only';
options formdlim='-' pageno=min
nodate;
proc mixed data = covtest
noclprint;
class School;
model MathAch = / solution;
random intercept / subject =
School;
run;
Fixed Effects
• Effects that are constant across schools.
• model MathAch =/
• No effect follows "=," the only parameter
estimated will be mean across schools.
Solution for Fixed Effects
Effect
Estimate Standard DF
Error
Intercept 12.6370 0.2443 159
t Value
Pr > |t|
51.72
<.0001
Random Effects
• Effects that vary across schools.
• random intercept / subject =
School;
Covariance Parameter Estimates
Cov Parm
Subject
Estimate
Standard
Error
Z Value
Pr > Z
Intercept
School
8.6097
1.0778
7.99
<.0001
39.1487
0.6607
59.26
<.0001
Residual
• There is significant variance across schools
(8.6097) in intercepts.
• And among students within schools (39.1487)
Intraclass Correlation
• the proportion of the variance in MathAch
that is due to differences among schools
• = (that due to schools) / (total variance)
• = 8.6097 / (8.6097 + 39.1487) = 18%.
Next, we are going to add a Level 2
predictor, MeanSES. Do note that this
variable has been centered to mean 0.
Add a Level 2 Predictor
 0 j   00   01MeanSES
j
 0 j
• The intercept (mean math achievement) at
the jth school =
• The average intercept across schools
when all predictors have value 0 (the
mean, since we centered MeanSES).
• + the effect of being in a school with the
MeanSES of school j
• + the effect of everything else on which j
differs from the other schools.
title 'Model 2: Including
Effects of School (Level 2)
Predictors';
title2 '-- predicting MathAch
from MeanSES'; run;
proc mixed covtest noclprint;
class school;
model MathAch = MeanSES /
solution ddfm = bw;
random intercept / subject =
school;
run;
• “bw” specifies between/within partitioning of df
Fixed Effects
• MeanSES was centered about zero. That is,
transformed to mean zero.
Solution for Fixed Effects
Effect
Estimate
Standard
Error
DF
t Value
Pr > |t|
Intercept
12.6495
0.1492
158
84.77
<.0001
MeanSES
5.8635
0.3613
158
16.23
<.0001
• Math Achievement = 12.6495 + 5.8635(School
MeanSES – GrandMean SES)
• Each 1 point increase in School’s mean SES
increases achievement by 5.86 points.
Random Effects
Covariance Parameter Estimates
Cov Parm Subject
Estimate
Intercept
Residual
School
2.6357
39.1578
Standard
Error
0.4036
0.6608
Z Value
Pr > Z
6.53
59.26
<.0001
<.0001
• Including school mean SES in model
reduced variance in intercepts from 8.6097
to 2.6357 = a drop of 5.974.
• School mean SES accounts for
5.974/8.6097 = 69% of the variance
among schools.
Unexplained Variance
• After accounting for SES, MathAch
intercepts (means) still differ significantly
across schools (z = 6.53)
• Residual variance = 2.6357 (among
schools) + 39.1578 (within schools) =
41.7935.
• 2.6357/41.7935 = 6.3% remains to be
explained by some other Level 2 predictor.
Use a Level 1 Predictor
Y ij   0 j   1 j SES
ij
 e ij
• Score for the ijth student
= the intercept for the jth school
+ the effect of this student’s SES
+ other things involving that student.
• Student SES will be centered by
subtracting from it the mean SES at the
student’s school
Level 2 Equations
• Have dropped the MeanSAS predictor
• Need a random intercept and a random
slope.
 0 j   00   0 j
• Intercept for School j =
Grand intercept
+ effect of being in School j
 1 j   10   1 j
• Slope (for relating student’s SES to
MathAch) at School j =
The grand slope
+ the effect (on slope) of being at School j
Combined Equation
Y ij  [ 00   10 ( SES
ij
 M c SES j )  e ij ]
ij
 M SES j )  e ij
• The fixed effects
 [  0 j   1 j ( SES
• The random effects
title 'Model 3: Including Effects of
Student-Level Predictors';
title2 '--predicting MathAch from
cSES';
data HLM2; set HLM; cSES = SES MeanSES; run;
proc mixed data = hsbc noclprint
covtest noitprint;
class School;
model MathAch = cSES / solution
ddfm = bw notest;
random intercept cSES / subject =
School type = un; run;
Fixed Effects
Solution for Fixed Effects
Effect
Estimate Standard DF
Error
Intercept 12.6493 0.2445
159
cSES
2.1932
0.1283
7024
t Value
Pr > |t|
51.75
17.10
<.0001
<.0001
• estimated MathAch for a student whose SES
is average for his or her school is 12.6493
• average slope, across schools, for predicting
MathAch from student SES is 2.1932, which is
significantly different from zero
Random Effects
Covariance Parameter Estimates
Cov Parm Subject Estimate Standard Z Value
Error
UN(1,1) School
8.6769
1.0786
8.04
Pr Z
<.0001
• The estimated variance in intercepts, across
schools, is a significant 8.6769, even after
controlling for student SES.
Covariance Parameter Estimates
Cov Parm Subject Estimate Standard Z Value
Error
UN(2,1) School
0.05075 0.4062
0.12
Pr Z
0.9006
• The correlation between School Intercept and
School Slope is a nonsignificant .051.
Covariance Parameter Estimates
Cov Parm Subject
Estimate Standard Z Value
Error
UN(2,2) School
0.6940
0.2808
2.47
Pr Z
0.0067
• The variance in slopes (for predicting
MathAch from student SES) is a significant
.694.
• The slopes differ significantly across
schools.
Covariance Parameter Estimates
Cov Parm Subject
Estimate Standard Z Value
Error
Residual
36.7006 0.6258
58.65
Pr Z
<.0001
• There remains significant within-school
variance, 36.7, after controlling for student SES.
• SES accounted for 39.1487-36.7006 – 2.4481
units of variance, or 2.4881/39.1487* = 6.25%
of the within-school variance.
*See Slide 9
Predictors at Both Levels
• Level 1: Student’s SES
• Level 2: School mean SES
• And a new Level 2 predictor, whether the
school is in the public sector (0) or is
Catholic (1).
title 'Model 4: Model with Predictors
From Both Levels and Interactions';
proc mixed noclprint covtest noitprint;
class School;
model mathach = MeanSES sector cSES
MeanSES*Sector MeanSES*cSES Sector*cSES
MeanSES*Sector*cSES /
solution ddfm = bw notest;
random intercept cSES /
subject = School type = un;
run;
Nonsignificant Fixed Effects
Solution for Fixed Effects
Effect
Estimate Standard
Error
MeanSES*Sector
-1.1276 0.7355
MeanSES*Sector*cSES -0.1888 0.5997
DF
t Value
Pr > |t|
156
7021
-1.53
-0.31
0.1273
0.7528
• Without further comment, I shall drop these
two nonsignificant interactions from the model
.
Our Reduced Model
Y ij   0 j   1 j cSES  e ij
• Level 1
• Level 2, intercepts
 0 j   00   01 MeanSES
j
  02 Sector
j
 0 j
• Level 2, slopes
 1 j   10   11 MeanSES
j
  12 Sector
j
 1j
Combined Equation
• Fixed Effects
Y ij  [ 00   01MeanSES
j
  10 cSES   11MeanSES
  0 j   1 j cSES
j
  02 Sector
j
cSES
j
j
  12 Sector j cSES j ]
 e ij
– Grand intercept
– Overall slope for School SES (Predictor 1)
– Overall slope School Sector (Predictor 2)
– Overall slope for Student SES
– Interaction between School SES and Student SES
– Interaction between School Sector and Student
SES
• Random Effects
[  0 j   1 j cSES
j
 e ij ]
• Effect on intercept of being at School j
• Effect on slope being at School j
• Everything else affecting Student i at
School j
title 'Model 5: Model with Two Interactions
Deleted';
title2 '--predicting mathach from meanses,
sector, cses and ';
title3 'cross level interaction of meanses and
sector with cses'; run;
proc mixed noclprint covtest noitprint;
class School;
model MathAch = MeanSES Sector cSES
MeanSES*cSES Sector*cSES /
solution ddfm = bw notest;
random intercept cSES /
subject = School type = un;
proc means mean q1 q3 min max skewness
kurtosis; var MeanSES Sector cSES; run;
Fixed Effects
Solution for Fixed Effects
Effect
Estimate
Intercept
MeanSES
Sector
cSES
MeanSES*cSES
12.1136
5.3391
1.2167
2.9388
1.0389
Standar
d Error
0.1988
0.3693
0.3064
0.1551
0.2989
DF
t Value Pr > |t|
157
157
157
7022
7022
60.93
14.46
3.97
18.95
3.48
<.0001
<.0001
0.0001
<.0001
0.0005
Sector*cSES
-1.6426
0.2398 7022
-6.85
<.0001
Interpret New Effects
• Sector: Math achievement is higher at
Catholic Schools
• MeanSES x cSES: the slopes for
predicting MathAch from cSES differ
across levels of MeanSES.
• Sector x cSES: the slopes for predicting
MathAch from cSES differ between public
and Catholic schools
Sector x cSES
• In the combined equation, substitute 0 for
value of sector to get equation for public
schools
• And 1 to get equation for Catholic schools
• Public: 12.11 + 5.34(MeanSES) +
2.94(cSES) + 1.04(MeanSES)(cSES)
• Catholic: 13.33 + 5.34(MeanSES) +
1.30(cSES) + 1.04(MeanSES)(cSES)
• Intercept higher for Catholic than for public
– MathAch higher at Catholic schools.
• Slope for student SES higher at public
schools than at Catholic schools.
MeanSES x cSES
• Find Q1, Q2, and Q3 for School SES
• Substitute the quartile values into the
combined equation to get one equation for
each quartile.
• For each of two values (-3, +3) of cSES,
predict MathAch at each value with each
equation.
• Prepare table and plot of predicted values.
MeanSES
Q1
Q2
Q3
cSES
-3
5.57
6.23
7.06
+3
16.43
19.25
21.88
Diffrnc
10.86
13.02
14.82
• Notice that the slope increases as
MeanSES increases
Random Effects
Covariance Parameter Estimates
Cov Parm Subject
Estimate Standard Z Value
Error
Pr Z
UN(1,1)
School
2.3817
0.3717
6.41
<.0001
UN(2,1)
School
0.1926
0.2045
0.94
0.3464
UN(2,2)
School
0.1014
0.2138
0.47
0.3177
36.7212
0.6261
58.65
<.0001
Residual
• UN(1,1): The intercepts still differ significantly
across schools.
• UN(2,1): No significant correlation
between intercepts and slopes.
• UN(2,2): The slopes (predicting MathAch
from cSES) do not differ significantly
across schools.
• I shall drop cSES from the random effects.
Trimmed Model
title 'Model 6: Simpler Model
Without cSES Slopes';
proc mixed noclprint covtest
noitprint;
class School;
model MathAch = MeanSES Sector
cSES MeanSES*cSES Sector*cSES /
solution ddfm = bw notest;
random intercept /
subject = School; run;
Effects of Trimming
• All of the fixed effects are still significant.
• Intercepts still differ significantly across
schools.
• The Log Likelihood statistic has increased
from 46503.7 to 46504.8, indicating
slightly poorer fit.
• We can evaluate the difference in Log
Likelihood statistics via Chi-square on 2 df.
p = .58.