MEASUREMENT MODELS
Download
Report
Transcript MEASUREMENT MODELS
MEASUREMENT MODELS
BASIC EQUATION
• x=+e
• x = observed score
• = true (latent) score: represents
the score that would be obtained
over many independent
administrations of the same item
or test
• e = error: difference between y
and
ASSUMPTIONS
• and e are independent
(uncorrelated)
• The equation can hold for an
individual or a group at one
occasion or across occasions:
• xijk = ijk + eijk (individual)
• x*** = *** + e*** (group)
• combinations (individual across
time)
x
x
e
RELIABILITY
• Reliability is a proportion of
variance measure (squared
variable)
• Defined as the proportion of
observed score (x) variance due
to true score ( ) variance:
• 2x = xx’
•
= 2 / 2x
Var()
Var(e)
Var(x)
reliability
Reliability: parallel forms
• x 1 = + e 1 , x 2 = + e2
• (x1 ,x2 ) = reliability
•
= xx’
•
= correlation between
parallel forms
x1
e
x
x
xx’ = x * x
x2
e
ASSUMPTIONS
• and e are independent
(uncorrelated)
• The equation can hold for an
individual or a group at one
occasion or across occasions:
• xijk = ijk + eijk (individual)
• x*** = *** + e*** (group)
• combinations (individual across
time)
Reliability: Spearman-Brown
• Can show the reliability of the
composite is
• kk’ = [k xx’]/[1 + (k-1) xx’ ]
• k = # times test is lengthened
• example: test score has rel=.7
• doubling length produces
rel = 2(.7)/[1+.7] = .824
Reliability: parallel forms
• For 3 or more items xi, same
general form holds
• reliability of any pair is the
correlation between them
• Reliability of the composite (sum
of items) is based on the average
inter-item correlation: stepped-up
reliability, Spearman-Brown
formula
RELIABILITY
Generalizability
coefficients
d ANOVA
g - coefficients
Cronbach’s alpha
test-retest
internal consistency
inter-rater
parallel form
Hoyt
dichotomous
scoring
KR-20
KR-21
split half
Spearman
Brown
average
inter-item
correlation
COMPOSITES AND FACTOR
STRUCTURE
• 3 MANIFEST VARIABLES REQUIRED
FOR A UNIQUE IDENTIFICATION OF A
SINGLE FACTOR
• PARALLEL FORMS REQUIRES:
– EQUAL FACTOR LOADINGS
– EQUAL ERROR VARIANCES
– INDEPENDENCE OF ERRORS
e
e
x1
x
e
x3
x
x2
x
xx’ = x * x
i
j
RELIABILITY FROM SEM
• TRUE SCORE VARIANCE OF THE
COMPOSITE IS OBTAINABLE FROM
THE LOADINGS:
K
= 2i
i=1
K = # items or subtests
•
2
= K
x
Hancock’s Formula
• Hj = 1/ [ 1 + {1 / (Σ[l2ij/(1- l2ij )] ) }
• Ex. l1 = .7, l2= .8, l3 = .6
• H = 1 / [ 1 +1/( .49/.51 + .64/.36 + .36/.64 )]
= 1 / [ 1 + 1/ ( .98
+1.67 + .56 ) ]
= 1/ (1 + 1/3.21)
= .76
Hancock’s Formula Explained
Hj = 1/ [ 1 + {1 / (Σ[l2ij/(1- l2ij )] ) }
now assume strict parallelism: then l2ij= 2xt
thus Hj = 1/ [ 1 + {1 / (Σ[2xt /(1- 2xt)] ) }
= k 2xt / [1 + (k-1) 2xt ]
= Spearman-Brown formula
RELIABILITY FROM SEM
• RELIABILITY OF THE COMPOSITE IS
OBTAINABLE FROM THE LOADINGS:
= K/(K-1)[1 - 1/ ]
• example 2x =
.8, K=11
= 11/(10)[1 - 1/8.8 ]
= .975
SEM MODELING OF
PARALLEL FORMS
• PROC CALIS COV CORR MOD;
•
LINEQS
•
X1 = L1 F1 + E1,
•
X2 = L1 F1 + E1,
•
…
•
X10 = L1 F1 + E1;
•
STD E1=THE1, F1= 1.0;
TAU EQUIVALENCE
• ITEM TRUE SCORES DIFFER BY A
CONSTANT:
i = j + k
• ERROR STRUCTURE UNCHANGED AS
TO EQUAL VARIANCES,
INDEPENDENCE
TESTING TAU EQUIVALENCE
• ANOVA: TREAT AS A REPEATED
MEASURES SUBJECT X ITEM DESIGN:
• PROC VARCOMP;CLASS ID ITEM;
• MODEL SCORE = ID ITEM;
• LOW VARIANCE ESTIMATE CAN BE
TAKEN AS EVIDENCE FOR
PARALLELISM (UNLIKELY TO BE
EXACTLY ZERO
CONGENERIC MODEL
• LESS RESTRICTIVE THAN PARALLEL
FORMS OR TAU EQUIVALENCE:
– LOADINGS MAY DIFFER
– ERROR VARIANCES MAY DIFFER
• MOST COMPLEX COMPOSITES ARE
CONGENERIC:
– WAIS, WISC-III, K-ABC, MMPI, etc.
e2
e1
x1
x
x
1
e3
x
3
x2
2
x3
(x1 , x2 )= x * x
1
2
COEFFICIENT ALPHA
•
•
•
•
•
•
•
•
xx’ = 1 - 2E /2X
= 1 - [2i (1 - ii )]/2X ,
since errors are uncorrelated
= K/(K-1)[1 - (s2i )/ s2X ]
where X = xi (composite score)
s2i = variance of subtest xi
sX = variance of composite
Does not assume knowledge of subtest ii
COEFFICIENT ALPHANUNNALLY’S COEFFICIENT
• IF WE KNOW RELIABILITIES OF EACH
SUBTEST, i
• N = K/(K-1)[s2i (1- rii )/ s2X ]
• where rii = coefficient alpha of each subtest
• Willson (1996) showed
N
SEM MODELING OF
CONGENERIC FORMS
MPLUS EXAMPLE: this is an example of a CFA
DATA: FILE IS ex5.1.dat;
VARIABLE: NAMES ARE y1-y6;
MODEL:
f1 BY y1-y3;
f2 BY y4-y6;
OUTPUT:
SAMPSTAT MOD STAND;
NUNNALLY’S RELIABILITY CASE
e2
e1
x1
x
s1
x
1
2
s2
e3
x
3
x3
s3
x2
X X = 2x
i
i
i
+ s2 i
CORRELATED ERROR PROBLEMS
e2
e1
s
x1
x
x
1
e3
x
3
x3
s3
x2
2
Specificities can be
misinterpreted as a
correlated error model if
they are correlated or a
second factor
CORRELATED ERROR PROBLEMS
e1
x1
e2
x
x
1
e3
x
3
x3
s3
x2
2
Specificieties can
be misinterpreted as
a correlated error
model if
specificities are
correlated or are a
second factor
SEM MODELING OF
CONGENERIC FORMSCORRELATED ERRORS
MPLUS EXAMPLE: this is an example of a CFA
DATA: FILE IS ex5.1.dat;
VARIABLE: NAMES ARE y1-y6;
specifies
residuals of
MODEL:
f1 BY y1-y3;
previous model,
f2 BY y4-y6;
correlates them
y4 with y5;
OUTPUT:
SAMPSTAT MOD STAND;
MULTIFACTOR STRUCTURE
• Measurement Model: Does it hold for each
factor?
– PARALLEL VS. TAU-EQUIVALENT VS.
CONGENERIC
• How are factors related?
• What does reliability mean in the context of
multifactor structure?
SIMPLE STRUCTURE
• PSYCHOLOGICAL CONCEPT:
• Maximize loading of a manifest variable on
one factor ( IDEAL = 1.0 )
• Minimize loadings of the manifest variables
on all other factors ( IDEAL = 0 )
SIMPLE STRUCTURE
Example
SUBTEST
FACTOR1
FACTOR2
FACTOR3
A
1
0
0
B
1
0
0
C
0
1
0
D
0
1
0
E
0
0
1
F
0
0
1
MULTIFACTOR ANALYSIS
• Exploratory: determine number,
composition of factors from empirical
sampled data
– # factors # eigenvalues > 1.0 (using squared
multiple correlation of each item/subtest i with
the rest as a variance estimate for 2x
i
– empirical loadings determine structure
MULTIFACTOR ANALYSIS
TITLE: this is an example of an exploratory
factor analysis with continuous factor
indicators
DATA:
FILE IS ex4.1.dat;
VARIABLE:
NAMES ARE y1-y12;
ANALYSIS:
TYPE = EFA 1 4;
MULTIFACTOR MODEL WITH
THEORETICAL
PARAMETERS
MPLUS EXAMPLE: this is an example of a CFA
DATA: FILE IS ex5.1.dat;
VARIABLE: NAMES ARE y1-y6;
MODEL:
f1 BY [email protected] [email protected] [email protected];
f2 BY [email protected] [email protected] [email protected];
f1 with [email protected];
OUTPUT:
SAMPSTAT MOD STAND;
e1
MINIMAL CORRELATED FACTOR
STRUCTURE
x1
e2
x2
x
x
1 1
2 2
e3
x 1
1
2
3
x3
x
x4
4 2
12
e4
FACTOR RELIABILITY
• Reliability for Factor 1:
= 2(x11 * x31 ) / (1 + x11 * x31 )
(Spearman-Brown for Factor 1
reliability based on the average inter-item
correlation
• Reliability for Factor 2:
= 2(x22 * x42 ) / (1 + x22 * x42 )
FACTOR RELIABILITY
• Generalizes to any factors- reliability is
simply the measurement model reliability
for the scores for that factor
• This has not been well-discussed in the
literature
– problem has been exploratory analyses produce
successively smaller eigenvalues for factors due
to the extraction process
– second factor will in general be less reliable
using loadings to estimate interitem r’s
FACTOR RELIABILITY
• Theoretically, each factor’s reliability should be
independent of any other’s, regardless of the covariance
between factors
• Thus, the order of factor extraction should be independent
of factor structure and reliability, since it produces
maximum sample eigenvalues (and sample loadings) in an
extraction process.
• Composite is a misnomer in testing if the factors are
treated as independent constructs rather than subtests for a
more global composite score (separate scores rather than
one score created by summing subscale scores)
CONSTRAINED FACTOR MODELS
• If reliabilities for scales are known
independent of the current data (estimated
from items comprising scales, for example),
error variance can be constrained:
• s2e = s[1 - i ]
i
e1
sx1 [1- 1 ]1/2
CONSTRAINED SEM- KNOWN
RELIABILITY
x1
x
x
2
1
e3
sx3 [1-
3 ]1/2
x
3
x3
x2
e2
sx2 [1- 2 ]1/2
CONSTRAINED SEM-KNOWN
RELIABILITY
MPLUS EXAMPLE: this is an example of a CFA with
known error unreliabilities
DATA: FILE IS ex5.1.dat;
VARIABLE: NAMES ARE y1-y6;
MODEL:
f1 BY y1-y3;
similar statement
f2 BY y4-y6;
for each item
[email protected];
[email protected];
OUTPUT:
SAMPSTAT MOD STAND;
SEM Measurement Procedures
• 1. Evaluate the theoretical measurement model for
ALL factors (not single indicator variables
included)
• Demonstrate discriminant validity by showing the
factors are separate constructs
• Revise factors as needed to demonstrate- drop
some manifest variables if necessary and not
theoretically damaging
• Ref: Anderson & Gerbing (1988)