Transcript Bridging the gap from LogR to IRT
Bridging the gap from LogR to IRT
Indebted to: Wu, A. D., & Zumbo, B.D. (2007). Thinking About Item Response Theory from a Logistic Regression Perspective: A Focus on Polytomous Models. In Shlomo S. Sawilowsky (Ed.),
Real Data Analysis
(pp. 241-269). Information Age Publishing, Inc.., Greenwich, CT..
Bridging the gap from LogR to IRT • The explanatory variable
– In IRT, the exposure is a cts latent variable – Hence IRT = generalized linear latent model
• The outcome variable(s)
– Logistic regression typically models ONE outcome, whereas IRT models a number of categorical outcomes simultaneously
Aim of IRT
• To relate a subjects’ responses to a number of test items, to an underlying ability (AKA trait) by way of a mathematical function • Due to the non-linear relationship, a logistic curve is often used, and is referred to as the Item Characteristic Curve or Item Response Function
Increasing prob of correct response
ICC / IRF
Increasing level of latent trait
Options for form of ICC Examples • Step function (Guttman) • 2 parameter normal ogive (Lord) • 2 parameter logistic (Birnbaum) • 1 parameter logistic (Rasch) • Nonparametric, monotone increasing (Mokken)
Back to the logistic form
• Two parameter binary logistic IRT model
P
(
u i
1 | ,
i
,
i
) 1 exp( exp(
i
(
i
(
i
))
i
)) •
θ
: ability level • • α i : the slope (AKA discrimination) for item i • β i : the threshold (AKA difficulty) for item I
(θ –
β i ): discrepancy between item & ability of respondent
For a single item:
P
(
u
1 | , , ) exp( 1 exp( ( ( )) ))
Let X = (θ – β
i ) & add intercept c
P
(
u
1 |
X
) exp(
c
1
X
exp(
c
)
X
)
Slope
Recall:
-6 -4 -2 bwt_z 0 2 4 -6 -4 -2 bwt_z 0 2 Value of covariate (ability) at point of inflection 4
So • For a uni-dimensional IRT model (a single trait
θ)
t he 2PL IRT model is a simple LogR model • In the binary IRT setting we simultaneously model a number of items • The parameters for each item may/may-not vary across items
Conditional Independence
Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Before Trait Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 After
The Rasch model A worked example across multiple packages
Abortion data
Analysis of Multivariate Social Science Data, Second Edition David J. Bartholomew Fiona Steele Irini Moustaki Jane Galbraith Dataset actually comes from the first edition so hope it’s still in the second!!!
Idea • Same Rasch model 4 ways
– R (LTM) – Mplus – Raschtest – GLLAMM (via long format data-prepping)
Table 7.1 – attitude towards abortion
Abortion should be permitted if: 1] The woman decides on her own that she does not wish to have the child 2] The couple agree that they do not wish to have the child 3] The woman is not married and does not wish to marry the man 4] The couple cannot afford any more children Response patterns 1111 0000 0111 0011 0001 1110 0010 0100 0110 1011 0101 1101 1100 1000 1010 1001 Total 3 3 1 0 0 379 10 9 7 6 6 Freq 141 103 44 21 13 12
Basic output
SUMMARY OF CATEGORICAL DATA PROPORTIONS WOMAN Category 1 0.562
Category 2 0.438
COUPLE Category 1 0.406
Category 2 0.594
NOT_MARR Category 1 0.364
Category 2 0.636
AFFORD Category 1 0.383
Category 2 0.617
100.00% 80.00% 60.00% 40.00% 20.00% 0.00% WOMAN COUPLE NOT_MARR AFFORD
Abortion: [R]
rasch(data = abortion[, c(2, 3, 4, 5)], IRT.param = FALSE) > summary(rasch1) Model Summary: log.Lik AIC BIC -657.7894 1325.579 1345.078
Coefficients: value std.err z.vals
woman -0.7843 0.2762 -2.8395
coupl 1.1288 0.2724 4.1437
nt.mr 1.7950 0.2969 6.0453
affrd 1.5129 0.2870 5.2716
z 4.9064 0.4264 11.5057
Integration: method: Gauss-Hermite quadrature points: 21 Optimization: Convergence: 0 max(|grad|): 0.00097 quasi-Newton: BFGS
plot(rasch1, type = c("ICC"))
par(mfrow = c(2, 2)) plot(rasch1, items = c(1), type = c("IIC"), ylim=c(0,7)) plot(rasch1, items = c(2), type = c("IIC"), ylim=c(0,7)) plot(rasch1, items = c(3), type = c("IIC"), ylim=c(0,7)) plot(rasch1, items = c(4), type = c("IIC"), ylim=c(0,7))
margins(rasch2, "two")
Response: (0,0) Item i Item j Obs Exp (O-E)^2/E 1 2 4 111 119.38 0.59 2 1 4 125 133.09 0.49 3 1 2 143 140.62 0.04 Response: (1,0) Item i Item j Obs Exp (O-E)^2/E 1 1 4 13 7.28 4.50 *** 2 1 2 5 9.57 2.18 3 2 4 27 20.99 1.72 Response: (0,1) Item i Item j Obs Exp (O-E)^2/E 1 2 4 37 30.81 1.24 2 1 4 80 72.37 0.80 3 3 4 19 21.27 0.24 Response: (1,1) Item i Item j Obs Exp (O-E)^2/E 1 1 4 147 152.26 0.18 2 1 2 155 149.97 0.17 3 3 4 208 203.36 0.11
margins(rasch2, "three")
Response: (0,0,0) Item i Item j Item k Obs Exp (O-E)^2/E 1 1 2 4 111 117.01 0.31 2 2 3 4 102 104.53 0.06 3 1 3 4 110 110.44 0.00 Response: (1,0,0) Item i Item j Item k Obs Exp (O-E)^2/E 1 1 2 4 0 2.37 2.37 2 1 2 3 0 2.04 2.04 3 2 3 4 10 7.63 0.74 Response: (0,1,0) Item i Item j Item k Obs Exp (O-E)^2/E 1 1 3 4 15 22.65 2.58 2 2 3 4 9 14.85 2.30 3 1 2 4 14 16.08 0.27 Response: (1,1,0) Item i Item j Item k Obs Exp (O-E)^2/E 1 1 2 4 13 4.90 13.38 *** 2 1 3 4 11 5.56 5.33 *** 3 2 3 4 17 13.36 0.99 Response: (0,0,1) Item i Item j Item k Obs Exp (O-E)^2/E 1 1 2 4 32 23.61 2.98 2 1 2 3 29 26.93 0.16 3 2 3 4 12 11.20 0.06 Response: (1,0,1) Item i Item j Item k Obs Exp (O-E)^2/E 1 1 3 4 2 4.19 1.15 2 2 3 4 7 10.07 0.94 3 1 2 3 5 7.53 0.85 Response: (0,1,1) Item i Item j Item k Obs Exp (O-E)^2/E 1 2 3 4 25 19.61 1.48 2 1 3 4 63 55.29 1.08 3 1 2 3 49 51.00 0.08 Response: (1,1,1) Item i Item j Item k Obs Exp (O-E)^2/E 1 1 2 3 151 146.10 0.16 2 1 2 4 142 145.07 0.06 3 1 3 4 145 148.07 0.06 '***' denotes a chi-squared residual greater than 3.5
Abortion: [Mplus]
Read the data into Mplus
data: file is "abortion_attitude.txt"; variable: names are woman couple not_marr afford num; categorical are woman couple not_marr afford; usevariables are woman couple not_marr afford; freqweight = num; analysis: type = basic;
Basic output – sample stats
FIRST ORDER SAMPLE PROPORTIONS : WOMAN COUPLE NOT_MARR AFFORD ________ ________ ________ ________ 1 0.438 0.594 0.636 0.617
SECOND ORDER SAMPLE PROPORTIONS WOMAN COUPLE NOT_MARR AFFORD ________ ________ ________ ________ WOMAN COUPLE 0.420
NOT_MARR 0.420 0.538
AFFORD 0.396 0.512 0.559
SAMPLE THRESHOLDS WOMAN$1 COUPLE$1 NOT_MARR AFFORD$1 ________ ________ ________ ________ 1 0.156 -0.237 -0.347 -0.299
Basic output – sample stats
SAMPLE TETRACHORIC CORRELATIONS WOMAN COUPLE NOT_MARR AFFORD ________ ________ ________ ________ WOMAN COUPLE 0.902
NOT_MARR 0.866 0.882
AFFORD 0.768 0.821 0.903
STANDARD DEVIATIONS FOR SAMPLE TETRACHORIC CORRELATIONS WOMAN COUPLE NOT_MARR AFFORD ________ ________ ________ ________ WOMAN COUPLE 0.125
NOT_MARR 0.158 0.137
AFFORD 0.217 0.181 0.120
Rasch model in Mplus
data: file is “...abortion_attitude.txt"; variable: names are woman couple not_marr afford num; usevariables are woman couple not_marr afford; categorical are woman couple not_marr afford; freqweight = num; analysis: ESTIMATOR = MLR; model: F by woman* (1) couple (1) not_marr (1) afford (1); F@1; plot: type = plot3;
Mplus results
TESTS OF MODEL FIT Loglikelihood H0 Value -709.937
H0 Scaling Correction Factor 1.009
for MLR Information Criteria Number of Free Parameters 5 Akaike (AIC) 1429.874
Bayesian (BIC) 1449.562
Sample-Size Adjusted BIC 1433.698
(n* = (n + 2) / 24) Chi-Square Test of Model Fit for the Binary and Ordered Categorical (Ordinal) Outcomes Pearson Chi-Square Value 22.788
Degrees of Freedom 10 P-Value 0.0116
Likelihood Ratio Chi-Square Value 22.595
Degrees of Freedom 10 P-Value 0.0123
Mplus results
Two-Tailed Estimate S.E. Est./S.E. P-Value F BY WOMAN 4.336 0.390 11.124 0.000
COUPLE 4.336 0.390 11.124 0.000
NOT_MARR 4.336 0.390 11.124 0.000
AFFORD 4.336 0.390 11.124 0.000
Thresholds WOMAN$1 0.776 0.311 2.496 0.013
COUPLE$1 -1.047 0.306 -3.417 0.001
NOT_MARR$1 -1.573 0.315 -4.994 0.000
AFFORD$1 -1.339 0.322 -4.161 0.000
Variance of F 1.000 0.000 999.000 999.000
IRT PARAMETERIZATION IN TWO-PARAMETER LOGISTIC METRIC WHERE THE LOGIT IS 1.7*DISCRIMINATION*(THETA - DIFFICULTY) Item Discriminations F BY WOMAN 2.551 0.229 11.124 0.000
COUPLE 2.551 0.229 11.124 0.000
NOT_MARR 2.551 0.229 11.124 0.000
AFFORD 2.551 0.229 11.124 0.000
Item Difficulties WOMAN$1 0.179 0.071 2.514 0.012
COUPLE$1 -0.241 0.072 -3.353 0.001
NOT_MARR$1 -0.363 0.074 -4.913 0.000
AFFORD$1 -0.309 0.074 -4.199 0.000
Variance of F 1.000 0.000 999.000 999.000
ICC’s
Abortion: [Stata] –Raschtest–
raschtest woman couple not_marr afford, meandiff icc
Estimation method: Conditional maximum likelihood ( CML ) Number of items: 4 Number of groups: 5 (3 of them are used to compute the statistics of test) Number of individuals: 365 (0 individuals removed for missing values) Number of individuals with null or perfect score: 242 Conditional log-likelihood: -131.2562 Log-likelihood: -320.5403 Difficulty Standardized Items parameters Std. Err. R1c df p-value Outfit Infit U ---------------------------------------------------------------------------- woman 1.64747 0.19064 1.940 2 0.3790 -1.232 -0.422 -1.411
couple -0.19486 0.16979 2.342 2 0.3100 -0.574 -0.313 -0.838
not_marr -0.87046 0.18302 1.580 2 0.4538 -1.272 -1.467 -0.854
afford -0.58216 0.17588 3.937 2 0.1397 2.336 2.113 3.015
---------------------------------------------------------------------------- R1c test R1c= 15.343 6 0.0177
Andersen LR test Z= 14.594 6 0.0237
---------------------------------------------------------------------------- The mean of the difficulty parameters is fixed to 0 You have groups of scores with less than 30 individuals. The tests can be invalid.
Ability Expected Group Score parameters Std. Err. Freq. Score ll ------------------------------------------------------------- 0 0 -2.560 2.860 102 0.38
------------------------------------------------------------- 1 1 -1.114 0.814 29 1.15 -31.4221
------------------------------------------------------------- 2 2 -0.109 0.664 33 1.97 -39.3159
------------------------------------------------------------- 3 3 1.054 0.984 61 2.84 -53.2214
------------------------------------------------------------- 4 4 2.833 3.626 140 3.66
--------------------------------------------------------------
Observed and Expected ICC for the item woman Observed and Expected ICC for the item couple -2 -1 0 Latent trait Observed ICC 1 Expected ICC Observed and Expected ICC for the item not_marr 2 -2 -1 0 Latent trait Observed ICC 1 Expected ICC Observed and Expected ICC for the item afford 2 -2 -1 0 Latent trait Observed ICC 1 Expected ICC 2 -2 -1 0 Latent trait Observed ICC 1 Expected ICC 2
raschtest woman couple not_marr afford, method(mml)
Estimation method: Marginal maximum likelihood ( MML ) Number of items: 4 Number of groups: 5 (5 of them are used to compute the statistics of test) Number of individuals: 365 (0 individuals removed for missing values) Number of individuals with null or perfect score: 242 Marginal log-likelihood: -665.8056 Log-likelihood: -281.5298 Difficulty Standardized Items parameters Std. Err. R1m df p-value Outfit Infit --------------------------------------------------------------------- woman 1.25298 0.26213 4.606 2 0.0999 -2.624 0.164
couple -0.66034 0.30265 18.408 2 0.0001 . -3.567
not_marr -1.27117 0.29512 11.668 2 0.0029 . -0.046
afford -1.02314 0.29784 26.037 2 0.0000 . -0.611
--------------------------------------------------------------------- R1m test R1m= 31.056 8 0.0001
--------------------------------------------------------------------- Sigma 4.12109 0.28776
--------------------------------------------------------------------- You have groups of scores with less than 30 individuals. The tests can be invalid.
Ability Expected Group Score parameters Std. Err. Freq. Score -------------------------------------------------- 0 0 -4.33624 1.60486 102 0.11
-------------------------------------------------- 1 1 -2.24208 0.68050 29 0.70
-------------------------------------------------- 2 2 -1.29596 1.34362 33 1.34
-------------------------------------------------- 3 3 2.05542 0.97822 61 3.55
-------------------------------------------------- 4 4 3.91275 1.56707 140 3.91
---------------------------------------------------
Abortion: [Stata] –GLLAMM–
Rasch model – data prepping
Difficulty
The data
+---------------------------------------+ | woman couple not_marr afford num | |---------------------------------------| 1. | 1 1 1 1 141 | 2. | 0 0 0 0 103 | 3. | 0 1 1 1 44 | 4. | 0 0 1 1 21 | 5. | 0 0 0 1 13 | 6. | 1 1 1 0 12 | 7. | 0 0 1 0 10 | 8. | 0 1 0 0 9 | |---------------------------------------| 9. | 0 1 1 0 7 | 10. | 1 0 1 1 6 | 11. | 0 1 0 1 6 | 12. | 1 1 0 1 3 | 13. | 1 1 0 0 3 | 14. | 1 0 0 0 1 | 15. | 1 0 1 0 0 | 16. | 1 0 0 1 0 | +---------------------------------------+
The data
+---------------------------------------+ | woman couple not_marr afford num | |---------------------------------------| 1. | 1 1 1 1 141 | 2. | 0 0 0 0 103 | 3. | 0 1 1 1 44 | 4. | 0 0 1 1 21 | 5. | 0 0 0 1 13 | 6. | 1 1 1 0 12 | 7. | 0 0 1 0 10 | 8. | 0 1 0 0 9 | |---------------------------------------| 9. | 0 1 1 0 7 | 10. | 1 0 1 1 6 | 11. | 0 1 0 1 6 | 12. | 1 1 0 1 3 | 13. | 1 1 0 0 3 | 14. | 1 0 0 0 1 | 15. | 1 0 1 0 0 | 16. | 1 0 0 1 0 | +---------------------------------------+ +---------------------+ |
i1 i2 i3 i4
num | |---------------------| 1. | 1 1 1 1 141 | 2. | 0 0 0 0 103 | 3. | 0 1 1 1 44 | 4. | 0 0 1 1 21 | 5. | 0 0 0 1 13 | 6. | 1 1 1 0 12 | 7. | 0 0 1 0 10 | 8. | 0 1 0 0 9 | |---------------------| 9. | 0 1 1 0 7 | 10. | 1 0 1 1 6 | 11. | 0 1 0 1 6 | 12. | 1 1 0 1 3 | 13. | 1 1 0 0 3 | 14. | 1 0 0 0 1 | 15. | 1 0 1 0 0 | 16. | 1 0 0 1 0 | +---------------------+
Reshape
+-----------------------------------+ | i1 i2 i3 i4 num pattern | |-----------------------------------| | 1 1 1 1 141 1 | | 0 0 0 0 103 2 | | 0 1 1 1 44 3 | | 0 0 1 1 21 4 | |-----------------------------------| gen pattern = _n reshape long i, i(pattern) j(item) rename i score +------------------------------+ | pattern item score num | |------------------------------| | 1 1 1 141 | | 1 2 1 141 | | 1 3 1 141 | | 1 4 1 141 | |------------------------------| | 2 1 0 103 | | 2 2 0 103 | | 2 3 0 103 | | 2 4 0 103 | |------------------------------| | 3 1 0 44 | | 3 2 1 44 | | 3 3 1 44 | | 3 4 1 44 | |------------------------------|
Create dummies for the 4 items
+------------------------------------------------------------------------------+ | pattern item score num d1 d2 d3 d4 negd1 negd2 negd3 negd4 | |------------------------------------------------------------------------------| | 1 1 1 141 1 0 0 0 -1 0 0 0 | | 1 2 1 141 0 1 0 0 0 -1 0 0 | | 1 3 1 141 0 0 1 0 0 0 -1 0 | | 1 4 1 141 0 0 0 1 0 0 0 -1 | |------------------------------------------------------------------------------| | 2 1 0 103 1 0 0 0 -1 0 0 0 | | 2 2 0 103 0 1 0 0 0 -1 0 0 | | 2 3 0 103 0 0 1 0 0 0 -1 0 | | 2 4 0 103 0 0 0 1 0 0 0 -1 | +------------------------------------------------------------------------------+ …etc.
tab item, gen(d) forvalues i=1/4 { } gen negd`i' = -d`i'
Relate this back to original equation
+------------------------------------------------------------------------------+ | pattern item score num d1 d2 d3 d4 negd1 negd2 negd3 negd4 | |------------------------------------------------------------------------------| | 1 1 1 141 1 0 0 0 -1 0 0 0 | | 1 2 1 141 0 1 0 0 0 -1 0 0 | | 1 3 1 141 0 0 1 0 0 0 -1 0 | | 1 4 1 141 0 0 0 1 0 0 0 -1 | |------------------------------------------------------------------------------|
Rasch model using GLLAMM
• Rename num wt2 (shows that weighting applies at level-2 i.e. person level) • constraint def 1 [patt1]_cons = 1 • gllamm score negd1-negd4, i(pattern) /// weight(wt) /// link(logit) /// family(binomial) /// frload(1) constr(1) /// nip(15) nocons adapt trace
Rasch results
log likelihood = -852.8413222195184
----------------------------------------------------------------------------- | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+--------------------------------------------------------------- negd1 | .333879 .1287913 2.59 0.010 .0814527 .5863054
negd2 | -.4827996 .1301057 -3.71 0.000 -.7378022 -.2277971
negd3 | -.7138122 .1321945 -5.40 0.000 -.9729087 -.4547158
negd4 | -.6117231 .131175 -4.66 0.000 -.8688214 -.3546249
----------------------------------------------------------------------------- Variances and covariances of random effects ----------------------------------------------------------------------------- ***level 2 (pattern) var(1): 1 (0) ----------------------------------------------------------------------------- We constrained this to unit variance
Plot ICC’s
--------------------------- | Coef. ------------------+-------- negd1 (woman) | .334 negd2 (couple) | -.483 negd3 (not_marr) | -.714 negd4 (afford) | -.612 --------------------------- Curves are parallel First item is most “difficult” -5 0 x Woman Not_married Couple Afford 5 twoway(function Woman =invlogit(x-[score]negd1), range(-6 6)) /// (function Couple =invlogit(x-[score]negd2), range(-6 6) lpatt(".")) /// (function Not_married =invlogit(x-[score]negd3), range(-6 6) lpatt("-")) /// (function Afford =invlogit(x-[score]negd4), range(-6 6) lpatt("_"))
GLLAMM versus raschtest • Raschtest
– Avoids the need to derive dummy variables – Needs complete dataset, not frequency-weights – Reformats the dataset in the background so no need to do it yourself – Can employ CML (an estimation specific to Rasch models) which requires no integration
Polytomous IRT
Extension to polytomous IRT • We now have a hierarchy of parameters to model 1. At the test level
• A number of items models simultaneously with the potential for parameters to vary across items
2. At the item level
• Contrasts are used to model the response categories within each item. Parameters may/may-not vary across response categories.
So when faced with a set of polytomous items We must decide
1. The payoff from not collapsing into binary items 2. The form of contrasts needed to model over response categories within items 3. Any constraints required across these response categories 4. Any parameter constraints across test items within a single
4 commonly used polytomous IRT models
• Partial Credit model (PCM) – Masters (1982) • Rating Scale model (RSM) – Andrich (1978a/b) • Graded Response model (GRM) – Samejima (1969) • Nominal Response model (NRM) – Bock (1972)
Partial Credit Model
Partial Credit Model (PCM) • Designed for items where you can obtain a “partial credit”, e.g.
0 = solved nothing, 1 = solved part A, 2 = solved parts A and B
• i.e. those who scored a ‘2’ can also be thought of as having achieved a ‘1’
Partial Credit Model (PCM)
– Here items scored from 0 to
m,
however it’s possible for items to have differing numbers of categories
P
( attain level
x
for item
i
|
)
exp[
r m
0
exp[
j x
0
(
j r
0
(
ij
)]
ij
)]
with
j
0 0 (
ij
) 0 – β ij i s referred to as the “step parameter”
Partial Credit Model (PCM)
E.g. for an item with three categories 0,1,2:
P
(
x
0 | ) 1 exp[ 1 ] 1 exp[( 1 ) ( 2 )]
P
(
x
1 | ) 1 exp[ 1 ] exp[ exp[( 1 ] 1 ) ( 2 )]
P
(
x
2 | ) 1 exp[ exp[( 1 ] 1 ) exp[( ( 1 ) 2 )] ( 2 )]
Same format as adjacent category ordinal model
P
(
u
0 |
X
) 1 exp(
c
1 exp((
c
0 1
X
) 0
X
) exp((
c
0 (
c
1 0
X
1 )
X
)) (
c
1 1
X
))
P
(
u
1 |
X
) 1 exp(
c
1 1
X
) exp(
c
1 exp((
c
0 1
X
) 0
X
) (
c
1 1
X
))
P
(
u
2 |
X
) 1 exp(
c
1 1 1
X
) exp((
c
0 0
X
) (
c
1 1
X
)) Hence, PCM is a DIRECT or divide-by-total model
Collapse to two category levels:
Or for an item with two categories 0,1:
P
(
x
0 | ) 1 1 exp[ 1 ]
P
(
x
1 | ) exp[ 1 exp[ ] 1 1 ] i.e. the familiar 1PL or Rasch model - the PCM is the polytomous extension of the Rasch model
Step parameters
β ij “ step parameters ” occur at the intersection of adjacent ICC’s - thresholds for transition from one category to the next Disordering of steps Suggests lack of ordinality e.g. 1/3/2/4 instead of 1/2/3/4
Partial Credit Model (PCM)
Adjacent category model Outcome Level 1 Level 2 Level 3 Level 4 C1 C1 C2 C2 C3 C3 Within item constraints Ideally want ordering of steps so that each response category is endorsed in turn as trait level increases Across item constraints Rasch-based hence items have equal discrimination. This can be relaxed to give Generalised PCM but this loses the desirable Rasch properties
Rating Scale Model
Rating Scale Model (RSM) • Traditionally used for attitudes data • Constrained form of PCM, hence is also Rasch
Rating Scale Model (RSM)
PCM
P ij
( ) exp[
r m
0 exp[
j x
0 (
j r
0 (
ij
)]
ij
)] RSM
P ij
( ) exp[
r m
0 exp[
j x
0
j r
0 (
i
(
i
j
)]
j
)] Probabilities estimated directly , as with PCM Not suitable for items with differing response formats All items share same step parameters δ j There is a location parameter β i which can vary across items
Rating Scale Model (RSM)
Adjacent category model Outcome Level 1 Level 2 Level 3 Level 4 C1 C1 C2 C2 C3 C3 Within item constraints Ideally want ordering of steps so that each response category endorsed in turn as trait level increases Across item constraints Constrained, more parsimonious version of PCM Only appropriate for instance where all items have same number of response categories + have same response options (text)
Graded Response Model
Graded Response Model (GRM)
P
(
u
j
| ) exp[ 1
i
exp[ (
i
(
ij
)]
ij
)] This is the IRT equivalent of the contrasts described earlier for the POM:
P
(
u
j
|
X
) exp(
c j
1 exp(
c j
X
)
X
)
Consequently
• The Graded Response Model is a difference model, in contrast to the PCM/RSM or indirect IRT • The β ij are thresholds with interpretation akin to the binary 1PL/2PL models – The ability level for which the probability of making a response equal to or greater than the threshold j is 50% • Plots of P(u ≥ j |θ) are Operating Characteristic Curves
ICC’s for GHQ-01 under GRM
GRM parameters correspond to where p=0.5 crosses each OCC in turn
ICC’s for GHQ-01 under GRM
ICC’s obtained by subtraction
ICC’s for GHQ-01 under GRM
1st and last GRM parameters can also be obtained from equivalent ICC plot: where p=0.5 crosses ICC’s #1, #4
Graded Response Model (GRM)
P
(
u
j
) 1 exp[
i
exp[ (
i
(
ij
)]
ij
)] Outcome Level 1 Level 2 Level 3 Level 4 C1 C2 C3 C1 C2 C3 Within item constraints Discrimination α i equal across item levels= homogeneous GRM ~ POM Without this, the OCC’s will cross at some trait level hence the difference can be negative Across item constraints Discrimination α i β ij differ across items differ across items β ij = c i + b j = Modified GRM Location Thresholds
Nominal Response Model
Nominal Response Model (NRM)
P ij
( ) exp(
c ij
ij
)
j J
0 exp(
c ij
ij
)
Nominal Response Model (NRM)
Contrasts relative to baseline As in multinomial logistic model Across items Outcome Level 1 Level 2 Level 3 Level 4 C1 C1 C2 C2 C3 C3 Within items
Some ordinal examples
Winsteps: PCM and RSM Ego-GHQ Odd items
Winsteps
• • Specialist package for fitting IRT models of the Rasch family • John M Linacre www.winsteps.com
• Estimation using Joint Maximum Likelihood • Impressive array of output tables and figures
[Winsteps] PCM
Partial Credit in Winsteps
The conventional representation of the Partial Credit model is log ( P nij / P ni(j-1) ) = θ n β ij Winsteps parameterizes β ij average ( β ij ). as β i + δ ij where sum( δ ij ) = 0. And β i is the log ( P nij / P ni(j-1) ) = θ n β i δ ij
Algebraically these two representations are identical.
Thus every item has a mean difficulty, supported by Winsteps.
β i . (~location) This simplifies communication, because the results of a Partial Credit analysis now have the same form as any other polytomous analysis
PCM step parameters (table 3.2)
FOR GROUPING "0" Item NUMBER: 2 ghq03 Item DIFFICULTY MEASURE OF 1.10 ADDED TO MEASURES ------------------------------------------------------------------ |CATEGORY OBSERVED|OBSVD SAMPLE|INFIT OUTFIT||STRUCTURE|CATEGORY| |LABEL SCORE COUNT %|AVRGE EXPECT| MNSQ MNSQ||CALIBRATN| MEASURE| |-------------------+------------+------------++---------+--------| | 0 0 116 10| -1.24 -2.27| 1.61 1.45|| NONE |( -4.36)| 0 | 1 1 783 71| -.41 -.27| 1.12 1.13|| -4.36 | -.43 | 1 | 2 2 167 15| 1.96 1.95| .98 .98|| 1.32 | 3.28 | 2 | 3 3 43 4| 3.27 3.52| 1.24 1.36|| 3.04 |( 5.35)| 3 ------------------------------------------------------------------ STRUCTURE CALIBRATN, the calibrated measure of the transition from the category below to this category. This is an estimate of the Rasch-Andrich model parameter, δ j . Use this for anchoring in Winsteps. (This corresponds to δ j in the β i + δ j similarly applied as the δ ij parameterization of the "Rating Scale" model, and is of the β ij = β i + δ ij of the "Partial Credit" model.) The bottom category has no prior transition, and so that the measure is shown as NONE.
CATEGORY PROBABILITIES: MODES - Structure measures at intersections P -+---------+---------+---------+---------+---------+---------+ R 1.0 + + O | 333| B | 111111 33 | A |0 111 111 33 | B .8 + 00 11 1 33 + I | 0 1 11 3 | L | 0 11 1 3 | I | 0 1 1 3 | T .6 + 0 1 1 3 + Y | 0 1 1 222 3 | .5 + * 1 22 22 3 + O | 1 0 2* *2 | F .4 + 1 0 2 1 3 2 + | 1 0 2 1 3 2 | R | 1 00 22 1 3 2 | E | 1 0 2 13 22 | S .2 + 11 0 22 311 2 + P |1 00 2 3 1 22 | O | 000 222 33 11 222 | N | 22****0 3333 1111 222| S .0 +*******************3333333*******00000000000000000***********+ E -+---------+---------+---------+---------+---------+---------+ -6 -4 -2 0 2 4 6 Person [MINUS] Item MEASURE
PCM – location/difficulty parameters (table 13.1)
------------------------------------------------------------------------------------------- |ENTRY TOTAL MODEL| INFIT | OUTFIT |PT-MEASURE |EXACT MATCH| | |NUMBER SCORE COUNT MEASURE S.E. |MNSQ ZSTD|MNSQ ZSTD|CORR. EXP.| OBS% EXP%| Item | |------------------------------------+----------+----------+-----------+-----------+-------| | 6 745 1119 2.34 .06|1.09 1.7| .94 -.6| .72 .73| 62.8 67.0| ghq11 | | 2 1267 1119 1.10 .07|1.23 4.3|1.22 3.4| .58 .68| 74.3 74.6| ghq03 | | 5 1398 1119 .69 .05| .76 -6.2| .80 -4.8| .83 .77| 69.5 59.6| ghq09 | | 3 1593 1119 .20 .05| .98 -.4| .98 -.6| .76 .75| 62.7 61.5| ghq05 | | 1 1548 1119 .08 .06| .90 -2.2| .88 -2.2| .73 .70| 76.6 71.7| ghq01 | | 4 1605 1119 -.01 .06| .97 -.7| .95 -.9| .73 .72| 70.9 68.1| ghq07 | |------------------------------------+----------+----------+-----------+-----------+-------| | MEAN 1359.3 1119.0 .73 .06| .99 -.6| .96 -1.0| | 69.4 67.1| | | S.D. 299.4 .0 .81 .01| .15 3.3| .13 2.4| | 5.3 5.3| | --------------------------------------------------------------------------------------------
PCM: Individual
expected score
ICC’s
PCM: Item Information Functions
PCM:
[Winsteps] RSM
PCM → RSM in Winsteps
Winsteps parameterization of PCM: log ( P nij / P ni(j-1) ) = θ n β i δ ij This is comparable to the familiar RSM formulation log ( P nij / P ni(j-1) ) = θ n β i – δ j Recall in the RSM, items share the same step parameters.
Winsteps deals with this by
grouping
the J items.
J groups → PCM, 1 group → RSM k (< J) groups → somewhere in between
PCM input file
&INST TITLE = C:\work\courses\summer_school\GHQ_IRT\Winsteps\ego_ghq12_id_0123.dta
PERSON = Person ; persons are ...
ITEM = Item ; items are ...
ITEM1 = 1 ; column of response to first item in data record NI = 6 ; number of items NAME1 = 8 ; column of first character of person identifying label NAMELEN = 1 ; length of person label XWIDE = 1 ; number of columns per item response CODES = 0123 ; valid codes in data file UPMEAN = 0 ; person mean for local origin USCALE = 1 ; user scaling for logits UDECIM = 2 ; reported decimal places for user scaling ISGROUPS = 000000 ; Item Scale Grouping for modeling rating scales ; STATA file created or last modified: 25/08/2009 18:28:37 ; : 25 Aug 2009 18:28 ; STATA Cases processed = 1119 ; STATA Variables processed = 14 TOTALSCORE = Yes ; Include extreme responses in reported scores ; Person Label variables: columns in label: columns in line &END ghq01 ghq03 ghq05 ghq07 ghq09 ghq11 END LABELS 311332 333333
RSM input file
&INST TITLE = C:\work\courses\summer_school\GHQ_IRT\Winsteps\ego_ghq12_id_0123.dta
PERSON = Person ; persons are ...
ITEM = Item ; items are ...
ITEM1 = 1 ; column of response to first item in data record NI = 6 ; number of items NAME1 = 8 ; column of first character of person identifying label NAMELEN = 1 ; length of person label XWIDE = 1 ; number of columns per item response CODES = 0123 ; valid codes in data file UPMEAN = 0 ; person mean for local origin USCALE = 1 ; user scaling for logits UDECIM = 2 ; reported decimal places for user scaling ISGROUPS = AAAAAA ; Item Scale Grouping for modeling rating scales ; STATA file created or last modified: 25/08/2009 18:28:37 ; : 25 Aug 2009 18:28 ; STATA Cases processed = 1119 ; STATA Variables processed = 14 TOTALSCORE = Yes ; Include extreme responses in reported scores ; Person Label variables: columns in label: columns in line &END ghq01 ghq03 ghq05 ghq07 ghq09 ghq11 END LABELS 311332 333333
RSM step parameters (
δ j ) • Single table for step parameters as they are shared by all items SUMMARY OF CATEGORY STRUCTURE. Model="R" ------------------------------------------------------------------ |CATEGORY OBSERVED|OBSVD SAMPLE|INFIT OUTFIT||STRUCTURE|CATEGORY| |LABEL SCORE COUNT %|AVRGE EXPECT| MNSQ MNSQ||CALIBRATN| MEASURE| |-------------------+------------+------------++---------+--------| | 0 0 1249 19| -3.10 -3.10| 1.06 1.02|| NONE |( -4.27)| 0 | 1 1 3278 49| -1.32 -1.29| .95 .92|| -3.15 | -1.42 | 1 | 2 2 1629 24| .74 .62| .89 .89|| .35 | 1.59 | 2 | 3 3 498 7| 2.43 2.62| 1.20 1.22|| 2.81 |( 3.96)| 3 -------------------------------------------------------------------
Single ICC plot since “person minus item measure” is constant for all items CATEGORY PROBABILITIES: MODES - Structure measures at intersections P -+-----+-----+-----+-----+-----+-----+-----+-----+-----+ R 1.0 + + O | | B | | A |00 | B .8 + 00 + I | 00 1111 33| L | 0 111 111 3 | I | 0 11 11 22 3 | T .6 + 00 1 1 222 222 33 + Y | 0 11 1 22 22 3 | .5 + * 112 23 + O | 1 0 221 322 | F .4 + 1 0 2 1 3 2 + | 11 00 2 1 3 22 | R | 1 0 22 1 3 2 | E | 11 0 2 11 33 22| S .2 + 11 00 22 13 + P |11 0022 3311 | O | 222000 333 111 | N | 22222 00000 3333 1111 | S .0 +**************3333333333333333*000000000000000000******+ E -+-----+-----+-----+-----+-----+-----+-----+-----+-----+ -5 -4 -3 -2 -1 0 1 2 3 4 Person [MINUS] Item MEASURE
RSM difficulty parameters (
β i ) TABLE 13.1 C:\work\courses\summer_school\GHQ_IRT\ ZOU638WS.TXT Aug 26 8:32 2009 INPUT: 1119 Persons 6 Items MEASURED: 1119 Persons 6 Items 4 CATS 3.67.0
------------------------------------------------------------------------------- Person: REAL SEP.: 1.91 REL.: .78 ... Item: REAL SEP.: 16.26 REL.: 1.00
Item STATISTICS: MEASURE ORDER ------------------------------------------------------------------------------------------ |ENTRY TOTAL MODEL| INFIT | OUTFIT |PT-MEASURE |EXACT MATCH| | |NUMBER SCORE COUNT MEASURE S.E. |MNSQ ZSTD|MNSQ ZSTD|CORR. EXP.| OBS% EXP%| Item | |------------------------------------+----------+----------+-----------+-----------+------| | 6 745 1119 2.85 .06|1.39 8.1|1.20 3.7| .74 .72| 60.6 69.3| ghq11| | 2 1267 1119 1.12 .06|1.03 .8|1.00 .1| .60 .73| 67.6 63.5| ghq03| | 5 1398 1119 .72 .05|1.10 2.4|1.11 2.5| .82 .73| 55.5 62.8| ghq09| | 1 1548 1119 .28 .05| .72 -7.4| .72 -7.2| .74 .73| 73.9 63.3| ghq01| | 3 1593 1119 .14 .05|1.05 1.1|1.04 .9| .75 .73| 62.7 63.6| ghq05| | 4 1605 1119 .11 .05| .83 -4.2| .83 -4.1| .73 .73| 70.7 63.6| ghq07| |------------------------------------+----------+----------+-----------+-----------+------| | MEAN 1359.3 1119.0 .87 .06|1.02 .1| .98 -.7| | 65.1 64.3| | | S.D. 299.4 .0 .95 .00| .21 4.9| .16 3.8| | 6.2 2.2| | -------------------------------------------------------------------------------------------
RSM: Individual
expected score
ICC’s
RSM: Individual
expected score
ICC’s
Compare parameters
Item ghq11 β i 2.34
ghq3 ghq9 1.10
0.69
ghq1 ghq5 ghq7 0.20
0.08
-0.01
δ 1 PCM -1.90
-4.36
-2.23
-4.51
-3.03
-4.04
δ 2 0.03
1.32
-0.06
1.01
0.20
0.76
δ 3 1.87
3.04
2.29
3.50
2.83
3.29
β i 2.85
1.12
0.72
0.28
0.14
0.11
RSM δ 1 -3.15
“ “ “ “ “ δ 2 0.35
“ “ “ “ “ Note in this set up, sum of step parameters = zero for each item δ 3 2.81
“ “ “ “ “
Compare item fit
PCM ------------------------------------------------------------------------------------------ |ENTRY TOTAL MODEL| INFIT | OUTFIT |PT-MEASURE |EXACT MATCH| | |NUMBER SCORE COUNT MEASURE S.E. |MNSQ ZSTD|MNSQ ZSTD|CORR. EXP.| OBS% EXP%| Item | |------------------------------------+----------+----------+-----------+-----------+------| | 2 1267 1119 1.10 .07|1.23 4.3|1.22 3.4|A .58 .68| 74.3 74.6| ghq03| | 6 745 1119 2.34 .06|1.09 1.7| .94 -.6|B .72 .73| 62.8 67.0| ghq11| | 3 1593 1119 .20 .05| .98 -.4| .98 -.6|C .76 .75| 62.7 61.5| ghq05| | 4 1605 1119 -.01 .06| .97 -.7| .95 -.9|c .73 .72| 70.9 68.1| ghq07| | 1 1548 1119 .08 .06| .90 -2.2| .88 -2.2|b .73 .70| 76.6 71.7| ghq01| | 5 1398 1119 .69 .05| .76 -6.2| .80 -4.8|a .83 .77| 69.5 59.6| ghq09| |------------------------------------+----------+----------+-----------+-----------+------| RSM ------------------------------------------------------------------------------------------ |ENTRY TOTAL MODEL| INFIT | OUTFIT |PT-MEASURE |EXACT MATCH| | |NUMBER SCORE COUNT MEASURE S.E. |MNSQ ZSTD|MNSQ ZSTD|CORR. EXP.| OBS% EXP%| Item | |------------------------------------+----------+----------+-----------+-----------+------| | 2 1267 1119 1.12 .06|1.03 .8|1.00 .1|c .60 .73| 67.6 63.5| ghq03| | 6 745 1119 2.85 .06|
1.39
8.1|1.20 3.7|A .74 .72| 60.6 69.3|
ghq11
| | 3 1593 1119 .14 .05|1.05 1.1|1.04 .9|C .75 .73| 62.7 63.6| ghq05| | 4 1605 1119 .11 .05| .83 -4.2| .83 -4.1|b .73 .73| 70.7 63.6| ghq07| | 1 1548 1119 .28 .05| .72 -7.4| .72 -7.2|a .74 .73| 73.9 63.3| ghq01| | 5 1398 1119 .72 .05|1.10 2.4|1.11 2.5|B .82 .73| 55.5 62.8| ghq09| |------------------------------------+----------+----------+-----------+-----------+------|
Test Information Function: PCM/RSM
2.50
2.00
1.50
1.00
0.50
0.00
-8 -6 -4 -2 0 RSM PCM 2 4 6 8
[R] PCM versus RSM Using the package –eRm–
eRm
• e
xtended
R
asch
m
odelling • Similar parameterization to Winsteps
– For both PCM/RSM the step parameter β ij β i and δ ij (or δ j )
• Different estimation to Winsteps
is split into – Uses Conditional Maximum Likelihood (CML), as used in -raschtest-, which is computationally easier than JML
PCM: Example with 2 items (sum0 = TRUE)
Model matrix W eta 1 eta 2 eta 3 eta 4 eta 5 beta ghq01.c1 -1 -1 -1 -1 -1 beta ghq01.c2 1 0 0 0 0 beta ghq01.c3 0 1 0 0 0 beta ghq03.c1 0 0 1 0 0 beta ghq03.c2 0 0 0 1 0 beta ghq03.c3 0 0 0 0 1 β ij = W*eta
i
,
j
ij
0 Item Easiness Parameters (beta): beta ghq01.c1 2.761 beta ghq01.c2 1.656
beta ghq01.c3 -1.627 beta ghq03.c1 1.848 beta ghq03.c2 -0.466
beta ghq03.c3 -4.172
Model parameters eta eta 1 1.6555484
eta 2 -1.6267047 eta 3 1.8482333
eta 4 -0.4656337 eta 5 -4.1723330
threshold i
1
threshold i
2
threshold i
3
location i
Design Matrix Location Threshold 1 Threshold 2 Threshold 3 ghq01 0.54223 -2.76089 1.10534 3.28225
ghq03 1.39078 -1.84823 2.31387 3.70670
i
1
i
1
i
2
j
i
2
i
3
threshold ij
j
j
RSM: Example with 2 items (sum0 = TRUE)
Model matrix W eta 1 eta 2 eta 3 beta ghq01.c1 -1 0 0 beta ghq01.c2 -2 1 0 beta ghq01.c3 -3 0 1 beta ghq03.c1 1 0 0 beta ghq03.c2 2 1 0 beta ghq03.c3 3 0 1
i
,
j
ij
β ij = W*eta Item Easiness Parameters (beta): beta ghq01.c1 0.498
beta ghq01.c2 -2.975
beta ghq01.c3 -8.367
beta ghq03.c1 -0.498
beta ghq03.c2 -4.966 beta ghq03.c3 -11.354
0 Model parameters eta eta 1 -0.4978386 eta 2 -3.9701867 eta 3 -9.8605819 Note, distance between thresholds is now constant over items
threshold i
1
threshold i
2
threshold i
3
location i
Design Matrix Location Threshold 1 Threshold 2 Threshold 3 ghq01 2.78902 -0.49784 3.47235 5.39256
ghq03 3.78470 0.49784 4.46803 6.38823
i
1
i
1
i
2
j
i
2
i
3
threshold ij
j
j
PCM: Example with 2 items (sum0 = FALSE)
Model matrix W eta 1 eta 2 eta 3 eta 4 eta 5 beta ghq01.c1 0 0 0 0 0 beta ghq01.c2 1 0 0 0 0 beta ghq01.c3 0 1 0 0 0 beta ghq03.c1 0 0 1 0 0 beta ghq03.c2 0 0 0 1 0 beta ghq03.c3 0 0 0 0 1 β ij = W*eta Item Easiness Parameters (beta): beta ghq01.c1 0.000 beta ghq01.c2 -3.866 beta ghq01.c3 -9.909 beta ghq03.c1 -0.913 beta ghq03.c2 -5.987 beta ghq03.c3 -12.455 Model parameters eta eta 1 -3.8662273 eta 2 -9.9093688 eta 3 -0.9126535 eta 4 -5.9874083 eta 5 -12.4549913
Design Matrix
threshold i
1
threshold i
2
threshold i
3
location i
i
1
i
1
i
2
j
Location Threshold 1 Threshold 2 Threshold 3 ghq01 3.30312 0.00000 3.86623 6.04314 ghq03 4.15166 0.91265 5.07475 6.46758
i
2
i
3
threshold ij
j
j
RSM: Example with 2 items (sum0 = FALSE)
Model matrix W eta 1 eta 2 eta 3 beta ghq01.c1 0 0 0 beta ghq01.c2 0 1 0 beta ghq01.c3 0 0 1 beta ghq03.c1 1 0 0 beta ghq03.c2 2 1 0 beta ghq03.c3 3 0 1 Model parameters eta eta 1 -0.4978386 eta 2 -3.9701867 eta 3 -9.8605819 Note, location + threshold parameters now comparable between PCM/RSM β ij = W*eta Item Easiness Parameters (beta): beta ghq01.c1 0.000
beta ghq01.c2 -3.970 beta ghq01.c3 -9.861 beta ghq03.c1 -0.996 beta ghq03.c2 -5.962 beta ghq03.c3 -12.848
threshold i
1
threshold i
2
threshold i
3
location i
Design Matrix Location Threshold 1 Threshold 2 Threshold 3 ghq01 3.28684 0.00000 3.97016 5.89036
ghq03 4.28251 0.99567 4.96584 6.88604
i
1
i
1
i
2
j
i
2
i
3
threshold ij
j
j
Sum0 = TRUE Sum0 = FALSE PCM RSM
Just to prove it’s the same model…
> ego_pcm_odd_erm2<-PCM(ego[,c(1,3,5,7,9,11)], sum0 = TRUE ) > LRtest(ego_pcm_odd_erm) Andersen LR-test: LR-value: 84.651 Chi-square df: 14 p-value: 0 > ego_pcm_odd_erm2<-PCM(ego[,c(1,3,5,7,9,11)], sum0 = FALSE ) > LRtest(ego_pcm_odd_erm2) Andersen LR-test: LR-value: 84.651 Chi-square df: 14 p-value: 0
Partial Credit Model with 6 items (odd-GHQ)
i
,
j
> ego_pcm_odd_erm<-PCM(ego[,c(1,3,5,7,9,11)]) beta ghq01.c1 3.46
beta ghq01.c2 2.29
beta ghq01.c3 -0.87
beta ghq03.c1 2.48
beta ghq03.c2 0.14
beta ghq03.c3 -3.41
beta ghq05.c1 2.03
beta ghq05.c2 1.49
beta ghq05.c3 -1.21
beta ghq07.c1 3.10
beta ghq07.c2 2.23
beta ghq07.c3 -0.68
beta ghq09.c1 0.95
beta ghq09.c2 0.27
beta ghq09.c3 -2.38
beta ghq11.c1 -0.66
beta ghq11.c2 -2.80
beta ghq11.c3 -6.43
ij
0 ghq01 ghq03 ghq05 ghq07 ghq09 ghq11 eta 1 2.29
eta 7 1.49
eta 13 0.27
eta 2 -0.87
eta 8 -1.21
eta 14 -2.38
eta 3 2.48
eta 9 3.10
eta 15 -0.66
eta 4 0.14
eta 10 2.23
eta 16 -2.80
eta 5 -3.41
eta 11 -0.68
eta 17 -6.43
eta 6 2.03
eta 12 0.95
Location 0.29
1.14
0.40
0.23
0.79
2.14
Threshold 1 -3.46
-2.48
-2.03
-3.10
-0.95
0.66
location i
Threshold 2 1 3
j
Threshold 3
j
1.17
2.34
3.16
3.56
0.54
0.87
0.69
2.14
2.70
2.91
2.65
3.63
> ego_rsm_odd_erm<-RSM(ego[,c(1,3,5,7,9,11)], sum0 = TRUE ) Design Matrix Block 1: Location Threshold 1 Threshold 2 Threshold 3 ghq01 2.11281 -0.49631 2.42362 4.41113
ghq03 2.81321 0.20409 3.12401 5.11152
ghq05 2.00262 -0.60650 2.31343 4.30094
ghq07 1.97330 -0.63582 2.28411 4.27162
ghq09 2.48345 -0.12567 2.79426 4.78177
ghq11 4.26932 1.66020 4.58013 6.56764
> ego_rsm_odd_erm2<-RSM(ego[,c(1,3,5,7,9,11)], sum0 = FALSE ) Warning messages: 1: In sqrt(diag(solve(parest$hessian))) : NaNs produced 2: In sqrt(diag(lres$W %*% solve(parest$hessian) %*% t(lres$W))) : NaNs produced Design Matrix Block 1: Location Threshold 1 Threshold 2 Threshold 3 ghq01 225.29491 0.00000 -270.3309 946.2156
ghq03 314.61332 89.31842 -181.0124 1035.5340
ghq05 -0.74024 -226.03514 -496.3660 720.1804
ghq07 -12.34834 -237.64325 -507.9741 708.5723
ghq09 187.89149 -37.40341 -307.7343 908.8122
ghq11 819.56596 594.27105 323.9402 1540.4866
?! Try changing starting values
Let’s try the GHQ even items instead
> ego_rsm_even_erm<-RSM(ego[,c(2,4,6,8,10,12)], sum0 = TRUE) Design Matrix Block 1: Location Threshold 1 Threshold 2 Threshold 3 ghq02 2.82054 -0.11586 3.35950 5.21797
ghq04 2.93927 0.00287 3.47823 5.33670
ghq06 3.14174 0.20535 3.68071 5.53918
ghq08 2.71370 -0.22270 3.25266 5.11113
ghq10 3.37903 0.44263 3.91800 5.77647
ghq12 2.62412 -0.31228 3.16308 5.02155
> ego_rsm_even_erm2<-RSM(ego[,c(2,4,6,8,10,12)], sum0 = FALSE) Design Matrix Block 1: Location Threshold 1 Threshold 2 Threshold 3 ghq02 2.93640 0.00000 3.47537 5.33383
ghq04 3.05513 0.11873 3.59410 5.45256
ghq06 3.25761 0.32121 3.79658 5.65504
ghq08 2.82956 -0.10684 3.36853 5.22699
ghq10 3.49490 0.55850 4.03387 5.89233
ghq12 2.73998 -0.19642 3.27895 5.13741
Phew!!
GHQ-even RSM versus PCM
> ego_pcm_even_erm2< PCM (ego[,c(2,4,6,8,10,12)], sum0 = FALSE) ghq02 ghq04 ghq06 ghq08 ghq10 ghq12 Location 2.36
2.56
2.67
2.29
2.72
2.19
Threshold 1 0.00
-1.50
-0.06
-1.82
0.90
-1.24
Threshold 2 2.44
3.84
3.07
3.40
2.54
3.00
Threshold 3 4.63
5.34
5.00
5.29
4.73
4.81
> ego_rsm_even_erm2< RSM (ego[,c(2,4,6,8,10,12)], sum0 = FALSE) ghq02 ghq04 ghq06 ghq08 ghq10 ghq12 Location 2.94
3.06
3.26
2.83
3.49
2.74
Threshold 1 0.00
0.12
0.32
-0.11
0.56
-0.20
Threshold 2 3.48
3.59
3.80
3.37
4.03
3.28
Threshold 3 5.33
5.45
5.66
5.23
5.89
5.14
GHQ-even RSM versus PCM
> ego_pcm_even_erm2< PCM (ego[,c(2,4,6,8,10,12)], sum0 = FALSE) ghq02 ghq04 ghq06 ghq08 ghq10 ghq12 Location 2.36
2.56
2.67
2.29
2.72
2.19
Threshold 1 0.00
-1.50
-0.06
-1.82
0.90
-1.24
Threshold 2 2.44
3.84
3.07
3.40
2.54
3.00
Threshold 3 4.63
5.34
5.00
5.29
4.73
4.81
Diff 1 2.44
5.34
3.13
5.23
1.64
4.24
Diff 2 2.19
1.50
1.93
1.89
2.19
1.81
> ego_rsm_even_erm2< RSM (ego[,c(2,4,6,8,10,12)], sum0 = FALSE) ghq02 ghq04 ghq06 ghq08 ghq10 ghq12 Location 2.94
3.06
3.26
2.83
3.49
2.74
Threshold 1 0.00
0.12
0.32
-0.11
0.56
-0.20
Threshold 2 3.48
3.59
3.80
3.37
4.03
3.28
Threshold 3 5.33
5.45
5.66
5.23
5.89
5.14
Diff 1 3.48
3.48
3.48
3.48
3.48
3.48
Diff 2 1.86
1.86
1.86
1.86
1.86
1.86
PCM/RSM are nested:
> LRtest(ego_pcm_even_erm2) Warning message: Persons with median raw scores are assigned to the lower raw score group!
Warning in LRtest.Rm(ego_pcm_even_erm2) : The following items were excluded due to inappropriate response patterns within subgroups: ghq08 Full and subgroup models are estimated without these items!
Andersen LR-test: LR-value: 74.267 Chi-square df: 14 p-value: 0 > LRtest(ego_rsm_even_erm2) Warning message: Persons with median raw scores are assigned to the lower raw score group!
Warning in LRtest.Rm(ego_rsm_even_erm2) : The following items were excluded due to inappropriate response patterns within subgroups: ghq08 Full and subgroup models are estimated without these items!
Andersen LR-test: LR-value: 317.87 Chi-square df: 6 p-value: 0
Linking IRT to “Factor Analysis”
[Mplus] EGO: GRM (1)
DATA : File is "C:\work\IRT experimenting\ego_ghq12_id.dta.dat" ; VARIABLE : Names are ghq01 ghq02 ghq03 ghq04 ghq05 ghq06 ghq07 ghq08 ghq09 ghq10 ghq11 ghq12 f1 id; Missing are all (-9999) ; usevariables = ghq01 ghq03 ghq05 ghq07 ghq09 ghq11; categorical = ghq01 ghq03 ghq05 ghq07 ghq09 ghq11; ANALYSIS : ESTIMATOR = MLR; LINK =
logit;
MODEL : f BY ghq01* ghq03 ghq05 ghq07 ghq09 ghq11; f@1; PLOT : TYPE = PLOT3; First item’s loading is freely estimated Trait variance fixed to one
[Mplus] EGO: GRM (1)
Estimate S.E. Est./S.E. P-Value F BY GHQ01 2.054 0.152 13.538 0.000
GHQ03 1.206 0.105 11.513 0.000
GHQ05 2.036 0.135 15.115 0.000
GHQ07 1.889 0.132 14.279 0.000
GHQ09 2.899 0.254 11.421 0.000
GHQ11 1.849 0.130 14.195 0.000
Thresholds GHQ01$1 -4.545 0.264 -17.199 0.000
GHQ01$2 0.943 0.106 8.860 0.000
GHQ01$3 3.966 0.209 19.016 0.000
GHQ03$1 -2.636 0.142 -18.549 0.000
GHQ03$2 1.740 0.095 18.270 0.000
GHQ03$3 3.723 0.167 22.251 0.000
GHQ05$1 -3.000 0.158 -19.018 0.000
GHQ05$2 0.363 0.100 3.637 0.000
GHQ05$3 3.320 0.179 18.593 0.000
GHQ07$1 -4.022 0.212 -19.015 0.000
GHQ07$2 0.601 0.097 6.193 0.000
GHQ07$3 3.461 0.181 19.139 0.000
GHQ09$1 -2.289 0.190 -12.038 0.000
GHQ09$2 0.819 0.134 6.130 0.000
GHQ09$3 4.204 0.310 13.579 0.000
GHQ11$1 0.155 0.094 1.651 0.099
GHQ11$2 2.340 0.122 19.125 0.000
GHQ11$3 4.564 0.219 20.835 0.000
1 3 5
[Mplus] EGO: GRM (1) – ICC’s
7 9 11
1 3 5
[Mplus] EGO: GRM (1) – IIF
7 9 11
[Mplus] EGO: GRM (1) – IIF
1.8
1.6
1.4
1.2
1 0.8
0.6
0.4
0.2
0 -6 -4 -2 0 2 Data extracted from Mplus and plot done in Excel 4 ghq_01 ghq_03 ghq_05 ghq_07 ghq_09 ghq_11
[Mplus] EGO: GRM (1) – IIF + TIF
3 2 6 5 4 1 0 -6 -4 -2 0 2 Data extracted from Mplus and plot done in Excel 4 ghq_01 ghq_03 ghq_05 ghq_07 ghq_09 ghq_11 Total
[R] EGO: GRM (1)
ego_grm_odd<-grm(ego[,c(1,3,5,7,9,11)], IRT.param = FALSE) coef(ego_grm_odd) beta.1 beta.2 beta.3 beta ghq01 -4.544 0.946 3.968 2.049
ghq03 -2.637 1.739 3.722 1.203
ghq05 -2.995 0.365 3.318 2.027
ghq07 -4.018 0.604 3.464 1.885
ghq09 -2.281 0.821 4.195 2.883
ghq11 0.159 2.348 4.574 1.849
[R] EGO: GRM (1)
ego_grm_odd<-grm(ego[,c(1,3,5,7,9,11)], IRT.param = FALSE) coef(ego_grm_odd) beta.1 beta.2 beta.3 beta ghq01 -4.544 0.946 3.968 2.049
ghq03 -2.637 1.739 3.722 1.203
ghq05 -2.995 0.365 3.318 2.027
ghq07 -4.018 0.604 3.464 1.885
ghq09 -2.281 0.821 4.195 2.883
ghq11 0.159 2.348 4.574 1.849
GHQ01 Results from Mplus $1 $2 $3 loading -4.545
0.943
3.966
2.054
GHQ03 -2.636
1.740
3.723
1.206
GHQ05 -3 0.363
3.320
2.036
GHQ07 -4.022
0.601
3.461
1.889
GHQ09 -2.289
0.819
4.204
2.899
GHQ11 0.155
2.340
4.564
1.849
par(mfrow = c(2, 3)) plot(ego_grm_odd, type = c("ICC"))
par(mfrow = c(1, 1)) plot(ego_grm_odd, type = c("IIC"))
Or individual IIF – ghost plots?
?
par(mfrow = c(2, 3)) plot(ego_grm_odd, type = c("OCCu"))
par(mfrow = c(2, 3)) plot(ego_grm_odd, type = c("OCCl"))
Compare IRF’s across items
[Mplus] EGO: GRM (2)
DATA : File is "C:\work\IRT experimenting\ego_ghq12_id.dta.dat" ; VARIABLE : Names are ghq01 ghq02 ghq03 ghq04 ghq05 ghq06 ghq07 ghq08 ghq09 ghq10 ghq11 ghq12 f1 id; Missing are all (-9999) ; usevariables = ghq01 ghq03 ghq05 ghq07 ghq09 ghq11; categorical = ghq01 ghq03 ghq05 ghq07 ghq09 ghq11; ANALYSIS : ESTIMATOR = MLR; LINK = logit; MODEL : f BY ghq01* ghq03 ghq05 ghq07 ghq09 ghq11 (1); f@1; All items have same loading First item’s loading is freely estimated PLOT : TYPE = PLOT3; Trait variance fixed to one Is equivalent to R’s ego_grm_odd<-grm(ego[,c(1,3,5,7,9,11)], constrained = TRUE, IRT.param = FALSE)
Mplus: alternative IRT models
• Two similar models can be fitted in Mplus Model A Model B • Estimation = MLR • Uses full information • Logit = default link function • Non-linear relationship between trait and items • Conditional probability method • Estimation = Least squares • Uses limited information • Probit = default link function • Linear relationship between trait and items • Underlying variable (UV) method There are 2*2*2*2*2 = 32 different models defined by these options, and most can be fitted in some software or other.
some models are equivalent, and some are good approximations for each other We will focus on two – because Mplus does
Comparison • Model A
– Models the categorical items as a multivariate set of ordinal responses (here using the GRM) – Here we’ll use the (non-default) probit-link to aid comparison
• Model B
– Uses the one/two-way margins present in the observed categorical data to estimate a (polychoric) correlation matrix for a set of underlying latent continuous variables – Uses this correlation matrix and standard CFA to estimate a trait which is linearly related to these latent continuous variables – Throws away information, but often a good approximation + estimation is much simpler (no need for integration)
A B
Item 1 – Information function
A B
Trait distribution
TIFs
1.8
1.6
1.4
1.2
1 0.8
0.6
0.4
0.2
0 -6 -4.8
-3.6
-2.4
-1.2
0 1.2
2.4
3.6
4.8
A B