Transcript How to - Chiang Mai University

MEQ
Analysis
Outline




Validity
Reliability
Difficulty Index
Power of Discrimination
Validity
“evidence present to support or
refute the meaning assigned to
assessment results”
face validity
content validity
criterion-related validity
construct validity

Face Validity

high face validity



knee jerk & nervous system



seemed valid (only)
~ person
for doctor => high face validity
for lay people => low face validity
MEQ => high face validity
Content Validity



~ sample & population
MCQ => high content
validity
MEQ => low content
validity
Validity
“evidence present to support or
refute the meaning assigned to
assessment results”
- don’t need the score
face validity
- before using the
test
content validity
criterion-related validity - score needed
- after using
construct validity
the test

Criterion-Related
Validity

Predictive validity

MEQ score & close observation
score of real practice at ER

Concurrent Validity

MEQ score & VIVA score in the
same topic
Statistic = correlation coefficient

Construct Validity

based on theory

communication skill ~
leadership


correlation OSCE : communication
VS questionnaire : leadership
good ethics ~ beloved doctor

MEQ : medical ethics VS
questionnaire : beloved doctor?
Reliability



stability
internal consistency
equivalent
Stability



test-retest reliability
parallel form reliability
intra-rater reliability


~ scoring key
statistics : correlation coeff.

0-1
Internal Consistency




[homogeneity]
item - item correlation
item - total correlation
split half correlation
Item-Item Correlation

each item

Dichotomous


Interval


Phi Correlation
Pearson’s Product Moment
Correlation
whole test

Mean of ...
Item-Item Correlation
Item-Total Correlation

each item

Dichotomous


Interval


Point Biserial Correlation
Pearson’s Product Moment
Correlation
whole test

Mean of ...
Item-Total Correlation
Spilt Half Reliability

Dichotomous


Kuder-Richardson 20 (KR 20)
Interval


Kuder-Richardson 21 (KR 21)
Cronbach’s alpha coefficient
Equivalent


parallel item on alternate
form reliability
inter-rater reliability


agreement
kappa
Difficulty Index

[p]

(mean H + mean L)/2(full score)



p = 1 => very easy
p = 0 => very difficult
expecting p = 0.2-0.8


must => 0.7
should => 0.5
Power of Discrimination






[r]
(mean H - mean L)/(full score)
> 0.40 => very good item
0.30 - 0.39 => good item
0.20 - 0.29 => borderline
< 0.20 => poor item
Conclusion




Validity
Reliability
Difficulty Index
Power of Discrimination