Transcript Slide 1

‫به نام خدا‬
How
to
Construct
welldesigned
MCQ’s ?
Types of MCQ’s:
1-True/false
questions(TFQ)
2-multiple-choice
questions (MCQ) OR
ONE-BEST-ANSWER
QUESTIONS
PURPOSE OF TESTING
WHAT SHOULD BE TESTED?
ISSUE OF SAMPLING
THE NATURE OF SAMPLING
DETERMINES THE EXTENT TO
WHICH THE ESTIMATE OF TRUE
ABILITY IS
REPRODUCIBLE(RELIABLE,
GENERALIZABLE) AND ACCURATE
(VALID).
IF THE SAMPLE IS NOT THE
REPRESENTATIVE OF THE
BROADER DOMAIN OF
INTEREST,EXAM RESULTS
WILL BE BIASED AND WILL
NOT PROVIDE A GOOD BASIS
FOR ACHIEVEMENT IN THE
DOMAIN OF INTEREST.
COMPONENTS:
1-STEM
2- LEAD-IN
3-OPTIONS:
CORRECT ANSWER
DISTARCTOR
TECHNICAL ITEM FLAWS:
A- GENERAL
B1- TESTWISENESS
2-IRRELEVANT DIFFICULTY
GENERALGUDELINE FOR ITEM
CONSTRUCTION
1- Make sure the item can be answered without looking
at the options(cover-the-options rule).The entire
question is included in the stem.
2-Include as much of the item as possible in the
stem:the stems should be long and the options short.
3- Avoid “tricky “ and overly complex items.
GENERALGUDELINE FOR
ITEM CONSTRUCTION
4-Avoid superfluous information.
5-Write options that are grammatically consistent
and logically compatible with the stem:list
them in logical or alphabetical order .Write
distractors that are plausible and the same
relative length as the answer.
GENERALGUDELINE FOR
ITEM CONSTRUCTION
6-Avoid using absolutes such as always,never
,and all in the options:also avoid using vague
terms such as usually and frequently.
7-Avoid negatively phrased terms (e.g. those
with except or not in the lead-in).
If you must use a negative stem,use only short
(preferebly single word )options.
GENERALGUDELINE FOR
ITEM CONSTRUCTION
And most important of all:
Focus on important
ConCepTs ,don’T wasTe
time testing trivila
facts.
STILL TWO MORE POINTS!
What is a
good Item
shape?
POORLY SHAPED ITEM
THE OPTIONS:
IN ORDER TO
RANK –ORDER
THE RELATIVE CORRECTNESS
OF OPTIONS ,THE OPTIONS
MUST DIFFER
ON A
SINGLE DIMENSION .
Issues relaet to
testwiseness:
What is test-wiseness?
Millman, Bishop and Ebel (1965)
defined
test-wiseness
as “a subject’s capacity to utilize the
characteristics and formats of
the test and/or the test taking situation to
receive a high score”.
Testwiseness
is independent of
the skills and/or knowledge
that the test is supposedly
measuring, therefore
differential test-wiseness
within a group of students is a
source of invalidity in test scores.
Issues relaTED to
testwiseness:
1- Grammatical cues
2- Logical cues
3- Absolute terms
4- Long correct answer
5- Word repeats
6-Convergence strategy
Issues relaTED to testwiseness:
1-GRAMMATICAL CUES:
2-LOGICAL CUES:
3-ABSOLUTE TERMS:
4-LONG CORRECT ANSWER:
5-WORD REPEATS(STEM &
CORRECT ANSWER):
6-CONVERGENCE STRATEGY:
WHICH ARE CORRECT:
1-PENCIL AND PEN
2-PENCIL AND
HIGHLIGHTER
3-PEN AND MARKER
4-PENCIL AND CRAYON
One of our main aims is
to reduce
the number of
lucky guesses
and
to increase
the number of
knowledge-based answers
ISSUES RELATED TO
IRRELEVANT
DIFFICULTY:
1-OPTIONS ARE
LONG,COMPLICATED OR
DOUBLE
THIS CAN SHIFT WHAT IS MEASURED
BY AN ITEM FROM CONTENT
KNOWLEDGE TO READING SPEED.
2-NUMERIC DATA ARE NOT
STATED CONSISTENTLY:
3-FRQUENCY TERMS IN THE
OPTIONS ARE
VAGUE(e.g.RARELY,USUALLY):
GUIDELINES FOR
BASIC SCIENCE
ITEM WRITING:
BASIC RULES
1- Each item should focus on an important
concept.
2-Each item should assess application of
knowledge.
3-The stem of the question must pose a clear
question.
4-All distarctors should be homogenous.
5-Avoid technical flaws that provide special
benefit to testwise eaminees.
Basic Science Recall Item Stem:
What area is supplied with blood
by the posterior inferior
cerebellar artery?
Basic Science Application of Knowledge
Item Stem:
A 62-year-old man develops left-sided
limb ataxia.Horner’s syndrome
,nystagmus,and loss of appreciation of
facial pain and temperature sensations .
What artery is most likely to be included?
GUIDELINES
FOR
CLINICAL SCIENCE
ITEM CONTENT
1-Test application of knowledge using clinical
vignettes to pose medical decisions in patient
care situations.
2- Focus items on common or potentially
catastrophic problems:avoid “zebras” and
esoterica.
3-Pose clinical decision -making tasks
that would be expected of a
successful examinee.
4- Avoid clinical situations that would
be handled by a (sub)specialist.
The structure of a
clinical vignette
An example of a flawed clinical item!
THIS IS A
WAITING ROOM
ITEM!
Another example of a flawed clinical
item!
Which of the following findings is
most likely to be seen in
postsurgical patients with
pulmonary embolism?
A clinically backward item!!
Patients rarely tell their
physician what disease they
have and then ask the
physician what their signs
and symptoms are!
THE
END RESULT
SHOULD BE
A
VALID
AND
RELIABLE
TEST.
2BOTH VALID AND RELAIBLE
1NEITHER VALID ,NOR RELAIBLE
2NOT VALID ,BUT RELAIBLE