Transcript Completion, Short-Answer, and True-False Items

Completion,
Short-Answer, and
True-False Items
Educational Assessment of Students
Chapter 7
Anthony J. Nitko
(Brenda Stewart)
Three Fundamental Principles for
Crafting Assessments



Focus each assessment task entirely on important
learning targets
Craft each assessment task to elicit from students only
the knowledge and performance that are relevant to the
learning targets you are assessing
Craft each assessment task to neither prevent nor
inhibit a student’s ability to demonstrate attainment of
the leaning targets you are assessing
Short-Answer Items


Require a word, short phrase, number, or
symbol response.
Three types of short-answer items:
Question – What is the capital of Texas?
 Completion – The capital of Texas is _________.
 Association – On the blank next to the state name,
write the abbreviation for that state.

TexasTX
 California

CA
Usefulness of Short-Answer
Items

Abilities Assessed

Lower-order thinking skills
Recall
 Comprehension of information


Higher-level abilities
Make simple interpretations of data and applications of
rules
 Ability to solve numerical problems in science and
mathematics
 Ability to manipulate mathematical symbols and balance
mathematical and chemical equations

Usefulness of Short-Answer
Items

Strengths
Relatively easy to construct
 Can be scored objectively
 Lowers the probability of getting the answer correct
by random guessing


Shortcomings
Not free of subjectivity
 Cannot anticipate all possible responses
 Tends to lower the reliability of the obtained score

Crafting Short-Answer Items






Importance of what is assessed
How does it match the test blueprint
Question format is the preferred format for a
short-answer item. Focus the item on the
specific knowledge sought
Word the items specifically and clearly
Put the blank near the end of the sentence
If possible do not copy statements verbatim
Crafting Short-Answer Items





A completion item should omit important words
and not trivial words
Limit blanks to one or two
Keep all blanks the same length
Specify the precision you expect in your answer
Avoid irrelevant clues
True-False Items


Consists of a statement or proposition the
student may judge as true or false
Six varieties:
True-False
 Yes-No
 Right-Wrong
 Correction
 Multiple True-False
 Yes-No with explanation

Usefulness of True-False Items

Advantages





Certain aspects of the subject matter readily lend themselves
to verbal propositions that can be judged true or false
Relatively easy to write
Scored easily and objectively
Can cover a wide range of content within a relatively short
period
Criticisms




Poorly constructed assess only specific, frequently trivial facts
Ambiguously worded
Answered by random guessing
Encourage students to study and accept only oversimplified
statements of truth and factual details
Usefulness of True-False Items






Generalizations in a subject area
Comparisons among concepts
Casual or conditional propositions
Relationships between two events, concepts,
facts, or principles
Explanations for why events or phenomena
occurred
Instances or examples of a concept or principle
Usefulness of True-False Items





Evidential statements
Predictions about phenomena or events
Steps in a procedure of process
Computations (or other kinds of results
obtained from applying a procedure)
Evaluations of events of phenomena
Validity of True-False Item Format




Essence of educational achievement is the
command of useful verbal knowledge
All verbal knowledge can be expressed in
propositions
Proposition is any sentence that can be said to
be true or false
Particular area of knowledge is indicated by
success in judging truth or falsity of
propositions
Guessing on True-False Items:
An Ill Wind?

Guessing – getting 50% correct
Blind (complete random guessing) quite unlike
informed guessing (based on partial knowledge)
 Well-motivated guess blindly only on a small
percentage of questions on a test
 Difficult to obtain a good score on a test by blind
guessing only
 A given true-false test has a high reliability
coefficient – evidence blind guessing does not
seriously affect the scores

Guessing on True-False Items:
An Ill Wind?


Random guessing – sometimes called blind
guessing
Partial knowledge – make an informed guess
Higher than 50-50 chance of success
 Measures of partial knowledge



Full knowledge – 100% chance of answering
correctly
Percentages of correct answers go down with
the number of blind guesses made
Suggestions for Getting Started
Properly




Create pairs of items, one true and one false
Rephrase evaluative judgments into a
comparative statement
Write false statements that reflect the actual
misconceptions held by students
Convert a multiple-choice item into two or more
true-false items
Suggestions for Improving
True-False Items




Importance of what is assessed
How does it match the test blueprint
Assess important ideas, rather than trivia,
general knowledge, or common sense
Make sure it is either definitely true or definitely
false
Use short statements when possible
 Use exact language
 Use positive statements and avoid double negatives

Suggestions for Improving
True-False Items






Avoid copying sentence verbatim
True and false statements should have approximately
the same number of words
Don’t present in a repetitive or easily learned pattern
Do not use verbal clues that give away the answer
Attribute the opinion in a statement to an appropriate
source
Focus on one idea
Crafting Multiple True-False Items




Looks like a multiple-choice item – followed by
several alternatives
No single correct or best answer
Response to each alternative is true or false
Each alternative is scored correct or incorrect
Crafting Multiple True-False Items

Advantages
Two or three multiple true-false responses in the
time it takes to make one multiple-choice response
 Multiple true-false tests created from multiple-choice
items have higher reliability than original multiplechoice
 Can assess same abilities as straight multiple-choice
items that are crafted to assess parallel content

Crafting Multiple True-False Items

Advantages
Students believe multiple true-false items do a better
job of assessing knowledge than multiple-choice
items
 Students perceive them to be harder
 May be easier to write – not limited to one correct
answer


Limitations

Shares some of same limitations as multiple-choice
Works Cited

Nitko, A. J., (2004). Educational Assessment of
Students, 4th ed. Upper Saddle River, NJ:
Pearson Education, Inc.