Transcript Testing your

Assessing Student Knowledge:
Going to the Next Level
Knowledge Network of Experiences
 Social Studies Idea
Meaningful Social Studies instruction,
“enables individuals to personally explain
relationships and decide how to encounter social
events in ways that are consistent with their values
and those of their culture…memorizing facts does
not encourage students to find meaning in what
they are learning, nor does it help them make and
test decisions.”
Sunal & Hass. A Constructivist Approach to Social Studies, 2005
How to Evaluate and Create
Quality Assessment Items

This training will introduce the basics of
aligning and evaluating assessment items.
Using Blooms II and the research of
James Popham, Hilda Taba and others,
participants will learn how to align
knowledge and cognitive levels and use
them to identify and develop quality
assessment items.
A Note about Test Banks
 Not
all test items are created equally
 Socials Studies textbook test banks
are predominantly (90%+)made up
of lower level questions.
– Why?
 Most
nationally produced banks are
include a majority of lower level
questions
Which of the following cognitive levels
is best aligned to the learning
objective below.
6.9B Identify
Identify problems and issues that may arise when
one or more of the assessments that students receive is
not aligned to the learning objective and explain
explain how this
might effect a student’s academic success.
A. Remember
B. Understand
C. Apply
D. Analyze
A Taxonomy for Learning, Teaching and Assessing
Lorin Anderson and David Krathwohl
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
2
Understand
3
Apply
4
Analyze
5
Evaluate
6
Create
Assessment Alignment
Read the TEKS SEs and Plot it on
the taxonomy chart
 Read the corresponding
question, plot it, and then decide
whether or not the assessment
item is aligned to the TEKS SE.
1.6A Identify and describe the physical characteristics of
places such as landforms, bodies of water, natural
resources, and weather
The Knowledge
Dimension
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
The Cognitive Process Dimension
1
Remember
2
Understand
Identify
and
describe
Identify
and
describe
Physical
Charac.
3
Apply
4
Analyze
5
Evaluate
6
Create
1st Grade Assessment Item
The picture shows a –
1. Plain
2. Valley
3. Mountain
4. Forest
1.6A Identify and describe the physical characteristics of
places such as landforms, bodies of water, natural
resources, and weather
The Knowledge
Dimension
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
The Cognitive Process Dimension
1
Remember
2
Understand
Identify
and
describe
Identify
and
describe
ITEM
Physical
Charac.
3
Apply
4
Analyze
5
Evaluate
6
Create
3.2B Compare ways in which people in the local community
and communities around the world meet their needs for
government, education, communication, transportation, and
recreation, over time and in the present
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
2
Understand
Compare
Need
Gov, Ed.
Com., etc.
3
Apply
4
Analyze
Compare
5
Evaluate
6
Create
3rd Grade Assessment Item
 In
1900, people traveling far from
 home could stay in touch with friends
 using a __________, but after 1980,
 they could use a __________.
 A. computer; pay phone
 B. newspaper; telegram
 C. radio; walkie-talkie
 D. telegram; cell phone
3.2B Compare ways in which people in the local community
and communities around the world meet their needs for
government, education, communication, transportation, and
recreation, over time and in the present
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
2
Understand
3
Apply
Compare
Compare
ITEM
Need
Gov, Ed.
Com., etc.
4
Analyze
5
Evaluate
6
Create
5.6B Translate geographic data into a variety of
formats such as raw data to graphs and maps.
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
2
Understand
3
Apply
4
Analyze
Geographic
Data
Translate
Translate
Geographic
Data
Translate
5
Evaluate
6
Create
5th Grade Assessment Item
The elevation at the western edge of the Caspian Sea
is below sea level. The elevation of the Himalayas is
4000 meters and above. Which map indicates this
information?
A. I
B. II
C. III
D. IV
5.6B Translate geographic data into a variety of
formats such as raw data to graphs and maps.
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
A.
Factual Knowledge
2
Understand
3
Apply
4
Analyze
Geographic
Data
Translate
Translate
ITEM
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
Geographic
Data
Translate
5
Evaluate
6
Create
6.3(B. pose and answer questions about geographic
distributions and patterns for selected world regions
and countries shown on maps, graphs, charts, models,
and databases
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
2
Understand
3
Apply
4
Analyze
A.
Factual Knowledge
Understand
data
B.
Conceptual
Knowledge
Geographic Pose
Geographic
distributions questions distributions
- regions
- regions
C.
Procedural
Knowledge
Map, chart
skills
D.
Metacognitive
Knowledge
Pose
questions
5
Evaluate
6
Create
Pose
questions
6th Grade Assessment Item
Which of the following is a
question a geographer might ask
about tea production in Sri Lanka
to find information that is not on
the map?
A. How many oceans surround Sri
Lanka?
B. What other crops are grown in
Sri Lanka?
C. How far is Sri Lanka from the
equator?
D. What are the natural features of
central Sri Lanka?
6.3(B. pose and answer questions about geographic
distributions and patterns for selected world regions
and countries shown on maps, graphs, charts, models,
and databases
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
2
Understand
3
Apply
4
Analyze
A.
Factual Knowledge
Understand
data
B.
Conceptual
Knowledge
Geographic Pose
Geographic
distributions questions distributions
- regions
- regions
C.
Procedural
Knowledge
Map, chart
skills
D.
Metacognitive
Knowledge
5
Evaluate
6
Create
ITEM
Pose
questions
Pose
questions
8.4 A analyze causes of the American Revolution, including
mercantilism and British economic policies following the
French and Indian War
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
2
Understand
3
Apply
4
Analyze
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
Analyze
causes:
mercantilism
5
Evaluate
6
Create
8th Grade Assessment Item
Which of the following was NOT part of the
mercantilist policy of England?
A. Regulating colonial trade by taxation
B. Requiring colonial businesses to trade
with England
C. Increasing exports of manufactured
goods to the colonies
D. Decreasing use of natural resources from
the colonies
8.4 A analyze causes of the American Revolution, including
mercantilism and British economic policies following the
French and Indian War
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
2
Understand
3
Apply
4
Analyze
ITEM
Analyze
causes:
mercantilism
5
Evaluate
6
Create
WH1(D. explain the significance of the following
dates: 1066, 1215, 1492, 1789, 1914-1918, and
1939-1945.
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
2
Understand
3
Apply
Explain
significance
of date
Sig. of
dateassess
4
Analyze
5
Evaluate
6
Create
World History Assessment Item
The events of 1914 to 1918 marked the first
time in history that —
A. the Western European nations fought
as allies
B. nations from five continents fought in
a single conflict
C. the United States took part in a war
outside its territory
D. nations forced men to fight by setting
up military drafts
WH1(D. explain the significance of the following
dates: 1066, 1215, 1492, 1789, 1914-1918, and
1939-1945.
The Knowledge
Dimension
The Cognitive Process Dimension
1
Remember
A.
Factual Knowledge
B.
Conceptual
Knowledge
C.
Procedural Knowledge
D.
Metacognitive
Knowledge
2
Understand
3
Apply
Explain
significance
of date
Sig. of
dateassess
ITEM
4
Analyze
5
Evaluate
6
Create
Learning Objective: Blooms II
 It
is not important which taxonomy
that you use but that you use one
 If you do not know the knowledge
and cognitive levels of your learning
objective then you cannot effectively
align your instruction, assessment
and questioning to it
L. Anderson, D. Krathwohl, Taxonomy for Learning, Teaching, and Assessing,
A: A Revision of Bloom's Taxonomy of Educational Objectives, (2001..
Break
Determining Difficulty
Level
Determining Difficulty Level
 There
are two different difficulty
levels that we are looking at with any
assessment item:
 Cognitive Difficulty
 Procedural Difficulty
Determining Difficulty Level
Cognitive Difficulty
This is basically: How hard is
question based on Blooms
Taxonomy?
No question on a test should be
above the level of the learning
objective (TEKS SE) **
It is OK to have questions that are
lower
**Unless you have a stated learning objective
that has been clearly expressed to the
students and taught.
Determining Difficulty Level
Cognitive Difficulty

EASY
– Remember
– Understand

MEDIUM
– Apply
– Analyze

HARD
– Evaluate
– Create
Determining Difficulty Level
Procedural Difficulty
This is basically: How many mental
processing steps does the student
have to go through to answer the
question?
The greater the number of
processing steps the higher the
difficulty level
Determining Difficulty Level
Procedural Difficulty
 EASY
– The item includes only the stem and the answer choices

MEDIUM
– The item includes a graphic, short reading selection,
map etc. (Stimulus piece) The student only has to
interpret the stimulus or pull information from it to
select the correct answer.

HARD
– The item includes a graphic, short reading selection,
map etc. (Stimulus piece) The student has to infer,
analyze, summarize, etc. and apply that to the stem or
answer choices to select the correct answer.
What does the student have to “do”
to answer the question?
What does the student have to “do”
to answer the question?
What
does the
student
have to
“do” to
answer
the
question?
Specificity Difficulty

We DO NOT want to use specificity to increase the
difficulty level of the question!
5.2(A) identify the contributions of significant individuals during the revolutionary
period, including Thomas Jefferson and George Washington;
George Washington was in charge of which army during
the American Revolution?
 American, German, Chinese, Russian
 Revolutionary, Continental, Virginia Regulars, American
George Washington defeated the British in which battle
that ended the Revolutionary War?
 Yorktown, Saratoga, Trenton, Cowpens
George Washington credited some of his ability to win the
battle of Yorktown to his horse  Faithful, Liberty, Constitution, Magnolia
Specificity Difficulty
 Choices
and Distracters
5.2(A) identify the contributions of significant individuals during the
revolutionary period, including Thomas Jefferson and George Washington;
Thomas Jefferson wrote which important historical
document  Declaration of Independence, Civil Rights Act of
1964, Bible, Quran


Declaration of Independence, Constitution,
Mayflower Compact, Gettysburg Address
Wall of separation letter, Manual of Military
Practice, Constitution for Virginia, Virginia
Resolutions of 1798
How to Evaluate and Create
Quality Assessment Items
 Questions
Question Alignment Application
 Move
to your grade level group
 Read the selected student
expectation
 Create at least two appropriate
questions that target the level of
thinking required by the TEKS
 Create one lower level question
 Create one question at a higher level
than the TEKS SE
Processing Info
Each grade level will share
their work
Social Studies
Item Review Process


Paper work:
Read and fill out Oath of Test
Security Form and return it to the
coordinator.
Item Review:

1.
2.
Make any corrections or
comments directly on the test.
Check to see if the item is properly
aligned to COL for the units
covered during this benchmark
period. (Pacing Guide)
Check for content alignment
Item Review:
3.
4.
Check to make sure that the item
TEKS SE is on the Blueprint
Check to see if the item is aligned
to the TEKS SE.
a. Check for content alignment
i. If the TEKS SE is focused on content then
be sure that the item tests that particular
content.
Item Review:
4. Check to see if the item is aligned to the
TEKS SE.
b) Check for alignment to the “verb” in the SE.
i.
The item can test at a higher level than the verb,
but not a lower level.
c) Check for skill alignment
i.
If the TEKS SE is asking for a specific skill to be
used make sure that that skill is what is being
tested.
ii. Some items will test both skills and content as is
the case on TAKS. Again, check for alignment to
the SE.
Item Review:
5.
6.
7.
Check for correct answer on key.
(Last Page)
Check for “Level” on the answer
key. (If you have question please
let me know.
Check for typos and grammar
problems.
Item Review:
8.
9.
10.
Check for “unreadable” graphics.
Check to make sure that graphics
and reading selections are at the
appropriate level.
Note: Formatting for all tests will
follow TAKS social studies based on
the 8th grade test.
Four Keys to Quality
Student Assessment
Key 1
Key 2
Arise from clear and
appropriate learning
targets
Serve focused and
appropriate purposes
Key 3
Based on clearly defined
performance
objective/descriptor
Key 4
Are clear, concise and
eliminate sources of bias
and distortion
Four Keys to Quality
Student Assessment
Key 1
Arise from clear and
appropriate learning
targets
The TEKS: Clarification
needed where it is not present.
Student Expectation
Clarification
6.4D identify and explain the
Include location of natural
geographic factors responsible resources, trade
for the location of economic
routes/transportation,
activities in places and regions. physical features, etc.
Four Keys to Quality
Student Assessment
Key 2
Serve focused and
appropriate purposes
The test item assesses
information that is important
for the student to understand:
TAKS test, local benchmark,
teacher benchmark, etc.
Four Keys to Quality
Student Assessment
Key 3
Based on clearly defined
performance
objective/descriptor
What does the TEK actually ask the
student to do?
Four Keys to Quality
Student Assessment
Key 3
Based on clearly defined
performance
objective/descriptor
History. The student
understands traditional
historical points of
reference in US history
through 1877. The
student is expected to
A. Identify major eras in US
history through 1877 and
describe their defining
characteristics:
Given a list of events,
people, or period by their
distinguishing characteristics
the student will sequence or
choose the correct sequence
Four Keys to Quality
Student Assessment
Key 4
Eliminate sources
of bias and
distortion
Evaluating Tests-Overview
1.
Do the items cover a range of
understanding for each TEKS SE?
2. Do the items reflect the weight given to
various TEKS? (TAKS/Benchmark
Blueprint)
Evaluating Tests-Overview
TAKS BLUEPRINT FOR GRADE 8 SOCIAL STUDIES
TAKS
Objectives
Number of Items
Objective 1—History ……………………………13
Objective 2—Geography ………………………...6
Objective 3—Economic and Social Influences .9
Objective 4—Political Influences ……………….12
Objective 5—Social Studies Skills ………………8
Total number of items …………………………48
TAKS Objectives
November 9-15
March 2-8
April 21
State TAKS Test
State TAKS
Blueprint (Grade
8.
BENCHMARK
BENCHMARK
Objective 1 –
History
13 questions
*All T3
7 questions
5 questions
7.1C, 7.2B, 7.2C,
7.2D, 7.2F
7.1C, 7.4A, 7.4B,
7.5B, 7.13B, 7.19B
10 questions
5 questions
7.8B 7.9A, 7.9B,
7.9C, 7.10A, 7.10B,
7.8A, 7.9A, 7.9B,
7.10A, 7.10B,
0 questions
1 question
Objective 2 –
Geography
6 questions
*All T3
Objective 3 –
Economic and
Social Influences
9 questions
*All T3
Objective 4 –
Political Influences
12 questions
*All T3
7.5B
0 questions
6 questions
7.5A, 7.14A, 7.14B,
7.16A
Objective 5 – Social
Studies Skills
Total number
of items
8 questions
*All T3
48 questions
8 questions
8 questions
7.21A, 7.21B,
7.21C, 7.21H
7.17A, 7.21A, 7.21B,
7.21C, 7.21E, 7.21H
25
25
Evaluating Tests-Overview
3.
Are the items easy to understand?
4.
Do test items ask for more than recall?
5. Do the items contain superfluous
information or art?
Evaluating Tests-Overview
6. Do items provide test-wise students
with an advantage?



Opposites as answer choices
One option much longer, shorter, or
more complex than others
Clues to answer in the item stem
Evaluating Tests-Overview
7. Do items favor students who are
good readers?



Negatives in stems or options
Answers including “all of the above”
or “none of the above”
Specific determiners such as “all”
The fatal pedagogical
error…
to throw answers like
stones at the heads of
those who have not yet
asked the questions.
Paul Tillich
Graphics
TAKS requires the use of more
graphics!
Is the graphic that is used aligned to the
performance descriptor?
Is the graphic clear (readable.?
Is the graphic at the appropriate level?
Do you use a variety of graphics?
Maps-http://www.nationalgeographic.com/siteindex/i
ndex.html#maps
Multiple Choice Questions
Multiple choice items consist of a direct
question or incomplete statement called the
stem.
 This is followed by a list of choices or the
options.
 The options consist of the correct option (the
answer. and the distracters.
 The distracters are plausible competitors of
the answer.
 The student must select the appropriate
response to the stem using the options given.
Multiple Choice Questions
I.
Be Clear and Concise.
You can't overstate the importance of
clarity. Think of this as a rule, not a
guideline.
BE CLEAR.
Multiple Choice Questions
Be Clear and Concise. (Cont.)

Without sacrificing clarity, be as concise as
possible. Keep a sense of focus.

Remember that your purpose is to measure
your students' knowledge, reasoning, and
ability.

The idea is to discriminate levels of
understanding, not to trap the unwary.
Multiple Choice Questions
I.
Be Clear and Concise (Continued)

Simply stated: It is not sufficient to write a
question that can be understood.
To paraphrase the advice of Gen. Douglas
MacArthur to the Corps of Cadets at West
Point:
Write questions that cannot be
misunderstood, not merely questions that
can be understood.


Multiple Choice Questions
II. Writing Multiple Choice Stems
A. The multiple choice stem needs to
present one problem to the student
before the options are considered.
Multiple Choice Questions
Example of a faulty item:
The Spanish Civil War
A. broke out in June 1931 and lasted until April
1936.
B. was fought between the upper class and the
working class.
C. was caused by the collapse of Rivera's military
dictatorship.
D. was a religious war to convert the country to
Catholicism.
Multiple Choice Questions
Example of a sound item:
Which statement describes an immediate effect of
the Spanish Civil War?
A. Spain was divided into two ideological groups.
B. The government banned the Catholic church.
C. Workers gained political power in the
government.
D. Spain adopted a democratic form of
government.
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
The Supreme Court
A. was established by the Judiciary Act of 1789.
B. first contained a Chief Justice and eight
Associate Justices.
C. was first called to assemble on Feb. 1, 1790.
D. as an organization is detailed in Article III of
the US Constitution
Multiple Choice Questions
B. Avoid writing stems that contain extra
information such as an introduction to the
question. The stem should only include the
question or the incomplete statement.
Multiple Choice Questions
Example of a faulty item:
The "new kid on the retirement block" is called the
Roth IRA. It was introduced as a vehicle for
retirement investments in the 1997 Taxpayer
Relief Act. What is the special feature of a Roth
IRA?
A. There is no dollar limit on contributions.
B. The withdrawals are tax-free.
C. There is no penalty for early withdrawal.
D. The contributions are tax-deductible.
Multiple Choice Questions
Example of a sound item:
What is the special feature of a Roth IRA?
A. There is no dollar limit on contributions.
B. The withdrawals are tax-free.
C. There is no penalty for early withdrawal.
D. The contributions are tax-deductible.
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
The National Park System now manages 379 parks throughout
the United States. Which of the following is a correct fact
about the history of the National Park Service?
A. The United States Department of the Interior was
responsible for parks and monuments before the National
Park Service.
B. Mesa Verde was the world's first area to be designated as a
national park.
C. President Franklin Roosevelt approved legislation to create
the National Park Service.
D. George Marks is credited with the idea of establishing a
national park service
Multiple Choice Questions
C. Include as much information in the stem
and as little information in the options as
possible.
Multiple Choice Questions
Example of a faulty item:
Hawaii:
A. is a group of 30 volcanic islands in the
Pacific Ocean that contains one of the largest
active volcanoes in the world.
B. contains the volcano called Mauna Loa that
is located on the Hawaiian island called
Kahoolawe.
C. entered the Union in the year of 1959 along
with Alaska and these were the last two
states to enter the Union.
D. has a state flower that is the gardenia and its
state flag has an orange blossom drawn on it.
Multiple Choice Questions
Example of a sound item:
Which state was the last one to be admitted
to the United States?
A. Alaska
B. New Mexico
C. Hawaii
D. Arizona
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Utah:
A. has largely been shaped by the ethnic and
cultural influence of Mormons.
B. has largely been shaped by the ethnic and
cultural influence of Hispanics.
C. has largely been shaped by the ethnic and
cultural influence of Native Americans.
D. has largely been shaped by the ethnic and
cultural influence of Nordic peoples.
Multiple Choice Questions
D. Restrict the use of negative terms in the
stem.
Multiple Choice Questions
Example of a faulty item:
Which of the following reasons is not a main
cause of the Civil War?
A. Political differences
B. Economic differences
C. Cultural differences
D. Landscape differences
Multiple Choice Questions
Example of a sound item:
Which of the following reasons was a main
cause of the Civil War?
A. Political differences
B. Ethnic differences
C. Language differences
D. Landscape differences
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
What is a health and welfare program that is not
administered by state governments?
A. Medicaid
B. Temporary Assistance to Needy Families
C. Social Security
D. Unemployment Compensation
Multiple Choice Questions
E. Construct stems that require the
selection of the best answer when each of
the options contain elements of
correctness.
Multiple Choice Questions
Example of a faulty item:
Which of the themes of geography listed
below relates to cultural diffusion?
A. Region
B. Movement
C. Human/Environment Interaction
D. Location
Multiple Choice Questions
Example of a sound item:
Which of the themes of geography listed
below relates most closely to the
concept of cultural diffusion?
A. Region
B. Movement
C. Human/Environment Interaction
D. Location
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
What is one of the roles delegated specifically to
the office of the President of the United States?
A. answering 911 emergency calls
B. being the pilot of Air Force One
C. inviting celebrities to the White House
D. nominate cabinet members
Multiple Choice Questions
III. Use the Active Voice.
Everybody processes information much
more easily in the active voice.
Various research studies have confirmed
this. (Olsen and Tichy.
Multiple Choice Questions
III. Use the Active Voice.
In sentences written in active voice, the subject
performs the action expressed in the verb; the subject
acts.
 The dog bit the boy.
 The boy was bitten by the dog.
 Historians researched the project before presenting it.
 The project was researched by Historians before
presenting it.
 Teachers will write good items because they care.
 Good items will be written by teachers, because they care.
Multiple Choice Questions
IV. Focus each test question on a specific
objective of the instructional unit.
Multiple Choice Questions
7.5B The student will analyze the political,
economic, and social effects of the Civil War and
Reconstruction in Texas.
Which of the following was a governor of Texas
during the Civil War?
A. Sam Houston
B. Francis Lubbock
C. John A. Wharton
D. Jefferson Davis
Multiple Choice Questions
Example of a sound item:
Which of the following was a political effect of the
Civil War on Texas?
A. Lack of a civil or military force to keep order
B. Increased amount of money flowing to the state
C. Lack of leadership by the governor
D. Lack of leadership by the congress
Multiple Choice Questions
V. Answer Choices
A. Each distracter should have about the
same number of words as the correct
option.
Multiple Choice Questions
Example of a faulty item:
Abraham Lincoln is famous for:
A. the years he served in the Illinois legislature.
B. issuing the Emancipation Proclamation on
January 1, 1863 stating that all persons held as
slaves in the United States shall be free and the
executive government will recognize all such
persons as free.
C. his law practice in Springfield, Illinois.
D. serving as captain in the Black Hawk War.
Multiple Choice Questions
Example of a sound item:
Abraham Lincoln is most famous for:
A. the years he served in the Illinois legislature.
B. issuing the Emancipation Proclamation.
C. his law practice in Springfield, Illinois.
D. serving as captain in the Black Hawk War.
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
William Barret Travis is known
A. for abandoning his wife
B. for buying land from Stephen F. Austin
C. as the colonel who was born in Saluda Co., SC,
and became the Texas commander at the Battle
of the Alamo
D. for being a newspaper publisher.
Multiple Choice Questions
B. Does the item have one clearly correct
answer?
Abraham Lincoln was president of the
United States during:
A. A time of strife
B. The Revolutionary War
C. A time of expansion
D. The Civil War
Multiple Choice Questions
C. Make all distracters plausible.
Multiple Choice Questions
Example of a faulty item:
Abraham Lincoln is famous for:
A. being the first man to walk on the moon.
B. issuing the Emancipation Proclamation.
C. his role in the musical Les Miserables.
D. inventing the electric light bulb.
Multiple Choice Questions
Abraham Lincoln is most famous for:
A. the years he served in the Illinois
legislature.
B. issuing the Emancipation Proclamation.
C. his law practice in Springfield, Illinois.
D. serving as captain in the Black Hawk
War.
Multiple Choice Questions
D. Do not repeat wording or common
elements from the stem in the correct
option.
Multiple Choice Questions
Example of a faulty item:
Which of the following occurred during the
Children's Crusade?
A. Crusaders loot Constantinople.
B. Crusaders take Antioch, then Jerusalem.
C. Crusaders win coast from Tyre to Jaffa.
D. Thousands of innocent children march to death
and slavery in North Africa
Multiple Choice Questions
Example of a sound item:
Which of the following events occurred
during the Fourth Crusade?
A. Constantinople was looted.
B. Antioch was defeated, then Jerusalem.
C. The entire coast from Tyre to Jaffa was
defeated.
D. Anatolia defeated the invaders.
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
Who designed the Eiffel Tower?
A. Gustave Eiffel
B. Adolf Loos
C. Frei Otto
D. Eero Saarinen
Multiple Choice Questions
E. To increase item difficulty include "none of
the above" as a final option.
Multiple Choice Questions
Example of a sound item:
Which of the following relationships describes the
relationship between Spanish and Anglo
settlements of Texas?
A. friendship
B. cooperation
C. hatred
D. None of the above
Multiple Choice Questions
F. When a list of choices is included in each
option increase the similarity of all the
answer choices to increase the difficulty of
the test.
Multiple Choice Questions
Example of a faulty item:
Who were the leaders of the Axis Powers
during World War II?
A. Grant, Adams, Carter
B. Michelangelo, Cortes, Einstein
C. Curie, Columbus, MacArthur
D. Hirohito, Hitler, Mussolini
Multiple Choice Questions
Example of a sound item:
Who were the leaders of the Axis Powers
during World War II?
A. Hirohito, Hitler, Stalin
B. Hitler, Stalin, Mussolini
C. Hirohito, Mussolini, Stalin
D. Hirohito, Hitler, Mussolini
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
The atomic bombs in World War II were dropped
on these two cities:
A. Washington and Fukuoka
B. Nagoya and Cairo
C. Hiroshima and Nagasaki
D. Kyoto and London
Multiple Choice Questions
VI. Tap Higher Level Cognitive Domains
Rote memorization of facts, laws, and
definitions has its place in the overall
scheme. However, at least 80-90% of the
test should be devoted to higher levels of
cognition.
Multiple Choice Questions
A. Watch Difficulty Levels
The ideal question will be answered
correctly by 60-65% of the tested
population. This level of difficulty
maximizes discrimination on exams.
Multiple Choice Questions
B. Avoid word choices that are above the
student's vocabulary level.
Multiple Choice Questions
Example of a faulty item:
If a numismatist was alive 4000 years ago,
what would he be looking for?
A. Wampum
B. Tally sticks
C. Cowrie shells
D. Manilla
Multiple Choice Questions
Example of a sound item:
What is the form of money that was used in
ancient Egypt?
A. Wampum
B. Tally sticks
C. Cowrie shells
D. Manilla
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Phrenologist suggested which of the following?
A. horses were faster than trains
B. whites were more intelligent than blacks
C. space travel could not happen
D. the theory of evolution
Multiple Choice Questions
VII. Avoid constructing negative options that
follow a negative stem.
Multiple Choice Questions
Example of a faulty item:
Which of the following characteristics is not
among the attributes of an
Human/Environment Interaction?
A. Lack of buildings.
B. Lack of pollution.
C. Lack of rivers.
D. Lack of recycling.
Multiple Choice Questions
Example of a sound item:
Which of the following characteristics is not
among the attributes of an
Human/Environment Interaction?
A. Buildings.
B. Pollution.
C. Rivers.
D. Recycling.
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
What is not a general characteristic of wars?
A. lack of shortages
B. lack of disease
C. not poor economic times for families
D. not loss of rights
Multiple Choice Questions
VIII.
Clear and Concise
A. Make sure the stem is grammatically
consistent with all of the options.
Multiple Choice Questions
Example of a faulty item:
Movement is an example of an:
A. application.
B. theme.
C. diffusion.
D. location.
Multiple Choice Questions
Example of a sound item:
Movement is an example ofA. application
B. theme
C. diffusion
D. location
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
An example of diffusion is anA. Jesuit church opening in Spain
B. McDonalds opening in Detroit
C. Hispanic song being played in Texas
D. Tex-Mex meal being eaten in the Congo
Multiple Choice Questions
B. Carefully proofread items.
Multiple Choice Questions
Example of a faulty item:
What is Hernan Cortes famous for?
A. He captured Mexico for Spain.
B. He explored Centrol America.
C. He explores the Amazon River.
D. He conqered the Incas for Spain.
Multiple Choice Questions
Example of a sound item:
What is Hernan Cortes famous for?
A. He captured Mexico for Spain.
B. He explored Central America.
C. He explored the Amazon River.
D. He conquered the Incas for Spain.
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
A word that modifies a noun is called:
A. an adverd
B. a conjunction
C. a verrb
D. an adjective
Multiple Choice Questions
C. Avoid using overlapping distracters.
Multiple Choice Questions
Example of a faulty item:
A person in the first tax bracket makes less
than:
A. $125,000
B. $100,000
C. $75,000
D. $50,000
Multiple Choice Questions
Example of a sound item:
A person in the third tax bracket makes:
A. $14,651-$24,650
B. $24,651-$59,750
C. $59,751-$124,650
D. $124,651-$271,050
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
Midnight occurs at what time of day?
A. evening
B. night
C. 12 PM
D. 12 AM
Multiple Choice Questions
D. Avoid the use of indefinite terms such as,
“would”, “could”, “usually” and “generally”
in the options and distracters.
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
The length of a senator's single term is:
A. 2 years
B. 3 years
C. Usually 4 years
D. Generally 6 years
Multiple Choice Questions
E. Avoid the use of absolute terms such as
“never” and “always” in the options and
distracters.
Multiple Choice Questions
Example of a faulty item:
Which of the following is a sound investment
principle?
A. Keep emergency money funds in highly
liquid assets.
B. Mutual funds are always the safest
investment vehicle.
C. Social security is a risk free income source
for retirement.
D. Pay only the minimum balance on the credit
card each month.
Multiple Choice Questions
Example of a good item:
Which of the following is a sound investment
principle?
A. Social security is a risk free income source for
retirement.
B. Mutual funds are the most growth oriented
investment vehicle.
C. Keep emergency money funds in highly liquid
assets.
D. Pay only the minimum balance on the credit
card each month.
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
An earthquake has the following property:
A. it always causes tidal waves.
B. it always cracks the earth.
C. is caused by plate tectonics.
D. is never explosive.
Multiple Choice Questions
F. Avoid using "all of the above."
Multiple Choice Questions
Example of a faulty item:
Which of the following are examples of
physical features?
A. Mountains
B. Bays.
C. Glaciers
D. All of the above
Multiple Choice Questions
Example of a sound item:
Which of the following is an example of a
physical feature?
A. Pollution.
B. Bay.
C. Culture.
D. Location.
Multiple Choice Questions
Revise the following faulty item based upon the
above guidelines.
Example of a faulty item:
An event leading to the American Revolution was
A. the Stamp Act
B. the Sugar Act
C. the Tea Act
D. all of the above
Contact Information
John L. David
 972-466-6180
 [email protected]

Jennifer Carlisle
 972-466-6138
 [email protected]
