Transcript Mitchell Assessment Workshop

Writing Quality
Multiple Choice Questions
Facilitated By
Sara Fridley
Region 3 Education Service Agency
[email protected]
QUESTION – ANSWER
RELATIONSHIP
LITERAL
INFERENTIAL
Right
There
Think and
Search
Writer
and Me
On My
Own
Answer is in
the text,
usually in one
sentence
Answer is in
text, but not
all in one
place
Answer is in
text and in
reader’s head
Answer is
totally in
reader’s head
Types of Questions
What Are You Assessing?





Detail/recall
Reading
Comprehension
Vocabulary
Chart/graph analysis
Problem-solving




Inference
Predict
Author’s purpose
Cause & effect
Summative & Formative
Assessment

Assessment of Learning



(Summative Assessment)
How much have students learned as of a
particular point in time?
Assessment for Learning


(Formative Assessment)
How can we use assessments to help
students learn more?
Assessment of Learning

Purpose is




to measure student achievement at a point in time for
reporting and accountability
to sort students in rank order
to maximize student learning through standardized tests
Results used primarily by



policy makers & program planners
educators
Parents & students
Assessment for Learning

The purpose is





To promote further improvement of student learning during
the learning process
To involve students in ongoing assessment of their own
achievement
Identify student needs
Help students see and hit a target
Results are used primarily by


Educators – to adjust teaching
Parents & students
Which is your purpose?


OF learning
FOR learning
Disadvantages of
Multiple Choice Questions






Quality items are difficult and time consuming to develop
Tendency for items to focus on low level learning
objectives
No credit for partial knowledge
Assessment results may be biased by students’ reading
ability and test-wiseness
May overestimate learning due to the ability to utilize an
elimination process for answer selection
Does not measure the ability to organize and express
ideas
With that in mind . . .

Allows for assessment of a wide range of learning
objectives from factual to evaluative understanding
Amenable to item analysis
Permits manipulation of difficulty level by adjusting the
degree of similarity among response options
Objective nature limits bias in scoring
Easily administered to large numbers of students
Limits assessment bias caused by poor writing skills

AND it is the format used on high-stakes tests





General Guidelines for
Writing Questions

Keep wording simple and focused




Avoid clues within items



Grammatical clues
Determiners such as “always” and “never”
Don’t lift wording directly from materials


Follow rules of grammar
Eliminate superfluous material
Test mastery of material, not ability to figure out what you’re
asking
write in your own words
Highlight critical words such as MOST, LEAST, EXCEPT

they are easily overlooked
Parts of the MC Question

Stem



Correct Answer


Provides introductory information
Direct question or incomplete statement
One and ONLY ONE is preferable
Distractors

Incorrect response options
General Guidelines for
Multiple Choice Questions


State whole question in the stem
Keep responses brief and parallel in:







Length
Use of specific determiners
Grammatical construction
Generality
Avoid repetition in response options
Limit use of “all” or “none of the above”
Be sure there is only one correct or best answer

Best answer items should be worded to ask for BEST
More Guidelines

Negative premises are undesirable



The answer to one question should not be obtainable
from another question.
Avoid potential sources of bias & distortion




may especially disadvantage ESL (English Second Language)
students.
Wording
stimuli (graphics, pictures, charts)
Use at least 3 distractors to reduce the chance of
guessing the correct answer.
Avoid using a pattern for the position of the correct
response.
School Bus — Multiple Choice
A school is planning a field trip and will be using
school buses to transport the students. A
school bus holds 36 students. If 1,128
students are being transported, how many
buses are needed?
a. 31
c.
32
b. 31.33
d.
36
Sample Questions
The Good, The Bad, and The Ugly
Two cyclists start biking from a trail’s start 3 hours
apart. The second cyclist travels at 10 miles per
hour and starts 3 hours after the first cyclist who is
traveling at 6 miles per hour. How much time will
pass before the second cyclist catches up with the
first from the time the second cyclist started
biking?
A.
B.
C.
D.
E.
2 hours
7 ½ hours
4 hours, 30 minutes
6 hours
5 ¾ hours
Practice question from ACT Prep booklet
If a circle has the diameter of 8, what is the
circumference?
A.
B.
C.
D.
E.
6.28
12.56
25.13
50.24
100.48
Practice question from ACT Prep booklet
The wound exhibited signs of copious drainage
requiring medical intervention.
Copious means
A.
B.
C.
D.
Minimal
Clarity
Maximal
Foul
Practice question from ACT Prep booklet
When New York freed its slaves, Isabella had
A.
B.
C.
D.
no children
five children
an education
three children
The concept of the plasma membrane as a
simple sieve-like structure is not inadequate
to explain the
A. passage of gases involved in respiration
into and out of the cell
B. ability of the cell to admit selectively
some inorganic ions while excluding
others
C. Failure of protein molecules to pass
through the membrane
D. Inability of the cell to use starch without
prior digestion
Released test questions from the 2003 and 2004 California Standards Tests (CSTs).
The details about the princess’s life before her
eighteenth birthday imply that she was -
A. Raised to value only material goods
B. Indifferent to the feelings of others
C. Easily bored
D. Both A and C
E. Both B and C, but not A
Sample question from Standardized Test Practice
Purpose of Questions
What are you asking the
student to know?
Detail
Where was Susan’s dad stationed overseas?
A. In Albany, New York
B. In Adu Jahm, near Baghdad
C. Among ancient ruins in southern Iraq
D. East of Iraq, in Iran
Sample question from Scholastic Scope
Vocabulary –
Multiple Meanings
The subject of today’s lesson is words with multiple
meanings. In which sentence does the word subject
mean the same thing as in the sentence above?
A. Susan’s mother is subject to fainting spells.
B. In Kenya, high school students take 13 subjects at a time.
C. If you disobey traffic laws, you may be subject to arrest.
D. Feel free to change the subject of your report.
Sample question from SAT9 Activities (McDougall-Littell).
Inference Sample
Released test questions from the 2003 and 2004 California Standards Tests (CSTs).
Infer/Predict Sample
Released test questions from the 2005 Massachusetts Standards Tests (CSTs).
Inference
The details about the princess’s life before her
eighteenth birthday imply that she was A. Raised to value only material goods
B. Indifferent to the feelings of others
C. Easily bored
D. Not interested in getting married
Sample question from Standardized Test Practice
Comprehension
Sample
Released test questions from the 2003 and 2004 California Standards Tests (CSTs).
Author’s Purpose
Released test questions from the 2003 and 2004 California Standards Tests (CSTs).
Author’s Purpose
Released test questions from the 2003 and 2004 California Standards Tests (CSTs).
Application Sample
Released test questions from the 2005 Massachusetts Standards Tests (CSTs).
Critical Analysis
You can conclude from the passage that Epimetheus
A. did not want to know the contents of the box.
B. already knew what the box contained.
C. wanted to know what was in the box.
D. never did learn the contents of the box.
Sample question from SAT9 Activities (McDougall-Littell).
Critical Analysis
The poem suggests that John Henry dies from
A. shock.
B. a broken heart.
C. disappointment.
D. pushing himself too hard.
Sample question from SAT9 Activities (McDougall-Littell).
Reading Comprehension
Which of the following questions is answered in the
article?
A. What causes heart damage during prolonged space travel?
B. Why might space travelers develop kidney stones?
C. What can space travelers do to avoid becoming depressed?
D. What will the interior of a passenger space plane look like?
Sample question from Standardized Test Practice
Write Questions &
Peer Review --- 25 minutes

10-15 minutes



10-15 minutes





Write 4 questions
Check guidelines
Read passage
Answer questions
Check guidelines
Provide feedback
Edit questions if needed