Transcript Test-Making Strategies

Oscar Vergara
Chihlee Institute of Technology
July 28, 2014
About Me

 Lecturer at CIT since 2012
 More than 10 years ESL/EFL teaching experience
 Co-author of a variety of EFL textbooks & test data
banks
 Certified IELTS examiner
Today’s Content

 Some Guiding Principles
 Multiple Choice Test Questions
 Workshop Activity: Suggestions for Creating Good
Tests
Q&A
Some Guiding Principles

 Clarity of questions & instructions
 Reliability & Validity
 Fairness
Clarity

 Instructions
 Clearly state what is required
 Use simple language / unambiguous
 Questions / Stems
 Content or structure should not prevent an informed
student from answering correctly
 Don’t include distracting or unnecessary details
Reliability & Validity

 Do the exam questions measure what they purport
to measure?
 Do the questions test your course goals?
 Does the exam accurately reflect the achievement of
what you intended to teach?
 Classes may differ, so different versions may be
needed.
Fairness

 Provide clear expectations about student
performance
 Provide examples / practice with mock tests or past
tests
 Students should know expectations of how their grade
on tests reflects their skills (not extraneous factors)
Guiding Principles

 Difficult to achieve due to:
 Demands on time
 Grading resources
 Require many versions
Guiding Principles:
Bloom’s Taxonomy

 Taxonomy of Educational Objectives
 Specifies different abilities and behaviors related to
thinking processes
 Contains 6 distinct categories
- Assess, criticize,
justify
- Formulate or
modify ideas
- Distinguish or
differentiate
between ideas
- Rote memorization
- Recall
Evaluation
Synthesis
Analysis
Application
- Apply concepts
to situations
- Interpret
information
- Test questions
on facts, rules &
principles
Comprehension
Knowledge
Guiding Principles

 Knowledge: Remember
 Knowledge of terms and concepts
 Recall of information
 Comprehension: Understand
 Comprehension including translating, summarizing,
demonstrating, discussing
 Application: Apply
 Apply what was learned
 Use problem-solving methods
Worksheet

 Please work in pairs or small groups
 Do only part A
 Discuss
Multiple choice

 Made up of a single question called a stem
 Many possible choices with one correct answer
 Several incorrect answers called distractors
 Distractors are plausible but not possible
How to Write Good
Multiple Choice Stems

 Match your learning goals
 Appropriate level of difficulty
 Be aware of common errors
Worksheet: Group
Activity

 Please work in groups of 3 or 4
 Do part B
 There is no single correct answer; only note what
you think could be improved
Worksheet Part A:
Suggestions

1. More than one possibility; change (d) to different
word formation
2. Double negative is confusing
3. Answers should all be of a similar length; (d) is too
long and obvious
4. Too complex; time-consuming; frustrates some testtakers; item value
Worksheet Part A:
Suggestions

5. Grammar can give away answer; always use a(n) as
necessary
6. Negatives should be emphasized; ie – NOT,
EXCEPT, etc.
7. Too many blanks; item value; misspelled or
nonsensical/ non-existent words
8. Avoid absolutes including All, None or more than
one answer
Multiple Choice:
Summary

What to avoid in the
stem
 Long complex
sentences
 Negatives / Double
negatives
 Unintentional clues
What to use in the stem
 Your own words (if
possible)
 Single idea & clearly
formulated question
Multiple Choice:
Summary

What to avoid in the
choices
 Statements too close to
being correct
 Completely implausible
answers
 Absolute answers (ie –
All of the above)
What to use in the
choices
 Plausible &
homogeneous
distractors
 Same option lengths
 True statements that do
not answer the question
 Answers distributed
evenly
True / False

 Composed only of statements with two possible
answers
 Assess familiarity with course content and general
misconceptions
 Test a range of broad concepts and can quickly
respond
 Easy to grade, but time-consuming to create
True / False

What to avoid
 Negatives / Double
negatives
 Long complex
sentences
 Ambiguous or trivial
material
What to use
 Your own words
 50/50 or 60/40 in favor
of false (students more
likely to answer true)
 One idea per item
Matching

 Contains equal number of stems and choices
 Assess recognition and recall
 Important if acquisition of detailed knowledge is a
learning goal
 Easy to grade, but students may require more time
than equal number of m/c or t/f
Matching

What to avoid
What to use
 Long stems and options
 Short responses; 10 to
15 items per page
 Heterogeneous content
(ie – testing grammar
and vocabulary)
 Clear directions
 Implausible responses
 Ordered choices (ie –
alphabetical or
chronological)
Q&A

References

 Bloom, Benjamin S. Taxonomy of Educational
Objectives (1956). Published by Allyn and Bacon,
Boston, MA. Copyright (c) 1984 by Pearson
Education.
 Airasian, Peter W.; Cruikshank, Kathleen A.; Mayer,
Richard E.; Pintrich, Paul R.; Raths, James; Wittrock,
Merlin C. (2000). Anderson, Lorin W.; Krathwohl,
David R., eds. A taxonomy for learning, teaching, and
assessing: A revision of Bloom's taxonomy of educational
objectives. Allyn and Bacon. ISBN 978-0-8013-1903-7.
Thank you!

E-mail:
[email protected]