Transcript Assessment - Characteristics of Good Testing.ppt

Are you the kind of
teacher who asks the
following questions?
The poem “The Raven” ______
a.
b.
c.
d.
was written by Edgar Allan Poe
was written by Elizabeth Browning
was written by Omar Khayyan
was written by Jose Garcia Villa
• Is it NOT true
that Magellan discovered the
Philippines?
When did the People Power
Revolution take in the
Philippines?
A.
B.
C.
D.
February 23, 1986
after the Snap Election
March 1, 1956
after Valentines’ Day in 1986
• Who was the author of the
book quoted in the
footnote of Chapter 1 of
the present textbook
If you answered “YES” to
any of the choices
presented then you have
a BIG PROBLEM!
YOUR STUDENTS ARE
NOT LEARNING!!!!
“13% of students who fail
in class are caused by
faulty test questions”
WORLDWATCH
The Philadelphia Trumpet
August 2005
• It is estimated that 90% of
all test questions asked in the
US are of “Low level” knowledge and comprehension
(Wilen, W.W., 1992)
• “Low level” doesn’t mean easy:
• Write an essay explaining the decline and
fall of the Roman Empire incorporating at
least five of the seven causes discussed in
class from the writings of Gibbon and
Toynbee
• “High level” doesn’t mean hard:
• Which movie did you like more,
WALL-E or Cars? Why?
TEST
CONSTRUCTION
Presented by:
Arnel O. Rivera
http://www.slideshare.net/ArnelSSI
Outline:
Part I
 Principles in Test
Construction
 Steps in Preparing
Test Questions
 Preparing Multiple
Choice Questions
 Preparing True or
False Questions
Part II
 Review of Part I
 Preparing Matching
Type Questions
 Preparing Sentence
Completion Questions
 Preparing Essay
Questions
 Other types of Test
Questions
 Wrap-up/
Things to Remember
“The evaluation of pupils’
progress is a major aspect of
the teacher’s job.“
Evaluating Educational Outcomes
(Oriondo & Antonio)
Explain the message of the comic strip.
The Purpose of Testing
To provide a record for assigning
grades.
To provide a learning experience for
students.
To motivate students to learn.
To serve as a guide for further study.
The Purpose of Testing
To assess how well students are
achieving the stated goals of the lesson.
To provide the instructor with an
opportunity to reinforce the stated
objectives and highlight what is
important for students to remember.
Characteristics of Good Tests
Validity – the extent to which the
test measures what it intends to
measure
Reliability – the consistency with
which a test measures what it is
supposed to measure
Usability – the test can be
administered with ease, clarity
and uniformity
Other Things to Consider
Scorability – easy to score
Interpretability – test results can
be properly interpreted and is a
major basis in making sound
educational decisions
Economical – the test can be
reused without compromising the
validity and reliability
“To be able to prepare a good
test, one has to have a
mastery of the subject
matter, knowledge of the
pupils to be tested, skill in
verbal expression and the
use of the different test
format”
Evaluating Educational Outcomes
(Oriondo & Antonio)
5 Most Commonly used
Test Format
1.
2.
3.
4.
5.
Multiple Choice
True or False
Matching Type
Fill-in the blanks (Sentence Completion)
Essay
Source: Turn-out of Test Questions in SSI (2003-2007)
General Steps in Test Construction
DRAFT
OUTLINE
ORDER
PRODUCE A
T.O.S.
TEST
ANALYZE
SUBMISSION
OUTLINE:
– the unit learning objectives or
– the unit content or major
concepts to be covered by the
test
Back to Main Menu
Table of Specifications
(TOS)
• A two way chart that relates the
learning outcomes to the course
content
• It enables the teacher to prepare a
test containing a representative
sample of student behavior in each of
the areas tested.
Tips in Preparing the Table
of Specifications (TOS)
•
Don’t make it overly detailed.
•
It's best to identify major ideas and skills
rather than specific details.
•
Use a cognitive taxonomy that is most
appropriate to your discipline, including
non-specific skills like communication skills
or graphic skills or computational skills if
such are important to your evaluation of
the answer.
Tips in Preparing the Table
of Specifications (TOS)
•
Weigh the appropriateness of the
distribution of checks against the students'
level, the importance of the test, the
amount of time available.
•
MATCH the question level appropriate to
the level of thinking skills
Examples of Student Activities and
Verbs for Bloom’s Cognitive Levels
Table 2.1 in Jacobs & Chase (1992:19)
Bloom’s Cognitive
Level
Student Activity
Words to Use in Item
Stem
Knowledge
Remembering facts,
terms, concepts,
definitions, principles
Define, list, state,
identify, label, name,
who?, when?, where?,
what?
Comprehension
Explaining/interpreting
the meaning of material
Explain, predict, interpret,
infer, summarize, convert,
translate, account for,
give example, paraphrase
Application
Using a concept or
principle to solve a
problem
Apply, solve, show, make
use of, modify,
demonstrate, compute
Examples of Student Activities and
Verbs for Bloom’s Cognitive Levels
Table 2.1 in Jacobs & Chase (1992:19)
Bloom’s Cognitive
Level
Student Activity
Words to Use in Item
Stem
Analysis
Breaking material down
into its component parts to
see interrelationships/
hierarchy of ideas
Differentiate,
compare/contrast,
distinguish ____from
____, how does
____relate to ___, why
does ____work
Synthesis
Producing something new or Design, construct, develop,
original from component
formulate, imagine,
parts
create, change, write a
poem or short story
Evaluation
Making a judgment based
on a pre-established set of
criteria
Appraise, evaluate,
justify, judge, which would
be better?
Tips in Preparing the Table
of Specifications (TOS)
• The following array shows the most
common questions types used at various
cognitive levels.
Factual Knowledge
Multiple Choice
True/False
Matching Type
S. Completion
Short Answer/RRT
Application
Multiple Choice
Short Answer
Problems
Essay
Analysis and
Evaluation
Multiple Choice
Essay
Activity:
• Prepare a short TOS using the
selection in your activity sheet.
Back to Main Menu
DRAFT the questions covering
the content in the outline
Back to Main Menu
ORDER the selected questions
logically.
 Place simpler items at the beginning
to ease students into the exam.
 Group item types together under
common instructions.
 If desirable, order the questions
logically from a content standpoint
(e.g. chronologically or by conceptual
groups, etc.)
Back to Main Menu
Test
• PUT the questions away for one or
two days before rereading them or
have someone else review them for
clarity.
• TEST the questions by actually
taking the test.
Back to Main Menu
• ANALYZE the items to give you
an idea whether the questions
were well-written or poorly
written as well as if there were
problems in understanding
instruction.
Back to Main Menu
General Rules in Writing Test
Questions
Number test questions continuously.
Keep your test question in each test group
uniform.
Make your layout presentable.
Do not put too many test questions in one
test group.




T or F: 10 – 15 questions
Multiple Choice: max. of 30 questions
Matching type: 5 questions per test group
Others: 5 – 10 questions
Some additional guidelines to consider
when writing items are described below:
1. Avoid humorous items. Classroom testing is
very important and humorous items may
cause students to either not take the exam
seriously or become confused or anxious.
2. Items should measure only the construct of
interest, not one’s knowledge of the item
context.
3. Write items to measure what students know,
not what they do not know. (Cohen & Wallack)
Multiple Choice Test
What to Look for on
Multiple Choice Tests
When checking the stems for correctness:
Ensure that the stem asks a clear
question.
Reading level is appropriate to the
students
The stem is grammatically correct.
Negatively stated stems are
discouraged.
What to Look for on
Multiple Choice Tests
Example:
What is the effect of releasing a ball in
positive gravity?
a) It will fall “down.” correct
b) It will retain its mass. true but unrelated
c) It will rise. false but related
d) Its shape will change. false and unrelated
Multiple Choice Questions
1. Use negatively stated stems sparingly and
when using negatives such as NOT,
underline or bold the print.
2. Use none of the above and all of the above
sparingly, and when you do use them, don't
always make them the right answer.
3. Only one option should be correct or clearly
best.
Multiple Choice Questions:
4. All options should be homogenous and
nearly equal in length.
5. The stem (question) should contain only
one main idea.
6. Keep all options either singular or plural.
7. Have four or five responses per stem
(question).
Multiple Choice Questions:
7. When using incomplete statements place
the blank space at the end of the stem
versus the beginning.
8. When possible organize the responses.
9. Reduce wordiness.
10. When writing distracters, think of incorrect
responses that students might make.
Examples
1. Sheldon developed a highly controversial
theory of personality based on body type and
temperament of the individual. Which of the
following is a criticism of Sheldon's work?
a. He was influenced too much by the
Freudian psychoanalysis.
b. His rating of physique and temperament
were not independent.
c. He failed to use empirical approach.
d. His research sample was improperly
selected.
Examples
Better: (Eliminate excessive wording and
irrelevant information)
1. Which of the following is a criticism of
Sheldon's theory of personality?
Examples
1. The receptors for the vestibular sense
are located
a. in the fovea.
b. in the brain.
c. in the middle ear.
d. in the inner ear.
Examples
Better: (Include in the stem any word(s) that
might otherwise be repeated in each
option.)
1. The receptors for the vestibular senses are
located in the _______.
a. fovea
b. brain
c. middle ear
d. inner ear
Examples
1. Which is not a major technique for
studying brain function?
a. Accident and injury
b. Cutting and removing
c. Electrical stimulation
d. Direct phrenology
Examples
Better: (Use negatively stated stems
sparingly. When used, underline
and/or capitalize the negative word.)
1. Which is NOT a major technique for
studying brain function?
Examples
4. ________________ is the least form of
behavior disorder.
a. Psychosis
b. Panic disorder
c. Neurasthenia
d. Neurosis
Examples
Better: (When using incomplete
statements avoid beginning with the
blank space.)
1. The least severe form of behavior
disorder is __________________.
Examples
1. The number of photoreceptors in the
retina of each human is about
a. 115 million
b. 5 million
c. 65 million
d. 35 billion
Examples
Better: (When possible, present
alternatives in some logical order.)
1. The number of photo receptors in the
retina of each human is about
a. 5 million
b. 35 million
c. 65 million
d. 115 million
Examples
6. Latane and Darley's smoke-filled room
experiment suggested that people are
less likely to help in groups than
alone, because people
a. in groups talk to one another.
b. who are alone are more attentive.
c. in groups do not display pluralistic
ignorance.
d. in groups allow others to define the
situation as a non-emergency
Examples
Better: (All alternatives should be approximately
equal in length.)
6. Latane and Darley's smoke-filled room
experiment suggested that people are less
likely to help in groups than alone, because
people in groups
a. talk to one another
b. are less attentive than people who are
alone
c. do not display pluralistic ignorance
d. allow other to define non-emergencies
Activity:
• Prepare two multiple choice questions
based on the selection in your activity
sheet.
True or False
What to Look for on
True/False Tests
Each statement is clearly true or
clearly false.
Trivial details should not make a
statement false.
Statements are written concisely
without more elaboration than
necessary.
Statements are NOT quoted exactly
from text.
Tips in Making True/False Tests
Give emphasis on the use of quantitative
terms than qualitative terms.
Avoid using of specific determiners which
usually gives a clue to the answer.


False = all, always, never, every, none,
only
True = generally, sometimes, usually,
maybe, often
Discourage the use of negative
statements.
Whenever a controversial statement is
used, the authority should be quoted.
Discourage the use of pattern for answers.
Examples:
Find the errors, and/or problems with the
following true-false tests.
____ 1. Repetition always strengthens the
tendency for a response to occur.
(Using "always" usually means the answer
is false.)
Examples:
_____ 2. The process of extinction is
seldom immediate but extends
over a number of trials.
(Words like "seldom" usually indicate a
true statement.)
Examples:
_____ 3. The mean, median, and mode are
measures of central tendency,
whereas the standard deviation and
range are measures of variability.
(Express a single idea in each statement.)
e.g.“The mean and standard
deviation are measures of central
tendency.”
Activity:
• Prepare two true or false questions
based on the selection in your activity
sheet.