Transcript Classroom Assessment Plan your assessment as you plan your

Designing the Test and Test Questions
Jason Peake
Principles of Testing
Validity
Does the test measure what it is supposed to
measure?
 Does the test require student performance as
described by the objective?

Principles of Testing
Reliability
Does the test generate the same results each
time it is given?
 Is the test affected by misinterpretation

Poorly written directions
 Vague questions
 Misleading words
 Not sure of response

Why do we evaluate students?
Motivate students
Measures achievement
Identify areas for review
Check effectiveness of materials, teaching
method, and teacher
Assign grades
Agenda
Aligning evaluation with level of learning.
2. Developing a test blueprint and preparing
to write items
3. Common pitfalls when writing test
questions
4. Writing questions at various levels of
performance
1.
Defining objectives using
Bloom’s Taxonomy
CATEGORY
•
Evaluation
•
Synthesis
•
Analysis
SKILL
• Judge
• Combine
• Break apart
•
• Apply principles
•
•
Application
Comprehension
Knowledge
• Understanding
• Recall
Sample Verbs
Knowledge

Cite, define, identify, label, list, match, name,
recognize, reproduce, select, state
Comprehension

Classify, convert, describe, distinguish between,
explain, extend, give examples, illustrate, interpret,
paraphrase, summarize, translate
Application

Apply, arrange, compute, construct, demonstrate,
discover, modify, operate, predict, prepare, produce,
relate, show, solve, use
Sample Verbs continued
Analysis

Analyze, associate, determine, diagram, differentiate,
discriminate, distinguish, estimate, infer, order,
outline, point out, separate, subdivide
Synthesis

Combine, compile, compose, construct, create,
design, develop, devise, formulate, integrate, modify,
organize, plan, propose, rearrange, reorganize, revise,
rewrite, tell, write
Evaluation

Appraise, assess, compare, conclude, contrast,
criticize, discriminate, evaluate, judge, justify,
support, weigh
Agenda
Aligning evaluation with level of learning.
2. Developing a test blueprint and preparing
to write items
3. Common pitfalls when writing test
questions
4. Writing questions at various levels of
performance
1.
Test blueprint (specifications)
•
•
•
Matrix of number of test questions by
topic and level of objective
Helps you plan your test
Do before writing items
Example blueprint
Topic A
Topic B
Topic C
Topic D
Total
Knowledge
1
2
1
1
5 (20%)
Comprehension
2
1
1
2
6 (24%)
Application
2
1
1
1
5 (20%)
Analysis
1
1
2
2
6 (24%)
1
2 (8%)
Synthesis
1
Evaluation
Total
1
6
(24%)
6
(24%)
6
(24%)
1 (4%)
7
(28%)
25
(100%)
Agenda
Aligning evaluation with level of learning.
2. Developing a test blueprint and preparing
to write items
3. Common pitfalls when writing test
questions
4. Writing questions at various levels of
performance
1.
Preparing to write items
•
•
•
•
•
Create test items while preparing class lessons
Make note of questions asked frequently by
students
Make note of misconceptions made frequently
by students during class or homework
Invite students to submit items at the end of each
class or at other times
Pull questions from formative assessments.
Agenda
Aligning evaluation with level of learning.
2. Developing a test blueprint and preparing
to write items
3. Common pitfalls when writing test
questions
4. Writing questions at various levels of
performance
1.
Common pitfalls
•
•
•
•
•
Providing unintended cues
Using trick questions
Using distractors that are obviously
incorrect
Using NOT or EXCEPT in the question
Including all of the above, none of the
above, or only A and B as option choices
Preparing for test-wise students
•
•
•
•
•
•
•
•
Longest answer
“C” as correct answer
Avoid picking “always” or “never”
Opposites
Scientific sounding
Avoid simple & obvious
Related word
Guess
Agenda
Aligning evaluation with level of learning.
2. Developing a test blueprint and preparing
to write items
3. Common pitfalls when writing test
questions
4. Writing questions at various levels of
performance
1.
Levels of performance
Lower levels (knowledge, comprehension, and application)

Recognize the principle in altered format.

Match component parts with vocabulary.

Identify example seen before.

Recognize an example never seen before.

Solve a problem using familiar format, but different specifics.
Higher levels (analysis, synthesis, evaluation)

Pick out components of a situation.

Problem solve by combining elements.

Make judgments of value.
Categories of Test Questions
Test can roughly be broken into
1.
Objective




2.
Multiple choice
Matching
Completion
True false
Subjective



Short answer and/or Essay
Oral test
Case Study
Sample multiple choice items
Lower levels

Which of the following is an example of erosion? (choices
include one or more used in class)

Identify the definition of water erosion?

Which of the following is an example of erosion? (choices
never seen in class)

Here is an erosion scenario. Which of the following principles
apply?
Higher levels

Which of the following solutions would be the best to apply in
the scenario described below to solve the erosion problem?

Here is an erosion problem situation. Identify which principle
from this class could be used to solve it.
Sample multiple choice distracters
A term used in the unit but referring to a different concept

Cause of erosion – no till farming
An everyday term that is close but misleading in meaning

Cause of erosion – hydrofoil
A common misconception or confusion students have

Cause of erosion – lack of diverse soil particle size
A plausible word or phrase that is meaningless but derives
from something in the stem

Cause of erosion – systemic erosion syndrome
Matching
Appropriate for only knowledge level
testing
Use a reasonable number of items 7 – 12
All premises in one list, all responses in
another list
Premises can be long, but try to keep
responses short
Use more responses than premises
Clear Directions
____ 1. List premise
____ 2. List premise
____ 3. List Premise
____ 4. List Premise
____ 5. List Premise
____ 6. List Premise
A. Response
B. Response
C. Response
D. Response
E. Response
F. Response
G. Response
Completion Items
Most useful for lower order levels of
learning
Use your own words
Be sure only one answer can apply
Keep all blanks the same length
Avoid grammatical clues
Complete the meaning of each
statement by writing the correct
word in the corresponding numbere
blank
1. A thermometer should be rinsed in
_____________ water.
2. The NPK in fertilizers stands for
_____________, _____________, and
_____________
True / False Questions
Can be used to check for knowledge and
comprehension
Keep questions simple
Make entirely true or entirely false
Avoid double negative items
Writing True / False Questions
For each of the statements below circle “T”
for true and circle “F” for false.

T F Swine gestation period is 33 days.
For each of the statements below circle “T”
for true and circle “F” for false and explain
in the blank why it is false.

T F Swine gestation period is 33 days.
_________________________________
Subjective Test Questions
Short answer and/or Essay
 Oral test
 Case Study

Short Answer and/or Essay
May require many different behaviors:
Comparison, for or against, cause and
effect, explanation, summary, analysis,
describe relationships, apply rules, or
discussion
Oral Test
All students must receive the same item
stated in clear concise manner designed to
elicit correct answers arranged in logical
order.
Case Study
Problem is practical and realistic
Requires comprehension of prior learning
and application of that learning
Subjective Test Questions
Short answer and/or Essay
 Oral test
 Case Study


Theses subjective test questions or subjective
test require a criteria to be graded. We will
work on grading criteria tomorrow.
Which Type of Question to Use?
Select the appropriate technique to test the
student competence at the level of cognition
required by the objective.
Avoid one type of test or test question
We know students have different strengths
Visual
 Audio
 kinesthetic

Materials in this presentation
developed in part by…
Brian Parr
University of Georgia
Dr. Dawn Zimmaro
University of Texas