Transcript Classroom Assessment Plan your assessment as you plan your
Developing Tests and Test Questions/Items
Why do we evaluate students?
Motivate students Measure achievement Identify areas for review Check effectiveness of materials, teaching methods, and teacher Differentiate between different levels of achievement Assign grades
Basic Principles of Testing
Validity
Does the test measure what it is supposed to measure?
Does the test require student performance as described by a corresponding objective?
Basic Principles of Testing
Reliability
Would the test generate the same results if it were given again (assuming no change in students)?
Is the test affected by misinterpretation?
Poorly written directions or instructions
Vague or tricky questions
Misleading words
Not sure of student response
Agenda
1.
2.
3.
4.
Aligning evaluation with level of learning
Developing a test blueprint or “table of specifications” Common pitfalls when writing test questions Writing questions at various levels of performance
Defining objectives using Bloom’s Cognitive Taxonomy
• • •
CATEGORY/level Evaluation Synthesis Analysis
• • •
SKILL (what students do) Judge Combine Break apart
• • •
Application Comprehension Knowledge
• • •
Apply principles Understanding Recall
Sample Verbs for Different Categories/Levels of Cognition Knowledge
Cite, define, identify, label, list, match, name, recognize, reproduce, select, state Comprehension
Classify, convert, describe, distinguish between, explain, extend, give examples, illustrate, interpret, paraphrase, summarize, translate Application
Apply, arrange, compute, construct, demonstrate, discover, modify, operate, predict, prepare, produce, relate, show, solve, use
Sample Verbs
continued Analysis
Analyze, associate, determine, diagram, differentiate, discriminate, distinguish, estimate, infer, order, outline, point out, separate, subdivide Synthesis
Combine, compile, compose, construct, create, design, develop, devise, formulate, integrate, modify, organize, plan, propose, rearrange, reorganize, revise, rewrite, tell, write Evaluation
Appraise, assess, compare, conclude, contrast, criticize, discriminate, evaluate, judge, justify, support, weigh
Agenda
1.
Aligning evaluation with level of learning 2.
3.
4.
Developing a test blueprint or “table of specifications”
Common pitfalls when writing test questions Writing questions at various levels of performance
Developing a Test Blueprint or a Table of Specifications Develop a matrix (much like a rubric) of number of test questions (or percentage of test) by topic and level of objective, and by amount of emphasis on each topic and objective
•
Helps you plan your test
•
Do before writing items
Example Test Blueprint or Table of Specifications Number of Objectives (Amount of Effort*} Level of Objective Topic A Topic B Topic C Topic D Knowledge Comprehension Application Analysis Synthesis* Evaluation* Total *Adjustment factor 1 2 2 1 6 (24%) 2 1 1 1 1 6 (24%) 1 1 1 2 1 6 (24%) 1 2 1 2 1 7 (28%) Adjust Total 5 (20%) 6 (24%) 5 (20%) 6 (24%) 2 (8%) 1 (4%) 25 (100%)
Levels of performance
Lower levels (knowledge, comprehension, and application)
Recognize the principle in altered format.
Match component parts with vocabulary.
Identify example seen before.
Recognize an example never seen before.
Solve a problem using familiar format, but different specifics.
Higher levels (analysis, synthesis, evaluation)
Pick out components of a situation.
Problem solve by combining elements.
Make judgments of value.
Preparing to write test items
• • • • •
Create test items while preparing class lessons.
Make note of questions asked frequently by students. Make note of misconceptions made often by students during class or homework.
Invite students to submit items at the end of each class or at other times.
Adapt questions from formative assessments.
Categories of Test Questions Test items can roughly be broken into 1.
Objective (“fixed” or single correct response)
True-false
Multiple choice Matching Completion or fill-in-the-blank.
2.
Subjective (extended response/multiple correct)
Short answer and/or Essay Oral test
Case Study
Common pitfalls in writing test directions —and avoiding them!
• •
Giving unclear directions: “Choose the best answer.” What is meant by “best”? Funniest? Simplest? Cheapest? Easiest?
“Choose the correct answer” is a much better statement of directions —and then ensure that there is a single correct answer!
Writing GOOD test directions
•
Indicate carefully and specifically how students should respond. For example, on a TRUE/FALSE section on a test: Circle “V” for statements that are VERDAD and “F” for statements that are FALSO.
•
Do NOT have students write V or F in a blank.
Give point values for items and sections.
Allow students to EXPLAIN on any fixed response (true-false, multiple choice, etc.) item.
Writing True / False Items for Tests and Exams Can be used to check for knowledge and comprehension.
Keep statements simple.
Make entirely true or entirely false.
Avoid double negative items.
In fact, best to avoid negative items.
Common pitfalls in writing true-false test items
•
Using NOT or EXCEPT in the statement
•
Using cues such as “ALWAYS” or “USUALLY”
•
Using unclear, vague terms such as “OFTEN” or “MANY” or “FEW”
•
Using double negatives in a statement
•
Using compound statements with one indpendent clause true and one, false.
Writing True / False Test Items For each of the statements below, circle “T” if true and circle “F” if false.
T F The length of gestation in swine is 33 days.
For each of the statements below, circle “T” if true and circle “F” if false. Also, explain in the blank why it is false.
T F The length of gestation in swine is 33 days.
__________________________________
Writing multiple choice items for tests and exams
•
Use distractors that are plausible
•
Do NOT use “all of the above” or “none of
the above
” or “only A and B” or “B and C on alternate Thursdays” as distractors
•
Use 4 or 5 distractors, but use the SAME number of distractors (4 or 5) for all items
•
Vary randomly where you put the correct answer.
•
Allow students to explain!
Preparing for students who are test-wise to some kinds of items
• • • • • • • •
Longest answer “C” as correct answer Process of elimination Use grammatical cues Scientific sounding Avoid simple and obvious Related word Guess by figuring out your pattern
Writing Matching Items for Tests and Exams Appropriate only for knowledge level testing Use a reasonable number of items: 7 – 12 All premises in one list, all responses in another list Premises can be long, but try to keep responses short Use more responses than premises Can approximate fill-in-the-blank items
Writing Clear Directions for Matching Items Write the LETTER of the correct Response in the Blank preceding the numbered premise.
____ 1. List premise A.
Response ____ 2. List premise ____ 3. List Premise B.
C.
Response Response ____ 4. List Premise ____ 5. List Premise ____ 6. List Premise D.
E.
F.
G.
Response Response Response Response
Writing Fill-in-the-Blank or Completion Test Items Most useful for lower order levels of learning Use your own words Try to ensure that only one answer is correct, but, if that is too hard to do, … You may consider “matching” using a word bank Keep all blanks the same length and at END of statement Avoid grammatical clues
Writing Directions for Completion Test Items Complete each statement by writing the correct word(s) in the blank(s) provided.
1.
A thermometer should be rinsed in _____________ water. Good or bad item?
2.
The letters “N-P-K” in fertilizers stand for, respectively, _____________, _____________, and _____________.
Extended Response Test Items
Short answer and/or Essay
Oral test
Case Study or “Word Problems/Situations”
These kinds of test questions require criteria and pre-determined descriptions of levels of performance to be graded reliably and fairly. We call that what?
Short Answer and/or Essay
May require many different behaviors:
Comparison, for or against, cause and effect, explanation, summary, analysis, describe relationships, apply rules, or discussion
Oral Test
All students must receive the same test item stated in a clear, concise manner designed to elicit correct answers arranged in logical order.
Determine beforehand how you may lead the students if they stumble
.
Case Study
Problem should be practical and realistic Requires recall and comprehension of prior information and, if well-written, may require application, analysis, synthesis and/or evaluation of that information “Word problems” are similar to a case study.
Which Type of Question to Use?
Select the appropriate technique to test the student competence at the level of cognition required by the objective.
Avoid using just one type of test question.
We know students have different strengths:
Visual
Audio
Kinesthetic
Materials in this presentation developed in part by… Jason Peake University of Georgia Brian Parr Auburn University Dawn Zimmaro University of Texas Gary Briers Texas A&M University