Transcript Slide 1

Prepare and Use Knowledge
Assessments
Introduction
•
Why do we give
knowledge tests?
•
What problems did
you have with tests
as a student?
•
As a faculty
member?
Quick Review
•
When assessing students’ knowledge, you will
use formative knowledge assessments to
help guide students toward meeting the
learning objectives.
•
You will use summative knowledge
assessments to ensure that students finish
the course with the required knowledge.
Objectives
•
•
•
•
•
Select methods for assessing students’
knowledge
Prepare knowledge assessments
Develop questions for objective written
examinations
Administer and score knowledge assessments
Use assessment results to improve
performance
Methods #1
•
Drills, quizzes, and practice tests
•
Written exercises
•
Case studies, clinical scenarios, and patient
management problems
•
Project reports
Methods #2
•
Essay examinations
•
Objective written examinations (e.g., true-false,
multiple-choice, matching and short-answer
questions)
•
Structured practical examinations
•
Oral examinations
Prepare Knowledge Assessments #1
•
Identify the learning objectives or outcomes.
•
Use simple and clear language.
•
Include at least one item per objective (too few
items – may not be valid, too many items –
students become test tired)
•
Use correct grammar in questions and
answers.
Prepare Knowledge Assessments #2
•
Ensure that questions reflect conditions stated
in the objective.
•
Is the question in any way controversial?
•
Are the distractors in multiple-choice questions
reasonable, and similar in structure and length
to the correct answers?
•
What is the difficulty of the questions?
Prepare Knowledge Assessments #3
•
Make each test item separate from other items.
•
Ask the same or similar questions (especially
for oral examinations and reports).
•
Divide long tests into several parts.
•
Provide clear directions for each type of item.
•
Decide on the model answers to questions.
Use answer keys or checklists for scoring.
Arranging Test Items
•
Subject matter
•
Type
•
Level of difficulty
Review the Assembled Test #1
•
Are there items for all of the objectives? Does
the number reflect the time spent on each
objective?
•
Is the number of items included in the test in
direct proportion to their importance in
learning?
•
Does each item really measure the students’
attainment of the objective?
Review the Assembled Test #2
•
Is each set of directions clear?
•
Is there ample space to write the response?
•
Are tricky, obvious, or irrelevant questions
avoided?
•
Is each item separate and independent from
the rest of the items?
Review the Assembled Test #3
•
Are similar items grouped together?
•
Is the test designed so that it is easy to score?
•
Will students be provided with meaningful
feedback about their answers?
Preparing a Test Bank
•
Work in a group to develop questions.
•
Sort questions based on subject, category,
level of knowledge, or type of test item.
•
Store the questions on a computer or cards.
•
Ensure that only authorized persons have
access to the question bank.
Activity
•
Work in 4 groups. Each group will be assigned
1 type of item (Table 10-1).
•
In your own words, describe the item and the
advantages and disadvantages of that item.
•
Briefly share with the other groups.
Writing True-False Items #1
•
The language of true-false test items should be
clear, concise, and understandable (avoid
words such as more, few, large, and good).
•
Avoid using negatively stated items (e.g., It is
not recommended that. . . .) as these are
confusing in a true-false question.
Writing True-False Items #2
•
Avoid words or expressions that frequently
identify a statement as true or false.
•
Words often found in false statements are only,
never, all, every, always, none, and no.
•
Words often found in true statements are
usually, generally, sometimes, customarily,
often, may, could, and frequently.
Writing True-False Items #3
•
Use statements that are either completely true
or completely false. Avoid a statement that is
partially true and partially false. True-false
items should focus on one idea and should
challenge, but not trick, the students.
•
Make the true statements equivalent in length
and number to the false statements.
Sample Directions
Directions: For each of the following
statements, print a capital T in the block to the
left of the number if the statement is true or a
capital F if the statement is false. You will
receive one point for each correct answer.
Parts of the Multiple Choice Question
•
Stem
•
Responses
•
Distractors
Writing Multiple-Choice Questions #1
•
Write simple stems that state the problem.
•
Avoid using negative terms in the stem (such
as no, never, none, and not) – if you do, boldface and/or underline the term to draw
attention to it.
Example: “Which of the following is not the. .”
•
Provide a coherent list of possible answers
(use a/an when needed).
Writing Multiple-Choice Questions #2
•
Write reasonable distractors.
•
All distractors should be of similar length to
avoid giving clues to the correct response.
•
Avoid use of “All” and “None of the above” in
responses (when used, use 25% of the time as
correct answer when there are 4 choices).
Sample Scenario MCQ
Mrs. B. is 20 years old and had an IUD
inserted a month ago. She came to the health
center 2 days ago with vaginal discharge and
abdominal and pelvic pain. She reports that
she does not have any fevers or chills.
1. What is your plan?
A.
B.
C.
D.
Gather history, send vaginal cultures, remove the IUD.
Gather history, send vaginal cultures, follow up in 3 days.
Gather history, treat with antibiotics for presumptive PID.
Gather history, send vaginal cultures, treat with antibiotics
for PID, and remove the IUD if the woman wishes.
Sample Directions
Directions: Each of the items on this
examination is followed by four possible
responses. For each item, select the best
response. Indicate your answer by circling the
appropriate letter next to your answer. Each
correct response is worth one point.
Matching Questions
Refer to the manual
for more information
on developing
matching questions.
Short-Answer Questions
Refer to the manual
for more information
on developing shortanswer questions.
Test Administration
•
•
•
•
•
•
Time allowed
How to select and record answers
Scoring system to be used
Physical environment
Remain in the room
Lighting and ventilation
Score the Test #1
•
With written objective examinations, students
can mark the answers directly onto the test or
onto a separate answer sheet.
•
When there are many test items, it may be
easiest for students to write their answers on a
separate answer sheet.
Score the Test #2
•
Use an answer key
(or a computer with
scanner) to score
the tests.
•
See the module for
more information on
scoring knowledge
assessments.
Using Assessment Results #1
•
Knowledge is assessed to determine whether
students are meeting the learning objectives
and have acquired the required information
base to become competent healthcare
providers.
Using Assessment Results #2
•
Formative knowledge assessments help
students decide what content areas they need
to spend more time studying to prepare for
summative assessments.
•
The results of summative assessments
determine if students are meeting the learning
objectives.
Using Assessment Results #3
•
Instruct students to review the material related
to the questions they missed.
•
Give students an opportunity to ask you
questions about any test items on which they
scored poorly or that they did not understand.
Using Assessment Results #4
•
If many students had trouble with the same
questions, either the teaching methods or
materials did not adequately address that
learning objective, or the question needs to be
rewritten.
•
Do not be afraid to revise problematic test
questions or adapt the teaching methods used
to better address the content.
Activity
•
If we have time, let’s try writing some MCQs
based on topics in your various curricula and
share your questions.
•
Work in small groups to develop 1 question
and then put this on a flipchart to share with
the other groups.
Summary
•
What questions do you have about developing,
administering and scoring knowledge
assessments?
•
How can we assist faculty members in
developing, administering and scoring
knowledge assessments to support
implementation of curricula?