Transcript Slide 1

Audience Response Systems:
Transforming your Lecture Hall into a Platform
for Interactivity
Alice Chuang, MD
University of North Carolina – Chapel Hill
Francis S. Nuthalapaty, MD
University of South Carolina - Greenville
2009 APGO Faculty Development Seminar
Objectives
• Demonstrate ARS in action*
• Discuss the rational for using Audience
Response Systems (ARS) in medical education
• Outline ARS question types and best practices
• Outline practical concerns regarding use of ARS
• Describe available ARS systems
Audience Response Systems
• Definition: Electronic tool which allows lecturer
to poll learners with immediate feedback.
• Synonyms:
– Classroom response system
– Student response system
– Personal response system
Instructions for Clickers
• To vote: Press 1/A, 2/B etc…
• Vote Status Light:
– Green if your vote was recorded
– Orange if not
• To change your vote – press another key
• Clickers work like an RF remote
• Receiver is plugged into USB port
Are you here today?
100%
o
N
s
0%
Ye
1. Yes
2. No
I am a…
56%
11%
0%
22%
11%
1.
2.
3.
4.
5.
Clerkship director
Program director
Chair
Clerkship coordinator
Other
What is your role in education?
1.
Teaching students in the
pre-clinical curriculum
2. Teaching students in the
clerkship
3. Administration (Dean,
Clerkship Director)
4. Other
44%
44%
11%
(D
st
ra
tio
n
en
ts
A
dm
in
i
st
ud
hi
ng
ac
Te
r
th
e
O
ea
n
th
e.
..
in
th
e.
..
in
en
ts
st
ud
hi
ng
ac
Te
,C
...
0%
How often do you currently use ARS in
your teaching?
1. Never used ARS
before
2. Rarely (<25% of my
lectures)
3. Often (>50% of my
lectures)
4. Very Often (>75% of
my lectures)
30%
0%
70%
Never used ARS before
Rarely (<25% of my lectures)
Often (>50% of my lectures)
Very Often (>75% of my lectures)
Roles of ARS
• Restarting the attention clock
–Average human attention span: 20 minutes
–Recall of information decreases after 15-20
minutes
–At maximum 20-25% of material presented in
standard lecture is retained
Is finding lecture
material somewhat
mundane, starts
previewing following
lecture for interesting
slides
Attention paid to lecturer (%)
Settling into
lecture, previewing
lecture slides
90
80
70
60
Adult Learner
(i.e. tired
overworked
medical
student)
50
40
30
20
Has renewed
sense of interest
in topic at hand
10
0
0
15
30
Time (min)
45
60
Listens intently,
hoping to catch key
summary points
Roles for ARS
• To increase interactions
– To assess learner’s attitudes
– To start or focus discussions
– To vote after a debate or panel discussion
• To assess preparation and ensure accountability
– Questions about reading or homework
Roles for ARS
• To engage learner throughout a lecture
– To assess learner’s understanding and misunderstanding of
topics during a lecture
– To find out more about student attitudes on a topic
– To assess need for more explanation or detail
• For quizzes or tests
– paying attention
– taking good notes
– preparing for class or labs
– keeping up with homework
– actively thinking
Prevalence of ARS
• Used ubiquitously in many
disciplines…mathematics, philosophy,
psychology, economics
• Used in many age groups…K-12, elementary,
undergraduate, graduate
History
• Pradhan et al
• RCT of ARS v. standard lecture format on the
topic of contraception
• 17 ob/gyn residents at UMDNJ
• Compared pretest/posttest (6 weeks later)
scores
– ARS: 21% improvement
– Standard lecture: 2% improvement
– P = 0.18
History
• Nayak and Erinjeri
• 27 medical students who were instructed to give a
presentation at the end of course
– From learners viewpoint
• Increased self-confidence with understanding material and verbally
answering questions
• No differences in end of course exam scores
– From presenters viewpoint
• Made them more comfortable that they were conveying information
effectively
• Prepared them for future presentations
Pros…based on the literature
• Instructor benefits
– Immediate feedback on learners understanding, identify
gaps
– Improved retention of knowledge
• Technical issues
– Decreased error in collecting data because of electronic
transmission
– Anonymous
• Can increase attendance
– in one study only if participation was worth at least 10% of
grade
– Can reduce attrition through the duration of a course (4% vs
8-12%)
– Particularly if class was midday
Pros…based on the literature
• Increased learner satisfaction by improving
interactivity and by engaging learner
– Student gave instructors and “learning environment” more
positive ratings
• Allows learner to compare own understanding with
others
• Increases active participation
– Learner must commit to answer, thus increasing attention
– Affects active participation in other ways (asking more
questions)
– Less sleeping during class
– Superior to show of hands technique with increased
participation
Cons
• Need initial investment in software and hardware
– Program to create questions
– “clickers” or other data entry modality
• Training and buy-in from faculty
• Willingness to participate by students
– Some negative responses
• Using ARS decreases time for actual delivery of
material during the course of lecture
• Can be distracting if overused in a single lecture (25/50 minute lecture)
Question Types
•
•
•
•
•
•
•
Factual Questions
Conceptual Questions
One-best-answer Questions
Opinion Questions
Data Gathering Questions
Questions Asking for Predictions
Feedback on Teaching
Question Types
Factual Questions
These questions might be used to see if students
did the reading, remember important points
from prior classes, or have memorized key facts.
Vanderbilt Center for Teaching
Question Types
Conceptual Questions
Multiple-choice questions that demonstrate
whether or not students understand important
concepts and principles.
Vanderbilt Center for Teaching
Question Types
One-best-answer Questions
Questions which include multiple answer choices,
more than one of which could be argued as
correct. Students are asked to select the one best
answer from these choices. Also referred to as
"partially correct multiple-choice questions"
(Burnstein & Lederman, 2006)
Vanderbilt Center for Teaching
Question Types
Opinion Questions
Evaluative and opinion questions which may not
have correct answers. Asking these questions
can engage students and provoke rich
discussions, particularly in response to ethical,
legal, or moral issues.
Vanderbilt Center for Teaching
Question Types
Data Gathering Questions
Used to quickly gather demographic, opinion, or
other data from a class full of students. This is
often useful in social science classes that utilize
social or economic experiments in the
classroom.
Vanderbilt Center for Teaching
Question Types
Questions Asking for Predictions
Students can be asked to predict the outcome of
an experiment prior to being shown the
experiment. This gives the teacher a sense of the
students’ preconceptions and increases the
surprise value of the experiment when students
see just how many of their classmates expected
different outcomes.
Vanderbilt Center for Teaching
Question Types
Feedback on Teaching
Ask students to provide direct feedback on how
well a particular class is going. For example, one
might ask students to rate their level of
understanding of the lecture thus far on a scale
of 1 to 4.
Vanderbilt Center for Teaching
Question Best Practices
• Apply Bloom’s Taxonomy to test concepts
rather than recall
• Test understanding of jargon separately from
understanding of concepts to be sure source of
confusion can be clearly identified
• Include “I don’t know” to ensure data truly
reflects understanding rather than some sample
of student guessing
Bloom’s Taxonomy
Bloom’s Taxonomy
6. Evaluation: think critically about and defend a position
– Judge, Justify, Defend, Criticize, Evaluate
5. Synthesis: transform, combine ideas to create
something new
– Develop, Create, Propose, Design, Invent
4. Analysis: break down concepts into parts
– Compare, Contrast, Distinguish
3. Application: apply comprehension to unfamiliar
situations
– Apply, Use, Compute, Solve, Predict
2. Comprehension: demonstrate understanding of ideas,
concepts
– Restate, Explain, Summarize, Interpret, Describe
1. Factual Knowledge: remember and recall factual
information
– Define, List, State, Name, Cite
Question Best Practices
• Use variety of types of questions
• Ask about linking a general concept to a specific
example
• Ask about linking a familiar idea to a new concept
• Ask about a concept yet to be introduced to increase
student curiosity
• Can ask question twice
– First for individuals
– Second after student discussion to promote exchange of ideas
How would you rate this workshop so
far?
1. Fascinating
2. Somewhat
interesting
3. Boring
4. A waste of time
75%
25%
So
of
or
in
g
tim
e
0%
A
w
as
te
B
g
tin
ti
nt
er
es
m
ew
ha
Fa
sc
in
a
tin
g
0%
Question Writing Exercise
35 yo G0 BMI 40 recently married with
long h/o oligoovulation. EMB shows
CAH. Propose next step:
yn
G
to
m
y
st
er
ec
h
re
...
hy
pa
t
es
ti.
O
nc
co
ns
t:
gm
m
al
ed
ic
M
ul
t/
U
pr
og
ou
nd
ltr
as
C
d
an
D
or
m
b
0%
..
0%
no
w
.
in
.
x
tb
re
pe
a
n,
ea
te
at
io
ep
R
6.
14%
0%
er
v
5.
14%
bs
2.
3.
4.
Observation, repeat bx in 3
mos
Repeat emb or D and C now
Ultrasound
Medical mgmt: progestin
Rx and eval for PCOS
Gyn Onc consult/path
review
hysterectomy
O
1.
71%
To use for future conference planning,
Palm Beach is a great location for a
conference because:
15%
54%
31%
0%
1.
2.
3.
4.
Accessible location
Pleasant weather
Luxurious accomodations
Various recreational activities
UNC Experience
• Questions should be similar difficulty as those
on examinations
• Students liked ARS especially if questions
provided hints to examination materials
• Participation was about 60-70%, not the 100%
we anticipated
Best Practices
• Planning
– Be sure to have tech support
– Decide how ARS will or will not affect student grades
– Define clear objectives for use of ARS in your
classroom and write questions with these objectives
in mind
– First year will take a little more time and effort
– Practice when first starting to use ARS
Best Practices
• Student communication
– If used for grading, be sure to have rules about use
(i.e. no use of other’s clickers)
• Consider throwing out some grades for inadvertent
clicking
• Consider making it worth 5-10% of total grade (i.e. low
stakes, but not too low)
– Plan for time to debrief after each question
ARS Options
• Different technologies
– IR transmitters
– RF transmitters
– Web-based
– Smartphone
• Presentation software integration
• LMS integration
ARS Options
Turning Point
Interwrite
iClicker
Acknowledgements
• Special thanks to:
Jake Achey
Instructional Technology Technician
UNC School of Medicine
Medical Students Teaching Lab
References
• 1.
BARBER M, NJUS D. Clicker evolution: seeking intelligent design. CBE
Life Sci Educ 2007;6:1-8.
• 2. CALDWELL JE. Clickers in the large classroom: current research and
best-practice tips. CBE Life Sci Educ 2007;6:9-20.
• 3. COLLINS J. Audience response systems: technology to engage learners.
J Am Coll Radiol 2008;5:993-1000.
• 4. COLLINS LJ. Livening up the classroom: using audience response
systems to promote active learning. Med Ref Serv Q 2007;26:81-8.
• 5. NAYAK L, ERINJERI JP. Audience response systems in medical student
education benefit learners and presenters. Acad Radiol 2008;15:383-9.
• 6. PRADHAN A, SPARANO D, ANANTH CV. The influence of an audience
response system on knowledge retention: an application to resident
education. Am J Obstet Gynecol 2005;193:1827-30.
• 7. TURPIN DL. Enhance learning with an audience response system. Am J
Orthod Dentofacial Orthop 2003;124:607
Session content, including narrated MS
Powerpoint slides available at:
http://www.obgynknowledgebank.net