Transcript View PowerPoint

Streaming Video Tutorials for Complex
Laboratory Techniques:
Lessons Learned in Sophomore and Junior
Biomechanics Courses
Kristen L. Billiar, Ph.D.
Dept of Biomedical Engineering, Worcester Polytechnic Institute
Dept of Surgery, University of Massachusetts Medical School
Goals
Laboratories in all BME courses


Making and interpreting measurements
Challenge-based, self-guided, self-paced
Particular challenge


Rapid growth of BME (40-50 per course)
Limited resources
Instructor time (and TA)
Volume of sophisticated equipment
Laboratory space (scheduling time)
Funds
Practical vs. theoretical

Expanding access, not assessing educational
theories or comparing methods
Share approach to problem and assessment



Discuss time and effort involved
Describe assessment methods and findings
Initiate dialog – chance to get your feedback
UMMS
Department of Biomedical Engineering
Approach and goal of study
Approach - teaching technology

Web-based streaming video tutorials
Provide the “toolbox”
Economical, sustainable

Allow students to perform labs


Self-paced, flexible schedule
Without constant instructor
or TA supervision
Goal - formative study


Assess student learning from the tutorials
Gather student feedback on their usefulness and quality as a
guide for future improvements
UMMS
Department of Biomedical Engineering
Tutorials – web-based
Tutorials




How? rather than What and Why
Large number (>20), short (< 5 minutes)
Easy to create and modify1
Format
MS Powerpoint, Camtasia, video
Slides, video, audio
Design team

Two professors
Biomedical and Mechanical Engineering

Academic Technology Center Staff
Video, editing, streaming servers

Two sophomore-level students
20 hrs/wk x 8wks x 2 = 320 hrs
Funding

Two small internal grants, paid for student time and provided access
1Billiar
and Shonat, PEBEL II, 2004
UMMS
Department of Biomedical Engineering
Example tutorial - safety
Posted on myWPI (Blackboard)
PPT file complete

(link)
23 MB (w/o video clip)
PPS file



1.2 MB
Streaming audio
Streaming video
Requires fast internet connection
X

UMMS
Department of Biomedical Engineering
Assessment
Sophomore-level “Foundations of Biomechanics”

large number (n=47), limited use, single laboratory
Junior-level “Experimental Biomechanics”

small number (n=17), extensive use as they were meant to be, challenge-based
Questionnaires (sophomore-level only)




Pre-tutorial
Post-tutorial – with answers and explanation
Controlled environment – 2 one-hour sessions
Automation with Blackboard
Safety practicum

10-20 minutes/group, Password given
Grade comparison (sophomore-level only)

Quiz assessing knowledge of concept of stress-strain analysis
Feedback – usefulness and quality


Likert-type
Written responses
UMMS
Department of Biomedical Engineering
Example questionnaire
UMMS
Department of Biomedical Engineering
Assessment of tutorials
Questionnaires


34% ± 9%
71% ± 12%
Pre-tutorial score
Post-tutorial score
p < 0.05 paired t-test in all groups (except M5)
post
pre
% correct answers
100%
80%
60%
40%
20%
0%
M1
M2
M3
M4
M5
M6
S1
S3
S4
S5
S6
S7
Tutorial questionnaire
UMMS
Department of Biomedical Engineering
Assessment of tutorials
Compliance with questionnaires

Completed 100%: n = 27 of 47 (57% compliance)
Completed >90%: n = 36 of 47 (77% compliance)
100%
% completed

R2 = 0.84
95%
90%
85%
80%
75%
1
2
3
4
5
6
7
8
9 10 11 12
Questionnaire #
UMMS
Department of Biomedical Engineering
Feedback survey
DON’T
KNOW
EXCELLENT
VERY GOOD
GOOD
TUTORIAL
FAIR
Please rate the general quality of
each tutorial by checking the one cell
for each tutorial that most closely
matches your opinion.
POOR
DON’T
KNOW
EXTREMELY
QUITE
SOMEWHAT
JUST A
LITTLE
NOT AT ALL
Please rate the usefulness of each
tutorial in helping you to successfully
complete the lab by checking the one
cell for each tutorial that most closely
matches your opinion.
Tutorial M1
Tutorial M2
Tutorial M3
Tutorial M4
Tutorial M5
Tutorial M6
Tutorial S1
Tutorial S3
Tutorial S4
Tutorial S5
Tutorial S6
Tutorial S7
UMMS
Department of Biomedical Engineering
Feedback survey
In each shaded section below, please check the one box that most closely matches your opinion.
In attempting to understand how to do the lab, I found these tutorials, in general and as a whole, were
 not at all useful.
 not very useful.
 neither not useful nor useful.
 somewhat useful.
 very useful.
 something that made me very confused.
 something that made me a little confused.
 something that neither confused nor clarified.
 something that made the material a little clearer.
 something that made the material much clearer.
 much less helpful than written materials.
 less helpful than written materials.
 as helpful as written materials.
 more helpful than written materials.
 much more helpful than written materials.
 much less helpful than one-on-one time with the prof.
 less helpful than one-on-one time with the prof.
 as helpful as one-on-one time with the prof.
 more helpful than one-on-one time with the prof.
 much more helpful than one-on-one time with the prof.
 very difficult to use.
 difficult to use.
 neither difficult nor easy to use.
 easy to use.
 very easy to use.
 very inconvenient to use.
 inconvenient to use.
 neither inconvenient nor convenient to use.
 convenient to use.
 very convenient to use.
What about the Tutorials would you change?
What did you especially like about the Tutorials and your experience using
them?
Please identify aspects of particular Tutorials that were especially
problematic.
UMMS
Department of Biomedical Engineering
Feedback – Likert scale
Sophomore
Junior
completion of survey
74% (35/47)
94% (16/17)
general quality either “good,” “very good,”
or “excellent.”
70% to 90%
80% to 93%
60%
100%
54% to 72%
73% to 100%
“as helpful,” “more helpful,” or “much more
helpful” than written materials
80%
100%
tutorials alone were adequate (or better) in
preparing them for their lab
60%
75%
“as helpful” or “more helpful” than one-onone time with the professor
30%
75%
“easy” or “very easy” to use
“quite” or “extremely” useful (all but one)
UMMS
Department of Biomedical Engineering
Quiz grade comparison
Uniaxial test on leather specimen
1000
Load (N)
800
1st cycle
10th cycle
600
400
200
0
0
# students
Average
SD
CV%
Uniaxial test data quiz
2003
2004
2005
(no lab or
(demo
(limited
demo)
only)
lab)
41
45
46*
74
74
85
18
17
13
24%
23%
15%
5
10
Extension (mm)
15
Final course grades
2003
2004
2005
(no lab or
(demo
(limited
demo)
only)
lab)
41
45
47
85
82
86
13
12
8
16%
15%
10%
For quiz: P<0.001 by ANOVA with 2005 different than 2003 and 2004 (P < 0.05, Dunn’s)
For final course grades: P = 0.175 by ANOVA (not significant).
*One student did not take this particular quiz
UMMS
Department of Biomedical Engineering
Discussion
Motivation



Provide labs to students with resource savings
in the long term
Available tutorials (MTS, Microsoft) not well
suited, much longer
General enough for multiple types of users at
WPI and beyond
Assessment




Effectiveness of tutorials and aspects to
improve
Not video vs. written or professor
Not lab vs. no lab
Controls for summative evaluations not
practical
Resource requirement



High initial investment (team, time)
Written materials require effort also
Advantage - technical staff and students
UMMS
Department of Biomedical Engineering
Discussion
Overall success – short term



Students passed practicum, completed laboratories
Sophomore, junior, and grad as well
Instructor time in lab minimal
Questionnaires


Controlled viewing, but lower scores than desired
Too many (12 x 2), too much time commitment
Feedback


Too broad for one lab in sophomore-level course
Well received when used as designed
Increased use of equipment - time logged


Sophomore-level course > 50 hours; ~ 1hr/student
Junior-level course >20 hours/student, relatively
few scheduling conflicts
Facilitated challenge-based laboratories
True time and resource savings - future
UMMS
Department of Biomedical Engineering
Acknowledgements
Allen Hoffman

tutorial content
Elias Wilson (TA)

questionnaire and practicum
administration
Megan Murphy and Dan Filipe


sophomores
tutorial creation
Paula Quinn

assessment consultant
Ross Shonat (NIH)

Funding

Adam Jacobi and David
Taranto (WPI ATC)

production support
introduction to formal CBI

WPI Educational Development
Council Grant
WPI Teaching Technology
Fellowship
UMMS
Department of Biomedical Engineering
Questions?
UMMS
Department of Biomedical Engineering
Educational framework
Challenge-based learning (self-guided)

Building on HPL

framework1

Communication
of findings
Generate
ideas
Testing &
analysis
Literature &
interview
Learning
Cycle
Learner-, community-, knowledge-, and
assessment-centered
Self-paced

The “Challenge”
& problem
identification
Aid in the development of
critical thinking – authentic problems
Allow for differences
Allow greater number of student-hours in
lab
1Bransford,
Brown, and Cocking, 2000
Research
& revise
UMMS
Department of Biomedical Engineering