Transcript Presentation: A comparative assessment of MPharm students...

Comparative assessment of
student pharmacists’
performance in computer- and
paper-based dispensing
exercises
Monika Sareen and Mike Daly
The School of Pharmacy, Wolverhampton
[email protected]
Background
 Computer simulations can be used to imitate
practical tasks
 Widely used in training airline pilots and medical
Anaesthetists
However….
Can computer simulation mimic common
pharmaceutical dispensing tasks ?
 Can or should they be used as tools for
assessment ?
PBEs versus CBEs
 Paper-based exercises (PBEs) are a common
method of teaching dispensing
 Large student cohorts mitigate against exposure
to practical dispensing tasks – around 30-40
items in one semester
 Computer-based exercises (CBEs) are being
developed to support PBEs
However….
 Does formative use provide appropriate
support ?
 Would summative use be fair and accurate ?
Methods
 A series of four prescription tasks were prepared
as PBEs
 These tasks were converted to a CBE using
WOLF
 Students sequentially undertook practical or
computer based dispensing tasks
 Results were marked using a standard pro-forma
already used in the department
 Students were asked for their comparative
opinions between the two teaching methods
WOLF screenshot
There are a number of compromises :
The date is static to save time
Options are presented as MCQs
Manual dexterity is not required
Label production is not required
Product selection is artificial
Skin preparations
Labels
Results - 1
 57 students undertook the two dispensing
exercises (31 male, 26 female)
Pass mark – standard 50%
Female students appeared to perform worse in
the CBE than in the (practical) PBE
Evaluation
 PBE : 16% of male students failed the pass mark of 50%, whereas
only 1 female failed (4%)
 CBE : The failure rate was 35% of males and 35% of females
There appears to be a gender-related difference :
Females performed better than males on the Paper Based Exercise
but
scored equally poorly as males on the Computer Based Exercise
Does this imply that the use of computerised
dispensing simulations could disadvantage
sub-groups of students ?
Preferences
 Students who preferred the use of the CBE over the PBE performed
better at the CBE
 No such correlation of preference and performance appeared with the
PBE
 All students stated their preference for practical paper-based
dispensing over computer simulations
 In general, students that scored higher electronically performed worse
in the manual exercise, and vice versa
Student performance measured electronically may not
automatically match their practical ability – use for
summative assessments requires proper validation
Results - 2
The exercise was repeated the following year
with a further cohort of 1st year MPharm
students (30males, 24 female).
The results were marked
against a standard
marking proforma
Females significantly outperformed males in the PBE
(P=0.0024)
Results - 3
The exercise was repeated again this year among
28 male and 39 female students.
Males :
Females :
Questions / Prompts
Questions may provide
‘answers’
which could compromise
summative assessments
Summary





These results indicate that a dispensing CBE appears to
‘work’ as a teaching tool
There does not appear to be equivalence between
students performance between the PBE and the CBE
There is currently no intention to use the CBEs
summatively
There appears to be a trend, across the studies, of
female students performing worse at CBEs than PBEs
Caution may be warranted moving to ‘digital teaching /
assessment’ to ensure that sub-groups of students are not
disadvantaged
Summary
There could be a range of factors involved :







Cultural attitudes to computers
Access to computers
Familiarity with computers
Linguistic / comprehension issues
Poor organisation of material on WOLF
Lack of ‘open book’ access to information
Awareness of being assessed electronically