Transcript Errors in Pharmacy - University of Portsmouth

Graphical methods for turning data
into information
Martin Utley
Clinical Operational Research Unit (CORU)
University College London
www.ucl.ac.uk/operational-research
Monitoring outcomes to improve outcomes
Data
Information
system
Care
process
Need to get every step right
Analysis of
data
Feedback
Steps discussed in this talk
Data
Information
system
Care
process
Analysis of
data
Feedback
Case study 1:
monitoring outcomes of cardiac surgery
Work done by:
Steve Gallivan
Chris Sherlaw-Johnson
Jocelyn Lovegrove
Tom Treasure
Oswaldo Valencia
CORU
St Georges / Guy’s
Mortality data for cardiac surgery
0000000000000000100000000
0000000000000000000000000
0000000000000100000000000
0000000100000000000000000
0000000000000000000100000
0000000000000000000001010
Mortality data for cardiac surgery
0000000000000000100000000
0000000000000000000000000
0000000000000100000000000
0000000100000000000000000
0000000000000000000100000
0000000000000000000001010
6 perioperative deaths in 150 cases
Graphical presentation of data
Cumulative deaths
10
First used in the context of surgery by DeLeval
8
6
4
2
0
0
20
40
60
80
100
120
Operation number
140
Graphical presentation of data
Cumulative deaths
10
Is this series of outcomes good or bad?
8
6
4
2
0
0
20
40
60
80
100
120
Operation number
140
Patient factors that contribute to risk of death
Age
Co-morbidity
Emergency
status
Risk of
of
Risk
perioperative
perioperative
death
death
LV function
Repeat
operation
To be fair, assessment of outcomes should account for case-mix
Compare outcomes to expectations
Cumulative deaths
10
Expected mortality (from risk model)
8
Actual mortality
6
Par for the course
4
Net life
gain
2
0
0
20
40
60
80
100
120
Operation number
140
Variable Life Adjusted Display (VLAD)
Net life gain
5
4
VLAD plot for a single surgeon
3
2
1
0
0
20
40
60
80
100
120
140
Operation number
Comparing three fictitious surgeons
Net life gain
5
4
3
2
1
0
-1
-2
-3
-4
Unexpected
death
Vlad the impaler
The venerable bleed
Hawkeye Pierce
Survivor
against
the odds
0
20
40
60
80
100
120
140
Operation number
Comparing surgeons within a unit
Net life gain
Operation number
Tools to assist interpretation
Net life gain
15
+1% tail
Net life gain
10
+5%
+10%
5
+25%
0
0
50
100
150
200
250
300
350
-5
-25%
-10%
-5%
-10
-15
-1%
Operation number
Operation number
VLAD adopted worldwide
Keys to success
• Surgeons say that visual display is intuitive
• Can be used to identify possible problems in real time
• Monitoring tool “rewards” good outcomes rather than
just punish bad outcomes
• Clinical champion
Case study 2:
monitoring prescription errors
Collaborators:
Steve Gallivan
Christos Paschalides
Bryony Dean Franklin
Ann Jacklin
Kara O’Grady
CORU
Hammersmith
Nick Barber London School of Pharmacy
Funded by the Trustees of Hammersmith Hospitals NHS Trust
Monitoring the prescribing process
Data
Care
process
Junior doctor writes
prescription
Ward pharmacist corrects
any errors that he or she
identifies
Feedback
Information
system
Analysis of
data
Monitoring the prescribing process
Data
Information
system
Care
process
Prescription errors deemed
sufficiently serious by
pharmacist are logged as
incidents
Analysis of
data
Feedback
Monitoring the prescribing process
Data
Information
system
Care
process
Extensive research on
nature and rates of reported
prescription errors
Analysis of
data
Feedback
The problem
Data
Information
system
Care
process
No systematic
feedback to
prescribers
Analysis of
data
With no feedback, how can we expect
prescribing practice to improve?
Feasibility study
...& records
consultant team,
number of new
orders and all
errors identified
Data entered onto
spreadsheet
Junior doctors write prescriptions
Ward pharmacist
checks new medication orders...
Graphical
summaries
prepared
Feedback sent
to head of specialty
Graphical summaries kept simple
Number of new orders
1200
Your specialty
93 (12%) of 773 new orders
had at least one error
1000
800
New medication orders with at least one error
New medication orders without any errors
600
400
200
0
Specialty
C
Specialty
How much statistics?
Proportion of new
Proportion
orders w ithof
annew
orderserror
with an error
16%
95% confidence
interval
14%
12%
10%
8%
6%
Proportion for whole directorate
4%
Proportion for whole directorate
2%
0%
AMUC
Admissions
Specialty
C
CXH
other specialties
AllAllother
specialties
Specialty
Performance over time
Cumulative number
of orders with an error
Admissions
Specialty
AMUC
C
CXH
100
23/05/05
90
09/05/05
80
Other specialties
70
25/04/05
60
50
11/04/05
40
30
30/03/05
20
16/03/05
10
02/03/05
16/02/05
0
0
200
400
600
800
1000
Cumulative number of new medication orders
Prototype feedback pages 1 and 2
Prototype feedback page 3
Comments written by the
Trust’s Principal Pharmacist
highlighting any issues that
arise from the data.
Representative examples
of prescribing errors
recorded over the period.
Hang on...
Data
Information
system
Care
process
Analysis of
data
Feedback
...does this
process lead
to improvement?
Planned study
Does monitoring and feedback reduce
errors in the prescribing process?
Error rate
Time
Monitor and feedback results
Summary
• Succinct graphical methods can be very useful in
the analysis of clinical data and in feeding back
information to clinical teams.
• Appropriate feedback cannot do any harm, can it?
• Evaluation of monitoring systems in terms of
clinical improvement is desirable.