Transcript Keystone ICU Project: Measurement

Measuring Progress in Patient Safety
Peter Pronovost, MD, PhD, FCCM
Johns Hopkins University
Exercise
Please answer each question with a score of 1 to 5.
1 is below average, 3 is average and 5 is above average
•
•
•
•
•
How smart am I
How hard do I work
How kind am I
How tall am I
How good is the quality of care we provide
Improving Sepsis
Care
Mortality
(n= 19 ICUs)
ICU LOS
14.0
50.0
41.8
12.0
40.0
10.0
10.0
Days
%
30.0
21.9
20.0
13.1
10.0
7.6
8.0
6.2
6.0
4.0
2.0
0.0
0.0
Oct - Dec
2003
Mar - May
2004
July - Sept
2004
69% Reduction (p < 0.001)
Oct - Dec
2003
Mar - May
2004
July - Sept
2004
36% Reduction (NS)
Improving Sepsis
Care
Mortality
(n= 19 ICUs)
ICU LOS
14.0
50.0
41.8
12.0
40.0
10.0
10.0
Days
%
30.0
21.9
20.0
13.1
10.0
7.6
8.0
6.2
6.0
4.0
2.0
0.0
0.0
Oct - Dec
2003
Mar - May
2004
July - Sept
2004
69% Reduction (p < 0.001)
Oct - Dec
2003
Mar - May
2004
July - Sept
2004
36% Reduction (NS)
Central Mandate
Scientifically
Sound
x
Feasible
Local Wisdom
Conceptual model for measuring safety
Structure
Have we reduced the
likelihood of harm?
Process
Outcome
How often do we do
what we are
supposed to?
How often do
we harm?
IT
Context
Have we created a culture of safety?
Adapted from Donebedian
Keystone ICU Safety Dashboard
2004
2006
How often did we harm (BSI)
2.8/1000
0
How often do we do what we
should
How often did we learn from
mistakes
% Needs improvement in
Safety climate
Teamwork climate
66%
95%
100s
100s
84%
82%
43%
42%
Pronovost JAMA 2007
Engage
Evaluate
Explain why the
interventions are
important
Regularly assess
performance
measures
Educate
Share the evidence
supporting the
interventions
Execute
Design an intervention “toolkit” targeted
to barriers employing standardization,
independent checks and reminders, and
learning from mistakes
Pronovost BMJ 2008
Comprehensive Unit-based Safety Program
(CUSP)
1. Educate staff on science of safety
http://www.jhsph.edu/ctlt/training/patient_safety.
html
2. Identify defects
3. Assign executive to adopt unit
4. Learn from one defect per quarter and
implement teamwork tools
Pronovost J, Patient Safety, 2005
What can be measured as a valid rate?
• Rate requires
– Numerator- event
– Denominator- those at risk for event
– Surveillance for events and those at risk
• Minimal and Known Error
– Random error
– Systematic error
Sources Variation in
Safety measures
• True variation in Safety
• V data quality/definition/methods of
collection
• V case mix
• V historical rates
• Chance
Measuring Preventable Harm
• Measure rate or counts directly
– High sensitivity low specificity
• Estimate observed/expected (O/E)
– Low sensitivity and specificity
• Link process and outcome
– High specificity and moderate sensitivity
Process Measures
• Validity of the construct
• Validity of how we measure construct
It is Ok to have non-rate
measures
Self reported measures are generally
not valid as rates
A common mistake is interpreting a
non-rate measure as a valid rate
Learning from Mistakes
•
•
•
•
What happened?
Why did it happen (system lenses)
What could you do to reduce risk
How to you know risk was reduced
– Create policy/process/procedure
– Ensure staff know policy
– Evaluate if policy is used correctly
Pronovost 2005 JCJQI
Patient Safety Learning Communities
•Identify Hazards
•(
4. Evaluate
Effectiveness of
Risk Reduction
2. Analyze &
Prioritize Hazards
3. Mitigate Risks
Patient safety learning communities relate to each other in a gear like fashion: as the
identified hazards require stronger levels of intervention to achieve mitigation, the next
learning community is engaged in action, eventually feeding back to the group that provided
the initial thrust. Each group (unit, hospital, industry) follows the same four- step process,
but they engage unique matrices of stakeholders to mitigate hazards that are within their
locus of control.
Pronovost Health affairs in press
GYN/OB
JHOC
Medicine
Neurosciences
Oncology
Ophthalmology
FAC: Fetal Assessment
Center/OB Ultrasound
GSS - Shared Specialty
Suite
Asthma & Allergy - Allergy
& Clinical Immunology
BRU
GSS - Medical Oncology
GSS - Wilmer 110
GSS - GYN/OB 420
JHOPC - Express Testing
Asthma & Allergy Pulmonary
EMU
IPOP Clinic - HIPOP
Location
GSS - Wilmer Laser Center
GSS - GYN/REI
JHOPC - OR
Asthma & Allergy Rheumatology
JHOPC Neurosciences
IPOP Clinic - IPOP
Location
WECP & ER
HAL-2
JHOPC - PACU
Blalock 4 - Endoscopy
MEY 8 (12)
Weinberg OPD - 1st Floor
Wilmer OR
JHOPC GYN/OB
WM - Shared Specialty
Suite
Blalock 5 Echo Lab (2)
MEY 9 (5)
Weinberg OPD - 2nd Floor
Wilmer PACU
MCE
Cardiac CT
NCCU7
WGA 5 (5)
Wilmer White Marsh
NEL-2 Nursery
CCP-5 (5)
WGB 5
Wilmer: Other - E Balt
Divisions
NEL-2 Obstetric OR
CCU-5 (7)
WGC-5 (3)
Wilmer: Other - Satellites
NEL-2 PACU
CVC
WGD 5
Nelson Harvey 2
CVIL- CardioVascular
Interventional Lab
OSL-2
Dialysis Unit
OSL-3 Nursery
GSS - Internal Medicine
OSL-3
HAL-5 (5)
WGB-4
HAL-8 (7)
Hospitalist Unit (5)
CAST
• Each contributing factor rate
– importance of the problem and contributing
factors in causing the accident
– importance of the problem and contributing
factors in future accidents
• Each Intervention rate
– How well the intervention solves the problem or mitigates
the contributing factors for the accident
– Rates the team belief that the intervention will be
implemented and executed as intended
What is Culture*?:
“The way we do things
around here”
1 attitude = opinion…everyone’s attitude = culture
*aka Climate
Factor: Definition
Example items
Job satisfaction: positivity about the work
experience
–I like my job
–This hospital is a good place to work
Teamwork climate: perceived quality of
collaboration between the personnel in this
unit
–Disagreements in the ICU are appropriately resolved
(i.e., not who is right, but what is best for the patient)
–Our doctors and nurses work together as a well
coordinated team
Safety climate: perceptions of a strong and
proactive commitment to patient safety in
this unit
–I would feel safe being treated in this ICU
–Medical errors are handled appropriately in this ICU
Perceptions of management: approval of
managerial action
–Hospital management supports my daily efforts in
the ICU
–Hospital management does not knowingly
compromise the safety of patients
Stress recognition: acknowledgement of
how performance is influenced by stressors
(permitted to be human)
–I am less effective at work when fatigued
–When my workload becomes excessive, my
performance is impaired
Working conditions: perceived quality of
the work environment and logistical support
(staffing, training, etc.)
–Trainees in my discipline are adequately supervised
–This hospital deals constructively with problem
personnel
Executive Perceptions vs.
Frontline Perceptions:
Executives overestimate:
Teamwork Climate 4X
Safety Climate 2.5X
Executive Confidence vs.
Executive Accuracy:
-Often wrong but rarely in doubt…
-Currently no incoming data-streams
-Halo Effects
-Frontline data fills the gap
*
*
*
*
*
*
* Statistically Significant
71 Teamwork Climate 2008
67 Teamwork Climate 2007
64
Teamwork
Climate2005
2006
62 Teamwork
Climate
70 Safety Climate 2008
65 Safety Climate 2007
60
Safety
Climate
2006
59 Safety Climate
2005
#4. “I Would Feel Safe Being Treated Here As A Patient.”
100
CSICU T2
90
70
60
50
40
30
20
10
0
CSICU T1
% of respondents within an ICU that agree
80
#3. “Nurse Input Is Well Received In This ICU.”
100
CSICU T2
90
70
60
50
40
30
20
10
0
CSICU T1
% of respondents within an ICU that agree
80
#26. “In This ICU, It Is Difficult To Speak Up If I Perceive A
Problem With Patient Care.”
100
90
CSICU T1
70
60
50
40
30
20
10
0
CSICU T2
% of respondents within an ICU that agree
80
#32. “Disagreements In This ICU Are Resolved Appropriately
(i.e. not who is right, but what is best for the patient).”
100
90
70
50
40
30
20
10
0
CSICU T2
60
CSICU T1
% of respondents within an ICU that agree
80
Questions for Reflection
• How do you know you are safer?
• How will you become more efficient in
your measurement efforts?
• How will you better tap into local wisdom?
Focus and Execute