Transcript Measurement for Safety Improvement

Measurement for Safety
Improvement
Day 1 Evaluation Feedback
• We didn’t talk about the stakeholder grid which would’ve
been something new for me, or the SWOT analysis
although this wouldn’t have been new to me.
• The morning was a bit disjointed, I think the interaction
between the two presenters was supposed to be a
lighthearted approach but actually it was a bit distracting
• Afternoon session rushed and therefore not as
interactive
• Too much presentation, need more participation
• I was expecting more coaching
Day 1 Evaluation Feedback
• Helped out with the project and discuss pathway of
project
• Very engaging and good interaction
• Both presenters had the ability to develop and keep
audience engaged
• Helped my understanding with quality improvement and
aim statements.
• Introduced me to understanding programme objectives
• All good, nice lunch, plenty to eat and drink
Todays Facilitators
Jodie Whittle
Amanda Huddleston
Acknowledgements
Mike Davidge
Director NHS Elect
Head of Improvement Methodology, 1000 Lives Improvement Service
Learning Objectives
To have a basic understanding of:
• The 7 steps to measurement for improvement.
Today 1-5
• How to collect, present and analyse data
effectively.
• Link current safety data and initiatives to safety
improvement projects.
Quiz Time
All data and no theory
Form
printed
Form
piloted
Pharmacy
included
May-08
Apr-08
Mar-08
Feb-08
Jan-08
Dec-07
Nov-07
Oct-07
Sep-07
Aug-07
Jul-07
Jun-07
Letter
from CDs
May-07
100
90
80
70
60
50
40
30
20
10
0
Apr-07
% reconciled
% medicines reconciled in a Medical Admissions Unit
8
The 3 reasons for measurement
Source: Solberg et al 1997
9
7 Steps to measurement
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
5 Analyse
& present
4 Collect
data
Measurement Sins
Having no baseline
(or having a
compromised one)
Measuring the
wrong thing
Only collecting data
at 2 points in time
Inappropriate or
mindless use of
statistics
Presenting results in
a misleading way
11
The Measures Checklist
• Part One
•
•
•
•
Why important?
Who owns?
Definitions
Goals
• Part Two
• Collect
• Analyse
• Review
12
Step 1: Decide your aim
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
5 Analyse
& present
4 Collect
data
Driver Diagrams
• A driver diagram helps you to think
about the aim that you want to
achieve and more importantly what
necessary changes you need to
make.
• It is a simple way of organising on
one page the actions that will help
you achieve your aim
Example: Driver Diagram
Primary Drivers
Care Planning
Aim
A reduction in
incidents of
violence&
aggression by
20% in the
Unit during
2012/13
Environment
Therapeutic
Interventions
Workforce
© 2010 AQuA
Primary drivers are the systems
changes which will contribute to
achieving the Aim outcome measure.
Secondary Drivers
A. Raise awareness
B. Introduce a SU advanced
statement re management of
V&A.
A. Post all records (agreed actions)
of the community meetings in a
central area.
B. Post a weekly activity programme
at a central point on the ward.
A. Develop a formal process
regarding the planning of social &
therapeutic activities.
B. Introduce a community meeting.
C. Redesign role of staff member –
activity co-ordinator.
A. Review and compare data – make
data easily available to staff..
B. Identify specific times/places/
personnel involved in V&A.
C. Provide poster for staff comments
re new PDSAs.
D. Provide staff with written updates
re V&A to inform staff on return
from days off.
E. Recruit permanent staff to vacant
posts.
Secondary drivers are interventions
associated with primary drivers. They
can be used to create projects or
change packages that will affect the
primary driver.
15
Aim Statement
Brief rationale.
(What’s the problem? Why is it important? What are we going to do about it?)
What exactly are you trying to achieve?
For whom are you going to improve it for?
By how much will you improve it?
By when are you aiming to achieve it?
Final Aim Statement
Adapted from
16
Group Work & Feedback
Comfort
Break
Step 2: Choose measures
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
5 Analyse
& present
4 Collect
data
Family of measures
• Outcome
• Process
• Balancing
If the family were a fruit, it would be an
orange, a circle of sections, held together
but separable - each segment distinct
Kate Cheema
Specialist Information Analyst
Quality Observatory
Central Southern CSU
Aim
Primary
drivers
Reduce price per
litre
Secondary
drivers
Tertiary
drivers
Cost
per litre
Fill up at
supermarket
Combine
journeys
Decrease
household fuel
costs by 50% by
December 2014
Reduce miles
driven
Use alternatives
Plan ahead
No of non
vehicle
journeys
Cycle or walk short
distances
Work from home
Number
of Miles
Fuel
Costs
Actions/
Interventions
Improve car
efficiency
Buy a diesel car
next time
Limit speed
Stick to 70 mph on
motorway
Use appropriate
gear
Get into highest gear
more quickly
Increase
mpg
Improve driving
patterns
Accelerate more
slowly
Drive
smoothly
Book onto AA
course
Other types of measures we use
KPIs
CQUINS /
QOFS
Improvement
Project
Clinical
Outcomes
Step 3: Define measures
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
5 Analyse
& present
4 Collect
data
An operational definition is a description,
in quantifiable terms, of what to measure
and the steps to follow to measure it
consistently
Understanding your population
• All Population
• Simple Random Sampling
• Proportional Stratified
Random Sampling
• Judgement Sampling
Step 4: Collect Data
1 Decide aim
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
5 Analyse
& present
4 Collect
data
Monthly data shows
improvement
3.5
Average length of pre-ward stay on Barnsley
Stroke Ward
from 01/2007 to 07/2007
3
2.5
2
The chart shows
the average
monthly length of
time before
patients got to
the Stroke ward
1.5
1
0.5
0
1
2
3
4
5
6
7
Months
28
Weekly data tells a
slightly different story
Average length of pre-ward stay on Barnsley
Stroke Ward
from 01/2007 to 07/2007
9.0
8.0
7.0
6.0
5.0
4.0
3.0
2.0
1.0
31
29
27
25
23
21
19
17
15
13
11
9
7
5
3
1
0.0
Weeks
29
Patient level data adds
another level of understanding
30
Activity –
Complete Part
1 of the
Measurement
Checklist for 1
measure
Lunch
Step 5: Analyse and present
1 Decide aim
“The type of
presentation you use
has a crucial effect
on how you react to
data”
2 Choose measures
3 Define measures
6 Review
measures
7 Repeat
steps
4-6
5 Analyse &
present
4 Collect
data
Importance of Collecting &
Presenting Timely Data
Number of Complaints Per Year
400
350
300
250
200
150
100
50
0
2010
2011
2011 Complaints distribution –
per quarter
Comparison of Complaints
70
60
50
40
2011
2012
30
20
10
0
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Oct
Nov
Dec
Sept
Variation
How long does it take you to get to
work?
39
Can we classify variation?
Common
Cause
(Random)
Special
Cause
(Assignable)
• Sum of many small variations from real but small causes
that are inherent in any process
• Therefore it cannot be traced back to a root cause
• Stable in time (follows the laws of probability) and
therefore relatively predictable
• Random variation is a matter of measurement, not goal
setting
• Represents "appropriate" variation
• Variation arising from a single cause that is not part of the
process
• Therefore can be traced, identified and eliminated (or
implemented)
• Irregular in time and therefore unpredictable
• Represents "inappropriate" variation
40
What’s a person’s
normal heart rate?
Age
Fitness Quotient for Women
Athlete Excellent Good Above Average Average Below Average Poor
18-25
54-60
61-65
66-69
70-73
74-78
79-84
85+
26-35
54-59
60-64
65-68
69-72
73-76
77-82
83+
36-45
54-59
60-64
65-69
70-73
74-78
79-84
85+
46-55
54-60
61-65
66-69
70-73
74-77
78-83
84+
56-65
54-59
60-64
65-68
69-73
74-77
78-83
84+
65+
54-59
60-64
65-68
69-72
73-76
77-84
84+
41
What is a Run Chart?
Run Charts
• Plot data in a time order
• Can analyse by studying where values lie and
why they are distributed around the median
• Help us understand if the variation is
‘common’ or ‘special’
• Illustrates the impact of an improvement
intervention
Run Chart Exercise
Week Date
% compliance
with hand
hygiene
Week Date
% compliance
with hand
hygiene
1
01-Apr
50
14
01-Jul
89
2
08-Apr
43
15
08-Jul
78
3
15-Apr
20
16
15-Jul
92
4
22-Apr
45
17
22-Jul
50
5
29-Apr
70
18
29-Jul
65
6
06-May
54
19
05-Aug
40
7
13-May
34
20
12-Aug
54
8
20-May
67
21
19-Aug
48
9
27-May
32
22
26-Aug
37
10
03-Jun
79
23
02-Sep
50
11
10-Jun
85
24
09-Sep
63
12
17-Jun
90
25
16-Sep
45
13
24-Jun
70
%
100
90
80
70
60
50
40
30
20
10
0
Wk Beginning
16-Sep
9-Sep
2-Sep
26-Aug
19-Aug
12-Aug
5-Aug
29-Jul
22-Jul
15-Jul
8-Jul
1-Jul
24-Jun
17-Jun
10-Jun
3-Jun
27-May
20-May
13-May
6-May
29-Apr
22-Apr
15-Apr
8-Apr
1-Apr
% Compliance with Hand Hygiene
The median value
The Median is the "middle number" (in a sorted list of numbers).
How to Find the Median Value
To find the Median, place the numbers you are given in value order and find
the middle number.
Same number in your list more than once?
Even if you have 2 numbers the same you still place them both in the list.
Two Numbers in the Middle?
BUT, if there are an even amount of numbers things are slightly different.
In that case we need to find the middle pair of numbers, and then find the
value that would be half way between them. This is easily done by adding them
together and dividing by two.
%
100
90
80
70
60
50
40
30
20
10
0
Wk Beginning
16-Sep
9-Sep
2-Sep
26-Aug
19-Aug
12-Aug
5-Aug
29-Jul
22-Jul
15-Jul
8-Jul
1-Jul
24-Jun
17-Jun
10-Jun
3-Jun
27-May
20-May
13-May
6-May
29-Apr
22-Apr
15-Apr
8-Apr
1-Apr
With Median
Run Chart Rules
Shift
Trend
Astronomical
point
Runs
With Rules Applied
Shift
Runs
What is SPC?
Statistical Process Control
250
Performance Report – Number of Admissions
No of Admissions
200
150
100
50
No Admissions
0
1
2
3
4
5
6
Mean
7
8
Lower Limit (66.5)
9
10
11
Week
12
13
14
Upper Limit (222.4)
15
16
17
18
19
20
51
Summarise – the mean
• Mean defined
– the arithmetic mean (or simply the mean) of a list
of numbers is the sum of all of the list divided by
the number of items in the list.
• Why do we use it?
– Provides a useful estimate of the typical value
– Easy to understand and calculate
• What do we need to beware of?
– A few extremely large values can inflate the mean
– It tells us little about the spread of values
52
Constructing the chart
There are 5 steps to constructing your
chart:
1. Plot the individual values
2. Calculate the mean (X) and plot it
3. Derive the moving range values
4. Calculate the average moving
range (R)
5. Derive upper and lower limits from
this and plot them
53
1. Plot individual values
% Compliance with hand hygiene (weekly) April – Sept 2008
100
90
80
70
60
% 50
40
30
20
10
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
1 Apr 8
15 22 29 6 May 13 20 27 3 Jun 10 17 24 1 Jul 8
15 22 29 5 Aug 12 19 26 2 Sep 9
16
Week
54
2. Calculate Mean & plot it
% Compliance with hand hygiene (weekly) April – Sept 2008
100
90
80
70
60
% 50
40
30
20
10
x
x
x
x
x
x
x
x
x
Mean is 58
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
1 Apr 8
15 22 29 6 May 13 20 27 3 Jun 10 17 24 1 Jul 8
15 22 29 5 Aug 12 19 26 2 Sep 9
16
Week
55
3. Derive moving range
These are required to calculate the control limits
The first row contains the chart data
Use the second row to record the difference between successive data values
X Data
Moving
Range
0
0
5
4
8
9
0
5
1
4
1
The difference is always recorded as a positive value
4. Calculate Average
Moving Range
R1
R2
R3
R4
R5
R6
R7
R8
7
23
25
25
16
20
33
35
Average Moving Range
R1
R23
R24
13
18
mR
= 7+23+25+25+……………………….+18
24
Sum of the R’s
Number of readings
= 463
24
57
5. Derive process limits
Derive measure of variation
(1 sigma) as:
Average moving range
1.128
19.3
1.128
Calculate upper limit as:
Mean + 3 sigma
=58 + (3*(19.3/1.128))
Calculate lower limit as:
Mean – 3 sigma
=58 - (3*(19.3/1.128))
58
5. Plot limits
% Compliance with hand hygiene (weekly) April – Sept 2008
100
90
80
70
60
% 50
40
30
20
10
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
1 Apr 8
Upper limit 109
x
x
x
x
x
Lower limit 7
15 22 29 6 May 13 20 27 3 Jun 10 17 24 1 Jul 8
15 22 29 5 Aug 12 19 26 2 Sep 9
16
Week
59
The XmR chart (or I-chart)
X – for individual values
mR – for moving range
Makes fewest assumptions about how the data
values are distributed around the mean
Will give you reliable results in almost all situations
60
Some software to help you
create SPC charts
Go to:
www.valuesystemdesign.com
Special causes
Rule 1 - Any point outside one of the control limits
X
Point above UCL
UCL
X
X
X
X
X
X
X
X
X
X
X
X
MEAN
X
X
X
X
X
X
LCL
Point below LCL
X
62
Special causes
Rule 2 - A run of seven points all above or all below the centre
line, or all increasing or all decreasing
UCL
7 Points above centre line
X
X
X
X
X
X
X
X
X
X
MEAN
X
X
X
X
X
X
X
X
X
X
7 Points below centre line
LCL
63
Special causes
Rule 2 - A run of seven points all above or all below the centre
line, or all increasing or all decreasing
UCL
7 Points upward direction
X
X
X
X
X
X
X
X
X
X
MEAN
X
X
X
X
X
X
X
X
X
X
7 Points downward direction
LCL
64
Special causes
Rule 3 - Any “unusual” pattern or trends within the control limits
Cyclic pattern
X
X
X
X
X
UCL
X
XX
X
XX
X
X
X
X
Trend pattern
X
X
X
X
MEAN
X
X
X
X
XX
XX
X
X
X
X
X
X
X
X
XX
LCL
65
Special causes
Rule 4
Less than 2/3 of all the
points fall in this zone
X
X
More than 2/3 of all the
points fall in this zone
UCL
X
X
X
X
X
X
MEAN
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
LCL
66
Process out of control
• These rules are important!
• They tell us if the process is stable or
unstable
• They tell us if common or special cause
• variation is present
Remember the rules!
• Outside control limits
• Run of 7 or more consecutive points
• Patterns
• Rule of thirds
67
Practical Exercise
•
Interpreting the SPC Rules
•
See handout
•
Now apply to your own chart
7/20/2015
© AQuA Academy 2013

68
Constructing a Pareto Chart
Count of Reasons for Complaints
250
200
150
100
203
165
112
50
85
66
50
0
Lack of Time
Lack of
Consistency
Poor
Information
Delayed
Treatment
Incompetence
Poor Care
Count & Percentage of Reasons of Complaints
250
200
Number
150
100
50
30
24
16
12
10
7
Poor
Information
Delayed
Treatment
Incompetence
Poor Care
0
Lack of Time
Lack of
Consistency
Count, % and cumulative % of Reasons of Complaints
250
100
100
93
90
83
200
80
70
70
60
Number
54
50
100
40
30
30
50
20
30
24
16
12
10
7
0
10
0
Lack of Time
Lack of
Consistency
Poor
Information
Delayed
Treatment
Incompetence
Poor Care
%
150
Pareto Chart
250
100
90
200
80
70
80% of problems
created by these
issues
60
50
100
40
30
50
20
10
0
0
Lack of Time
Lack of
Consistency
Poor
Information
Delayed
Treatment
Incompetence
Poor Care
%
Number
150
Comfort
Break
Current Safety Data
© 2014 AQuA
Current Safety Data Sources
• Safety Thermometer – Health & Social
Care Information Centre
• Incident Reporting – Patient Safety, NHS
England
• Summary Hospital Mortality Indicator Health & Social Care Information Centre
• Infection Rates – Public Health England
So what?..........Tying it all together!
Open & Honest Care
• NHS Safety Thermometer
• Information on healthcare associated infection,
(MRSA and C Diff)
• Pressure ulcers
• Falls causing moderate or greater harm
• Information on staff experience
• Information on patient experience including Friends and Family Test
• A patient story
• An improvement story describing what the trust has learnt and what
improvements they are making.
• Some Trusts may choose to add additional information
• Staffing levels
• Never events
Measuring & Monitoring Safety
Past Harm
Integration
and Learning
Reliability
Safety
measurement
and
monitoring
Anticipation
and
Preparedness
Sensitivity to
Operations
Activity –
Complete Part 2
of the
Measurement
Checklist
81
Quiz Answers
Learning Objectives
To have a basic understanding of:
• The 7 steps to measurement for improvement.
Today 1-5
• How to collect, present and analyse data
effectively.
• Link current safety data and initiatives to safety
improvement projects.