Transcript No Slide Title

Analyzing & Presenting
Performance Improvement
(PI) Data
Objectives
Demonstrate an exercise that uncovers how leaders
make managerial decisions based upon data
Identify barriers to effective analysis and reporting of
PI data
Share 2 data analysis/reporting educational tools
targeted for staff
Provide sample PowerPoint slides for staff training
re: data analysis and process variability
Discuss PI information needs of leadership
CSR ©2011
1
2
3
CSR ©2011
Why aggregate and analyze?
Transform data into information
Identify current performance levels, patterns, or
trends
Determine
Whether or not improvement is needed
If a strategy to stabilize or improve performance
was effective
If design specifications met
Judge a particular process’s stability or a particular
outcome’s predictability in relation to performance
expectations
Problem #1
Lumping data together is usually not
enough!
 Aggregate #’s do not show any
“unusual” circumstances.
 If leaders take action based on data
assumptions without taking into account
unusual circumstances – they can
muck up a perfectly good process!
Should I change the route to work each day?
Time to work each day
M
i
n
u
t
e
s
September’s Rates –
Minutes to work
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
450
400
350
300
250
200
150
100
50
0
Problem #2
Before and after measures aren’t enough!
 Two aggregate measures taken before and
after a change do not in themselves
demonstrate that a process has improved.
 One needs to know the stability of the
processes that produced these aggregate
measures.
 To determine process stability, it is
necessary to look at data over time i.e., in
a time series design.
Staff Turnover
Intervention Begins 10/2009
Staff Turnover – the same data
using the # of staff over time!
3
2.5
2
1.5
1
0.5
0
2009
2010
2009
Intervention Begins 10/2009
Sheward & Deming Points
 Variation exists in all we do
 Processes that exhibit common causes of variation
are predictable within statistical limits
 Processes often have both common and special
cause variation
 How we respond to special causes is different than
our response to common cause variation
 Attempting to improve processes that contain special
causes will increase variation and waste resources
 Once special causes have been “eliminated”, it is
appropriate to consider changing the process
Common vs.
Special Cause Variation
Common Cause
 Is inherent in the design of the process.
 Is due to regular, natural, or ordinary causes.
 Results in a stable process. The variation is predictable.
 Also known as random or unassignable causes.
Special Cause
 Is due to causes not inherent in a process.
 Results in an unstable process, because the variation is not
predictable.
 Also know as non-random or assignable causes.
 Might be described as a “signal” that the process has changed.
CSR ©2011
Neither type of variation
is “good” or “bad” in
itself!
Common Cause
 Only tells you that a process is stable and predictable within
certain limits
 However, it may be functioning at an unacceptable level!
Special Cause
 Usually undesirable when you did not plan for it.
 Can also be a “signal” that a planned change was effective.
When people do not
understand variation
 See trends where there are no trends
 Blame and give credit to others for things
over which they have little or no control
 Build barriers, decrease morale, and create
an atmosphere of fear
 Never be able to fully understand past
performance, make predictions about the
future and make significant improvements in
processes
How Do We Analyze
Variation?
 Run charts and control charts are the
tools used to determine whether
variation is:
Common cause, or
Special cause
 They tell us what the process is actually
doing –
Not what we would like it to do!
Bar Graphs
 A preliminary
exploration of
data
may be timeordered
1400
2009
1200
2010
1000
800
600
 Are a common
graphical display
format
 Can be difficult for
trend detection
400
200
0
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
20
OCT
NOV
DEC
Same bar graph data displayed
in a simple Line Graph
 Offers a preliminary
view of time ordered
data
1300
LINE GRAPH
1200
 Stock market trends are
viewed in line graphs
1100
1000
900
800
 Common mistake is to
see trends where they
statistically don’t exist
700
600
500
400
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
DEC
JAN
21
FEB
MAR
APR
MAY
JUN
Run Chart
 Used to detect common
cause vs. special cause
variation
1300
RUN CHART: Monthly Calls Received
1200
1100
 Easy to construct and
evaluate
1000
900
800
700
 Less sensitive than
control charts for
identifying extreme data
points as a special
cause
MEDIAN
600
500
"COMMON CAUSE VARIATION"
400
JAN
1999
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
DEC
JAN
FEB
2000
22
MAR
APR
MAY
JUN
Run Chart = Line Graph + Center
Line*
RUN CHART: Monthly Requests for Services
1300
1200
1100
1000
900
800
MEDIAN
700
600
500
"COMMON CAUSE VARIATION"
400
JAN
1999
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
DEC
JAN
FEB
MAR
APR
MAY
JUN
2000
*The center line in a run chart is typically the median point for the data, but some
use the process average or mean as the center line
23
Run Chart Terminology
 RUN
Defined as one or more consecutive data
points occurring on the same side of the
center line
 TREND
Defined as an unusually long series of data
points steadily increasing or decreasing
24
EASY Run Chart Tests for
Special Causes
 TREND of 6 consecutive data points
steadily increasing or decreasing
 RUN of 8 consecutive data points on
one side of the center line (median or
mean)
 OUTLIER POINTS – use your judgment
whether to expend resources to
investigate further to understand cause
and determine if improvement is needed
CSR ©2011
25
Test #1
TREND of 6 consecutive data points
steadily increasing or decreasing
10
Median
2
Sequential Run of only 5 –
Common Cause Variation
1
Feb
March
April
May
June
July
Aug
Sept
Oct
Nov
Dec
Test #2
RUN of 8 consecutive data
points on one side of the center
line (median or mean)
10
Median
2
Run of 9 –
Special Cause Variation
1
Feb
March
April
May
June
July
Aug
Sept
Oct
Nov
Dec
Test #3
OUTLIER POINTS – use your
judgment whether to expend
resources to investigate further to
understand cause and determine
if improvement is needed
median
Is May’s result a special
cause????
Improvement Strategies: After making
a run or control chart, what’s next?
The type of variation determines your approach:
Special cause variation?
If negative, eliminate it.
If positive, emulate it.
But don’t change the process!
Common cause variation?
If process is functioning at an unacceptable level, change the
process!
Don’t “tamper” with individual data points!
29
How will you know your intervention is a
success?
A Special cause in the desired direction will
signal that the old process is changed for the
better.
A Special cause in the wrong direction will
indicate that your intervention was
counterproductive.
Continued common cause variation will
indicate that your intervention did not help –
CSR ©2011
but
did not hurt either.
30
Improvement Strategy
Conduct Initial
Investigation
Standardize
The
Process
Introduce
Improvement - 1
Introduce
Improvement - 2
Time 1
Time 2
Time 3
Time 4
31
Targeting Your Message
Hospital boards should hold accountable and
require full and complete explanations from
management when safety and quality
performance levels differ significantly from
national benchmarks or fall below expectations,
with specific attention devoted to the
organization’s plan for improvement (e.g., its
development, performance expectations, and the
basis on which expectations are established).
Hospital Governing Boards and Quality of Care: A Call to
Responsibility. Washington, DC: National Quality Forum; 2004.
Leadership should….
• Create alignment between organizational strategy,
measures, and improvement projects
• Unify Quality Improvement Plans, Strategic Plans, and
Financial Plans within the organization
• Ensure that the daily work of employees is organized to
support deployment of strategies and improvement projects
chosen because of their direct impact on system-level
measures or direct support of strategic objectives. Leaders
should then implement, monitor, and revise the strategy as
needed if the desired changes are not occurring.
Botwinick L, Bisognano M, Haraden C. Leadership Guide to Patient
Safety. IHI Innovation Series white paper. Cambridge, Massachusetts:
Institute for Healthcare Improvement; 2006. (Available on www.IHI.org)
Use of Lean/6 Sigma
Potential Topics to Report:
• Voice of the customer, suppliers and process workers
• Key critical customer requirements
• Outputs which are “Critical to Quality” (CTQ)
• Rating of relationship between the process steps
[inputs] to the customer requirements
• Current Process Controls Prevention
• Current Process Controls Detection
• FMEAs – findings about severity, occurrence and
detection
• Description of standard work – current and future states
(i.e., value stream map)
Questions?
Richard Scalenghe, CPQH
[email protected]
630-740-7914