Transcript Lies Damned Lies and Statistics.ppt

Lies, Damned Lies, and
Statistics
Hanford Performance Indicator Forum
December 11, 2003
Steven S Prevette
Fluor Hanford Safety and Health
509-373-9371
[email protected]
http://www.hanford.gov/safety/vpp/trend.htm
Introduction
• Numbers abound in the Safety profession
• We have all seen great uses and misuses
• How can we make the best use of the data
we have and make effective decisions that
will improve the safety of ourselves, our
coworkers, and our companies?
Liars Figure and Figures Lie 1
18
16
14
12
10
8
6
4
2
0
Jan-04
Oct-03
Jul-03
Apr-03
Jan-03
Oct-02
Jul-02
Apr-02
Jan-02
Injuries by Month
Here are the past 2 years of injury data.
What is your interpretation? What is your
prediction of future rates?
Liars Figure and Figures Lie 2
18
16
14
12
10
8
6
4
2
0
Injuries by Month
Jan-04
Oct-03
Jul-03
Apr-03
Jan-03
Oct-02
Jul-02
Apr-02
Jan-02
y = -0.132x + 11.987
Let’s add a “trend line”. What is your
interpretation now? What is your
prediction of future rates?
Liars Figure and Figures Lie 3
Injuries by Month
18
16
14
12
10
8
6
4
6 Month Moving Average
2
Jan-04
Oct-03
Jul-03
Apr-03
Jan-03
Oct-02
Jul-02
Apr-02
Jan-02
0
Let’s add a 6 month moving average. What
is your interpretation now? What is your
prediction of future rates?
Interlude – Fun with Charts
• It appears I can make this data say
anything I want it to say . . .
• Courses are taught on “chartsmanship”
• Can we interpret the data with a sound,
rigorous method that is repeatable and
gives credible results?
Liars Figure and Figures Lie 4
Average = 10.3
(Jan02 - Jan04)
25
20
Upper Control Limit
15
10
5
Lower Control Limit
Jan-04
Nov-03
Sep-03
Jul-03
May-03
Mar-03
Jan-03
Nov-02
Sep-02
Jul-02
May-02
Mar-02
Jan-02
0
How about the Control Chart?
- The data are stable, within certain bounds.
The Red Bead Experiment
• Many of you have experienced the “Red
Bead Experiment”
• Results changed from person to person,
but were random and predictable
• The data here were generated by the Red
Beads
• Which chart gave the proper interpretation
and predictions?
The Control Chart
• Separates random noise from signal
• Provides prediction capability
• When coupled with management theory
and action can lead to significant
improvements
• Does not require great costs to perform
• Actually has saved money
Management Theory For
Improvement
• When there is a trend, find “special cause”
and act upon the trend
• When there is no trend, find “common
causes” and act upon the system, the
overall performance
Pareto Charts
OSHA Cases FY 1997
Fluor Hanford
"Overall 25% Reduction to Date"
OSHA Cases FY 1998 through May 1998
0.40
0.35
0.30
0.25
0.20
0.15
-48%
-26%
-8%
72%
0.10
-63%
0.05
0.00
CES
PRO
R
A
CLE
NU
R
ATO
PER
O
S
E
PIP
T ER
FIT
R
N
GE
CIA
/RIG
T RI
R
C
E
E
EL
RK
WO
HP T
N
IRO
Pareto charts can be used to analyze
common causes – FH “Top 5 Occupation”
initiative from 1998
What is a Trend?
•
•
•
•
•
•
One point outside the control limits
Two out of Three points two standard deviations
above/below average
Four out of Five points one standard deviation
above/below average
Seven points in a row all above/below average
Ten out of Eleven points above/below average
Seven points all increasing/decreasing.
See http://www.hanford.gov/safety/vpp/trend.htm
Leading Indicators
•
This system can be used with leading indicators and
Behavior Based Safety
•
A personal example – my group targeted computer
ergonomics
•
Goal was to improve behaviors on the computer, to
reduce risk of Carpal Tunnel / Overuse Syndrome
Leading Indicator Example
BEHAVIOR TO OBSERVE
Backrest of chair is supporting person's back
Forearms are parallel to floor
Knees at angle of 90 - 110 degrees
Legs supported by chair seat
Person's eyes are 18 - 30 inches from screen
Top of monitor at or below eye level
Monitor viewed straight ahead with no neck turn
Wrists are in proper position (Wrist Support in use, or wrists held at level they
would be if there was a Wrist Support)
Typing stand (document holder) at same height and distance as screen [leave
blank if no stand in use]
No glare of outside light, overhead lights, or work lights on screen.
YES
NO
Leading Indicator Results
• Initial rate was 10% safe behaviors
• Plotted on Control Chart
• Pareto chart of individual items
established
• Safe behavior increased to 83% within 5
months, a significant increase
Barriers to Good Data Use
• Higher ups will use it as a “hammer”
• Subjected to quotas and targets imposed from above
• Fear (“accountability”) used as a “motivator”
• Actions and Explanations as a result of random fluctuations
• Perceived loss of control over portrayal of performance
The Biggest Barrier = FEAR
Data Sources
• Worker and Customer Opinion
• Expert Review
• Process Measures
– Top-Down (start with mission/vision)
– Process Approach
– Bottom-Up (start with available data)
Data Sources
Budget
Process
Cycle Time
Product
Service
In-Process
Inventory
Schedule
Procedure
Waste
Idle Time
Rework
Dollars
Hours
Materials
Data
THEORY
OUTCOMES
Mission Progress
Commitments Met
Stewardship
Profit
Safety
Satisfaction
Compliance
More Fun with Numbers
•
Boston Herald (Dec 23, 1996):
"Murders Sink To 30-Year Low "
• "....major crime such as homicide is down.
Over the last two years there has been a
52% reduction in shootings..... that's a
dramatic difference....Police
Commissioner Paul Evans has succeeded
where others have fell short...."
More Fun with Numbers 2
Homicides per Year, Boston MA
Average = 88.5
(1976 - 1989)
140
c-chart UCL
120
100
80
60
c-chart LCL
40
New Chief
in 1993
20
Year
1996
1994
1992
1990
1988
1986
1984
1982
1980
1978
1976
1974
1972
1970
1968
1966
0
1964
Number of Homicides
160
“Murder rate rises in Boston”
BOSTON (AP) — The number of murders in
Boston rose in 2000 after years of steady
decline, partially due to an alarmingly violent first
six months.
The final tally was 37 — up from 31 last year.
[Police Commissioner] Evans noted that there
were no homicides in October or November
For 2001, AP reported a 67% increase
Conclusion – Data Sanity
We can either react to numbers, with explanations
of every percent change, with the inherent
frustrations, fear, and failure
Or
We can understand our data, put it to good use,
and apply valid management principles
The choice is ours.