Quality Improvement Science and Patient Safety Research

Dan France, Ph.D., MPH Center for Clinical Improvement Vanderbilt University Medical Center

Outline

• Quality Improvement – Need for the engineering mentality/systems thinking in healthcare • Patient Safety • Student Project

Engineering

• Design/Analysis • Systems Engineering • Engineering Management • Quality Management • Quality Engineer • Industrial Engineer • Health System Engineer

Part I. Quality Improvement

Background

• Institute of Medicine (IOM) reports – Nov 1999:

To Err is Human

– March 2001:

Crossing the Quality Chasm

• Brief History of QI – Scientific Management (Taylor, 1911) • Assembly lines – Statistical Process Control (Shewhart, 1931) – Quality Improvement (Deming, 1955) – Lean Production (Womack, 1990) • Mass customization

Improvement in Healthcare

Expert knowledge Content knowledge System Thinking Statistical Variation Scientific Method Psychology of Change

Traditional Improvement Continuous Quality Improvement Paul Batalden MD

What is Quality

Quality

is the degree to which we meet or exceed customer expectations

Quality Assurance versus Quality Improvement

• Quality Assurance – meet a specification or standard – Take sample measurements to measure performance • Quality Improvement – continual process to improve current performance – Continual measurement and data feedback

The Relation Between Quality Inspection, Regulation, Management, and Improvement Design Redesign Management & Improvement Number of Providers Sanctions

0

Inspection & Regulation for Public Safety Research & Development Level of Quality

IOM Definition of Quality

• Six Dimensions of Quality in Healthcare – Safe – Effective – Timely – Patient centered – Efficient – Equitable

QI is a Science Defined Methodology

• Focus on systems (Systems theory) • Develop ideas for change and test them (Scientific method) • Understand the variation of data measured continuously over time (SPC) • Understand reasons and motivation of people to act on data (Common cause, special cause variation, diffusion of innovation) • Use a balanced set of measures (Value compass)

QI is a Discipline

• QI research is funded by AHRQ and NIH • QI research is published in peer review journals such as NEJM and JAMA • QI science is taught in schools of public health, business schools, graduate programs in engineering, management and education, medical schools in health services research, biostatistics, public health • There is a national Quality Scholars program in healthcare

Variation in Practice Institute of Medicine

• Overuse (eg. Antibiotics, C-Section) • Underuse (eg. Mammography, Beta Blockers) • Misuse (eg. Medical errors) The issue is unnecessary variation i.e., appropriateness of care

Six Sigma

• Domestic Airline Fatality – 6  – 99.99966% “Right” • Mammography Screening – 1.7  – 56% “Right”

QI is a Science: Statistical Approach Variation and Improvement

Lessons about Variation

• Once we begin to measure important quality characteristics and outcomes, we notice variation.

• We question measurements that display no variation.

• Often, single data points alone are uninformative, but data displayed over time can provide information for improvement.

• The primary purpose of understanding variation is to enable prediction.

• Interaction among process variables produces sources of variation: materials, methods, procedures, people, equipment, information, measurement, and environment.

A process

... a series of linked steps, often but not necessarily sequential, designed to ...

   

cause some set of outcomes to occur transform inputs into outputs generate useful information add value



Walter Shewhart:

a system of causes

Constant (convergent) systems



follow the laws of mathematical probability: How the process behaved in the past predicts how it should behave in the future



non-constant (divergent) systems follow the laws of chaos theory: How the process behaved in the past does not predict how it should behave in the future

Random variation



represents the sum of many small variations, arising from real but small causes that are inherent in —and part of —any real process



follows the laws of probability — behaves statistically as a random probability function



because random variation represents the sum of many small causes, it cannot be traced back to a root cause



is a physical attribute of the process

 

different processes have different levels of random variation random variation is a matter of measurement, not goal setting



represents " appropriate " variation

Assignable variation



represents variation arising from a single cause that is not part of the process causes) (system of



therefore can be traced, identified, and eliminated (or implemented)



represents " inappropriate " variation

Registration Times

– These are actual times it took triage level 2 patients to register in the Emergency Department of a hospital: 15 12 14 10 67 54 83 53 4 3 54 14 7 17 10 11 20

Parametric frequency distribution

Value observed

Parameters: mean and variance

center (mean, median) spread (variance, standard deviation, range)

Value observed

Probability-based boundaries

Frequency Distribution 99% 0.5%

2.575 std. devs.

2.575 std. devs.

Value observed

0.5%

Statistical Process Control Chart Time

Random variation

Process Control Chart

(How the process behaves over time)

T1 T2 T3 T4 T5 Time T6 T7 T8 T9

Assignable variation

Process Control Chart

(How the process behaves over time)

T1 T2 T3 T4 T5 Time T6 T7 T8 T9

Managing assignable variation



Find a data point that probably represents assignable variation (usually a statistical outlier)



track it to root causes



eliminate (or implement) the assignable cause ( React to individual fluctuations in the data)

RISK MANAGEMENT PROJECT PEOPLE

Communication (8) MD to MD MD to family RN to family Judgement accuracy of diagnosis (10) timing of treatment (5) Technical issues (7) mgmt of cultures from lab placing/monitoring central lines giving or not giving meds fluid management DKA Following IV policies (4)

EQUIPMENT

Central Lines (2) Use of non-VUMC equipment (2) Accepting test results from referring hospitals (3) Frequency of tests/x-rays (3) On-Call system for sub-specialties (4)

PROCEDURES POLICIES ENVIRONMENT

Staffing (6) Access to MD & treatment after regular hours (7)

SUMMARY UNDESIRABLE OUTCOMES Number patient encounters: 281,000 Total number of claims: 25 Total Cost Incurred: $7,426,815 Median Case: $ 100,000

d:\RiskMgtProjFishbone

Tampering:

Using assignable methods in an attempt to manage random variation Shewhart proved that tampering does not just waste time and effort - it seriously harms process performance

Statistical process control charts



Show the probability that an observation arose from the underlying process — that is,



the probability that a particular point's deviation from the center represents only "random" variation arising from the system of causes that make up the process, as opposed to "assignable" variation representing an identifiable, intruding cause.



They



separate random from assignable variation



based on statistical probability



using control limits, runs, trends, and other patterns in longitudinal data.

A trend

UCL 76.56000

# patients 55.000000

LCL 33.44000

UCL 27.77571

# patients 8.500000

LCL 0.00000

Psych Inpatient Admits / Month

Outcome

QI is a Science: Statistical Approach Overall Improvement Strategy

Process change Remove special causes Process change

Unstable process Special causes present Average is too high Stable process Common cause variation is high Average is too high Stable process Common cause variation reduced Average too high Stable process Common cause variation low Average reduced

CAP protocol compliance

0.8

Implementation Group -- Loose Abx Compliance Baseline Implementation

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0 -23 -21 -19 -17 -15 -13 -11 -9 -7 -5 -3 -1 1 3 5 7 9 11 13 15 17 Month relative to CPM implementation P chart - 0.01 control limits

Using data to improve



The minimum standard: an annotated time series 1.

Start with a run chart (80% of total value) 2.

Add center and goal lines (anchors the eye - now 95% of total value) 3.

Add control limits (in appropriate zones)

"Teen use turns upward"

% high school seniors who smoke daily

1992 1993 17.3% 19.0%

USA Today

, June 21, 1994

"Teen use turns upward"

% high school seniors who smoke daily

1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 18.8% 19.6% 18.7% 18.6% 18.1% 18.9% 19.2% 18.2% 17.3% 19.0%

(average moving range = 0.778)

USA Today

, June 21, 1994

% high school seniors smoking

25 20 15 10 5 0 84 85 86 87 88 Year 89 90 91 92 93

% high school seniors smoking

25 20 15 10 5 0 84 85 Mean = 18.64% 86 87 88 Year 89 90 91 92 93

% high school seniors smoking

25 20 15 10 5 0 84 85 86 87 Mean = 18.64% Avrg Moving Range = 0.778% Upper Process Limit = 20.71% Lower Process Limit = 16.57% 88 Year 89 90 91 92 93

Part II. Patient Safety

Heinrich Triangle

Knowledge

1 Major Error 29 Minor Errors Information on Major Errors Information on Minor Errors Information on Near Misses 300 Intercepted Errors (Near Misses)

Error

Parallel Universe

Essential System Characteristics

• Uses available technologies • Real-time data • Feedback providing (closing the loop) • Designed to succeed (safe)

ALCOA

“At ALCOA I have a real fine data system so that I knew every minute of every day the health and safety condition of 140,000 people. We shared the information across the whole place so that we had real-time learning among the people. The information was not there for me. It was for 140,000 people to learn from shared experiences. Without information having to travel up through some appointment process and maybe some day gets distributed so you can learn something. It was there every day.

If we had an incident in Sumatra, the people in Jamaica knew it tomorrow morning and they did something about it to avoid the same kind of circumstances.

When I asked for the data at Treasury, it took them a long time to get it for me and when they did, it turned out that their lost workday rate in the Treasury, that has about the same number of employees, was 20 times higher than ALCOA’s.” Paul O’Neill, Treasury Secretary

J.T. Reason

“major residual safety problems do not belong exclusively to either the technical or the human domains. Rather, they emerge from as yet little understood interactions between the technical and social aspects of the system” J.T. Reasons,

Safety at Sea and in the Air Taking Stock Together Symp

., Nautical Institute, 1991

Disney

“But, ultimately, even the most conscientious Cast Members cannot do it alone. Guests, too, have an essential role to play in making every visit to our parks safe.” Paul S. Pressler, Chairman, Walt Disney Parks and Resorts

Aviation Safety Network

“Without a doubt 2001 was the year with the highest aviation caused fatalities ever. However, when we take a closer look at the figures we can see that 34 fatal multi engined airliner accidents were recorded, which was an all time low since 1946.”

Learning Objectives

• Implement a blame-free reporting culture • Improve or expand chemotherapy taxonomy/definitions – Preventable adverse drug events and near misses – Operational barriers (i.e., delays) as errors?

• Evaluate wireless technologies as an electronic resources and reporting tool – Integrate into daily workflow – Extend to bedside • Apply Computerized Order Entry/Decision support • Quality Improvement via multidisciplinary teamwork based on data feedback

Intelligent Chemo Delivery System

Decision Support System

(Inbedded safety logic)

Blame-Free ADE Reporting

(Process focused)

Perfect Chemotherapy Delivery Clinical Improvement

(generate hypotheses, tests of change)

Chemotherapy Registry

(tracks metrics over time)

Chemo Events – Data Capture

Standardized Reporting

Lesson Learned

• Leadership and organizational culture are as critical for patient safety as structure • Vertical and horizontal organization communication are essential components of surveillance, prioritization • Team communication (chatter) is key to developing safe culture and trust; Foundation for safety “pattern recognition” • Timely data feedback drives safety improvement • Healthcare can learn much about systems thinking from other industries and cultures • Tightly coupled systems are more prone to failure than highly adaptive systems

Student Project

1. Develop and test taxonomy for systems errors in Emergency Medicine – What system factors in the ER increase the likelihood that care providers will commit errors?

– How to measure these factors?

2. Clinic Redesign - Orthopedics – Room Utilization tracking program