Transcript Developing Templates for Monthly Reports

Presenting Data
&
Standardized Infection Ratios
Tim Wiemken PhD MPH CIC
Assistant Professor of Medicine
Assistant Director of Epidemiology and Biostatistics
University of Louisville
School of Medicine, Division of Infectious Diseases
Clinical and Translational Research Support Center
[email protected]
Phone: 502.852.4627
www.ctrsc.net
Pre-Presentation Comment
• Do not collect any data you do not plan to
use (unless your boss tells you to).
Pre-Presentation Comment
• What constitutes ‘use’?
1. Displaying somewhere other than just a
meeting
2. Preliminary to a follow-up to collect data that
will be used
3. Education
4. Targeted prevention
5. Surveillance (only if you educate with it!)
Overview
•
•
•
•
•
What makes a good report?
How to choose a chart
Components of good charts
Tips for reports
Standardized Infection Ratios
Overview
•
•
•
•
•
What makes a good report?
How to choose a chart
Components of good charts
Tips for reports
Standardized Infection Ratios
What Makes a Good Report?
The catch all answer – it depends!
There are some key ideas you should think
about when preparing any report
What Makes a Good Report?
Create reports that can be USED, not just
looked at.
…more later
What Makes a Good Report?
1. How much time do you have?
What Makes a Good Report?
2. Who is your audience?
• How sophisticated are they?
• What do they already know?
• Remember that people think and speak differently
What Makes a Good Report?
3. Why are you giving the presentation?
•
•
•
•
•
To show data? (I hope not!)
To provide the status of something?
To prove your worth?
To affect change?
To get someone fired?
What Makes a Good Report?
Each of these helps you decide one key
piece of information:
What do you NEED to present?
What Makes a Good Report?
A few rules:
1. Never present more than you need to
present.
What Makes a Good Report?
A few rules:
1. Never present more than you need to
present.
2. Never present less than you need to
present.
What Makes a Good Report?
A few rules:
1. Never present more than you need to
present.
2. Never present less than you need to
present.
3. Keep it as short as possible.
What Makes a Good Report?
Therefore, focus on what is necessary to
meet your goals.
What Makes a Good Report?
Yeah, so smarty pants, what is necessary?
What Makes a Good Report?
IT DEPENDS!
What Makes a Good Report?
Although what is needed depends on how
much time you have, your audience, and
your goal:
1. It is rarely OK to present raw data by
itself:
What Makes a Good Report?
Infection Prevention and Control Report,
May 2012:
CLABSI: 5
VAP: 4
CAUTI: 7
MRSA: 10
What Makes a Good Report?
Infection Prevention and Control Report,
May 2012:
CLABSI: 5
VAP: 4
CAUTI: 7
MRSA: 10
What Makes a Good Report?
Infection Prevention and Control Report,
May 2012:
This is only slightly better.
What Makes a Good Report?
Although what is needed depends on how
much time you have, your audience, and
your goal:
2. It is rarely OK to present significant
amounts of text
– E.g. policies, procedures, etc.
What Makes a Good Report?
2. It is rarely OK to present significant
amounts of text
– No one wants a six page policy handed to
them in a meeting to review.
– It will not get reviewed (at least not well).
– Send these out early… and many times.
What Makes a Good Report?
The Holy Order of Data Quality (worst to best).
1.
2.
3.
4.
5.
6.
7.
Text
Raw numbers
Raw rates
Tables of chronological numbers or rates
Charts (hierarchy of charts later…)
Charts with #4
Charts with #1, #3, and more fun statistics
Overview
•
•
•
•
•
What makes a good report?
How to choose a chart
Components of good charts
Tips for reports
Standardized Infection Ratios
Choosing a Chart
• Charts
– Numbers alone are bad
Choosing a Chart
• Charts
– Visual depiction is easier to quickly
comprehend
Choosing a Chart
• Charts
– Visual depiction is easier to quickly
comprehend
– Choosing the correct chart is extremely
important
Choosing a Chart
• Charts
– Visual depiction is easier to quickly
comprehend
– Choosing the correct chart is extremely
important
– Some charts show trends or comparisons in
data
Choosing a Chart
• Charts
– Visual depiction is easier to quickly
comprehend
– Choosing the correct chart is extremely
important
– Some charts show trends or comparisons in
data
– The best chart does not have to be explained
Choosing a Chart
• Most common/useful types of charts
1.
2.
3.
4.
Pie
Bar
Line
Radar
Choosing a Chart
• Most common/useful types of charts
1. Pie
Choosing Charts
• Pie
– Less appropriate than one would think
– Categorical data that comprise of portions that
add to 100%
– Never use 3D pie charts
– Do not use a pie chart if you have a lot of
categories
Choosing Charts
• Pie
– Do not use a pie chart if you have a lot of
categories
Choosing Charts
• Pie
“Use a pie chart when you don’t have
anything to say”
- Dr. Julio Ramirez
Pie Charts
No!
15.0%
20.0%
5.0%
60.0%
Surgery
Anesthesia
Medicine
Cardiac
Choosing a Chart
• Most common/useful types of charts
2. Bar
Choosing Charts
• Bar
– Snapshots of data [rates in one month]
– Comparing data across different categories
– Never use 3D charts unless you have a 3rd
dimension!
Bar Charts
No!
Cardiac
Medicine
Anesthesia
Surgery
0%
10%
20%
30%
40%
50%
60%
Bar Charts
• Is it ever appropriate to use 3D charts?
Yes! (Really?)
Choosing a Chart
• Most common/useful types of charts
3. Line
Choosing a Chart
• Line
– Longitudinal data (data over time)
– Contiguous points
Choosing a Chart
• Line
– Longitudinal data (data over time)
– Contiguous points
– Rates, Counts, etc. by Month/Quarter/Year
• Run charts
• Statistical Process Control charts
Choosing a Chart
• Run charts
– Use when you have few time periods (e.g.
<25 months).
Choosing a Chart
• Run charts
– Anatomy
• Center Line / Median –represents the median of
all of the data points.
• X-axis –represents the time period of interest
(days, weeks, months, quarters, years).
• Y-axis –represents the scale of the plotted data
points (e.g. rate or count of infection).
• Data points – the actual data values.
Choosing a Chart
• Run charts
Y-axis (Rate)
Data Points (<25)
Center Line
(Median)
X-axis (Month)
Choosing a Chart
• Run charts
– Use
• Use to identify when the data are different than you
expect (for better or worse) through detecting
abnormal variation
Choosing a Chart
• Run charts
– Rules for abnormal variation
1. Seven or more consecutive points on
either side of the Center Line (median).
2. Five or more consecutive points
increasing or decreasing.
3. Fourteen or more consecutive points
alternating up and down.
Choosing a Chart
• Run charts
Rule 2
5 consecutive points
increasing
Rule 1
7 points below median
Choosing a Chart
• Statistical Process Control (SPC) charts
– Use when you have many time periods (e.g.
≥25 months).
– These are much better than run charts.
Choosing a Chart
• SPCcharts
– Anatomy
• Center Line / Mean–represents the average of all of
the data points.
• X-axis –represents the time period of interest (days,
weeks, months, quarters, years).
• Y-axis –represents the scale of the plotted data points
(e.g. rate or count of infection).
• Data points – the actual data values.
• Standard deviation lines (control limits) – represent 1,
2 or 3 standard deviations on each side of the center
line
SPC Chart Components
Mean
SPC Chart Components
+1 SD
-1 SD
SPC Chart Components
+2 SD
-2 SD
SPC Chart Components
Upper Control Limit (UCL)
+3 SD
-3 SD
Lower Control Limit (LCL)
Types of SPC Charts
Choosing the
correct chart can
be difficult - it is
also very
important.
Control limits are
calculated
differently for
each chart.
...although they
are marked in
standard
deviations, the
formulas vary...
Types of SPC Charts
Choosing a Chart
• SPC charts
– Use
• Same as Run Charts, except
• Identify different types of variation
Variation
Special
Cause ‘Out of
Control’
Common Cause - ‘ In Control’
Rules to Detect Special-Cause
Variation
• one point above or
below 3SD
• two of three points
above/below 2SD
• four of five points
above/below 1SD
• eight points in a row on
either side of the mean
• trends of 6 points in a
row increasing or
decreasing
• fifteen points in a row
within 1SD
• fourteen points in a row
alternating up and
down
• eight points in a row
outside of 1SD
Variation
Special
Cause ‘Out of
Control’
Common Cause - ‘ In Control’
A Special SPC Chart
• Often it is useful to
provide reports that
identify the ‘time
since the last event’.
• This motivates and
provides some
competition
A Special SPC Chart
• Rare events
make terrible
charts.
• Rates for rare
events do not
show you much.
A Special SPC Chart
• You can combine the benefits of the ‘time
between events’ and the issues with rare
events into an SPC g Chart.
g chart (time between events)
C chart (number of events)
Choosing a Chart
• Most common/useful types of charts
4. Radar (Spider)
Choosing Charts
• Radar
– Rarely appropriate – confusing
– May work for showing multiple percentages
in one small chart – Bar chart may still be
better.
Radar Charts
Overview
•
•
•
•
•
What makes a good report?
How to choose a chart
Components of good charts
Tips for reports
Standardized Infection Ratios
Components of Good Charts
• Good charts tell the complete picture of
what they contain
Components of Good Charts
• Good charts tell the complete picture of
what they contain
• Ideally, someone who is not familiar with
the data being presented should be able to
easily and quickly understand what the
chart is representing
Components of Good Charts
• Never use 3D charts unless you have a 3rd
dimension
Components of Good Charts
• The scale used for charts (that have a
scale) should begin at zero and end near
the highest value
– Changing the axis scale can tell a drastically
different story to those that are just glancing
Same Data, Different Scale
25%
MRSA colonization rate by service
20%
15%
10%
5%
0%
Anesthesia
Medicine
What would you say about these two services?
Same Data, Different Scale
25%
MRSA colonization rate by service
20%
15%
MRSA colonization rate by service
10%
21%
5%
20%
0%
Anesthesia
Medicine
19%
18%
17%
How about now?
16%
15%
14%
Anesthesia
Medicine
Overview
•
•
•
•
•
What makes a good report?
How to choose a chart
Components of good charts
Tips for reports
Standardized Infection Ratios
Tips for Reports
• Title must be very specific
– Include person (organism/disease
state/compliance measure, etc.), place and
time
• Example:
– Hospital-associated Methicillin-resistant
Staphylococcus aureus Isolates: Hospital X,
Acute Care ICU, January 2007-January 2010
Tips for Reports
• Data definitions should be included as a
footnote or on an introductory page.
Tips for Reports
• Data definitions should be included as a
footnote or on an introductory page.
• Example: Clostridium difficile
– Definition for numerator
• Are you measuring toxin only? Antigen? By PCR?
On liquid stool only?
– Definition for denominator
• Patient-days? Admissions?
Example introductory slide for MRSA rates
Hospital-associated Methicillinresistant Staphylococcus aureus
(MRSA)
Case of MRSA (Numerator): A case of MRSA was defined as a
new and unique, hospital-associated (isolated >48 hours after
admission), microbiological isolate from a patient admitted to hospital
x during the month of interest without a prior history of MRSA.
Patient days (Denominator): The denominator for the calculation
of the rate of MRSA was defined as the number of patient-days for
hospital x during the month of interest, regardless of risk status.
Rate: (Numerator / Denominator) * 1,000
Tips for Reports
• For charts, you should also include a
formula for how the rate was calculated.
Tips for Reports
• For charts, you should also include a
formula for how the rate was calculated.
– # Unique, hospital-associated MRSA
isolates divided by # Patient-days of care X
1,000
Tips for Reports
• For charts, you should also include a
formula for how the rate was calculated.
– # Unique, hospital-associated MRSA
isolates divided by # Patient-days of care X
1,000
– Be specific. Isolates, Infections,
Colonizations, etc.
Tips For Reports
• Include the appropriate chart!
0.0045
0.0040
UCL
0.004
Rate Per Bed-day of Care
0.0035
0.0030
0.0025
0.0020
0.0015
CL
0.001
0.0010
0.0005
0.0000
Date
Tips for Reports
• Raw data is important to be able to
understand what is in the chart and to
verify the data.
Tips for Reports
• Raw data is important to be able to
understand what is in the chart and to
verify the data.
• Some people just like the numbers.
Tips for Reports
• Raw data is important to be able to
understand what is in the chart and to
verify the data.
• Some people just like the numbers.
• Raw data should include numerator,
denominator and standardized rate.
Tips for Reports
• Raw data is important to be able to
understand what is in the chart and to
verify the data.
• Some people just like the numbers.
• Raw data should include numerator,
denominator and standardized rate.
Tips for Reports
• Including aggregated data for quick
comparisons is also a good idea.
• Yearly average rates may work.
• When possible, include P-values to detect
statistically significant differences (rate
comparison example to come..)
Average Rate per 1000 Bed-days of Care Average Rate per 1000 Bed-days of Care Average Rate per 1000 Bed-days of Care
2008
2009
2010 YTD
1.3
1.7
0.7
Tips for Reports
• Always include a description of the
assessment of the chart and your plans.
Tips for Reports
• Always include bubbles or call-out boxes
that adequately describe dates and
components of any interventions.
Tips for Reports
• You may use a bubble to describe if your
surveillance definition changes.
Tips for Reports
• You may use a bubble to describe if your
surveillance definition changes.
• A better idea is to just start over.
Hospital-Associated Methicillin-resistant Staphylococcus aureus Isolates: Hospital X, MICU,
January 2006 – January 2010
# HA MRSA Isolates divided by # Bed-days of Care
0.0050
June 2006:
Rate peaked to 4.30. Suspect
increase was due to a
change in MRSA precautions
implemented affected staff
compliance with when to
wear PPE.
0.0045
0.0040
Rate Per Bed-day of Care
0.0035
0.0030
UCL
0.003
CL
0.001
0.0025
0.0020
0.0015
0.0010
0.0005
0.0000
Date
Date
Jan
10
Feb
10
Number of Isolates
0
2
Number of Bed-Days of Care
1319
1117
Rate Per 1000 Bed-days of Care
0.0
1.8
Mar
10
Apr
10
May
10
Jun
10
Jul 10
Aug
10
Sep
10
Oct
10
Nov
10
Dec
10
Average Rate per 1000 Bed-days of Care Average Rate per 1000 Bed-days of Care Average Rate per 1000 Bed-days of Care
2008
2009
2010 YTD
0.4
0.6
0.9
Assessment: Process is in statistical control.
Plan: Continue surveillance activities.
Tips for Reports
• Some reports must be text, although every
attempt to transform into a graphic should
be made.
Tips for Reports
• Some reports must be text, although every
attempt to transform into a graphic should
be made.
• Keep text to a minimum with summary
information presented.
Tips for Reports
• Some reports must be text, although every
attempt to transform into a graphic should
be made.
• Keep text to a minimum with summary
information presented.
• Email out early and often.
Tips for Reports
• Another option is a dashboard.
• This may be particularly useful for
providing back to staff to post in their work
areas.
• After all – a report that isn’t used for
something other than looking pretty is not
particularly worth your time!
Tips for Reports
Tips for Reports
• KHA has a
wonderful
template
for you!
Overview
•
•
•
•
•
What makes a good report?
How to choose a chart
Components of good charts
Tips for reports
Standardized Infection Ratios
Standardized Infection Ratio
• SIR
• A way to standardize/risk adjust rates for
better comparability across institutions
Standardized Infection Ratio
• SIR
• Why are rates not comparable?
– Different types of institutions have different
patient populations
– These patients have different risks.
– Therefore, comparing a University Hospital
MICU to a 4 bed Cardiac Unit in Wyoming is
not useful.
Standardized Infection Ratio
• SIR
• SIR takes into account the types of
patients seen in various different types of
units or wards in a healthcare facility.
Standardized Infection Ratio
• SIR
• Compares the number of infections you
have to the number you expect to see,
based on some benchmark rate.
Standardized Infection Ratio
• SIR
– SIR>1 : Worse than expected numbers
– SIR <1 : Better than expected numbers
– SIR = 1 : As expected
Standardized Infection Ratio
• Advantages
– Adjusts based on
benchmarked rates
– Adjusts for various
locations [ICU vs Ward]
or types of surgeries,
etc.
– Excellent for comparing
low numbers [removes
high variation]
– Greater precision for
adjustment
ICHE April 2006, 27(4): 427-429.
• Disadvantages
– Statistical bias = may not
make appropriate
comparisons
– Utility beyond SSI is
relatively unstudied
– Wide understanding is
just not there.
ICHE January 2005, 26(1):8-9.
SIR for CLABSI
1. Get observed numerator as usual
SIR for CLABSI
1. Get observed numerator as usual
2. Get number of patient-days (line-days,
vent-days, etc) as usual
SIR for CLABSI
1. Get observed numerator as usual
2. Get number of patient-days (line-days,
vent-days, etc) as usual
3. Get NHSN benchmark rate from the
relevant publication
SIR for CLABSI
1. Get observed numerator as usual
2. Get number of patient-days (line-days,
vent-days, etc) as usual
3. Get NHSN benchmark rate from the
relevant publication
4. Multiply benchmark rate by the number of
patient days (line days, ventilator-days,
etc)= expected number of infections
SIR for CLABSI
Divide your observed number by the
expected number = SIR!
SIR for CLABSI
Example
1. Get observed number of CLABSIs as
usual: 5
SIR for CLABSI
Example
1. Get observed number of CLABSIs as
usual: 5
2. Get number of line-days as usual: 1,234
SIR for CLABSI
Example
3. Get NHSN benchmark rate from the
relevant publication
Can compare
to different
percentile
rates if that is
your goal!
SIR for CLABSI
Example
1. Get observed number of CLABSIs as usual:
5
2. Get number of line-days as usual: 1,234
3. Get NHSN benchmark rate from the relevant
publication: 1.9
4. Multiply benchmark rate by the number of
line-days and divide by 1,000 = expected
number of infections:
(1.9 *1,234) / 1,000 = 2.35
SIR for CLABSI
Example
Divide your observed number by the
expected number = SIR!:
Observed (5)/ Expected (2.35) = 2.13
SIR for CLABSI
Example
SIR = 2.13
You are doing worse than expected
given the benchmark.
SIR for CLABSI
• This method calculated an SIR for one
particular unit.
• You can calculate an overall SIR that
adjusts for multiple types of units. This
makes it more comparable to other
institutions.
Lets see how this works…
SIR for CLABSI
Example Using Free Software
http://innovationsforglobalhealth.com/ipstat/
Standardized Infection Ratio and
SPC
• SIRs can be plotted on an XmR chart and
may be more sensitive to detect special
causes (very much controversy here!)
Final Thoughts
• Charts are good – use them whenever
possible.
Final Thoughts
• Charts are good – use them whenever
possible.
• Knowing your audience is important –
don’t assume they know what you know or
are interested in what you are interested
in.
Final Thoughts
• Charts are good – use them whenever
possible
• Knowing your audience is important –
don’t assume they know what you know or
are interested in what you are interested
in.
• SIRs may be useful in some situations
Final Thoughts
• Make sure everyone knows how to read
your charts
POST YOUR REPORTS WHEREVER YOU CAN
More time?
• Example of how to make a control chart
using Excel.
• Calculating P-values to compare rates
• You can download this file at:
http://innovationsforglobalhealth.com/files/Stat_Tests_v13.xls
Questions?
Thanks!