Transcript A Risk Score for In-Hospital Ischemic Stroke Mortality Derived and Validated within the Get With The Guidelines®-Stroke Program Eric E Smith, Nandavar Shobha, David.

A Risk Score for In-Hospital
Ischemic Stroke Mortality
Derived and Validated within the
Get With The Guidelines®-Stroke Program
Eric E Smith, Nandavar Shobha, David Dai, DaiWai M Olson,
Mathew J Reeves, Jeffrey L Saver, Adrian F Hernandez,
Eric D Peterson, Gregg C Fonarow, Lee H Schwamm
Smith EE et al. Circulation. Epub Sept. 27, 2010
Disclosures
• The Get With The Guidelines®–Stroke (GWTG-Stroke)
program is provided by the American Heart
Association/American Stroke Association. The GWTGStroke program is currently supported in part by a charitable
contribution from Bristol-Myers Squib/Sanofi Pharmaceutical
Partnership and the American Heart Association
Pharmaceutical Roundtable. GWTG-Stroke has been funded
in the past through support from Boeringher-Ingelheim and
Merck.
• The individual author disclosures are listed in the manuscript
Smith EE et al. Circulation. Epub Sept. 27, 2010
Background
• Acute ischemic stroke results in substantial
morbidity and mortality
– In-hospital case fatality rate is approximately 5%
• Determining an individual patient’s risk of mortality
at admission:
– Could aid clinical care by providing valuable prognostic
information
– Could identify those at high risk for poor outcomes who
may require more intensive resources.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Background
• Increased interest by 3rd party payers and
regulatory agencies in tracking stroke mortality as
one of the measure of stroke quality of care, so
adjustment for baseline risk of mortality will be
necessary to avoid penalizing hospitals that admit
less healthy patients.
• Therefore, there is an increasingly important need
to develop well validated models that are useful in
predicting patient risk of mortality and can be
efficiently utilized in actual practice.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Introduction
• There are few validated models for prediction of inhospital mortality following ischemic stroke.
• Prior prediction models have not been incorporated
into clinical practice.
• There remains a need for an accurate and practical
clinical risk tool to predict ischemic stroke mortality
that overcomes limitations and can be readily
incorporated into clinical practice without need for
hand calculation.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Objective
• To develop a practical user-friendly web-enabled bedside
tool for risk stratification at the time of presentation for
patients hospitalized with acute ischemic stroke
• To derive and validate prediction models for a patient’s risk
of in-hospital ischemic stroke mortality using data from the
Get With The Guidelines-Stroke Program
• To test the added value of a measure of stroke severity by
using the patients who were documented in the NIHSS*, the
most commonly used standardized stroke scale.
*NIHSS: National Institutes of Health stroke scale score
Smith EE et al. Circulation. Epub Sept. 27, 2010
Methods
• Hospitals participating in GWTG-Stroke who
utilize the web-based patient management
tool for data collection.
• Outcome Sciences, Inc. served as the data
collection and coordination center
• The Duke Clinical Research Institute (DCRI)
served as the data analysis center
Smith EE et al. Circulation. Epub Sept. 27, 2010
Methods
Study Population and Study Measurements
• Hospitals were instructed to record data from consecutive
stroke and TIA admissions.
• Case ascertainment was through clinical identification during
hospitalization, retrospective identification by ICD-9 codes or
both.
• Eligibility of each case was confirmed at chart review
• Trained hospital personnel abstracted data using an internetbased system that performs checks to ensure the reported
data is complete and internally consistent.
• Data quality was monitored for completeness and accuracy.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Methods
Study Population and Study Measurements
• Hospital characteristics were based on American
Hospital Association data.
• Presentation during daytime regular hours was
defined at 7am to 5pm, Monday through Friday.
• Past medical history was defined based on preexisting conditions, excluding newly diagnosed
conditions during the hospital stay.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Methods
Study Population and Study Measurements
•
•
•
•
Between Oct. 1, 2001 and Dec. 30, 2007
1042 hospitals contributed data
320,635 ischemic stroke discharges
Only patients with ischemic stroke were included
(excluded hemorrhagic stroke or TIA)
• After exclusions:
– 274,988 patients from 1,036 hospitals were analyzed
Smith EE et al. Circulation. Epub Sept. 27, 2010
Methods
Study Population and Study Measurements
• Exclusions:
– Patients transferred out to another acute care hospital or
transferred in.
– Patients with missing data on discharge destination
– Small number of patients who did not present via the
Emergency Department because of direct floor admission
or because of new acute stroke occurring during
hospitalization.
– Few patients with missing gender information
Smith EE et al. Circulation. Epub Sept. 27, 2010
Methods
•
•
•
•
Statistical Analysis
The sample was randomly divided into a derivation (60%) and validation
(40%) sample.
Logistic regression was used to determine the independent predictors of
mortality and to assign point scores for a prediction model.
We also separately derived and validated a model in the 109,187
(39.7%) with NIH stroke scale score (NIHSS) recorded.
Model discrimination was quantified by calculating the c statistic from the
validation sample.
– The c statistic is equivalent to the probability that the predicted risk of death is
higher for patients that died than for patients that survived.
– A c stat of 1.0 indicates perfect prediction, while a c stat of 0.5 indicates no
better than random prediction.
• In-hospital mortality was 5.5% overall and 5.2% in the subset where
NIHSS was recorded.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Results
•
•
•
•
•
•
Study Population: 274,988 ischemic stroke patients
Admissions submitted: 1,036 hospitals
Mean Age: 71.6 years
Women: 53.4%
Treated in academic teaching hospitals: 60.6%
Median Hospital bed size: 372
Smith EE et al. Circulation. Epub Sept. 27, 2010
Results
• In-Hospital Death occurred: 5.51%
– 15,152 out of 274,988 patients
• Many differences between patients who died and
who survived
• Patients who died were more likely to:
– Be older
– Have arrived by ambulance
– Have a history of atrial fibrillation or coronary artery
disease
• Weekend or Night Admission was also associated
with higher mortality
Smith EE et al. Circulation. Epub Sept. 27, 2010
Characteristics of Ischemic Stroke Patients who Died or Survived to Hospital Discharge
Characteristic
Overall (N=274,988) %
Died (N=15,143) %
74 (62-83)
80 (70-86)
46.52
42.33
46.77
White/Caucasian
73.89
77.48
73.68
African American
15.12
11.48
15.33
Asian
2.28
2.38
2.27
American Indian
0.15
0.09
0.16
Hawaiian/Pacific Islander
0.27
0.31
0.27
Other or unable to
determine
3.95
4.61
3.91
Hispanic
4.16
3.48
4.2
EMS from scene
53.44
79.26
51.94
Private transport/walk-in
36.13
8.85
37.72
Did not present via ED
5.67
7.69
5.55
Age
Male Gender
Survived (N=259,845) %
Race
Mode of Arrival
Smith EE et al. Circulation. Epub Sept. 27, 2010
Characteristics of Ischemic Stroke Patients who Died or Survived to Hospital Discharge
Characteristic
Overall (N=274,988) %
Died (N=15,143) %
Survived (N=259,845) %
5 (2-11)
18 (11-24)
4 (2-10)
Atrial fibrillation
18.15
34.61
17.2
Prosthetic heart valve
1.47
1.98
1.44
Previous stroke or TIA
30.77
31.81
30.71
Coronary artery disease
27.53
35.47
27.07
History of carotid stenosis
(>50%)
4.74
4.21
4.77
Diabetes mellitus
29.92
29.42
29.95
Peripheral vascular disease
5.22
6.92
5.12
Hypertension
73.99
73.31
74.03
Dyslipidemia
35.2
27.04
35.68
Atrial fibrillation in hospital
15.91
30.27
15.07
Smoker, current/within past year
17.11
10.66
17.49
Arrived daytime regular hours
46.83
42.68
47.07
NIH stroke scale score
History of:
Smith EE et al. Circulation. Epub Sept. 27, 2010
Results
• The derivation sample and the validation sample were well matched with
respect to patient characteristics and overall mortality, with no significant
differences between the two.
• The multivariable-adjusted independent predictors of increased risk of
mortality were:
–
–
–
–
–
Increasing age (for each year greater than 60)
Atrial fibrillation
Coronary artery disease
Diabetes mellitus
Peripheral vascular disease
• Independent predictors of lower risk of mortality were:
–
–
–
–
–
–
history of previous stroke or TIA
Known carotid stenosis
Hypertension
Dyslipidemia
Current smoking
Presentation during weekday regular hours
Smith EE et al. Circulation. Epub Sept. 27, 2010
Risk Score
• Point scores were derived that could be used
to predict a patient’s risk of dying in hospital
• The probability of in-hospital mortality can be
estimated for an individual patient by
summing points assigned to the value of
each predictor to create a total point score
ranging from 0 to 204.
Smith EE et al. Circulation. Epub Sept. 27, 2010
The Predicted in-hospital mortality plotted as a continuous function of the developed risk score
Smith EE et al. Circulation. Epub Sept. 27, 2010
Risk Score
• This prediction model was validated in the remaining 40% of
the population.
• The risk score demonstrated good discrimination in the
validation sample
• Similar good discrimination was seen in the pre-specified
subgroups in the validation sample.
• A graph of observed vs. predicted mortality, in 6 groups
according to pre-specified categories of mortality risk,
showed an excellent correlation between observed and
predicted mortality in the validation sample, grouped into 10
deciles of predicted risk, also showed excellent calibration.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Model Without
Including NIHSS
Smith EE et al. Circulation. Epub Sept. 27, 2010
Risk Score that Includes NIHSS
• The median hospital-level percent of patients with NIHSS
recorded was 22.2%.
• Very large sample size so many differences between the
groups that reached conventional levels of statistical
significance but that were actually small in both relative and
absolute terms.
• Larger differences were seen for only a few characteristics:
NIHSS was more likely to be documented in patients who
were:
– younger, male, arrived by ambulance, arrived during daylight hours
• Overall mortality was slightly lower in the group with NIHSS
documented vs. the group without NIHSS documented.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Characteristics of ischemic stroke patients with or without NIHSS recorded
Characteristic
NIHSS recorded
(N=109,187) %
NIHSS not recorded
(N=165,801) %
73 (61,82)
75 (62,83)
48.19
45.43
White/Caucasian
74.72
73.64
African American
14.14
15.76
Asian
2.59
2.07
American Indian
0.15
0.15
Hawaiian/Pacific Islander
0.31
0.25
Other or unable to determine
3.78
4.07
Hispanic
4.31
4.06
EMS from scene
57.68
50.64
Private transport/walk-in
37.91
42.85
Did not present via ED
4.41
6.50
Age
Male Gender
Race
Mode of Arrival
Smith EE et al. Circulation. Epub Sept. 27, 2010
Characteristics of ischemic stroke patients with or without NIHSS recorded
Characteristic
NIHSS recorded
(N=109,187) %
NIHSS not recorded
(N=165,801) %
Atrial fibrillation
18.58
17.87
Prosthetic heart valve
1.52
1.44
Previous stroke or TIA
29.72
31.46
Coronary artery disease
27.47
27.57
History of carotid stenosis (>50%)
4.66
4.79
Diabetes mellitus
28.51
30.86
Peripheral vascular disease
4.88
5.45
Hypertension
74.01
73.98
Dyslipidemia
36.85
34.11
Atrial fibrillation in hospital
17.31
14.97
Smoker, current/within past year
18.26
16.35
Arrived daytime regular hours
47.83
46.18
History of:
Smith EE et al. Circulation. Epub Sept. 27, 2010
Characteristics of ischemic stroke patients with or without NIHSS recorded
Characteristic
NIHSS recorded
(N=109,187) %
NIHSS not recorded
(N=165,801) %
380 (267, 564)
365 (256, 507)
60.60%
60.61%
5.19
5.72
Hospital Characteristics:
Number of Beds
Academic Teaching Hospital
Died in-Hospital
Smith EE et al. Circulation. Epub Sept. 27, 2010
Risk Score
• 60% of the sample of patients with NIHSS recorded was
used for derivation and 40% were used for validation.
• The 2 samples were well matched with respect to patient
characteristics and overall mortality, with no significant
differences.
• NIHSS was strongly associated with mortality; median
NIHSS was 18 in those who died compared to 4 in those
who survived.
• Higher NIHSS was strongly associated with increased
mortality after controlling for other predictors.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Point Scores for the prediction modeling including NIHSS
Smith EE et al. Circulation. Epub Sept. 27, 2010
Risk Score
• Model discrimination and calibration was
again excellent across a wide range of prespecified predicted risk categories in the
derivation and validation samples.
• A plot of observed vs. predicted mortality in
the validation sample, grouped into 10 deciles
of predicted risk, again showed excellent
calibration.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Model
Including
NIHSS
Smith EE et al. Circulation. Epub Sept. 27, 2010
Risk Score
• The validation sample c statistic for the model
including NIHSS was greater than the c
statistic for the model derived without NIHSS.
• The c statistic for a model including NIHSS
alone, without any predictors, was also very
high.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Model Discrimination – Importance of
NIHSS Information
Model
C statistic
All variables with NIHSS
0.85
NIHSS alone
0.83
All variables but without NIHSS
0.72
Smith EE et al. Circulation. Epub Sept. 27, 2010
Impact of NIHSS on Predictions
For model with vs. without NIHSS,
Integrated Discrimination Index (IDI) = 9.4%.
IDI = (EY1-EY0) – (EX1-EX0)
Where
• EY1 and EY0 are the mean predicted probabilities of death from persons
who died (EY1) or survived (EY0) in model Y (with NIHSS)
• EX1 and EX0 are the mean predicted probabilities of death from persons
who died (EX1) or survived (EX0) in model X (with NIHSS)
Smith EE et al. Circulation. Epub Sept. 27, 2010
Limitations
• Voluntary participation—may not be representative of all
patients/hospitals.
• Study data were collected based on the medical record and
depend on the accuracy and completeness of clinical
documentation and chart abstraction.
• NIHSS data were not missing at random therefore the
relationship between predictors and outcome observed in
the subset with NIHSS documented may not be
generalizable to all ischemic stroke patients.
• Could not control for all potential predictors of ischemic
stroke mortality but nonetheless were able to predict inhospital mortality with similar discrimination as previously
published models.
• No post-discharge information.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Conclusions
• The GWTG-Stroke risk model provides
clinicians with a well validated, practical
bedside tool for mortality risk stratification.
• The NIHSS provides substantial incremental
information regarding patient’s short term
mortality risk and is the strongest predictor of
mortality.
Smith EE et al. Circulation. Epub Sept. 27, 2010
Conclusions
• A validated and accurate risk stratification tool
should also be of value to health services
researchers who are interested in comparing inhospital mortality rates across different hospitals
and systems;
• The tool has been incorporated into routine clinical
practice by implementing real-time reporting of
individual predicted mortality in GWTG-Stroke
participating hospitals via the web-based Patient
Management Tool
Smith EE et al. Circulation. Epub Sept. 27, 2010