SCreening for Occult REnal Disease (SCORED), a simple
Download
Report
Transcript SCreening for Occult REnal Disease (SCORED), a simple
SCreening for Occult REnal
Disease (SCORED)
Simple Algorithms to Predict
Kidney Disease: ready to be used
in the real world?
Heejung Bang, PhD & Madhu Mazumdar, PhD
Division of Biostatistics and Epidemiology
Department of Public Health
Weill Medical College of Cornell University
1
Overview
Background
Objectives
Methods: model development and
validation
Results
Discussion
2
Background
Prevalence of Kidney Disease (1999-2004)
6
5.7
5.4
5.4
5
Stage 1 GFR>90
Stage 2 90-60
Stage 3 59-30
Stage 4 29-15
Stage 5 <15
4
3
2
1
0.3
0.1
0
Stages 1 and 2 with kidney damage
3
Background
End-Stage Renal Disease (ESRD) Counts
4
Background
Total Cost of Medicare for ESRD (in billions)
30
95% CL
Projection
Cost
24
28.3
18
14.2
12
6
0
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
5
Background
Chronic kidney disease (CKD) is a global health problem.
Low-awareness and late detection are common problems.
It is progressive disease. Yet, most affected individuals are
asymptomatic with known risk factors and are not routinely
tested.
Identifying individuals with CKD should be ‘simple’ with
serum creatinine concentration that is widely available and
inexpensive ($10-20), in combination with urinalysis.
Systematic methods to predict disease in other chronic
conditions such as cardiovascular disease (e.g.,
Framingham, Reynolds scores, stroke instrument), cancer
(e.g., Gail model), diabetes exist but not for CKD.
6
Objectives
To develop risk prediction model for prevalent CKD
Important prerequisites in our investigation:
Easy to use but accurate
Cumulative effects of concurrent risk factors
Demographic + medical history + modifiable risk factors
To test the validity of the model internally as well as using
independent large databases (i.e., external validation)
To compare the performance of the model with the current
clinical practice guidelines
To develop risk prediction model for incident CKD
7
Kidney Early Evaluation Program (KEEP)
by the National Kidney Foundation
if a persons is ≥ 18 years old and has one or
more of the following:
1. diabetes
2. high blood pressure
3. a family history of diabetes, high blood
pressure or kidney disease
http://www.kidney.org/news/keep/
8
SCreening for Occult
REnal Disease
(SCORED)
Bang et al. (2007)
9
Methods
Cross-sectional analysis of a nationally
representative population based survey, the
National Health and Nutritional Examination
Surveys (NHANES) 1999-2002
Adult subjects only (≥20 years old)
Potential risk factors searched from literature
Endpoint: CKD stage 3 or higher, i.e., glomerular
filtration rate (GFR) < 60 ml/min/1.73m2 (using the
MDRD formula)
10
Methods (Cont’d)
Split-sample method to create a development and
validation dataset using a 2:1 ratio.
Standard diagnostic characteristics: # at high risk,
sensitivity, specificity, positive & negative predictive
values, area under ROC curve
Multiple logistic regression model (with proper
weighting and complex survey design)
e.g., proc surveylogistic in SAS.
11
Methods (Cont’d)
‘Categorical scoring system’ derived by assigning an
integer for the regression coefficients
‘Continuous probability’ of having CKD from the fitted
regression model
External validation using the Atherosclerosis Risk in
Communities (ARIC) Study, Cardiovascular Health Study
(CHS) and NHANES 2003-2004.
Comparison between SCORED vs. KEEP using standard
diagnostic measures
A number of sensitivity analyses (e.g., missing info,
different definitions)
--- important to be used in the real world!
12
Results
NHANES 1999-2002 gave 10,291 individuals
After exclusions (based on unmeasured or
missing data, etc.), dataset included 8,530
observations
A total of 601 individuals had CKD (5.4%
weighted proportion)
13
Final SCORED model in development data
(N= 5,666, AUC=0.88)
14
Diagnostic characteristics of SCORED in
internal validation dataset (N=2,864)
(cutpoint ≥4 to define high risk group)
15
Event rate by risk score
16
Fitting SCORED model to ARIC dataset
(N= 12,038, AUC=0.71)
17
Sample questionnaire
18
Advantages of SCORED
Estimate the cumulative likelihood of having disease
with multiple risk factors
Accuracy and high sensitivity.
Simple to use (implemented by the pen & pencil
method) so foresee a variety of uses
e.g., mass screenings
public education initiatives, health fair
medical emergency departments
web-based medical information sites
patient waiting room in clinics
19
Limitations of SCORED
Inability to assess family history of kidney disease
-- many large national and community studies do not
enquire about history of kidney disease.
For prevalent disease, not incident disease (a new
risk score is needed, later in this talk)
Some variables may be commonly missing (e.g.
proteinuria)
Low PPV (but prediction is HARD!)
Kidney disease: multiple definitions, different stages
20
Diagnostic performance of SCORED vs. KEEP
using external validation data
(Bang, Mazumdar et al. 2008)
Screening guidelines
% high risk
Sensitivity
Specificity
PPV
NPV
AUC
NHANES
40
95
65
20
99
0.88
ARIC/CHS
51
88
52
14
98
0.78
ARIC/CHS*
53
89
50
13
98
0.79
ARIC/CHS*
53
90
50
13
98
0.80
NHANES
67
90
35
12
97
0.75
NHANES*
69
92
33
12
98
0.77
ARIC
76
88
24
3
98
0.67
ARIC/CHS
77
86
24
9
95
0.65
SCORED
KEEP
21
* some sensitivity analyses
A simple algorithm to predict
incident kidney disease
(aka, SCORED II)
by Kshirsagar, Bang et al. In Press
22
Prediction is very hard, especially
about the future - Yogi Berra
23
Background
Another important issue is to predict a new disease in
disease-free population.
In many asymptomatic diseases, both prevalent and
incident diseases are important. (in contrast, for hard
outcomes such as heart attack, only incident disease
makes sense)
Incident disease is less urgent so less user-friendliness
is acceptable.
--- 3 different models developed: 1) best-fitting
continuous, 2) best-fitting categorical, 3) simplified
categorical.
Beyond AUC. We also used AIC/BIC.
24
Background (Conti’)
We need prospective studies to develop the
models.
Internal validation only using Split-sample, no
external validation.
Same logistic regression --- so observed
outcome among survivors.
Cutpoint for high risk group might be less
important.
25
Simplified categorical model
(AUC=0.69, AIC=6295, BIC=6374)
Beta coefficient
(standard error)
Odds Ratio
(95% Cl)
P value
Assigned
score
Age 50-59
0.63 (0.12)
1.9 (1.5, 2.4)
<0.0001
1
60-69
1.33 (0.12)
3.8 (3.0, 5.8)
<0.0001
2
70 or older
1.46 (0.14)
4.3 (3.3, 5.6)
<0.0001
3
Female
0.13 (0.07)
1.1 (1.0, 1.3)
0.05
1
Anemia
0.48 (0.20)
1.6 (1.1, 2.4)
0.02
1
Hypertension
0.55 (0.07)
1.7 (1.5, 2.0)
<0.0001
1
Diabetes mellitus
0.33 (0.10)
1.4 (1.2, 1.7)
0.0006
1
History of cardiovascular
disease
0.26 (0.10)
1.3 (1.1, 1.6)
0.009
1
History of heart failure
0.50 (0.25)
1.6 (1.0, 2.7)
0.04
1
Peripheral vascular disease
0.41 (0.13)
1.5 (1.2, 1.9)
0.002
1
Covariate
26
Risk prediction table for
up to 10 years
Total score
Estimated Risk (%)
≤1
2
3
≤5
8
13
4
5
6
7
20
25
30
35
≥8
≥50
27
Discussion
Evidence-based medicine = Science (theory) + Data +
Statistics.
Risk score = Statistics + Art + Reality
--- SCORED is a good example.☺
Performed well in a variety of different settings.
Seems to provide the enhanced guidelines upon the current
clinical practice guidelines.
It started be utilized in the ‘real world’.
SCORED II yet to be validated but strong consistency/
similarities observed in SCORED I and II.
28
Discussion (Conti’)
Categorization can be a bad idea (Royston et al. 2005;
Greenland 1995) but is crucial for risk scoring algorithms to be
useful in the real world.
More than 1 model may be justified and we can let
consumers/users to choose because
All models are wrong, but some are useful ---George Box
Relying on only 1 measure (e.g., AUC) can be problematic
(Cook et al. 2006; Cook 2007).
Trade-offs between accurate vs. easy medical terms.
Risk scores for internet vs. physician’s office vs. Walmart can
be different.
29
Current and future research
Evaluation of SCORED in vascular patients
because detection of CKD in patients with or at
increased risk of CVD was emphasized by a
science advisory from the American Heart
Association and National Kidney Foundation
(2006).
Relationships SCORED with other risk scores
Testing SCORED in community settings
30
References
Bang, Vupputuri, Shoham et al. (2007). SCreening for Occult REnal
Disease (SCORED). A simple prediction model for chronic kidney disease.
Archives of Internal Medicine.
Bang, Mazumdar, Kern et al. (2008). Validation and Comparison of a novel
prediction rule for kidney disease: KEEPing SCORED. Arch Int Med.
Kshirsagar, Bang, Bomback et al . A simple algorithm to predict incident
kidney disease. In Press. Arch Int Med.
Bang, Mazumdar, Newman et al. Screening for kidney disease in vascular
patients. Submitted.
Building and Using Disease Prediction Models in the Real World.
Roundtable discussion led by H. Bang at JSM, Utah, 2007. Slides at:
http://www.med.cornell.edu/public.health/conference_presentations.htm
31
Exposed to and used by public
Covered by the CBS Early Show (on World Kidney
Day 2007)
SCORED questionnaire is posted in some health
information websites
Distributed by ESRD network, KidneyTrust, Am
Kidney Fund, UK Dept of Health, and UNC Kidney
Center for Kidney Education Outreach Program
“Research Highlights” in Nature Clinical Practice
Nephrology (2007)
Lead Story in Physician’s Weekly (2007)
32