Transcript Document 7504851

Medical Epidemiology
Interpreting Medical Tests
and Other Evidence
1
Interpreting Medical Tests and Other Evidence






Dichotomous model
Developmental characteristics
– Test parameters
– Cut-points and Receiver Operating Characteristic
(ROC)
Clinical Interpretation
– Predictive values: keys to clinical practice
– Bayes’ Theorem and likelihood ratios
– Pre- and post-test probabilities and odds of disease
– Test interpretation in context
– True vs. test prevalence
Combination tests: serial and parallel testing
Disease Screening
Why everything is a test!
2
Dichotomous model
Simplification of Scale

Test usually results in continuous or complex
measurement

Often summarized by simpler scale -reductionist, e.g.
– ordinal grading, e.g. cancer staging
– dichotomization -- yes or no, go or stop
3
Dichotomous model
Disease
Yes (D+) No (D-) Total
Positive (T+)
a
b
a+b
Test
Negative (T-)
Total
c
d
c+d
a+c
b+d
n
Test Errors from Dichotomization
Types of errors
•False Positives = positive tests that are wrong = b
•False Negatives = negative tests that are wrong = c
4
Developmental characteristics: test parameters
Positive (T+)
Test
Negative (T-)
Total
Disease
Yes (D+) No (D-) Total
a
b
a+b
c
d
c+d
a+c
b+d
n
Error rates as conditional probabilities
 Pr(T+|D-) = False Positive Rate (FP rate) =
b/(b+d)

Pr(T-|D+) = False Negative Rate (FN rate) =
c/(a+c)
5
Developmental characteristics: test parameters
Disease
Yes (D+) No (D-) Total
Positive (T+)
a
b
a+b
Test
Negative (T-)
Total
c
d
c+d
a+c
b+d
n
Complements of error rates as desirable test properties
Sensitivity = Pr(T+|D+) = 1 - FN rate = a/(a+c)
Sensitivity is PID (Positive In Disease) [pelvic
inflammatory disease]
Specificity = Pr(T-|D-) = 1 - FP rate = d/(b+d)
Specificity is NIH (Negative In Health) [national
institutes of health]
6
Typical setting for finding
Sensitivity and Specificity





Best if everyone who gets the new test
also gets “gold standard”
Doesn’t happen
Even reverse doesn’t happen
Not even a sample of each (casecontrol type)
Case series of patients who had both
tests
7
Setting for finding Sensitivity
and Specificity




Sensitivity should not be tested in
“sickest of sick”
Should include spectrum of disease
Specificity should not be tested in
“healthiest of healthy”
Should include similar conditions.
8
Developmental characteristics: Cut-points and
Receiver Operating Characteristic (ROC)
Healthy
9
Developmental characteristics: Cut-points and Receiver
Operating Characteristic (ROC)
Healthy
Sick
10
Developmental characteristics: Cut-points and Receiver
Operating Characteristic (ROC)
Fals pos= 20% True pos=82%
11
Developmental characteristics: Cut-points and Receiver
Operating Characteristic (ROC)
Fals pos= 9% True pos=70%
12
Developmental characteristics: Cut-points and Receiver
Operating Characteristic (ROC)
F pos= 100% T pos=100%
13
Developmental characteristics: Cut-points and Receiver
Operating Characteristic (ROC)
F pos= 50% T pos=90%
14
15
Developmental characteristics: Cut-points and Receiver
Operating Characteristic (ROC)
Receiver Operating Characteristic (ROC)
16
Developmental characteristics: Cut-points and Receiver
Operating Characteristic (ROC)
Receiver Operating Characteristic (ROC)
17
Receiver Operating Characteristic (ROC)


ROC Curve allows comparison of
different tests for the same condition
without (before) specifying a cut-off
point.
The test with the largest AUC (Area
under the curve) is the best.
18
19
Developmental characteristics: test parameters
Problems in Assessing Test Parameters

Lack of objective "gold standard" for testing, because
– unavailable, except e.g. at autopsy
– too expensive, invasive, risky or unpleasant

Paucity of information on tests in healthy
– too expense, invasive, unpleasant, risky, and
possibly unethical for use in healthy
– Since test negatives are usually not pursued with
more extensive work-ups, lack of information on
false negatives
20
Clinical Interpretation: Predictive Values
Most test positives below are sick. But this is because
there are as many sick as healthy people overall. What
if fewer people were sick, relative to the healthy?
21
Clinical Interpretation: Predictive Values
Now most test positives below are healthy. This is because
the number of false positives from the larger healthy group
outweighs the true positives from the sick group. Thus, the
chance that a test positive is sick depends on the
prevalence of the disease in the group tested!
22
Clinical Interpretation: Predictive Values
But
•the prevalence of the disease in the group tested
depends on whom you choose to test
•the chance that a test positive is sick, as well as the
chance that a test negative is healthy, are what a
physician needs to know.
These are not sensitivity and specificity!
The numbers a physician needs to know are the
predictive values of the test.
23
Clinical Interpretation: Predictive Values
Sensitivity (Se)
Pr{T+|D+}
true positives
total with the disease
Positive Predictive Value (PV+, PPV)
Pr{D+|T+}
true positives
total positive on the test
24
Positive Predictive Value







Predictive value positive
The predictive value of a positive test.
If I have a positive test, does that mean I
have the disease?
Then, what does it mean?
If I have a positive test what is the chance
(probability) that I have the disease?
Probability of having the disease “after” you
have a positive test (posttest probability)
(Watch for “OF”. It usually precedes the denominator
Numerator is always PART of the denominator)
25
Clinical Interpretation: Predictive Values
D+
T+
T+
and
D+
26
Clinical Interpretation: Predictive Value
Specificity (Sp)
Pr{T-|D-}
true negatives
total without the disease
Negative Predictive Value (PV-, NPV)
Pr{D-|T-}
true negatives
total negative on the test
27
Negative Predictive Value





Predictive value negative
If I have a negative test, does that mean
I don’t have the disease?
What does it mean?
If I have a negative test what is the
chance I don’t have the disease?
The predictive value of a negative test.
28
Mathematicians don’t Like PV




PV- “probability of no disease given a
negative test result”
They prefer (1-PV-) “probability of disease
given a negative test result”
Also referred to as “post-test probability” (of a
negative test)
Ex: PV- = 0.95 “post-test probability for a
negative test result = 0.05”
Ex: PV+ = 0.90 “post-test probability for a
positive test result = 0.90”
29
Mathematicians don’t Like
Specificity either

They prefer false positive rate, which is
1 – specificity.
30
Where do you find PPV?

Table?
 NO
 Make new table
 Switch to odds
31
Use This Table ? NO
Test
Result
+
Total
Disease
+
95
5
100
8
92
100
Total
103
97
200
You would conclude that PPV is 95/103 = 92%
32
Make a New Table
Test
Result
+
Total
Disease
+
95
5
100
72
828
900
Total
167
833
1000
33
Make a New Table
Disease
Test
Result
+
Total
+
-
Total
95
5
100
72
828
900
167
833
1000
Probability of having the disease before testing was
10%. (pretest probability  prevalence)
Posttest probability (PPV) = 95/167 = 57%
So we went up from 10% probability to 57% after
having a positive test
34
Switch to Odds

1000 patients. 100 have disease. 900
healthy. Who will test positive?

Diseased 100__X.95 =_95
Healthy 900 X.08 = 72
We will end with 95+72= 167 positive
tests of which 95 will have the disease
PPV = 95/167


35
From pretest to posttest odds




Diseased 100
X.95 =_95
Healthy
900 X.08 = 72
100 = Pretest odds
900
.95 = Sensitivity__ = prob. Of pos test in dis
.08
1-Specificity prob. Of pos test in hlth
95 =Posttest odds. Probability is 95/(95+72)
72
36

Remember to switch back to probability
37
What is this second fraction?




Likelihood Ratio Positive
Multiplied by any patient’s pretest odds
gives you their posttest odds.
Comparing LR+ of different tests is
comparing their ability to “rule in” a
diagnosis.
As specificity increases LR+ increases
and PPV increases (Sp P In)
38
Clinical Interpretation: likelihood ratios

Likelihood ratio =
Pr{test result|disease present}
Pr{test result|disease absent}

LR+ = Pr{T+|D+}/Pr{T+|D-} = Sensitivity/(1-Specificity)

LR- = Pr{T-|D+}/Pr{T-|D-} = (1-Sensitivity)/Specificity
39
Clinical Interpretation: Positive Likelihood Ratio and PV+
O = PRE-TEST ODDS OF DISEASE
POST-ODDS (+) = O x LR+ =
 SENSITIVITY 
Ox

1
SPECIFICIT
Y


POST- ODDS(+)
PV+ = PPV =
1 + POST- ODDS(+)
40
Likelihood Ratio Negative





Diseased 100_ X.05 =_5__
Healthy
900 X.92 = 828
100 = Pretest odds
900
.05 = 1-sensitivity = prob. Of neg test in dis
.92
Specificity
prob. Of neg test in hlth
(LR-)
Posttest odds= 5/828. Probability=5/833=0.6%
As sensitivity increases LR- decreases and NPV
increases (Sn N Out)
41
Clinical Interpretation: Negative Likelihood Ratio and PVPOST-ODDS (-) = O x LR- =
1 - SENSITIVITY 
Ox

SPECIFICIT
Y


42

Remember to switch to probability and
also to use 1 minus
43
Post test probability given a negative
test
= Post odds (-)/ 1- post odds (-)
POST- ODDS(-)
PV- = NPV = 1 1 + POST- ODDS(-)
44
Value of a diagnostic test depends on
the prior probability of disease






Prevalence
(Probability) = 5%
Sensitivity = 90%
Specificity = 85%
PV+ = 24%
PV- = 99%
Test not as useful
when disease
unlikely






Prevalence
(Probability) = 90%
Sensitivity = 90%
Specificity = 85%
PV+ = 98%
PV- = 49%
Test not as useful
when disease likely
45
Clinical interpretation of post-test
probability
Probability of disease:
Don't
treat for
disease
Do further
diagnostic
testing
Treat for
disease
0
1
Testing
threshold
Disease
ruled out
Treatment
threshold
Disease
ruled in
46
Advantages of LRs





The higher or lower the LR, the higher or
lower the post-test disease probability
Which test will result in the highest post-test
probability in a given patient?
The test with the largest LR+
Which test will result in the lowest post-test
probability in a given patient?
The test with the smallest LR47
Advantages of LRs

Clear separation of test characteristics
from disease probability.
48
Likelihood Ratios - Advantage


Provide a measure of a test’s ability to
rule in or rule out disease independent
of disease probability
Test A LR+ > Test B LR+
– Test A PV+ > Test B PV+ always!

Test A LR- < Test B LR– Test A PV- > Test B PV- always!
49
Using Likelihood Ratios to Determine PostTest Disease Probability
Pre-test
probability
of disease
Pre-test
odds of
disease
Post-test
odds of
disease
Post-test
probability
of disease
Likelihood
ratio
50
51
Predictive Values
Alternate formulations:Bayes’ Theorem
PV+ =
Se  Pre-test Prevalence
Se  Pre-test Prevalence + (1 - Sp)  (1 - Pre-test Prevalence)
High specificity to “rule-in” disease
PV- =
Sp  (1 - Pre-test Prevalence)
Sp  (1 - Pre-test Prevalence) + (1 - Se)  Pre-test Prevalence
High sensitivity to “rule-out” disease
52
Clinical Interpretation: Predictive Values
PV+ And PV-1 Of Electrocardiographic Status2
For Angiographically Verified3 Coronary Artery
Disease, By Age And Sex Of Patient
Sex
Age
PV+ (%)
PV- (%)
F
F
F
<40
40-50
50+
32
46
62
88
80
68
M
M
M
<40
40-50
50+
62
75
85
68
54
38
1. Based on statistical smoothing of results from 78 patients referred to NC
Memorial Hospital for chest pain. Each value has a standard error of 6-7%.
2. At least one millivolt horizontal st segment depression.
3. At least 50% stenosis in one or more main coronary vessels.
53
Clinical Interpretation: Predictive Values
54
If Predictive value is more useful
why not reported?




Should they report it?
Only if everyone is tested.
And even then.
You need sensitivity and specificity from
literature. Add YOUR OWN pretest
probability.
55
So how do you figure pretest
probability?






Start with disease prevalence.
Refine to local population.
Refine to population you serve.
Refine according to patient’s presentation.
Add in results of history and exam (clinical
suspicion).
Also consider your own threshold for testing.
56
Why everything is a test




Once a tentative dx is formed, each piece of new
information -- symptom, sign, or test result -should provide information to rule it in or out.
Before the new information is acquired, the
physician’s rational synthesis of all available
information may be embodied in an estimate of
pre-test prevalence.
Rationally, the new information should update that
estimate to a post-test prevalence, in the manner
described above for a diagnostic test.
In practice it is rare to proceed from precise
numerical estimates. Nevertheless, implicit
understanding of this logic makes clinical practice
more rational and effective.
57
Pretest Probability: Clinical
Significance




Expected test result means more than
unexpected.
Same clinical findings have different
meaning in different settings
(e.g.scheduled versus unscheduled
visit). Heart sound, tender area.
Neurosurgeon.
Lupus nephritis.
58
What proportion of all patients
will test positive?




Diseased X sensitivity
+ Healthy X (1-specificity)
Prevalence X sensitivity +
(1-prevalence)(1-specificity)
We call this “test prevalence”
i.e. prevalence according to the test.
59
SENS = SPEC = 95%


What if test prevalence is 5%?
What if it is 95%?
60
61
Combination tests: serial and parallel testing
Combinations of specificity and sensitivity superior to the
use of any single test may sometimes be achieved by
strategic uses of multiple tests. There are two usual
ways of doing this.
Serial
testing: Use >1 test in sequence, stopping at the
first negative test. Diagnosis requires all tests to be
positive.
Parallel
testing: Use >1 test simultaneously, diagnosing
if any test is positive.
62
Combination tests: serial testing

Doing the tests sequentially, instead of together with
the same decision rule, is a cost saving measure.

This strategy
– increases specificity above that of any of the individual tests,
but
– degrades sensitivity below that of any of them singly.

However, the sensitivity of the serial combination may
still be higher than would be achievable if the cutpoint of any single test were raised to achieve the
same specificity as the serial combination.
63
Combination tests: serial testing
Demonstration: Serial Testing with Independent Tests

SeSC = sensitivity of serial combination
SpSC = specificity of serial combination

SeSC = Product of all sensitivities= Se1X Se2X…etc
Hence SeSC < all individual Se

1-SpSC = Product of all(1-Sp)
Hence SpSC > all individual Spi

Serial test to rule-in disease
64
Combination tests: parallel testing
Parallel Testing
 Usual decision strategy diagnoses if any test positive.
This strategy
– increases sensitivity above that of any of the individual tests,
but
– degrades specificity below that of any individual test.

However, the specificity of the combination may be
higher than would be achievable if the cut-point of
any single test were lowered to achieve the same
sensitivity as the parallel combination.
65
Combination tests: parallel testing
Demonstration: Parallel Testing with Independent Tests

SePC = sensitivity of parallel combination
SpPC = specificity of parallel combination

1-SePC = Product of all(1 - Se)
Hence SePC > all individual Se

SpPC = Product of all Sp
Hence SpPC < all individual Spi

Parallel test to rule-out disease
66
Clinical settings for parallel
testing


Parallel testing is used to rule-out serious but
treatable conditions (example rule-out MI by
CPK, CPK-MB, Troponin, and EKG. Any
positive is considered positive)
When a patient has non-specific symptoms,
large list of possibilities (differential
diagnosis). None of the possibilities has a
high pretest probability. Negative test for each
possibility is enough to rule it out. Any positive
test is considered positive.
67

Because specificity is low, further
testing is now required (serial testing) to
make a diagnosis (Sp P In).
68
Clinical settings for serial testing

When treatment is hazardous (surgery,
chemotherapy) we use serial testing to
raise specificity.(Blood test followed by
more tests, followed by imaging,
followed by biopsy).
69
Calculate sensitivity and
specificity of parallel tests
(Serial tests in HIV CDC exercise)
 2 tests in parallel
 1st test sens = spec = 80%
 2nd test sens = spec = 90%
 1-Sensitivity of combination =
(1-0.8)X(1-0.9)=0.2X0.1=0.02
 Sensitivity= 98%
 Specificity is 0.8 X 0.9 = 0.72
70
Typical setting for finding
Sensitivity and Specificity





Best if everyone who gets the new test
also gets “gold standard”
Doesn’t happen
Even reverse doesn’t happen
Not even a sample of each (casecontrol type)
Case series of patients who had both
tests
71
EXAMPLE




Patients who had both a stress test and
cardiac catheterization.
So what if patients were referred for
catheterization based on the results of
the stress test?
Not a random or even representative
sample.
It is a biased sample.
72
73
If the test is used to decide
referral for gold standard?
Disease
No
Disease
Total
Test
Positive
95
72
167
Test
Negative
5
828
833
Total
100
900
1000
Sn95/100
=.95
Sp 828/900 =
.92
74
If the test is used to decide
referral for gold standard?
Test
Positive
Test
Negative
Total
Disease
No
Disease
Total
95
85
5
1
100
86
72
65
828
99
900
164
167
167150
833
833 100
1000
Sn85/86=.99 Sp 99/164=.4
75
If the test is used to decide
referral for gold standard?
Disease
No
Disease
Total
Test
Positive
85
65
150
Test
Negative
1
99
100
Total
86
164
250
Sn85/86=.99 Sp 99/164=.4
76