Transcript A diagnostic test study
Diagnostic Test Studies
Tran The Trung Nguyen Quang Vinh
Why we need a diagnostic test?
We need “information” to make a decision “Information” is usually a result from a test Medical tests: y To screen for a risk factor (screen test) y y To diagnosse a disease (diagnostic test) To estimate a patient’s prognosis (pronostic test) When and in whom, a test should be done?
y When “information” from test result have a value.
Value of a diagnostic test
The ideal diagnostic test: y Always give the right answer: y x Positive result in everyone with the disease x Negative result in everyone else Be quick, safe, simple, painless, reliable & inexpensive But few, if any, tests are ideal.
Thus there is a need for clinically useful substitutes
Is the test useful ?
Reproducibility (Precision) Accuracy (compare to “gold standard”) Feasibility Effects on clinical decisions Effects on Outcomes
Determining Usefulness of a Medical Test Question
1. How
reproducible
is the test?
Possible Designs Statistics for Results
Studies of: -
intra- and inter observer &
-
intra- and inter laboratory variability
Proportion agreement, kappa, coefficient of variance, mean & distribution of differences (avoid correlation coefficient)
Determining Usefulness of a Medical Test Question Possible Designs
2. How
accurate
the test?
is Cross-sectional, case control, cohort-type designs in which
test result is compared with a “gold standard”
Statistics for Results
Sensitivity, specificity, PV+, PV-, ROC curves, LRs
Determining Usefulness of a Medical Test Question
3. How often do
test results affect clinical decisions
?
Possible Designs
Diagnostic yield studies, studies of pre & post test clinical decision making
Statistics for Results
Proportion abnormal, proportion with discordant results, proportion of tests leading to changes in clinical decisions; cost per abnormal result or per decision change
Determining Usefulness of a Medical Test Question
4. What are the
costs, risks, & acceptability
of the test?
Possible Designs Statistics for Results
Prospective or retrospective studies Mean cost, proportions experiencing adverse effects, proportions willing to undergo the test
Determining Usefulness of a Medical Test Question
5. Does doing the test
improve clinical outcome
, or
having adverse effects
?
Possible Designs Statistics for Results
Randomized trials, cohort or case-control studies in which the predictor variable is receiving the test & the outcome includes morbidity, mortality, or costs related either to the disease or to its treatment Risk ratios, odd ratios, hazard ratios, number needed to treat, rates and ratios of desirable and undesirable outcomes
Common Issues for Studies of Medical Tests
Spectrum of Disease Severity and Test Results: y Difference between Sample and Population?
y y Almost tests do well on very sick and very well people.
The most difficulty is distinguishing Healthy & early, presymtomatic disease.
Subjects should have a spectrum of disease that reflects the clinical use of the test.
Common Issues for Studies of Medical Tests
Sources of Variation: y Between patients y y Observers’ skill Equipments => Should sample several different institutions to obtain a generalizable result.
Common Issues for Studies of Medical Tests
Importance of Blinding: (if possible) y Minimize observer bias y Ex. Ultrasound to diagnose appendicitis (It is different to clinical practice)
Studies of Diagnostic tests
Studies of Test Reproducibility Studies of The Accuracy of Tests Studies of The Effect of Test Results on Clinical Decisions Studies of Feasibility, Costs, and Risks of Tests Studies of The Effect of Testing on Outcomes
Studies of Test Reproducibility
The test is to test the precision y Intra-observer variability y Inter-observer variability Design: y y y Cross-sectional design Categorical variables: Kappa Continuous variables: coefficient of variance Compare to it-self (“gold standard” is not required)
Studies of the Accuracy of Tests
Does the test give the right answer?
“Tests” in clinical practice: y Symptoms y Signs y y
Laboratory tests Imagine tests
To find the right answer.
“Gold standard” is required
How
accurate
is the test?
Validating tests against a gold standard:
New tests should be validated by comparison against an established gold standard in an appropriate subjects Diagnostic tests are seldom 100% accurate (false positives and false negatives will occur)
Validating tests against a gold standard
A test is valid y y if: It detects most people with disorder (high Sen ) It excludes most people without disorder (high Sp ) y a positive test usually indicates that the disorder is present (high PV+ ) The best measure of the usefulness of a test is the LR : how much more likely a positive test is to be found in someone with, as opposed to without, the disorder
A Pitfall of Diagnostic test
A test can separate healthy does the very sick from the very not mean that it will be useful distinguish patients with mild cases of the disease from others with similar symptoms in
Sampling
The spectrum of patients should be representative of patients in real practice.
Example: Which is better? What is the limits?
y y Chest X-ray to diagnose aortic aneurism (AA). Sample are 100 patients with and 100 without AA that ascertained by CT scan or MRI.
FNA to diagnose thyroid cancer. 100 patients with nodule > 3cm and had indication to thyroidectomy (biopsy was the gold standard).
“Gold standard”
“Gold standard” test: often confirm the presence or absence of the disease : D(+) or D(-).
Properties of “Gold standard”: y Ruling in the disease (often doing well) y y y Ruling out the disease (maybe not doing well) Feasible & ethical ? (ex. Biopsy of breast mass) Widely acceptable.
The test result
Categorical variable: y Result: Positive or Negative y Ex. FNA cytology Continuous variable: y y Next step is: find out “cut-off point” by ROC curve Ex. almost biochemical test: pro-BNP, TR-Ab,..
Analysis of Diagnostic Tests How
accurate
is the test?
Sensitivity & Specificity Likelihood ratio: LR (+), LR (-) Posterior probability (Post-test probability) / Positive, Negative Predictive value (PPV, NPV); given Prior probability (Pre-test probability)
Sensitivity and Specificity
Sens
a a
c Spec
d b
d
Test Result
+ -
Disease D “Gold standard”
+ -
a c b d
Positive & Negative Predictive Value
PV (+): positive predictive value PV (-): negative predictive value
PV a a
b
Test Result
+ -
Disease D
+
a c
-
b d
PV d c
d
) ) )
Posterior odds
When combined with information on the prior probability of a disease result: * , LRs can be used to determine the predictive value of a particular test Posterior odds = Prior odds x Likelihood ratio *expressing the prior probability [p] of a disease as the prior odds [p/(1-p)] of that disease. Conversely, if the odds of a disease are x/y, the probability of the disease is x / (x + y)
Choice of a cut-off point for continuous results
Consider the implications of the two possible errors: If false-positive results must be avoided (such as the test result being used to determine whether a patient undergoes dangerous surgery), then the cutoff point might be set to test's specificity maximize the If false-negative results must be avoided (as with screening for neonatal phenylketonuria), then the cutoff should be set to ensure a test sensitivity high
Choice of a cut-off point for continuous results
Using receiver operator characteristic (ROC) curves: y Selects several cut-off points , and determines sensitivity and specificity at each point the y Then, graphs sensitivity (true-positive rate) as a function of 1-specificity (false-positive rate) Usually, the best cut-off point is where the ROC curve " turns the corner ”
RECEIVER OPERATING CHARACTERISTIC (ROC) curve
ROC curves (Receiver Operator Characteristic) Ex. SGPT and Hepatitis SGPT < 50 50-99 100-149 150-199 200-249 250-299 >300 Sum D + 10 15 25 30 35 120 65 300 D 190 135 65 30 15 10 5 450 Sum 200 150 90 60 50 130 70 750
1 Sensitivity 1-Specificity 1