Transcript Document

Personalized Medicine
• Detection
• Diagnosis
• Treatment
• Survival
Prediction is very difficult,
especially about the future.
Niels Bohr
Danish physicist (1885 - 1962)
Biomarkers
1. Discrimination
(sensitivity, specificity, predictive value,
ROC analysis)
2. Utility
Test
Information
Decision
Outcome
(disease free survival, recurrence rates,
survival etc)
Diagnostic tests
Describing test performance
Properties of a test
• Sensitivity:
– a/a+c
Test
Result
Disease
No
disease
Total
Positive
a
b
a+b
Negative c
d
c+d
Total
c+d
a+b+c+d
a+c
• Specificity:
– d/c+d
• Positive predictive value:
– a/a+b
• Negative predictive value:
– d/c+d
The importance of disease
prevalence
• Screening
mammography
Test
result
No
breast
cancer
Total
4,980
5,340
Negative 40
94,620
94,660
Total
99,600
100,000
Positive
Breast
cancer
• Properties of the test
360
400
Sensitivity: 90%
a/a+c = 360/400
Specificity: 95%
d/c+d = 94,620/99,600
Positive predictive value:
a/a+b = 360/5340 = 7%
Negative predictive value:
d/c+d =94,620/94,660 = 100%
Desiderata for studies of
diagnostic tests.
•
•
•
•
•
•
•
“Gold” standard
Test result before outcome known
“Blind” reading
Pre-determined cut-off
Sensitivity and specificity.
Predictive value.
Receiver operating. characteristic
curves (ROC).
Diagnostic tests and the spectrum of
disease.
• Spectrum of patients.
• Clinical spectrum
• Co-morbid spectrum
• Pathologic spectrum
• Potential biases in test
evaluation.
•
•
•
•
Exclusion of equivocal cases
Work up bias
Test review bias
Incorporation bias
Clinical value of tests
Test
Information
Decision
Outcome
INITIAL STATE,
RECIPIENTS OF
PRINCIPAL AGENT
INITIAL STATE,
RECIPIENTS OF
COMPARATIVE AGENT
PRINCIPAL
AGENT
COMPARATIVE
AGENT
SUBSEQUENT
EVENTS,
RECIPIENTS OF
PRINCIPAL AGENT
SUBSEQUENT
EVENTS,
RECIPIENTS OF
COMPARATIVE
AGENT
Research Designs-General Structure
• Purpose of research
• Types of manoeuver
(initial states)
• Prevention.
• Prediction of risk in healthy.
• Treatment response or toxicity
• Inherited (eg genetic variant).
• Acquired
– Self selected (smoking,
alcohol)
– Other (treatment).
in those with disease.
• Imposed (atomic irradiation).
• Identify factors that influence
outcome (prognosis).
Principal research designs
Cohort study
Disease
Present
Absent
Present
a
b
Absent
c
d
Exposure
Passage of time
Relative risk = a/a+b ÷ c/c+d
Nested case control studies
Screening programs: NBSS, SMPBC, OBSP
Baseline mammogram
Risk factors
case
control
case
control
case
control
case
control
6-8 years follow-up
How many subjects (or samples) do
you need?
• Number of events (eg deaths).
• Willingness to risk a false positive (Type I)
error.
• Willingness to risk a false negative result
(Type II) error.
• Magnitude of difference worthwhile to
detect.
• Time for accrual and follow-up.
Sample size to detect an improvement in
survival (alpha=0.05; 1-beta=0.90)
P2-P1
P1
0.10
0.30
0.50
0.10
395
76
41
0.30
879
118
51
0.50
1020
116
-
Sample size for genetic studies
Odds ratio Allele %
1.2
5%
12,217
20%
3730
30%
2896
1.3
5702
1763
1380
1.5
2249
712
566
2.0
687
377
188
PRINCIPAL AGENT
INITIAL STATE
{
}
R
COMPARATIVE AGENT
SUBSEQUENT
EVENTS
A trial to change diet
Screening
Randomization
4,693
Low-fat
diet
Usual
diet
>8 years counseling
and follow-up
•
•
•
•
•
Vancouver + Surrey
Windsor
London + Sarnia
Hamilton + KW
Toronto
• Funding: Ontario Ministry of Health,
Medical Research Council, Canadian
Breast Cancer Research Alliance,
National Institutes of Health,
American Institute for Cancer
Research
0.150
All invasive breast cancer
(A)
HRa = 1.05 (95% CIb : 0.83 - 1.33)
Cumulative hazard
0.125
0.100
0.075
Intervention
0.050
Comparison
0.025
0.000
0
2
4
6
8
10
12
14
16
18
# eventsc I:
16
25
20
18
23
16
10
6
6
20
Year
3
16
9
24
18
18
20
20
4
10
1
2341
2312
2269
2228
2190
2148
1858
1194
740
324
2349
2323
2300
2258
2221
2181
1878
1194
742
329
C:
# at riskd I:
C:
Association or causation?
• Not all associations
are causal
• All causal factors
show association
• May be due to bias or
confounding
• Genetic associations
– Causal
– In linkage
disequilibrium with the
causal variant
– Population
stratification
Population stratification
• Type of confounding
• Ethnicity
– associated with disease
– associated with genotype
– gives spurious association between genotype
and disease
• Can be controlled in analysis (if recognized)
• Dispute about importance
Analysis
P<0.05
What does this mean?
The meaning of p-values.
If the TRUE difference between the compared
groups is zero (the null hypothesis), the
PROBABILITY of obtaining a difference as large
or larger than the one observed by CHANCE is p.
Multiple comparisons
• The problem.
• What protection?
• If alpha = 0.05
• 20 comparisons can be
expected to generate one
p<0.05.
• Few, a priori hypotheses
• Correction for number of
tests eg Bonferroni
– Alpha/number of tests
• (1-(1-alpha)k, where alpha
is the level for
significance and
k=number of tests.
• Stringent alpha eg E 10-8
• Replication/validation
Francis Galton’s ox and the “Winner’s
curse”.
• Country fair in 1906 • At auction, most bids
800 bought tickets and
cluster around the
predicted the weight of
“true” value of the
an ox.
object.
• Actual weight was 1,198 • The winning bid is
lbs.
always higher than the
“true” value.
• None were close to the
actual weight.
• Mean predicted weight
(N=787) was 1,197 lbs.
Replication -validation
• “leave one out”
– Applied to “learning set”
– Not an independent sample
– May help avoid overfitting
• Independent data set
– Preferably also an independent investigator
How to get a statistically significant
result.
• Count or ignore differences in
follow-up time.
• Censor at different time points.
• Exclude specific causes of death.
• Exploit sub-group analysis.
• Use different cut-offs for gene
expression (or other test result).
• Note: all of the above increases the
number of statistical tests you can
do!
Can you believe the literature?
• Publication bias (author and editor
bias).
• Multiple statistical testing.
• The “Winner’s curse”.
• Bias in the sampling,
measurement or analysis of the
data.
• Most published reports are never
replicated.
The “Winners Curse”
False positives more likely:
Small studies
Small effects
Early, hypothesis generating studies
Financial interest
“Hot” field
Ioannidis PLos Medicine 2005
How to stay out of trouble
•
•
•
•
•
•
Define target population.
Standardize sample collection.
Collect samples at zero time.
Define outcomes at the outset.
Random selection of cases and controls.
Analyze samples without knowledge of
case/control status.
• Replicate.