Transcript Prevalence of Breast and Bottle Feeding

Design and Analysis of Clinical Study
12. Meta-analysis
Dr. Tuan V. Nguyen
Garvan Institute of Medical Research
Sydney, Australia
Overview
• What is meta-analysis
• Two types of data
• Statistical procedures
Why Meta-analysis/Systematic Reviews?
• “. . . the mass of new information makes it difficult for
practicing physicians to follow the literature in all areas
that might be relevant to their practices. New methods to
synthesize and present information from widely dispersed
publications are needed
. . . .”
Jerome Kassirer. Clinical trials and meta-analysis: what
do they do for us? N Engl J Med 1992; 327:273-4.
Why Need Meta-analysis? Information Explosion
• 10-fold Increase in Number of Professional Journals
• Psychology Journals:
91 (1951) --> 1,175 (1992)
•
• Math Science Journals:
91 (1953) --> 920 (1992)
• Biomedical Journals:
2,300 (1940)--> 23,000 (1993)
Problem – Conflicting Information
• Not only is there more information, but . . .
• Not all information is of equal quality
• Information does not necessarily = evidence
• There is often conflicting information & reports Traditional
narrative reviews can be very “impressionistic”
Problems With Traditional Literature Reviews Addressed in Metaanalysis
• Selective inclusion of studies, often based on the
reviewer's own impressionistic view of the quality of the
study
• Differential subjective weighting of studies in the
interpretation of a set of findings
• Misleading interpretations of study findings
• Failure to examine characteristics of the studies as
potential explanations for disparate or inconsistent results
across studies
• Failure to examine moderating variables in the relationship
under examination
Rationale for Systematic Reviews
• “provide summaries of what we know, and do not know,
that are as free from bias as possible.” (Chalmers et al
1999)
• “research that uses explicit & transparent methods to
synthesise relevant studies, allowing others to comment
on, criticise or attempt to replicate the conclusions
reached. Systematic reviews follow same set of
procedures as any individual study, & are often reported in
the same way. . . .” (Petrsino et al 1999)
4 Basic Questions That a SR/MA Tries to Answer
• Are the results of the different studies similar?
• To the extent that they are similar, what is the best overall
estimate of effect?
• How precise and robust is this estimate?
• Can dissimilarities be explained?
Lau J, Ioannidis JPA, Schmid CH. Quantitative Synthesis in
Systematic Reviews. Annals of Internal Medicine 1997;
127:820-826.
What is a Systematic Review?
• Assemble the most complete dataset feasible, with
involvement of investigators
• Analyse results of eligible studies.
Use statistical synthesis of data
(meta-analysis) if appropriate & possible
• Perform sensitivity analyses, if appropriate & possible
(including subgroup analyses)
• Prepare a structured report of the review, stating aims,
describing materials & methods, & reporting results
Cochrane Library
•
•
•
•
•
•
•
Cochrane Library
CD (&
WWW)
Cochrane Database of
Systematic Reviews
(CDSR)
Database of Abstracts of
Reviews of Effectiveness
(DARE)
Cochrane Central Register of
Controlled Trials (CENTRAL)
Cochrane Review Methodology
Database
Health Technology Assessment
DB (HTA)
NHS Economic/Evaluation
Database (NHS EED)
Search Strategy – References & Databases
• Studies were identified from
– Cochrane Airways Group's Special Register of Controlled
Trials comprised of references from
– MEDLINE (1966-2000)
– EMBASE (1980-2000)
– CINAHL (1982-2000)
• hand searched airways-related journals
• PsychINFO
• Reference lists from relevant review articles that were
identified (ancestry approach
Search Strategy - Terms
• Congestive Heart Failure OR Heart Failure* AND
• clinical trial* OR beta blocker*
• placebo* OR trial* OR random* OR double-blind OR
double blind OR single-blind OR single blind OR controlled
study OR comparative study.
Identification of Trials
• Potentially relevant studies from literature search and
hand searches
• Excluded on basis of abstract, e.g., not randomised or
controlled clinical trials Articles selected for full text review
• Excluded after full text review
• Eligible trials
Main Outcome Measures
• Mortality / death
Beta-blocker and Congestive Heart Failure
Study
(i)
Beta-blocker
Placebo
N1
Deaths (d1)
N2
Deaths (d2)
1
25
5
25
6
2
9
1
16
2
3
194
23
189
21
4
25
1
25
2
5
105
4
34
2
6
320
53
321
67
7
33
3
16
2
8
261
12
84
13
9
133
6
145
11
10
232
2
134
5
11
1327
156
1320
228
12
1990
145
2001
217
13
214
8
212
17
Tổng cộng
4879
420
4516
612
Model of Meta-analysis
•
For each study
– Relative risk
RRi 
p1
p2
– Variance and standard error of logRR
1
1
1
1

 
d1 N1  d1 d 2 N 2  d 2
var log RRi  
SE log RRi  
1
1
1
1

 
d1 N1  d1 d 2 N 2  d 2
– 95% confidence interval of RR
log RR 1.96SElog RR
e
– Weight
Wi 
1
var log RRi 
Model of Meta-analysis
•
For all studies
– Overall relative risk
log RR 
W  log  RR 
i
i
W
i
– Variance and standard error
var  log RR  
1
W
i
SE log RR  
1
W
i
– 95% confidence interval
log RR 1.96  SE log RR
Meta-analysis: an example
Wi
Wi×log[RRi]
0.264
3.79
-0.69
-0.118
1.304
0.77
-0.09
1.067
0.065
0.079
12.61
0.82
0.080
0.500
-0.693
1.415
0.71
-0.49
0.038
0.059
0.648
-0.434
0.709
1.41
-0.61
6
0.166
0.209
0.794
-0.231
0.026
38.30
-8.86
7
0.091
0.125
0.727
-0.318
0.729
1.37
-0.44
8
0.046
0.155
0.297
-1.214
0.142
7.03
-8.54
9
0.045
0.076
0.595
-0.520
0.242
4.13
-2.15
10
0.009
0.037
0.231
-1.465
0.688
1.45
-2.13
11
0.118
0.173
0.681
-0.385
0.009
110.78
-42.63
Study
p1
p2
RRi
logRRi
1
0.200
0.240
0.833
-0.182
2
0.111
0.125
0.889
3
0.119
0.111
4
0.040
5
Var[logRR]
12
0.073
0.108
0.672
-0.398
0.010
96.13
-38.23
13
0.037
0.080
0.466
-0.763
0.174
5.75
-4.39
284.24
-108.42
Meta-analysis: an example
log wRR 
W  log  RR 
i
i
W

108.42
 0.38
284.24

1
 0.0035
284.24
i
Var  log wRR  
1
W
i
SE log wRR  
1
W
 0.0035  0.06
i
95% CI of logRR = -0.38 ± 1.96×0.06
= -0.498, -0.265
95% of RR:
exp(-0.498) = 0.61 to exp(-0.265) = 0.77
Meta-analysis using R
library(meta)
n1 <- c(25.9.194.25.105.320.33.261.133.232.1327.1990.214)
d1 <- c(5.1.23.1.4.53.3.12.6.2.156.145.8)
n2 <- c(25.16.189.25.34.321.16.84.145.134.1320.2001.212)
d2 <- c(6.2.21.2.2.67.2.13.11.5.228.217.17)
bb <- data.frame(n1.d1.n2.d2)
res <- metabin(d1.n1.d2.n2.data=bb.sm=”RR”.meth=”I”)
res
plot(res. lwd=3)
Meta-analysis using R
> res
RR
95%-CI %W(fixed) %W(random)
1 0.8333 [0.2918; 2.3799]
1.26
1.26
2 0.8889 [0.0930; 8.4951]
0.27
0.27
3 1.0670 [0.6116; 1.8617]
4.47
4.47
4 0.5000 [0.0484; 5.1677]
0.25
0.25
5 0.6476 [0.1240; 3.3814]
0.51
0.51
6 0.7935 [0.5731; 1.0986]
13.08
13.08
7 0.7273 [0.1346; 3.9282]
0.49
0.49
8 0.2971 [0.1410; 0.6258]
2.49
2.49
9 0.5947 [0.2262; 1.5632]
1.48
1.48
10 0.2310 [0.0454; 1.1744]
0.52
0.52
11 0.6806 [0.5635; 0.8221]
38.81
38.81
12 0.6719 [0.5496; 0.8214]
34.31
34.31
13 0.4662 [0.2056; 1.0570]
2.07
2.07
Number of trials combined: 13
RR
95%-CI
z p.value
Fixed effects model 0.6821 [0.6064; 0.7672] -6.3741 < 0.0001
Random effects model 0.6821 [0.6064; 0.7672] -6.3741 < 0.0001
Quantifying heterogeneity:
tau^2 = 0; H = 1 [1; 1.45]; I^2 = 0% [0%; 52.6%]
Test of heterogeneity:
Q d.f. p.value
11
12
0.5292
Forest Plot
1
2
3
4
5
6
7
8
9
10
11
12
13
0.05
0.10
0.20
0.50
1.00
Relative Risk
2.00
5.00
10.00
An Inverted Funnel Plot to Detect Publication Bias
An Inverted Funnel Plot to Detect Publication Bias
Heterogeneity
• Common, to be expected, not the exception
• Should do test for homogeneity, but . . . interpret
heterogeneity cautiously in spirit of exploratory data
analysis
– Exploring sources of heterogeneity can lead to insights
about modification of apparent associations by various
aspects of
– Study design
– Exposure measurements
– Study populations
Heterogeneity
• Relations discovered in process of exploring heterogeneity
may be useful in planning & carrying out new studies
• Excluding outliers solely on basis of disagreement with
other studies can lead to seriously biased summary
estimates (avoid)
• Easier to interpret sources of heterogeneity when
identified in advance of data analysis
(not when suggested only by data)
Fixed & Random Effects
• Fixed effects models assume that an intervention has a
single true effect
• Random effects models assume that an effect may vary
across studies
Random Effects
• Assumes sample of studies randomly drawn from
population of studies
• This is NOT typically true because:
– All trials are included
– Trials are systematically (e.g., conveniently) sampled and
not randomly sampled
Random Effects
• Primary value of M-A is in search for predictors of
between-study heterogeneity
• Random-effects summary is last resort only when
predictors or causes of between-study heterogeneity
cannot be identified
• Random-effects can conceal fact that summary estimate
or fitted model is poor summary of the data Sander
Greenland.
Am J Epidemiol 1994;140;290-6.
Random Effects
• Sometimes needed, but more sensitive to publication bias
than fixed-effects
• Random effects weights vary less across studies than
fixed-effects weights
• W = 1/v versus w = 1/(v + t2)
• Leads to reduced variation in weights
• Thus smaller studies given larger relative weights when
random effects models used
• Thus influenced more strongly by any tendency NOT to
publish small statistically insignificant studies
 biased estimate, spuriously strong associations
Random Effects
•
•
•
•
Fixed effects weights vs. random effects weights
W = 1/v versus w = 1/(v + t2)
Identical when there is little or no between study variation
When differ, confidence intervals are larger for randomeffects than fixed effects
• Smaller studies given larger relative weights in random
effects models & > influence
• Conversely, influence of larger studies is less
• May result in type II (beta error), e.g., Finding no
significant difference when one truly exists
Methodologic Choices & Their Implications in Dealing With Heterogeneous
Data in a Meta-analysis
Lau J, Ioannidis JPA, Schmid CH. Quantitative Synthesis in Systematic
Reviews. Annals of Internal Medicine 1997; 127:820-826.