Transcript Dia 1 - EMGO

Meta-analysis of
observational studies
Nicole Vogelzangs
Department of Psychiatry
& EMGO+ institute
Outline
• RCTs vs. observational studies
• Data extraction
• Meta-analysis
– Combining results (statistical pooling)
– Studying sources of heterogeneity (subgroup analysis
and meta-regression analysis)
Observational II-2
RCT vs. observational research
•
•
•
•
•
hypothesis
methodology
validity design
risk of confounding
analyses
RCT
Observational research
therapy
etiology, diagnosis, prognosis
clear
divers, different designs
high
varying
low
high
clear
univariable
adjustment for confounding
multivariable
Observational II-3
Heterogeneity
• More heterogeneity in observational studies
compared with RCTs:
–
–
–
–
Design
Population (less strict selection criteria)
Different ways of assessing exposure and disease
Adjustment for confounding
• Studying the sources of heterogeneity is an
important aim of analyses in reviews of
observational studies
Observational II-4
An example
Depression as a risk factor for the onset of type 2
diabetes mellitus - Knol, Twisk et al. Diabetologica 2006
• Selection criteria
– Longitudinal studies on depression and onset of DM2
– Exclusion:
• studies including prevalent DM2 cases
(only persons ‘at risk’)
• Insufficient data to calculate RR, OR or HR
Observational II-5
Publication bias?
Studies with large SE (small N) and weaker association
appear absent
Observational II-6
Outline
• RCTs vs. observational studies
• Data extraction
• Meta-analysis
– Combining results (statistical pooling)
– Studying sources of heterogeneity (subgroup analysis
and meta-regression analysis)
Observational II-7
Data extraction
For each study:
1. Effect estimate (association determinant – outcome)
For confounding corrected effect!
2. Variance or standard error (SE) of the effect
estimate (=> weight in meta-analysis)
3. Information on potential sources of
heterogeneity
Observational II-8
1. Effect estimate
• Effect estimate based on multivariable analysis,
adjusted for all possible confounders (age, sex,
BMI, blood pressure, etc.)
• Cohort study:
– Logistic regression: OR
– Cox regression: hazard ratio (HR ≈ RR)
• Patient-control study
– Logistic regression: OR
Observational II-9
Are OR and RR/HR exchangeable?
• Only when risk of outcome is small!
• Odds = p / 1-p
large risk, e.g. p=0.30: odds = 0.30 / 0.70 = 0.43
small risk, e.g. p=0.01: odds = 0.01 / 0.99 ≈ 0.01
=> when risk is large, OR is overestimation of RR
• alcohol consumption - bladder cancer: OR is OK
• computer job - neck/shoulder complaints: OR not OK
Observational II-10
2. Standard error of the effect estimate
• SE often not reported
• Calculate from
– confidence interval
– p-value (need sufficient decimals; p < .05 is
insufficient)
Observational II-11
3. Sources of heterogeneity
• Characteristics of study population
• Method of measurement for exposure to risk
factor / prognostic factor
• Method of measurement outcome or disease
• Aspects of design
• Analysis
Observational II-12
Sources of heterogeneity - example
• Population: differences in age, gender, potential
confounders
• Measurement of exposure: method for
assessment of depression (questionnaire, interview,
diagnosis care provider)
• Outcome: method for assessment diabetes
(screening or self-report)
Observational II-13
Sources of heterogeneity - example
• Aspects of design: cohort study
–
–
–
–
Duration of follow-up?
Response / drop-out rate?
Which confounders are measured?
Which methods are used to exclude diabetes patients
at baseline?
• Analysis
– How is exposure defined? (score on depression scale
or dichotomous [cut-off])
– Which confounders are used in analysis?
Observational II-14
Reducing heterogeneity
Some study differences can be canceled out by
converting published data:
– Defining exposure
(e.g. merge several depression categories into yes/no
depression)
– Degree of adjustment for confounding
=> Possibilities strongly depend on reporting
Observational II-15
Outline
• RCTs vs. observational studies
• Data extraction
• Meta-analysis
– Combining results (statistical pooling)
– Studying sources of heterogeneity (subgroup analysis
en meta-regression analysis)
Observational II-16
Meta-analysis
• Combining results (statistical pooling)
Enter for each study:
– effect estimate
– weight (1 / SE2)
• Fixed effects model (often not realistic)
Random effects model
• Study influence of sources of heterogeneity
– Subgroup analysis
– Meta-regression analysis
Observational II-17
Meta-analysis - example
Q-test: p = 0.02
RR (REM):
1.37 (1.14-1.63)
Observational II-18
Study heterogeneity: subgroup analysis
• 1 characteristic at once (e.g. duration of follow-up)
• Not too many categories (e.g. > or < 5 year)
• Stratify for these categories
• Pool effect estimates within each category
Observational II-19
Subgroup analysis - example
RR
2.0
1.8
1.6
1.4
1.2
1.0
0.8
Yes (n = 3)
No (n = 6)
Exclusion undetected diabetes at baseline
Observational II-20
Study heterogeneity: meta-regression
• Several characteristics can be studied
simultaneously (e.g. follow-up duration, design type,
age of population)
• Efficient and more power
• Also possible for ordinal and continuous
measures
Observational II-21
Meta-regression
What is meta-regression?
• Weighted linear regression
• Studies are subject of analysis
• Dependent variable (Y) = effect estimate of study
• Weight = 1 / SE2
• Independent variables (X) = sources of
heterogeneity (effect modifiers)
Y = b0 + b1X + unexplained variance
Observational II-22
Dependent variable (outcome)
• Linear regression: dependent variable (Y) should
be +/- normally distributed
• OR en RR: skewed distribution
0
1
2
3
4
Log-transformation:
5
-4
-3
-2
-1
0
1
2
3
4
ln (OR) = b0 + b1X
ln (RR) = b0 + b1X
Observational II-23
Meta-regression:
sleep position and sudden infant death
3,0
stomach vs. back
2,5
ln(OR)
2,0
1,5
1,0
0,5
0
1955 1960 1965 1970 1975 1980 1985 1990 1995
Year of study
Observational II-24
Meta-regression:
depression and incidence of diabetes
No association between duration of follow-up and effect estimate
Observational II-25
Meta-regression (fictive data)
• Association depression and incidence diabetes
• 6 (fictive) cohort studies
• 3 studies adjust for overweight (BMI > 25),
3 do not adjust for BMI
• Enter for each study:
– ln (RR)
– weighting factor: 1 / SE 2
– confounder adjustment (yes = 1, no = 0)
Observational II-26
Meta-regression - example
Regression equation: ln(RR) = b0 + b1* adjconf
• Results regression analysis (output)
b0 = 0.750, b1 = - 0.250
• Pooled ln(RR) for studies without adjustment for
overweight (adjconf = 0)
ln(RR) = 0.750 + (-0.250*0) = 0.750 => RR = e0.750 = 2.11
• Pooled ln(RR) for studies with adjustment for
overweight (adjconf = 1)
ln(RR) = 0.750 + (-0.250*1) = 0.500 => RR = e0.500 = 1.65
Observational II-27
Bottlenecks meta-regression I
• Limited number of variables that can be studied
at once: usually limited number of studies
• Necessary data are not always available
• Ecological fallacy (aggregation bias):
Associations are analyzed at aggregated level:
do not necessarily reflect the true association
within studies
Observational II-29
Ecological fallacy
Effect of age within studies,
not found when analyzed at
aggregated level
No effect of age within studies,
effect found when analyzed at
aggregated level
See: Thompson SG & Higgins JPT. Stat Med 2002;21:1559-73.
Observational II-30
Bottlenecks meta-regression II
• For subgroup analyses and meta-regression:
studying more variables increases risk of type I
error (false-positive results)
• Restrict the number of subgroups / variables:
– Hypothesis testing: a priori analyses:
describe in methods
– Hypothesis generating: post hoc analyses:
discuss in discussion
Observational II-31
Individual patient data (IPD) meta-analysis
• The best solution?
– Request all original data
– Clean data and recode if necessary
– Meta-analysis and subgroup analysis / meta-regression
– Very powerful
– Cheaper than new trials
• But:
– Very labor intensive
– Data sometimes (often?) no longer available
Observational II-33
In sum
• Meta-analysis of observational studies
– Large heterogeneity
– Prework necessary
• Statistical analysis
– Statistical pooling: fixed/random effects model
– Analysis of sources of heterogeneity (subgroup
analysis, meta-regression)
– IPD meta-analysis
• Keep in mind limitations of subgroup analysis /
meta-regression!
Observational II-34
Meta-analysis of
observational studies
THE END
Observational II-35