Transcript PPTX
EVAL 6970: Meta-Analysis
Subgroup Analysis
Dr. Chris L. S. Coryn
Kristin A. Hobson
Fall 2013
Agenda
β’ Subgroup analyses
β In-class activity
Subgroup Analyses
β’ Three methods
β Method 1: π-test
β Method 2: π-test based on ANOVA
β Method 3: π-test for heterogeneity
β’ All three methods are used to assess
differences in subgroup effects relative
to the precision of the difference
β’ All three are mathematically equivalent
Subgroup Analyses
β’ The computations for each of the
three methods vary slightly
depending on how subgroups are
analyzed
β Fixed-effect model within subgroups
β Random-effects model with separate
estimates of π 2
β Random-effects model with pooled
estimate of π 2
Subgroup Analyses
Model
Method 1
Method 2
Method 3
Fixed-effect
1
2
3
Random-effects with
separate estimates of
π2
4
5
6
Random-effects with
pooled estimate of π 2
7
8
9
Fixed-Effect Model within
Subgroups: Method 1
β’ Method 1: π-test (similar to t-test in
primary studies)
β’ Used when there are only two
subgroups
β In the fixed-effect model ππ΄ and ππ΅ are
the true effects underlying groups and
ππ΄ and ππ΅ are the estimated effects
with variance ππ΄ and ππ΅
Fixed-Effect Model within
Subgroups: Method 1
β’ The difference between the two
effects is
π·πππ = ππ΅ β ππ΄
β’ Which is tested as
π·πππ
ππ·πππ =
ππΈπ·πππ
β’ Where
ππΈπ·πππ =
πππ΄ + πππ΅
Fixed-Effect Model within
Subgroups: Method 1
β’ Where
π»0 βΆ ππ΄ = ππ΅
β’ For a two-tailed test
π = 2 1 β Ξ¦ |π|
ππ΅ β ππ΄
=(1-(NORMDIST(ABS(Z))))*2
Fixed-Effect Model within
Subgroups: Method 1
ππ΄ and ππ΅
ππ΄ and ππ΅
Fixed-Effect Model within
Subgroups: Method 2
β’ Method 2: π-test based on ANOVA
β For comparisons between more than
two subgroups
β An analogy to ANOVA in primary studies
β Used to partition the total variance into
variance within groups and variance
between groups
Fixed-Effect Model within
Subgroups: Method 2
β’ The following quantities are required
β ππ , the weighted SS of all studies about the
mean for all p subgroups (separately; e.g.,
ππ΄ , ππ΅ )
β ππ€ππ‘βππ , the sum of all ππ subgroups (e.g.,
ππ΄ + ππ΅ )
β ππππ‘ , the weighted SS of the subgroup
means about the grand mean (ππππ‘ = π β
ππ€ππ‘βππ
β π, the weighted SS of all effects about the
grand mean
Fixed-Effect Model within
Subgroups: Method 2
β’ Each source of variance (π statistic)
is evaluated with respect to the
corresponding degrees of freedom
=CHIDIST(π,ππ)
β’ Which returns the exact π-value
associated with each source of
variance
Fixed-Effect Model within
Subgroups: Method 2
π
ππ
π
A
8.4316
4
0.0770
B
4.5429
4
0.3375
Within
12.9745
8
0.1127
Between
13.4626
1
0.0002
Total
26.4371
9
0.0017
π-test ANOVA table
Fixed-Effect Model within
Subgroups: Method 2
Variance components
Fixed-Effect Model within
Subgroups: Method 3
β’ Method 3: π-test for heterogeneity
β Each subgroup is the unit of analysis
β Subgroup summary effects and
variances are tested for heterogeneity
using the same method for testing the
dispersion of single studies about the
summary effect
Fixed-Effect Model within
Subgroups: Method 3
π, ππ, and π
Use Total between
Magnitude of Subgroup Differences
β’ With
π·πππ = ππ΅ β ππ΄
β’ The 95% confidence interval is
β’ Where
π·πππ ± 1.96 × ππΈπ·πππ
ππΈπ·πππ =
πππ΄ + πππ΅
Random-Effects Model with Separate
2
Estimates of π
β’ For all three methods (π-test, π-test
based on ANOVA, and π-test for
heterogeneity) the same
computations are used, but with
random-effects weights and a
separate estimate of π 2 for each
subgroup
Random-Effects Model with Separate
2
Estimates of π
Random-effects model with
separate estimates of π 2
Random-Effects Model with Pooled
2
Estimate of π
β’ For all three methods (π-test, π-test
based on ANOVA, and π-test for
heterogeneity) the same
computations are used, but with
random-effects weights and a pooled
2
estimate of π 2 , referred to as ππ€ππ‘βππ
Random-Effects Model with Pooled
2
Estimate of π
2
β’ The pooled estimate of π 2 , ππ€ππ‘βππ
, is
2
ππ€ππ‘βππ
=
π
π
π=1 ππ β π=1 πππ
π
π=1 πΆπ
β’ Where
πΆ=
ππ β
ππ2
ππ
Random-Effects Model with Pooled
2
Estimate of π
π
ππ
πΆ
A
8.4316
4
269.8413
B
4.5429
4
241.6667
12.9745
8
511.5079
Total
2
ππ€ππ‘βππ
12.9745 β 8
=
= 0.00974
511.508
Random-Effects Model with Pooled
2
Estimate of π
π and ππ
Use A and B
and Total within
Random-Effects Model with Pooled
2
Estimate of π
Random-effects model with
pooled estimate of π 2
Random-Effects Model with Pooled
2
Estimate of π
Fixed-effect model for quantities
to calculate πΆπ‘ππ‘ππ (πΆπ΄ + πΆπ΅ )
πΆπ΄ and πΆπ΅
Proportion of Explained Variance
β’ Unlike the traditional interpretation
of π
2 (i.e., the ratio of explained
variance to total variance), π
2 as
used in meta-analysis is interpreted
as proportion of true variance to total
variance explained by covariates
β’ Computational model assumes that
π 2 is the same for all subgroups (i.e.,
pooled π 2 )
Proportion of Explained Variance
β’ In a meta-analysis, π
2 is the
between-studies variance within
subgroups divided by the total
between-studies variance (withinsubgroups plus between-subgroups)
2
π
π€ππ‘βππ
2
π
=1β
2
ππ‘ππ‘ππ
Calculating π
2
Random-effects model with
pooled estimate of π 2
Calculating π
2
2
ππ‘ππ‘ππ
Variance Explained by Subgroup
Membership
Within studies 34%
Between studies (I2) 66%
Within groups 33%
Between groups (R2) 67%
Summary Effects in Subgroup
Analyses
β’ Depends on questions and the nature
of data
β’ If question is one of superiority, best
not to report a summary effect across
subgroups
β’ If question is one of equivalence, a
summary effect across subgroups may
or may not be warranted
β’ Most important are substantive
implications and what summary effect
represents
Models for Subgroup Analyses
β’ Three models
β Fixed-effect analysis: Fixed-effect model
within and across subgroups
β Mixed-effect analysis: Random-effects
model within subgroups and fixed-effect
model across subgroups (generally
recommended model)
β Fully random-effects analysis: Randomeffects model within and across
subgroups
Todayβs In-Class Activity
β’ From the βTutoring Subgroup.CMAβ
data set
β Using Method 1 (Z-test) calculate the
mean difference, p, and the LL and UL
of the mean difference between
subgroups A and B for the fixed-effect
model, random effects model with
separate estimates of π 2 and random
effects model with pooled estimate of π 2
β Calculate and interpret π
2