Transcript META-ANALYSIS OF RESEARCH

Lecture 9
EPSY 642 Meta Analysis Fall 2009
Victor L. Willson, Instructor
Current Issues
 Multi-level models: Raudenbush & Bryk analysis
in HLM6
 Structural equation modeling in meta-analysis
 Clustering of effects: cluster analysis vs. latent
class modeling
 Multiple studies by same authors- how to treat
(beyond ignoring follow-on studies), the study
dependence problem
 Multiple meta-analyses: consecutive, overlapping
 Multiple outcomes per study
Multilevel Models
 Raudenbush & Bryk HLM 6
 One effect per study
 Two level model, mediators and moderators at the
second level
 Known variance for first level (wi)
 Mixed model analysis: requires 30+ studies for
reasonable estimation, per power analysis
 Maximum likelihood estimation of effects
Multilevel Models
 Model:
Level 1:
gi = gi + ei
where there is one effect g per study i
Level 2:
gi = 0 + 1W + ui
where W is a study-level predictor such as design
in our earlier example
Assumption: the variance of gi is known = wi
Structural Equation Modeling in SEM
 New area- early work in progress:
 Cheung & Chan (2005, Psych Methods), (2009, Struc
Eqn Modeling)- 2-step approach using correlation
matrices (variables with different scales) or covariance
matrices (variables measured on the same
scale/scaling)
 Stage 1: create pooled correlation (covariance) matrix
 Stage 2: fit SEM model to Stage 1 result
Structural Equation Modeling in SEM
 Pooling correlation matrices:
 Get average r:
rmean(jk) = wi riij/ wijk
I
i
where j and k are the subscripts for the correlation
between variables j and k,
where i is the ith data set being pooled
Cheung & Chan propose transforming all r’s to Fisher Zstatistics and computing above in Z
If using Z, then the SE for Zi is (1-r2)/n½ and
Structural Equation Modeling in SEM
 Pooling correlation matrices: for each study,
COVg(rij, rkl) = [ .5rij rkl (r2ik + r2il + r2jk + r2jl) +
rik*rjl + ril*rjk –
(rij*rik*ril + rji*rjk*rjl + rki*rkj*rkl +
rli*rlj*rlk)]/n
Let i = covariance matrix for study i, G = {0,1} matrix
that selects a particular correlation for examination,
Then G = [ |G1|’ G2 |’…| Gk |’]’
and  = diag [1, 2, … k]
Structural Equation Modeling in SEM
Beretvas & Furlow (2006) recommended
transformations of the variances and covariances:
SDrtrans = log(s) + 1/(2(n-1)
COV(ri,rj)trans = r2ij/(2(n-1))
The transformed covariance matrices for each study are
then stacked as earlier
Clustering of effects: cluster analysis vs.
latent class modeling
 Suppose Q is significant. This implies some subset
of effects is not equal to some other subset
 Cluster analysis uses study-level variables to
empirically cluster the effects into either
overlapping or non-overlapping subsets
 Latent class analysis uses mixture modeling to
group into a specified # of classes
 Neither is fully theoretically developed- existing
theory is used, not clear how well they work
Multiple studies by same authors- how to treat (beyond
ignoring follow-on studies), the study dependence problem
 Example: in storybook telling literature,
Zevenberge, Whitehurst, & Zevenbergen (2003) was
a subset of Whitehurst, Zevenbergen, Crone,
Schultz, Velging, & Fischel (1999), which was a
subset of Whitehurst, Arnold, Epstein, Angell,
Smith, & Fischel (1994)
 Should 1999 and 2003 be excluded, or included
with adjustments to 1994?
 Problem is similar to ANOVA: omnibus vs.
contrasts
 Currently, most people exclude later subset articles
Multiple meta-analyses:
consecutive, overlapping
 The problem of consecutive meta-analyses is now
arising:
 Follow-ons typically time-limited (after last m-a)
 Some m-a’s partially overlap others: how should they be
compared/integrated/evaluated?
 Are there statistical methods, such as the correlational
approach detailed above, that might include partial
dependence?
 Can time-relatedness be a predictor? Willson (1985)
Multiple Outcomes per study
 Multilevel Approach to study dependency:
 Requires assumption of homogeneous error variance
across studies


Variation within “cluster” (study) is the same for all studies
If above is reasonable, MLM may be a reasonable model
 Weight function acts as “sample size” equivalent for
second level analysis- use weighted average W1/2 or N for
the effects within the study (unclear which will be more
appropriate here)

If all effects within-study have the same sample size(s) the W
for all are equal to each other
Multiple Outcomes per study
Indep
Var
Effect

W1/2
Study effect
e
CONCLUSIONS
 Meta-analysis continues to evolve
 Focus in future on complex modeling of outcomes
(SEM, for example)
 More work on integration of qualitative studies with
meta-analysis findings