Transcript Slide 1
How to calculate: - Average Effect Size (step 4) - Moderators (step 5) Overview of PPT slides We will first go through “how” to do this. Then, we will repeat everything and explain “why” we do each step Overview of Data Analysis Central tendency – Effect size Variability – Homogeneity test What is the average effect and is it significant? Does average affect have variability? (If so, then you test moderators) Prediction Does the average effect differ with moderators? Average effect size (ES) Overview Conceptually… First, transform “r” into Fisher’s z-to-r Second, weight them by sample size/inverse variance Third, sum them together Fourth, divide by sum of total sample size. In Practice… You transform “r” into Fisher’s z-to-r You calculate inverse variance Then use Macro to do the rest At this point you have: (1) Effect sizes. (2) Sample sizes. (3) moderators Step 1 - Transform “r” into Fisher’s z-to-r See formulas from “Example-DataSet2” Step 2 - Weight by inverse variance See formulas from “Example-DataSet2” Step 3 - Upload to SPSS in SPSS, file-open Step 4 - Initiate “MeanES” macro Download from Lipsey/Wilson website Open YOUR DATASET Open a new syntax Put the following at the end of the syntax file INCLUDE 'C:\Documents and Settings\Desktop\metaclub\MeanES.SPS' . Highlight the sentence Click run (blue triangle at top) Step 5 - Calculate “MeanES” In syntax, type the following sentence: MEANES ES = ES_zr /W = weight /PRINT IVZR. fyi – “ES_zr” is my name for the effect sizes “weight” is my name for the weight “/PRINT IVZR” converts output back to “r” Highlight and click run Step 6 – Interpret output ES = .1225, p = .0000 Q = 1140.11, p = .0000 Now, let’s repeat and explain “why” Transform into Fisher’s z-to-r Weight by inverse variance Why? “r” does not have a normal distribution because it is skewed at the tails. Fishers’ z-to-r has a normal distribution, so it is the preferred metric. Why? The larger the sample size of a particular study, the larger the impact. Weighting by the inverse takes into account sample size. Control for standard error Why? When you weight by the inverse variance, then the standard errors are incorrect, so you can’t use the traditional tests within SPSS. The macros by Lipsey/Wilson control for the problem with standard error. Now, let’s move to Moderators Overview Conceptually… First, need to ascertain “homogeneity” which tells you if variance exists in average effect size Can test categorical moderators (categories like college student versus actual juror) similar to ANOVA Can test continuous moderators (such as length of stimulus) similar to Regression In Practice… Use macros Macros exist for ANOVA & Regression Macros for moderators METAF – Categorical Moderators INCLUDE 'C:\Documents and Settings\Desktop\metaclub\MetaF.SPS' . METAF ES = ES_zr /W = weight /GROUP = ev1_subjecttype /PRINT IVZR. METAF – Continuous Moderators INCLUDE 'C:\Documents and Settings\Desktop\metaclub\MetaREG.SPS' . METAREG ES = ES_zr /W = weight /IVS = ev1_subjecttype /PRINT IVZR. fyi – can only run 2 macros in same session fyi – I run categorical moderators using both METAF and METAREG because answer different questions Interpreting Categorical Macro Sig difference = 22.87, p = .0000 Interpreting Continuous Macro beta = -.0958, p = .0014 Now, to Multivariate METAREG can handle multiple variables METAREG ES = ES_zr /W = weight /IVS = ev1_subjecttype ev2_stimulustype /PRINT IVZR. fyi – Can include as many variables as you wish. fyi - I believe you can include CATEGORICAL moderators IF: (1) they are dichotomous, (2) they are continuous and linear relationship Interpreting Continuous Macro beta for each one, p value for each one overall r-squared Finally, Interaction analysis Center each variable Create interaction term by multiplying together Enter all three into METAREG If interaction term is sig, then interaction exists How to graph the interaction? (next week) FYI Our website has my excel file (Example-DataSet2) and the accompanying SPSS file (SPSS-ExampleDataSet2) You also have my quals paper, so you can use the SPSS file to practice and see if your data match the quals paper. HOWEVER, the data will only match the CATEGORICAL moderator analysis (not the continuous moderator analysis) for reasons I don’t have time to go into.