Transcript Statistics for the Terrified  University of Colorado Denver
Statistics for the Terrified
Paul F. Cook, PhD Center for Nursing Research
What Good are Statistics?
• “How big?” (“how much?”, “how many?”) – Descriptive statistics, including effect sizes – Describe a population based on a sample – Help you make predictions • “How likely?” – Inferential statistics – Tell you whether a finding is reliable, or probably just due to chance (sampling error)
Answering the 2 Questions
• Inferential statistics tell you “how likely” – Can’t tell you how big – Can’t tell you how important – “Success” is based on a double negative • Descriptive statistics tell you “how big” – Cohen’s d =
x 1 – x 2 SD pooled
– Pearson r (or other correlation coefficient) – Odds ratio
How Big is “Big”?
• Correlations – 0 = no relationship, 1 = upper limit – + = + effect,  =  effect – .3 for small, .5 for medium, .7 for large – r
2
= “percent of variability accounted for” • Cohen’s d – Means are how many SDs apart?: 0 = no effect – .5 for small, .75 for medium, 1.0 for large • Odds Ratio – 1 = no relationship, <1 =  effect, >1 = + effect • All effect size statistics are interchangeable!
How Likely is “Likely”?  Test Statistics
A ratio of “signal” vs. “noise” z
=
x 1 – x 2 s 1 2 /n 1 + s 2 2 /n 2
X 1 X 2 “signal” : AKA, “betweengroups variability” or “model” “noise” : AKA, “withingroups variability” or “error”
How do We Get the pvalue?
Chebyshev’s Theorem:
z > + 1.96
is the critical value for
p < .05
(
half above, half below: always use a 2tailed test unless you have reason not to
) 2.5% 2.5%
1.96 SD 1.96
1.96 SD 1.96
Hypothesis Testing – 5 Steps
1.
2.
State null and alternative hypotheses Calculate a test statistic 3.
4.
Find the corresponding pvalue “Reject” or “fail to reject” the null hypothesis
(your only 2 choices)
5. Draw substantive conclusions Red = statistics , blue = logic , black = theory
How Are the Questions Related?
• “Large” z = a large effect (d) and a low p • But z depends on sample size; d does not – Every test statistic is the product of an effect size and the sample size – Example: f = c
2
/ N • A significant result (power) depends on: – What alpha level ( a ) you choose – How large an effect (d) there is to find – What sample size (n) is available
What Type of Test?
• Nlevel predictor (2 groups): ttest or ztest • Nlevel predictor (3+ groups): ANOVA (Ftest) • I/Rlevel predictor: correlation/regression • Nlevel dependent variable: c
2
or logistic reg.
Correlation answers the “how big” question, but can convert to a ttest value to also answer the “how likely” question
The F test
• ANOVA = “analysis of variance” • Compares variability between groups to variability within groups
F
= MSEb MSEw = avg. difference among means avg. variability within each group • Signal vs. noise
Omnibus and Post Hoc Tests
• The Ftest compares 3+ groups at once • Benefit: avoids “capitalizing on chance” • Drawback: can’t see individual differences • Solution: post hoc tests – Bonferroni correction for 13 comparisons (uses an “adjusted alpha” of .025 or .01) – Tukey test for 4+ comparisons
F and Correlation (etasquared)
The FTable: SS Between Within Total (= SS b + SS w ) df Between Within Total (= df b + df w ) MS SS b SS w / df b / df w F MS b / MS w
p
.05
eta 2
= SS b SS total = % of total variability that is due to the IV (i.e., Rsquared)
Correlation Seen on a Graph
Same Direction, Moderate Correlation Same Direction, Strong Correlation
Regression and the Ftest
The line of best fit
(minimizes sum of squared residuals)
Actual value Error variance (residual) Predicted value Model variance (predicted)
F
= Avg. SS model variance Avg. SS error variance
Parametric Test Assumptions
• Tests have restrictive assumptions: – Normality – Independence – Homogeneity of variance – Linear relationship between IV and DV • If assumptions are violated, use a nonparametric alternative test: – MannWhitney U instead of t – KruskalWallis H instead of F – Chisquare for categorical data
ChiSquare
• The basic nonparametric test • Also used in logistic regression, SEM • Compares observed values (model) to observed minus predicted values (error) c
2
= S (
F o

F e
) 2
F e
• Signal vs. noise again • Easily converts to phi coefficient: f = √ c
2
/ N
2by2 Contingency Tables
Dependent Observations
• Independence is a major assumption of parametric tests (but not nonparametrics) • Address nonindependence by collapsing scores to a single observation per participant: – Change score = posttest score – pretest score – Can calculate SD (variability) of change scores • Determine if the average change is significantly different from zero (i.e., “no change”): – t = (average change – zero) / (SDchange / √ n ) – Nonparametric version: Wilcoxon signedrank test
ANCOVA / Multiple Regression
• Statistical “control” for confounding variables – no competing explanations • Method adds a “covariate” to the model: – That variable’s effects are “partialed out” – Remaining effect is “independent” of confound • One important application: ANCOVA – Test for posttest differences between groups – Control for pretest differences • Multiple regression: Same idea, I/Rlevel DV – Stepwise regression finds “best” predictors
This is the amount of variability in the DV that can be accounted for by its association with IV1 “Unique Variability” for IV1 Independent Variable #1 This circle represents “Shared Variability” for IV1 & IV2
dependent variable
Independent Variable #2 “Unique Variability” for IV2 This is the amount of variability in When two IVs account for the same variability in the DV (i.e., when there is shared variability), they are “multicollinear” with each other.
This graph can also be used to show the percentage of the variability in the DV that can be accounted for by each IV.
IV1 If … is 30% of … DV … then the Total R 2 is .30
“Total” R 2 for IV1 & IV2
together
= IV2 for by an IV is the definition of R 2 —the coefficient of determination.
A related concept is the idea of a “semipartial R 2 , which Semipartial R 2 for IV1 = for by each IV
on its own
,
not
counting any shared variability.
Total SS for the DV IV1 If … is 20% of … … then the semipartial R 2 for IV1 is .20
DV IV2 The semipartial R 2 is the percentage of variability in the DV that can be accounted for by one
individual
predictor,
independent of
the effects of all of the other predictors.
Lying with Statistics
• Does the sample reflect your population?
• Is the IV clinically reasonable?
• Were the right set of controls in place?
• Is the DV the right way to measure outcome?
• Significant pvalue = probably replicable – APA task force: always report exact pvalue • Large effect size = potentially important – APA, CONSORT guideline: always report effect size • Clinical significance still needs evaluation
What We Cover in Quant II
• Basic issues – Missing data, data screening and cleaning – Metaanalysis – Factorial ANOVA and interaction effects • Multivariate analyses – MANOVA – RepeatedMeasures ANOVA • Survival analysis • Classification – Logistic regression – Discriminant Function Analysis • Data simplification and modeling – Factor Analysis – Structural Equation Modeling • Intensive longitudinal data (hierarchical linear models) • Exploratory data analysis (cluster analysis, CART)