Transcript Statistics 2 - American Foundation for Suicide Prevention

Power Analysis in Grant Writing
Jill Harkavy-Friedman, Ph.D.
Is there a difference?
Observed
Yes
Expected
Yes
Expected
No
Observed
No
Type II Error
True Positive
β
Type I Error
α
True Negative
POWER IS KNOWLEDGE
Power: How likely are you to detect an effect?
Sample Size: n
How many people will you need?
Effect Size: σ (e.g., R2)
How much of a difference are you trying to detect?
Significance Level (Type I Error): α
How much of a risk are you willing to take of saying there is a difference when
there none?
Type II Error: β or (1- α)
How much risk there is of saying that there is no difference when there is a
difference.
Sample Size: n
Depends on Nature of the question & statistic approach



Group differences: m1-m2=0
Correlations: r=0
Regression: R2=0
Feasibility

Economic, staffing, recruitment
Power needed
Effect Size: σ
Amount of difference want to detect



Based on previous literature: average SD
Based on pilot data: SD
Based on size of difference
Small:
Medium:
Large:
d=.20
d=.50
d=.80
Statistical difference ≠ Clinical difference
Significance Level (Type I Error): α
Normal Curve and
Distribution of Sample
Means
10.0
Count
7.5
5.0
2.5
2.5%
11%
34%
34%
11%
2.5%
0.0
3
4
5
6
The larger the sample size the greater the
power
The larger the effect size the greater the
power
The larger the significance level the greater
the power
Power and Sample Size Table at
α 1=.05
ES
Power
.70
.75
.30
.40
.50
.60
108
123
62
70
41
46
29
33
.80
.85
.90
140
163
193
80
93
110
52
61
72
37
43
51
.95
.99
243
353
138
200
90
129
63
91
Data Analysis comes first power second
Determine your hypotheses
Determine your analyses
Determine the parameters for analysis by
hypothesis (i.e., power, ES, α)
Conduct power analysis
Power analysis will require:
Type of analysis
Sample size
Effect size
Significance level
Number of groups or factors
Plug in the numbers
Considerations
What is your question?
What type of data do you have?
What are your hypotheses?
What are your resources?
Clinical vs. statistical significance
How will you present your data?
What you would like the news headline to be?
SAGES Research Committee, August 2006
RE: Suggestions to Assist with Completing
a Winning Application
Power Analysis:

In order to minimize the reporting of false-negative data, a
power analysis should be performed for sample size
determination. Power is the capability of a study to detect
a difference if the difference really exists. A type II error
occurs when a true difference exists between study
populations but there are insufficient numbers of subjects
to detect this difference.
Any grant submitted without one of the items below
will not be eligible for review.
Power analysis. Please provide the following data: alpha and beta,
sample size needed in each group, what difference is expected.
(Example: "A power analysis was performed with a beta of .20 and
an alpha of 05. Assuming that a 10% difference exists between
patient and control groups, 150 subjects will be needed in each
arm. Thus the study would provide an 80% chance that a difference
would be detected if one exists.")
If a power analysis in not appropriate for the submitted project, a
statement should be included explaining why a power analysis is
not appropriate for the study.
Consultation with a statistician is recommended. However, there
are many statistical software programs available
What to do when you need
more power
Increase sample size
Reduce number of variables
Show your data graphically
Power Analysis
With n=400, alpha2 = .05, and a medium effect size (.30) the power will be >.99 for analyses of
variance and .96 for zero order correlations120. The power for a regression analysis that
includes 11 variables (i.e. sex, ethnicity, positive symptoms, negative symptoms, aggression,
impulsivity, depression, premorbid adjustment, gene marker, family history and substance
abuse) with n=400, alpha=.05 and a medium effect size (R2=.10) will be greater than .90. We do
not anticipate that all 11 variables will contribute significantly to the model. With 6 variables, a
more likely model, the power will be > .90. For the exploratory regression analyses conducted
within the group of attempters (n=200) the power will be at least .80 to detect a medium
effect size (R2=.10). For correlational studies among the genetic and biochemical measures
(approximate n=100) the power will be .80 for a medium effect size (R2=.10) and .95 for a
larger medium effect size (R2=.25). The increased sample size will now provide the power
necessary to consider attempters and nonattempters separately. Analyses examining single
attempters, multiple attempters and nonattempters (approximate sample sizes: 120, 80, 200)
will still maintain adequate power (power>.70).
Grant application by Jill M. Harkavy-Friedman, PhD
Free Power and Sample Size
Calculation
Cohen J. Statistical Power Analysis for the Behavioral Sciences (2nd
edition). Hillsdale, New Jersey: Lawrence Erlbaum Associates,
Publishers, 1988

http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleS
ize
G*Power 2

http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/
(limited)
G*3

http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/