Transcript Research Curriculum Session III – Estimating Sample Size and Power Jim Quinn MD MS Research Director , Division of Emergency Medicine Stanford University.

Research Curriculum
Session III – Estimating Sample Size
and Power
Jim Quinn MD MS
Research Director , Division of Emergency Medicine
Stanford University
Overview
Funding Issues
- ACEP.org - 2004-2005 Research Grant Program
Overview
- Kaiser - Mid December
Sample Size Calculations
- Basic statistical testing
- Variables
- Assumptions
- Strategies for minimizing sample size
Estimating Sample Size
Clearly stated simple question
One predictor and one outcome measure
Ensure that our sample is representative
of the population we are basing our
hypothesis on.
Hypothesis Testing
-
-
-
Null Hypothesis
There is no difference between the predictor and
outcome variables in the population
Assuming there is no association, statistical tests
estimate the probability that the association is due to
chance
Alternate Hypothesis
The proposition that there is an association between the
predictor and outcome variable
We do not test this directly but accept it by default if the
statistical test rejects the null hypothesis
Hypothesis testing
Statistical Principles
Always use two sided tests
Level of statistical significance
Type I and II errors
Effect Size
Variability of the population/sample
Level of Significance
Set at 0.05 for alpha and 0.20 for beta
“If there is less than a 1/20 chance that
difference between two group is due to
chance alone we reject the Null
hypothesis and accept the Alternate
hypothesis that they are different”
For two sided tests that is 0.025 in each
tail
Type I and II Errors
Many types of errors, not just statistical
False negative and false positive can occur
because of errors due to bias
Type I (statistical false positive)- reject the null
hypothesis but in fact it is true. (or you think
there is a difference but there really isn’t one)
Type II (statistical false negative) – accept the
null hypothesis but in fact there is a difference
Type I and II Errors
Type I and II errors are
usually avoidable by
having adequate sample
size or manipulating the
design of the study and
measure of outcomes.
0.05 and 0.20 are
arbitrary and many
believe beta should be
0.10
Effect Size
“What is a meaningful difference between the groups”
It is truly an estimate and often the most challenging
aspect of sample size planning
Large difference – small sample size
Small differences – large sample size
- Find data from other studies
- Survey people
- Cost/benefit
Variability
The greater the variability in the outcome
measure the more likely the groups will overlap
Less precise measures and measurement error
increase the variability
Variability is decreased by increasing the sample
size
For sample size calculations of continuous
variables the variability needs to be estimated
- Can get from other studies or small pilot study
Sample Size Calculation
Comparative Studies
State the Null Hypothesis
Determine appropriate statistical test
(For simplicity use T-test for continuous of
chi square for dichotomous)
Predict effect size and variability
Set α and ß
Use the appropriate formula or table
Sample Size Calculation for
Descriptive Studies
-
-
Continuous
Estimate std deviation
Specify precision (width of CI)
Select the confidence level for the interval
Dichotomous
Estimate the expected proportion of the variable
of interest (if > 50% calculate based on
proportion not expected to have the
characteristic)
Select the CI width
Select the confidence for the interval
Other Considerations
Account for dropouts
Ordinal variables especially if 5-6 groups
can be treated as continuous
Survival analysis
Matching
Equivalence studies
Strategies for Minimizing
Sample Size
Use continuous variables
Paired measurements (consider
measuring the change)
Use more precise variables
Use unequal group sizes
N = [(c+1)/2c] x n (c = controls per cases)
Use more common outcome
Errors to Avoid
Dichotomous outcomes can appear continuous when
expressed as a percentage
Sample size is for those who complete the study not
those enrolled
Tables assume equal numbers in both groups (if in doubt
use formulae)
For continuous variables use the standard deviation best
associated with the outcome
Do the calculation before you start your study and use it
to plan
Cluster data is confusing and needs a statistical
consultation
Questions and Answers