Transcript Sample size calculations

Sample size calculations
Marie-Pierre Sylvestre
[email protected]
Material adapted from
http://www.sgul.ac.uk/depts/phs/guide/size.htm
June 2007
Why bother?


Sample size calculations are important to ensure that
estimates are obtained with required precision or confidence.

E.g. a prevalence of 10% from a sample of size 20 ... 95%CI is
1% to 31%...

... a prevalence of 10% from a sample of size 400 ... 95%CI is 7%
to 13%
In studies concerned with detecting an effect

if an effect deemed to be clinically or biologically important exists,
then there is a high chance of it being detected, i.e. that the
analysis will be statistically significant.

If the sample is too small, then even if large differences are
observed, it will be impossible to show that these are due to
anything more than sampling variation.
Some terminology

Significance level


One-sided and two-sided tests of significance


Two-sided tests should be used unless there is a very good reason for doing
otherwise.
Power


Cut-off point for the p-value, below which the null hypothesis will be
rejected and it will be concluded that there is evidence of an effect. Typically
set at 5%.
Power is the probability that the null hypothesis will be correctly rejected
i.e. rejected when there is indeed a real difference or association. It can
also be thought of as "100 minus the percentage chance of missing a real
effect" - therefore the higher the power, the lower the chance of missing a
real effect. Power is typically set at 80% or 90% but not below 80%.
Effect size of clinical importance

This is the smallest difference between the group means or proportions (or
odds ratio/relative risk closest to unity) which would be considered to be
clinically or biologically important. The sample size should be set so that if
such a difference exists, then it is very likely that a statistically significant
result would be obtained.
Example (1)
Estimating a single proportion



Scenario: The prevalence of dysfunctional breathing amongst
asthma patients being treated in general practice is to be assessed
using a postal questionnaire survey (Thomas et al. 2001).
Required information:
 Primary outcome variable = presence/absence of dysfunctional
breathing
 'Best guess' of expected percentage (proportion) = 30% (0.30)
 Desired width of 95% confidence interval = 10% (i.e. +/- 5%,
or 25% to 35%)
Formula for sample size for estimation of a proportion is
n = 15.4 * p * (1-p)/W2
 where n = the required sample size
 p = the expected proportion - here 0.30
 W = width of confidence interval - here 0.10
Example (2)
Estimating a single proportion
 Here we have: n = 15.4 * 0.30 * (0.70)/ 0.102 = 324
 "A sample of 324 patients with asthma will be
required to obtain a 95% confidence interval of +/5% around a prevalence estimate of 30%. To allow for
an expected 70% response rate to the questionnaire,
a total of 480 questionnaires will be delivered."
 Note: The formula presented below is based on
'normal approximation methods', and, should not be
applied when estimating percentages which are close
to 0% or 100%. In these circumstances 'exact
methods' should be used.
Prevalence/Proportion
 http://www.cs.uiowa.edu/~rlenth/Po
wer/
 Then, test for 1 proportion
Cohort studies
 Epi Info: http://www.cdc.gov/epiinfo/
Unmatched case-controls
 http://stat.ubc.ca/~rollin/stats/ssize/
caco.html
 http://calculators.stat.ucla.edu/power
calc/binomial/case-control/index.php
Clinical Trials
 http://hedwig.mgh.harvard.edu/samp
le_size/quan_measur/assoc_quant.ht
ml
Simple Survival Analysis and
Regression
 PS:
http://biostat.mc.vanderbilt.edu/twiki
/bin/view/Main/PowerSampleSize
Which variables should be included
in the sample size calculation?
 The sample size calculation should
relate to the study's primary outcome
variable.
 If the study has secondary outcome
variables which are also considered
important (as is often the case), the
sample size should also be sufficient
for the analyses of these variables.
Allowing for response rates and
other losses to the sample
 The sample size calculation should relate to
the final, achieved sample.
 Therefore, the initial numbers approached
in the study may need to be increased in
accordance with the expected response
rate, loss to follow up, lack of compliance,
and any other predicted reasons for loss of
subjects.
 The link between the initial numbers
approached and the final achieved sample
size should be made explicit.
Consistency with study aims
and statistical analysis
 If the aim is to demonstrate that a new drug is superior
to an existing one then it is important that the sample
size is sufficient to detect a clinically important
difference between the two treatments.
 However, sometimes the aim is to demonstrate that
two drugs are equally effective. This type of trial is
called an equivalence trial or a 'negative' trial.
 The sample size required to demonstrate equivalence
will be larger than that required to demonstrate a
difference.
 The sample size calculation should also be consistent
with the study's proposed method of analysis, since
both the sample size and the analysis depend on the
design of the study.
Pitfalls to avoid (1)
 "The throughput of the clinic is around 50 patients a year,
of whom 10% may refuse to take part in the study.
Therefore over the 2 years of the study, the sample size
will be 90 patients. "
 Although most studies need to balance feasibility with
study power, the sample size should not be decided on the
number of available patients alone.
 Where the number of available patients is a known limiting
factor, sample size calculations should still be provided, to
indicate either


The power which the study will have to detect the desired
difference of clinical importance, or
The difference which will be detected when the desired power
is applied.
Pitfalls to avoid (2)
 "Sample sizes are not provided because there
is no prior information on which to base
them."
 Where prior information on standard
deviations is unavailable, sample size
calculations can be given in very general
terms, i.e. by giving the size of difference
that may be detected in terms of a number of
standard deviations.
Pitfalls to avoid (3)
 "A previous study in this area recruited 150 subjects and
found highly significant results (p=0.014), and therefore
a similar sample size should be sufficient here."
 Previous studies may have been 'lucky' to find significant
results, due to random sampling variation.
 Calculations of sample size specific to the present,
proposed study should be provided, including details of

power

significance level

primary outcome variable

effect size of clinical importance for this variable


standard deviation (if a continuous variable)
sample size in each group (if comparing groups)
References
 http://www.sgul.ac.uk/depts/phs/gui
de/size.htm