Transcript Sample Size - clinepi.mcgill.ca

Sample Size Estimations
Demystifying Sample Size Calculations
Graphics contributed by
Dr. Gillian Bartlett
© Nancy E. Mayo 2004
Choosing the Study Population
Question
Background
Population
Reasonable
Question
© Nancy E. Mayo 2004
Study Population
Exposure
?
© Nancy E. Mayo 2004
1.
2.
3.
COMMON QUESTIONS
How many subjects (specimens) do I need?
How do I analyze my data?
What do I put in the data analysis section?
1.
2.
3.
4.
5.
COMMON ANSWERS
What is your question?
What is your outcome?
How is it measured?
How big an effect do you want to see?
Is the effect meaningful?
© Nancy E. Mayo 2004
Clinically Meaningful Change
Meaningful to whom?
• Clinician - usually impairments
• Patient – function (disability), quality of life
• Society - health services utilization, cost
• Payer – disability, prescription medication
© Nancy E. Mayo 2004
Clinically Meaningful Change
• Norm referenced
– refers to changes that would put someone within
normal values or within a % of normal
• Criterion referenced
– change anchored in future benefit
– change is associated with increased probability of
distant outcomes
– relevant when impact is on pathology but benefit
not reaped for years
© Nancy E. Mayo 2004
Clinically Meaningful Change
• Content referenced
– for outcomes measured by scales
– translates change into what would have had to
have changed on the scale
– e.g. 5 points on Barthel Index - changed 1 level
on 1 item.
• Minimally detectable change
– Subjects can detect improvement
© Nancy E. Mayo 2004
How BIG is BIG?
Effect size: ratio of change to variability
0.2 - 0.3 – small
0.5 – moderate
0.8 - large
© Nancy E. Mayo 2004
Change greater than “noise”
signal is difficult to detect against
excessive background noise
© Nancy E. Mayo 2004
Raw vs. Cooked Data (order rare)
Raw Data
Cooked Data (<50, >=50)
39
0
43
0
68
1
56
1
78
1
22
0
34
0
49
0
50
1
51
1
35
0
55
1
48
0
29
0
33
0
78
1
56
1
69
1
53
1
66
1
Mean 50.6
SD
15.8
Ratio 3.2
Mean 0.55
SD
0.51
Ratio 1.1
© Nancy E. Mayo 2004
Examples of the Pitfalls of Cooking Data
Raw Data
Cooked Data (<50, >=50)
22
0
29
0
33
0
34
0
35
0
39
0
43
0
48
0
49
0
50
1
51
1
53
1
55
1
56
1
56
1
66
1
68
1
69
1
78
1
78
1
Mean 50.6
SD
15.8
Ratio 3.2
Mean 0.55
SD
0.51
Ratio 1.1
© Nancy E. Mayo 2004
DEMYSTIFIED
Sample Size Formula = SD / delta
Effect size = delta / SD
Delta = difference
© Nancy E. Mayo 2004
Relationship between Effect Size
and Sample Size
Sample Size per Group
100
80
60
40
20
0
0
0.5
1
1.5
Effect Size
(Two group design)
© Nancy E. Mayo 2004
2
2.5
Calculation of Sample Size for Comparing
Two Independent Means
n= 2
( za – zb ) SD
___________
xexp - xcon
2
Where:
Za = z value for the risk of a Type I error (significance level) 1.96 for 0.05
Zb = z value for the risk of a Type II error (power)
1.96 for 0.95 (two-tailed)
-1.65 for 0.95 (one-tailed)
SD = standard deviation of outcome in the general population
xcon = mean of control group
xexp = mean of experimental group
n = number of subjects per group
© Nancy E. Mayo 2004
Calculation of Sample Size for Comparing
Two Independent Proportions
n=
za √ 2 pcon (1 - pcon ) – zb √ pexp (1 – pexp ) + pcon (1 - pcon )
______________________________________________
pexp - pcon
2
Where:
za = z value for the risk of a Type I error (significance level)
1.96 for 0.05
zb = z value for the risk of a Type II error (power)
1.96 for 0.95 (two-tailed)
-1.65 for 0.96 (one-tailed)
pcon = prevalence of outcome in control group
pcon = prevalence of outcome in experimental group
n = number of subjects per group
© Nancy E. Mayo 2004
Colton (pg 168-169)
Sample Size Required Per Group for
Comparing Two Independent Means
Ratio of SD to difference ∆
between means (∆/SD)
POWER
.80
.90
.95
0.50 (2.0)
5
7
8
1.0 (1.0)
17
23
27
1.25 (0.8)
26
34
42
1.50 (0.67)
37
49
60
2.0 (0.5)
60
86
105
© Nancy E. Mayo 2004
Sample Size Required Per Group for Comparing Two
Independent Proportions: 80% Power
PREVALENCE OF OUTCOME IN CONTROL GROUP
Prevalence of outcome in experimental group
.05
.10
.20
.30
.40
.10
475
.15
160
726
.20
88
219
.25
59
113
1134
.30
43
72
313
.35
51
151
1417
.40
38
91
376
.45
30
62
176
1574
.50
25
45
103
408
68
186
.55
.50
.60
107
408
.65
70
183
.70
49
103
.75
36
66
.80
28
45
.90
17
25
© Nancy E. Mayo 2004
Sample Size Required Per Group for Comparing Two Independent
Proportions: 95% Power
PREVALENCE OF OUTCOME IN CONTROL GROUP
Prevalence of outcome in experimental group
.05
.10
.20
.30
.40
.50
.10
758
.15
251
1174
.20
137
349
.25
90
177
`850
.30
65
111
505
.35
78
241
2318
.40
58
144
609
.45
45
97
281
2578
.50
36
70
163
661
.55
53
107
299
2630
.60
41
75
170
661
.65
33
56
109
293
.70
43
75
163
.75
34
55
103
41
70
25
36
.80
.90
© Nancy E. Mayo 2004
More complex data situations
• Convert each component to the simple 2group comparison or correlation
• Estimate (calculate) sample size for the
contrast that has the smallest effect and build
up
• Remember if using correlation as the base,
you are not testing it against 0 you are testing
it against a correlation that you do not think
is important
© Nancy E. Mayo 2004
…More
• Consider the impact on power to maintain a
given effect size if other variables are in the
model
© Nancy E. Mayo 2004
Regression
• Green indicates that adequate power (80%)
can be achieved for moderate effect sizes
with a sample size N > 50 + 8m, where m is
the number of covariates to be modeled.
© Nancy E. Mayo 2004
Adjustment only
• no parameters are estimated
• no hypotheses tested
• to maintain the same degree of power, only 1
additional subject is required per level (l) or
per degree of freedom (df) inherent in the covariate
© Nancy E. Mayo 2004
Summary
•
•
•
•
Variable under study
N > 50 + 8m (moderate effect size, 80% power)
Adjustment only
Continuous = + 1 per covariate df
Dichotomous = 5-9 events per covariate
• Sub-group analysis
• Sample size for main effect * 4 for interaction with
group
© Nancy E. Mayo 2004
Marking Scheme for Protocol
•
•
•
•
•
•
•
•
•
Background 10
Question 5
Population 5
Design 5
Procedures 5
Measures 10
Analysis 5
Sample Size 5
Bonus points – above and beyond the call of duty
© Nancy E. Mayo 2004