Transcript www.rhoworld.com

Stratified Randomization in
Clinical Trials
Katherine L. Monti, Ph.D.
Senior Statistical Scientist and
Director of the Massachusetts Office, Rho, Inc.
Adjunct Associate Professor, Biostatistics
University of North Carolina
Outline
•
•
•
Introduction
Motivation for this work:
“Stratification can’t hurt”
Literature search
– Why stratify? Advantages
– Why not stratify? Disadvantages
– Considerations if stratification is going to take place
– Alternatives
– Limitations of the literature
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
2
Outline
•
Explore the notion that “stratification can’t
hurt”
– Description of the simulation
– Results
•
Conclusions
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
3
Introduction
•
•
Stratification is generally undertaken so that
treatment comparisons can be made within
relatively homogenous groups of
experimental units.
Stratification in clinical trials is different from
classical stratification in survey sampling, or
from blocking in experimental design.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
4
Introduction
•
•
In survey sampling, the population is divided into
subgroups (strata). There is a defined sampling
frame. Each stratum is randomly sampled with a
known sample size.
In experimental design, treatments are assigned
within blocks, which are defined by factors that are
generally determinable and often controllable (e.g.,
temp, water level in a greenhouse setting). Again,
the sample size in each block is part of the design.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
5
Introduction
In clinical trials…
• The sample size for each factor level is often
unknown until the end of the study.
• Exception: when sampling is halted differentially by strata
to force balanced strata.
•
The “blocking” factors are generally not
controllable (e.g., stage of disease,
concomitant medication usage).
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
6
Introduction
•
•
Sometimes stratification is beneficial in
clinical trials.
Some trialists maintain that it is never
harmful.
– Is that the case?
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
7
Motivation
•
A drug company’s design:
– 120 subjects
–
4 treatments (placebo, three drug doses)
– 30 sites
–
1 prognostic factor with 2 levels (hi and low levels,
continuous covariate)
•
Randomization:
– At each site, NOT centralized
– In blocks of 4 within factor level within site
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
8
Motivation
•
120 subjects / (30 sites) =
4 subjects per site
With 4 treatments, perfect balance overall would occur without
stratification.
•
120 subjects / (30 sites x 2 levels) =
2 subjects per site for each level
With 4 treatments, balance is not assured if randomization
occurs within level.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
9
Motivation
•
•
Do we really expect 4 subjects to enroll per
site?
No
There will be some imbalance among the
treatments even if randomization is
performed just within site, without regard to
level.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
10
Motivation
•
Those designing the study thought that
randomizing within factor level
– would increase balance in the design
– “couldn’t hurt”
•
Others argued that randomizing within factor
level would increase the overall imbalance in
the design.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
11
Literature
• What does the literature have to say
about stratification in clinical trials?
– When is stratification beneficial?
– When is stratification harmful?
– Does the literature suggest that
stratification “couldn’t hurt”?
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
12
Why stratify? Advantages
•
•
Keep variability of subjects within strata as
small as possible and between-strata
variability as large as possible in order to
have the most precision of the treatment
effect (Chow and Liu, 1998)
Avoid imbalance in the distribution of
treatment groups within strata
– Efficiency, credibility
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
13
Why stratify? Advantages
•
•
•
•
Protect against Type I and Type II errors
Avoid confounding
Satisfy prevailing investigator preconceptions
about study design
Provide credibility to choice of analysis
covariates
– Stratification variables are definitely specified a priori.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
14
Why not stratify? Disadvantages
•
Gains (power/efficiency) that can occur with
stratification is often small, particularly once
(# subjects) / (# treatments) > 50
•
More costly
•
More complicated trial
– Greater opportunity to introduce randomization
error
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
15
One Alternative:
Adaptive allocation
•
Dynamic allocation / adaptive allocation
-
Minimization by Taves
Pocock and Simon’s method
Zelen’s method
Begg and Iglewicz
Others
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
16
Minimization
•
•
Keep track of the current imbalance and
assign the treatment to a new subject to
reduce the existing imbalance between strata
Advantages:
- Produces less imbalance than simple permuted
blocks
- Can accommodate more factors
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
17
Minimization
•
Disadvantages:
- Need to keep track of current imbalance (central
randomization)
- None of the assignments are completely random
- Since it only aims to balance marginal totals of
multiple factors, precision is only increased if the
interaction between prognostic factors is not
pronounced. (Tu et. al., 2000)
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
18
Another Alternative:
Post-stratification
•
If stratification is not done at randomization,
covariate analysis can be performed.
- Easier and less costly to implement
- Often nearly as efficient
- May be less convincing, particularly if covariate
was not mentioned in the protocol
- Cannot correct for cases of extreme imbalance or
confounding of covariates
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
19
If you want to stratify …
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
20
Consider
•
How well is the stratification variable measured?
– If the covariates used to stratify are imprecisely assessed,
then may introduce error.
•
Is the stratification variable related to outcome?
– If not, the gain in efficiency may be small or negative.
•
How many strata will there be?
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
21
Number of Strata
The number of strata to allow depends on:
-
Total number of subjects in the trial
Expected number to be in each stratum
Predictive capability of prognostic factors
Type of allocation scheme (permuted blocks
vs. dynamic allocation)
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
22
Number of Strata
•
The number of strata should be less than
(total sample size) / (block size). (Hallstrom and Davis, 1988)
– In our case, N=120, B=4,
•
• Recommendation: < 30 strata
• Design: 60 strata
Stratification begins to fail (in terms of balance) if the
total number of strata is greater than approximately
N/2 (for 2 treatments). (Therneau, 1993)
– or N/k, k= number of treatments
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
23
Number of Strata
•
“One can inadvertently counteract the
balancing effects of blocking by having too
many strata.”
“…, most blocks should be
filled because unfilled blocks permit
imbalances.” (Piantadosi,1997)
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
24
Number of Strata
•
“If ‘institution effect’ were to be introduced as
a further prognostic factor, …, the total
number of strata may then be in the hundreds
and one would have achieved little more than
purely random treatment assignment.” (Pocock
and Simon, 1975)
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
25
Number of Strata
•
•
And thus we see that there are some
warnings in the literature about employing too
many strata.
However….
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
26
Conclusions of the
Literature Search
Authors are still concluding that
“Stratification is … harmless always,
useful frequently, and important rarely”.
(Kernan et al., 1999)
(Caveat: Elsewhere in the article, Kernan et al recommend against
overstratification, but this is the topic sentence of their discussion
section.)
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
27
Limitations of the Literature
•
•
Literature refers mostly to trials of two
treatments.
In the statistical literature, little attention is
paid to operational disadvantages of more
complex designs.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
28
Conclusions of the
Literature Search
Consider stratifying only if:
•
•
•
Prognostic factors are known to be related to the
outcome and are easy to collect prior to
randomization.
Operational costs justify any gain.
Sample size is small ( N < 100), but the stratified
design does not induce imbalance.
-
The number of strata should be less than
(total sample size) / (block size). (Hallstrom and Davis, 1988)
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
29
“Stratification can’t hurt.”
The notion that stratification “couldn’t
hurt”
–remains in current literature
–is being advanced by some trialists
This conclusion should be reconsidered.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
30
“Stratification can’t hurt.”
•
The remainder of the talk will
– Review the motivating example
– Describe a simulation to explore the notion that
“stratification can’t hurt”
– Summarize the results
– Provide conclusions
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
31
Motivation
•
A drug company’s design:
– 120 subjects
–
4 treatments (placebo, three drug doses)
– 30 sites
–
•
1 prognostic factor with 2 levels (hi and low levels,
continuous covariate)
Randomization:
– At each site, NOT centralized
– In blocks of 4 within factor level within site
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
32
Operational Difficulties
Randomization was actually done within strata within
site
•
•
•
•
•
Drug supply requirements increased. (~ 33%)
Packaging/shipping costs increased.
Additional training visits to sites were needed in order
to explain the more complex randomization scheme.
The project management burden increased
considerably.
The misassignment of subjects to treatment was
more likely.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
33
Imbalance
•
•
120 subjects/ (30 sites) =
4 subjects per site
Perfect balance with 4 treatments
However, 4 subjects enrolling at each site is
not really expected!
Perfect balance overall is not expected.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
34
Imbalance
•
•
Additional restriction on randomization to
within level of the prognostic factor within site
could only increase the imbalance.
How much worse does it get?
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
35
Simulation
•
I set out to compare the magnitude of the
treatment imbalance if randomization were
performed in permuted blocks of 4:
– within site (WS) (30 strata)
– within level of the factor within site (WLWS)
(60 strata)
•
I used simulation.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
36
Simulation
• The general approach was this:
– I don’t now how many subjects would enroll in each
site, so I used a variety of guesses regarding the
enrollment pattern.
– I don’t know how many subjects in each strata are
going to enroll, but I assumed an underlying 50:50 ratio
and forced 50:50 enrollment in some cases.
– I don’t know in what order subjects will enter the trial,
but I assumed that subjects enter randomly with
respect to their strata status in each site.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
37
Simulation
– Once I “identified” the number of subjects in each
strata at each site and the order in which they enrolled,
I randomized them to treatment twice: once randomly
WS and once randomly WLWS.
– Did that 10,000 times for each enrollment pattern.
– Compared the balance of WS to WLWS randomization.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
38
Enrollment Patterns
•
To assign subjects to treatments, we need to
know the the number of subjects ( Nij )
– in site i (i=1-30) who have
– factor level j (j=1,2)
The Nij are unknown….
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
39
Enrollment Patterns
…So we make assumptions
•
However, instead of prescribing the exact Nij
for each i and j in the simulation, I defined 9
different enrollment patterns for the 60
site/level combinations.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
40
Enrollment Patterns
•
Each enrollment pattern assumed a distribution of the
number of subjects in the 60 site/levels, so that there
were:
– 30 sites, 2 levels per site
– 0-8 subjects in each of the 60 site/level strata
– 120 subjects
•
Some enrollment patterns forced balance between
the factor levels:
N.1 = N.2 = 120/2 = 60 subjects per stratum
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
41
Entries are the number of strata having the indicated number of subjects.
No.
Subjects
Enrollment Plan
1
2,3*
4,5*
6,7*
8
9
6
10
5
30
20
14
12
11
20
16
16
15
20
14
12
8
6
10
6
0
1
2
3
4
60
5
2
6
1
7
1
8
1
30
* Plans forced a balance between strata across sites (60 subjects per strata overall).
An Enrollment Pattern
•
EX: Pattern 2:
– 20 site/levels strata w/ 3 subjects
– 20 site/levels strata w/ 2 subjects
– 20 site/levels strata w/ 1 subject
Total of 60 site/level strata w/ 60+40+20 = 120 subjects
Here, Nij = 1, 2 or 3
•
Given that pattern
– The 60 strata were randomly paired to construct 30 sites.
– Ni1 / Ni2 for site i could be any of the following:
3/3, 3/2, 3/1, 2/3, 2/2, 2/1, 1/3, 1/2, or 1/1
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
43
An Enrollment Pattern
•
•
In Pattern 2, it is unlikely that the factor levels
will be exactly balanced overall.
EX: Pattern 3 forces a 60/60 split
– 20 site/levels strata w/ 3 subjects (10/10 split)
– 20 site/levels strata w/ 2 subjects (10/10 split)
– 20 site/levels strata w/ 1 subject (10/10 split)
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
44
Enrollment Patterns
•
To compare the balance in treatment
assignment when randomizing WS and WLWS,
I assigned subjects to treatments
– at each site
and then reassigned
– in each level at each site
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
45
Simulation
•
Used SAS PROC PLAN to generate
treatment assignments in permuted blocks
of 4
– for 30 sites and
– for 30 sites with 2 factor levels per site
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
46
Evaluation Criteria
•
For each enrollment pattern, we want to be
able to compare the treatment balance
when randomization is performed WS and
WLWS using permuted blocks of 4
treatments.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
47
Evaluation Criteria
•
There are two types of treatment balance:
– Overall balance
The extent to which the treatment assignments are
balanced overall.
– Within-level balance
The extent to which the treatment assignments are
balanced within each of the 2 factor levels.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
48
Evaluation Criteria
•
4 criteria used to assess the randomization
results of each simulated run are reported
here:
– N[1] = smallest N of the 4 treatments
– N[1] + N[2] = smallest total sample size for any
comparison of treatments
– % loss of power compared to a completely balanced
design for the comparison based on N[1] + N[2] for a
study designed for 90% power
– N[4] – N[1] = maximum difference in sample sizes
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
49
Evaluation Criteria
•
Overall, treatment balance would be
achieved if
– N1 = N2 = N3 = N4 = 120/4 = 30
– N[1] = 30
– N[1] + N[2] = 60
– No loss of power relative to complete balance
– N[4] – N[1] = 0
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
50
Simulation Scheme
•
Do for enrollment pattern (EP) = 1 to 9
– Do 10,000 replications:
• Generate 60 strata of subjects (per EP).
– Assign random numbers to each subject
(to determine order of enrollment at the site
and the order of enrollment in the level at the
site).
– Assign a random number to each stratum
(to identify the levels, 1 vs 2).
• Randomly pair the 60 strata into 30 sites.
(The stratum with the lower random number is level 1.)
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
51
Simulation Scheme
• Randomly assign treatments WS based on the
•
order of enrollment in the site, then
-determine the sample size of each treatment
-compute the evaluation criteria
• Randomly assign treatments WLWS based on the
order of enrollment in the site/level, then
-determine the sample size of each treatment
-compute the evaluation criteria
– Retain all the evaluation criteria, go to next
iteration
Go to next enrollment pattern
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
52
Results Reported
•
For each EP there are 10,000 values of the
evaluation criterion
– when randomizing WS
– when randomizing WLWS
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
53
Entries are the number of strata having the indicated number of subjects.
No.
Subjects
Enrollment Plan
1
2,3*
4,5*
6,7*
8
9
6
10
5
30
20
14
12
11
20
16
16
15
20
14
12
8
6
10
6
0
1
2
3
4
60
5
2
6
1
7
1
8
1
30
* Plans forced a balance between strata across sites (60 subjects per strata overall).
Overall Balance, Statistic = N[1]
30
Resul t
25
20
15
10
1-1
1-2
1-3
1-4
1-5
1-6
1-7 1-8 1-9
2-1 2-2
Randomization Type_Enrollment Plan
Type 1 = Randomized Within S ite, Overall Results
2-3
2-4
2-5
2-6
2-7
2-8
2-9
Type 2 = Randomized Within S trata, Overall Results
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
55
Overall Balance, Statistic = N[1] + N[2]
60
Resul t
55
50
45
40
1-1
1-2
1-3
1-4
1-5
1-6
1-7 1-8 1-9
2-1 2-2
Randomization Type_Enrollment Plan
Type 1 = Randomized Within S ite, Overall Results
2-3
2-4
2-5
2-6
2-7
2-8
2-9
Type 2 = Randomized Within S trata, Overall Results
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
56
Overall Balance, Statistic = % Change in Power
0
Resul t
-5
-10
-15
-20
1-1
1-2
1-3
1-4
1-5
1-6
1-7 1-8 1-9
2-1 2-2
Randomization Type_Enrollment Plan
Type 1 = Randomized Within S ite, Overall Results
2-3
2-4
2-5
2-6
2-7
2-8
2-9
Type 2 = Randomized Within S trata, Overall Results
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
57
Overall Balance, Statistic = Max Difference: N[4] - N[1]
30
25
Resul t
20
15
10
5
0
1-1
1-2
1-3
1-4
1-5
1-6
1-7 1-8 1-9
2-1 2-2
Randomization Type_Enrollment Plan
Type 1 = Randomized Within S ite, Overall Results
2-3
2-4
2-5
2-6
2-7
2-8
2-9
Type 2 = Randomized Within S trata, Overall Results
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
58
Within Strata Balance, Statistic = N[1] + N[2]
*
30
Resul t
25
20
15
10
3-1
3-2
3-3
3-4
3-5
3-6
3-7 3-8 3-9
4-1 4-2
Randomization Type_Enrollment Plan
4-3
4-4
4-5
4-6
4-7
4-8
4-9
Type 3 = Randomized Within S ite, S trata Results Type 4 = Randomized Within S trata, S trata Results
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
* For EP 9, there are 4 subjects for each stratum, but the strata are not balanced, so it is not necessarily the case that each
treatment has 15 observations. Therefore, the sum N{1}+N[2} can be less than 30.
59
Within Strata Balance, Statistic = Max Difference: N[4] - N[1]
20
Resul t
15
10
5
0
3-1
3-2
3-3
3-4
3-5
3-6
3-7 3-8 3-9
4-1 4-2
Randomization Type_Enrollment Plan
4-3
4-4
4-5
4-6
4-7
4-8
4-9
Type 3 = Randomized Within S ite, S trata Results Type 4 = Randomized Within S trata, S trata Results
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
60
Conclusions
•
With a relatively large number of treatments:
– Randomizing within numerous small sites can lead to some
treatment imbalance and loss of power.
– Randomization within levels of a prognostic factor in those
small sites will generally
•
•
•
•
•
Increase the treatment imbalance overall
Increase the loss of power in overall pairwise comparisons
Do little to reduce the treatment imbalance within the levels of
the prognostic factor
Increase cost of conducting the trial
Increase the complexity of the trial and the chance of errors in
randomization
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
61
Conclusions
•
Is it the case that stratification by prognostic factor
“can’t hurt”?
– NO: In some cases, stratification can hurt
• Statistically (power)
• Operationally (money)
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
62
Contact Information
[email protected]
Slides: www.rhoworld.com
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
63
Acknowledgements
Gretchen Marcucci, M.S.
Stephen Gilbert, Ph.D.
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
64
References
•
•
•
•
•
•
•
•
•
•
Begg CB, Iglewicz B. A treatment allocation procedure for sequential clinical trials.
Biometrics 36: 81-90, 1980
Chow SC, Liu JP. Design and Analysis of Clinical Trials. John Wiley and Sons; 1998.
Hallstrom A, Davis K. Imbalance in treatment assignments in stratified block randomization.
Control Clin Trials 1988; 9:375-382.
Kernan WN, Viscoli CM, Makuch RW, et al. Stratified randomization for clinical trials. J Clin
Epidemiolol 1999; 52: 19-26.
Piantadosi, S. Clinical Trials. A methodologic perspective. John Wiley and Sons; 1997.
Pocock SJ, Simon R. Sequential treatment assignment with balancing for prognostic factors
in the controlled clinical trial. Biometrics 1975; 31:103-115.
Taves DR. Minimization: A new method in assigning patient to treatment and control group.
Clinical Pharmacology and Therapeutics 15: 443-453, 1974.
Therneau TM. How many stratification factors is "too many" to use in a randomization plan?
Control Clinical Trial 14: 98-108, 1993.
Tu D, Shalay K, Pater J Adjustment of treatment effect for covariates in clinical trials:
Statistical and Regulatory Issues Drug Info Journal 34:511-523, 2000.
Zelen M. The randomization and stratification of patients to clinical trials. Journal of Chronic
Dis, 27:365-375, 1974 .
2004 Rho, Inc. All rights reserved. No part of this document may be copied without express written consent.
65