Transcript Randomized Clinical Trial

Outline of
Randomization Lectures
1. Background and definitions
2. Generation of schedules
3. Implementation (to ensure allocation
concealment, sometimes called blinded
randomization)
4. Theory behind randomization
Randomization Schedule
A list showing the order in which
subjects are to be assigned to the
various treatment groups
Implementation Schemes
1. Sealed envelopes
- Opaque
- Sequentially numbered
2. Telephone
- Answering service
- Coordinating center
- IVRS
3. Personal computers
- Local
- Through communication with coordinating center
4. International coordinating centers in HIV treatment trials
use web-based system
5. Through electronic medical record for “point-of-care” or
“clinically integrated” randomized trials.
Urokinase-Pulmonary Embolism Trial
(UPET)
Circulation, 1973
1. Telephone answering service in New York City;
24-hour coverage
2. Assignments obtained through hospital
pharmacy
3. Sealed envelopes as back-up
Multiple Risk Factor Intervention Trial
(MRFIT)
JAMA, 1982
1. Assignments obtained by calling coordinating
center after:
a. Three screening visits
b. Informed consent
c. Eligibility checklist
2. Sealed envelopes used as back-up
Treatment of Mild
Hypertension Study (TOHMS)
1. Assignment (bottle no.) obtained using personal computer
to call coordinating center computer after:
a. Three screening visits
b. Informed consent
c. Eligibility checklist
2. Call coordinating center for back-up
3. Unique bottle no. for each participant
4. Bottle no. not assigned in sequence
Amer J Cardiol, 1987
Community Programs for Clinical
Research on AIDS (CPCRA)
1. Assignments obtained by calling Statistical Center:
– Minimal data collection
– Usually no data at Statistical Center prior to
randomization
– Eligibility checklist reviewed on telephone call
2. Pharmacist telephones to confirm assignment
3. Unique study ID number (SID) for each patient
4. SID numbers not assigned in sequence
Components of CPCRA Randomization
System
1. Randomization schedule, based on randomly permuted
blocks
2. SID numbers, sheets, and notebooks
3. Randomization logbooks
4. Eligibility checking program
5. Pharmacy checking program
6. Backup procedures
7. Training (local and for clinical sites)
Controlled Onset Verapamil Investigation
of Cardiovascular Endpoints (CONVINCE)
• Interactive Voice Response System (IVRS)
– Touch-tone keypad used for data entry of key eligibility
data
– System verifies eligibility and assigns medication code
(bottle number)
– Caller re-enters medication code as a double-check
– System also used for medication refills
IVIG Trial Randomization
Procedure summarizes data entered
& asks you to re-enter weight
If randomization is successful,
3 documents are available
to save and print
Treatment
Prescription
Double-check
dose against
your calculation
on the Baseline
CRF, and
complete
bottom portion
Timing of Randomization
Usual Sequence of Events
1. Verify eligibility, informed consent, and
completeness of baseline data.
2. Obtain assignment.
3. Record assignment on log and case report forms.
4. Initiate treatment as soon as possible after
randomization.
Alprenolol vs. Placebo in Post-MI
No. randomized
Alprenolol
193
Placebo
200
2 weeks
No. given treatment
Excluded:
Disease history
Rx contraindication
Dead
Other
69
93
124
84
11
17
12
107
74
10
18
5
Ahlmark, Eur J Pharmac, Vol. 10, 1976
Induction and Maintenance Treatment
for Non-Hodgkin’s Lymphoma
Non-Hodgkin’s Lymphoma Trial
Cytoxan-Prednisone
Response
BCVP
No
Response
Chlorambucil
BCNU-Prednisone
Response
BCVP
See Pocock, Clinical Trials: A Practical Approach, Page 72.
No
Response
Chlorambucil
Adjuvant Chemotherapy for Breast
Cancer
(A)
(B)
1 year of
chemotherapy
OR
1 year
of
chemotherapy
2 years
of
chemotherapy
Stop
Rivkin N, et al. J Clin Oncology, 11:1710-1716;1993.
Continue
1 more
year
Recommendations
• Make assignments close to the onset of
treatment from a central source after checking
eligibility
• Implement the randomization with a method that
ensures allocation concealment
• Never deviate from the schedule
• Verify assignments
Examples of Problems with
Allocations Concealment
• Hypertension Detection and Follow-up
Program (HDFP) – a single site (envelopes
that were opened in advance)
• Heparin for acute MI (N Engl J Med 1960) –
(envelopes not opaque or consecutively
numbered)
• Captopril for hypertension (Lancet 1999)
(large baseline differences indicating
envelopes opened in advance)
Documentation and Reporting of
Randomization Methods
• Document methods for generating schedules,
but do not share details with the investigators
• Describe allocation ratio and stratification
variables in the protocol
• Report how randomization was done in the trial
report
Example: Strategies for Management of
AntiRetroviral Therapy (SMART) Study
• Protocol:
“Eligible patients will be randomized in a 1:1 ratio to
either the DC or VS group. Randomization will be
stratified by clinical site. Randomization schedules will
be constructed to ensure that approximately equal
numbers of patients are assigned each treatment within
clinical site.”
• Trial Report (N Engl J Med 2006; 355:2283-96):
“Randomization was stratified by clinical site with the
use of permuted blocks of random sizes.”
Reporting Example That Includes Method
of Implementation: HIV Trial in South
Africa (Phidisa II)
• Trial Report (JID 2010; 202:1529-1537):
“Randomization was stratified by site, using
randomly mixed permuted blocks of different
sizes. Assignments were obtained by calling a
central toll-free number”
Outline of
Randomization Lectures
1. Background and definitions
2. Generation of schedules
3. Implementation (to ensure allocation
concealment, sometimes called blinded
randomization)
4. Theory behind randomization
Advantages of Randomization
Bradford Hill:
1. Eliminates bias from treatment assignment
2. Balances known and unknown differences
between groups on average
3. More credible study
RA Fisher:
1. Assures validity of statistical tests (type 1
error)
Fisher and the Validity of Statistical
Tests (1)
• Randomization guarantees that statistical
tests will have the valid significance levels.
• Even though groups may not be exactly
balanced with respect to covariates,
randomization permits a probability
distribution to be determined for comparing
treatments for outcomes of interest
Fisher and the Validity of Statistical
Tests (2)
• Randomization provides a basis for an
assumption free statistical test of the
equality of treatments – need to analyze
your data taking into account the way the
randomization schedule was prepared.
• Such tests are referred to as randomization
tests or permutation tests
Test of Significance
at the End of a Trial
Statistically Significant?
Yes
No
Reject
null hypothesis (HO)
Do not reject
HO
Sampling variation
is an unlikely
explanation for the
discrepancy
Sampling variation
is a likely
explanation for the
discrepancy
Relationship of Study Sample to
Study Population and Population at
Large
Population at Large
Definition of
Condition
Population without
Condition
Population with Condition
Entry Criteria
With Condition
but Ineligible
Study Population
Eligible but
not Enrolled
Enrollment
Study Sample
Source: Chapter 4, Friedman, Furberg and DeMets.
Population Model as a
Basis for Statistical Testing
Population A
y ~ G(y | A)
Population B
y ~ G(y | B)
Random Sample
Random Sample
nA patients
nB patients
yAj ~ G(y | A)
yBj ~ G(y | B)
Example
G is normal, i ~ N(i , 2)
Student’s t-test is most powerful test for
testing Ho : A = B
Invoked Population Model –
Randomization Model
Nonrandom Selection of Clinics in a
Nonrandom Selection of Communities
Undefined Sampling Procedure for Patients
(a variety of sources are used)
N = NA + NB patients
Randomization
NA patients
Source: Lachin J. Cont Clin Trials, 1988.
NB patients
Randomization Model Assumptions
• Under HO responses are assumed to be fixed (nonrandom) values – each patient’s response is what it
would have been regardless of treatment A or B
• The observed difference between A and B only
depends on the way treatments were assigned
(independent of other patient characteristics)
• To assess whether observed difference is “unusual”,
consider all possible ways patients could have been
assigned A or B (permutation test)
• Under simple randomization, permutation test is
asymptotically equal to homogenous population model.
Randomization or Permutation Test
1. Calculate test statistic for sample data, e.g.,
A - B difference, t-statistic
2. Determine the number of possible
randomization sequences
3. Enumerate all of these permutations;
calculate the test statistic for each and their
cumulative distribution
4. Determine where the test-statistic for sample
lies on distribution of all possible values
Example 3: Eight experimental units are
randomly allocated to receive treatment A or B
Treatment Group
A
B
18
9
13
16
3
17
17
17
n
mean
(sd)2
pooled (sd)2
4
12.75
46.92
4
14.75
14.92
30.92
t-statistic with 6 degrees of freedom
12.75 - 14.75
t(6) =
30.92

1+ 1
4 4

= -0.51, p = 0.628
The number of permutations using simple
random allocation (1:1) of NA and NB
assignments is given by:
(
NA + NB
NA
)
= (NA + NB)!/ NA ! NB!
NA = NB = 4 and number of permutations =70
Cumulative Distribution of t-statistic Obtained
from Randomization and Students’ Distribution
t
-2.48
-2.15
-1.88
-1.45
-1.26
-1.09
-.78
-.64
-.51*
-.25
-.125
0.0
.125
.25
.51
.64
.78
1.09
1.26
1.45
1.88
2.15
2.48
Cumulative Distribution
Randomization
1/70
4/70
5/70
8/70
12/70
15/70
18/70
22/70
25/70
28/70
32/70
38/70
42/70
45/70
48/70
52/70
55/70
58/70
62/70
65/70
66/70
69/70
70/70
.014
.057
.071
.114
.171
.214
.257
.314
.357*
*.400
.457
.543
.600
.643
.686
.743
.786
.828
.886
.928
.943
.986
1.000
* sample value, 2-sided p-value 50/70 = 0.71 versus 0.63
Students’ t(6)
.024
.038
.055
.097
.127
.159
.233
.273
.314*
.405
.452
.500
.548
.595
.686
.727
.767
.841
.873
.901
.945
.962
.976
Impact on P-value of Ignoring Blocking in the
Analysis
Simple Randomization of 20 Patients
Outcome
Accession No. Treatment
(Alive/Dead)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
A
B
A
B
B
B
A
A
B
B
A
A
B
A
A
B
A
B
B
A
A
D
D
D
D
D
D
A
D
D
A
A
D
A
A
D
A
A
A
A
Alive
Dead
A
8
2
B
2
8
Fisher’s exact test p-value =
0.0115 (1-tailed)
P-value = Probability 2 or fewer of the 10
deaths were randomly allocated to A
A
B
A
B
A
B
Alive
Dead
8
2
2
8
Alive
Dead
9
1
1
9
Alive
Dead
10
0
0
10
or
or
Fisher’s Exact Test
P - value =
1010 1010 1010
           
2 8  1 9  0 10
20 + 20 + 20
 
 
 
10 
10 
10
= .01096 + .00054125 +.00000541
= 0.0115
Restricted Randomization (block size = 4)
Accession No.
Outcome
(Alive/Dead)
Treatment
1
2
3
4
A
B
A
B
A
D
D
D
5
6
7
8
B
B
A
A
D
D
D
A
9
10
11
12
B
B
A
A
D
D
A
A
13
14
15
16
B
A
A
B
D
A
A
D
17
18
19
20
A
B
B
A
A
A
A
A
Block 1
Block 2
Block 3
Block 4
Block 5
p-value =
A
B
A
B
A
B
Alive
1
0
Alive
1
0
Alive
2
0
Alive
Dead
1
2
1
2
Dead
1
2
1
2
Dead
0
2
2
0
Dead
0
2
Alive
A
2
B
2
Dead
0
0
A
B
Probability
 12   12   16   16  1 
1
6
1
6
1
= 0.0069
General Setup
A
B
Alive
Dead
r
n-r
R-r
R
n
(N - R) –
N-n
(n - r)
N-R
N
 R  N - R
 

 r  n - r 
=
Prob (r alive on A)
N 
 
n
Based on hypergeometric distribution.
Randomization Theory Summary
• Guarantees control of type I error in hypothesis
tests
• Permutation or randomization tests are motivated
by the random assignment of patients
• The more restrictions imposed on the
randomization, the harder it is to determine the
permutation distribution.
• Permutation tests are not routinely used in the
analysis of trials (conservative). Can be useful to
consider blocking if population is heterogeneous
over time.