Transcript LECTURE 23

Interim Analyses of
Clinical Trials
A Requirement
Outline
• Background and how DSMBs
function
• Group sequential methods
• Examples
Suggested Reading
DeMets DL, Furberg CD, Friedman LM
Data Monitoring in Clinical Trials. A
Case Studies Approach
Springer, 2006
Example: ddI/ddC Trial, Good Thing We Did Not
Stop Early
ddC Better
ddI Better
FINAL 9/20/92
157:152 (ddI:ddC)
Confidence Intervals:
Repeated
95%
DSMB #5 8/21/92
130:130 (ddI:ddC)
DSMB #4 2/13/92
77:91 (ddI:ddC)
DSMB #3 11/08/91
50:66 (ddI:ddC)
DSMB #2 8/29/91
19:39 (ddI:ddC)
5.7
3.6
0.80
1.00
1.25
Relative Risk (on a log scale)
Fleming TR, Neaton JD, et al, JAIDS, 1995.
Lessons Learned: ddI/ddC
• Early findings can be unreliable
• Statistical monitoring can be useful for
assessing whether interim trends are
conclusive.
• Importance of restricting interim results of
trials to an independent DMC.
Example: Prevention Trial of Toxoplasmic
Encephalitis, A Trial Stopped 2, Maybe 3, Times
750 Patients
Unblinded
375 Clindamycin Arm
Blinded
250 Active
Treatment
375 Pyrimethamine Arm
Blinded
125
Placebo
250 Active
Treatment
125
Placebo
TOXO Protocol History
September 1990:
Enrollment begins
February 1991:
1st DSMB review
March 1991:
2nd DSMB review
Clindamycin arm discontinued
due to dose-limiting toxicities; and
protocol amended
Case 25 in Case Studies Book
Lancet 1992;339:333-334.
JID 1994; 169:384-394.
TOXO Trial:
Revised Design and Recruitment Goals
600
Patients
Pyrimethamine
N=400
Placebo
N=200
Study re-powered for one placebo group (2:1) and patients
in the clindamycin arm were offered re-randomization.
TOXO Protocol History (cont.)
August 29, 1991:
3rd DSMB review. Low overall TE
event rate noted
February 13, 1992:
4th DSMB review – DSMB
recommends stopping the study
due to low TE event rate among
controls, and higher mortality on
active drug. DSMB uncertain,
also recommend un-blinding the
chair to obtain his opinion.
TOXO Protocol History (cont.)
February 19, 1992: Protocol chair proposes to
continue study based on
review of interim analyses. He will
no longer enroll patients.
March 17, 1992:
5th DSMB review (special
teleconference) to discuss
chair’s position and new
data; DSMB recommends
termination of study again.
Number of Deaths and Rate
by Treatment Group and DSMB Review
Date of
Review
Cutoff
Date
Pyrimethamine
No.
Placebo
Rate*
No.
Rate*
08-29-91
07-31-91
3 (5)
6.5(10.8)
2 (2)
7.8 (7.8)
02-13-92
12-31-91
22 (28)
19.3(24.5)
8 (8)
13.7 (13.7)
03-17-92
01-31-92
28 (35)
21.4(26.7)
10 (10) 14.1(14.1)
( ) = actual numbers updated after review
*Per 100 person years
TOXO Protocol History (cont.)
March 30, 1992:
Chair and Protocol Team concur
with DSMB recommendation.
April 3, 1992:
Clinical alert sent to sites.
Number of Deaths and Rate
by Treatment Group and Report
Report
Cutoff
Date
Pyrimethamine
No.
Placebo
Rate*
No.
Rate*
DSMB
01-31-92
28 (35)
21.4 (26.7)
10 (10)
14.1 (14.1)
Clinical Alert
03-24-92
34 (46)
20.9 (28.3)
12 (14)
13.7 (16.0)
Final
03-30-92
48
28.9
( ) = actual numbers updated after review
*Per 100 person years
14
15.7
Lessons Learned: TOXO
• Importance of ensuring timely reporting of
outcomes.
• Unexpected results may require unblinding of
one or more investigators before making a
decision.
Example: CAST Study, A Trial
Stopped for Harm
• 1st DSMB meeting before enrollment started
– Recommended =0.025 (one-sided) for benefit
– Added =0.025 lower symmetric boundary for
harm
– Lan-DeMets spending function used with expected
number of cardiac sudden deaths (425) used to
estimate information fraction
Case 13 in Case Studies Book
CAST Sequential Boundaries
CAST Study: 2nd DSMB Meeting
• 6 months after study began
• No outcome data available for review
• DSMB decided to remain partially blinded in
the review of future interim data to maintain
objectivity
• Could be unblinded if necessary for
deliberations
CAST Study: 3rd DSMB Meeting
• 15 months after study began
• 1147 patients randomized, ~ ¼ of target
• Treatment assignment blinded to DSMB in reviewing data
• Sudden Death:
- DSMC report: 3/576 in drug X, and 19/571 in Drug Y
- Data sweep: 10 in drug X, and 22 in Drug Y
• DSMB decided no recommendations were appropriate
• Planned to meet again 6 months later
CAST Study: Events Following 3rd DSMB
Review
• CAST Coordinating Center monthly summarized the data
for internal monitoring
• 3 months after 3rd review, the results became more
extreme
• One month later, unblinded primary data presented to
NHLBI
• One month later, DSMB was notified the unblinded harmful
findings by conference call
• DSMB requested verification of treatment coding,
additional detailed analysis and sweep of the clinical
centers for yet unreported primary outcome
CAST Study: 4th DSMB review
• 33 sudden deaths in encainide and flecainide arms combined
and 9 on placebo
• Logrank Z=-3.22, crossed the harm boundary -3.04
• DSMB recommended dropping the encainide and flecainide
arms
• NHLBI, principal investigators, drug regulatory agencies
were notified the harmful findings
• The public was alerted because many non-study patients
were being treated with these drugs
• Primary report was published rapidly
CAST Study: Interim and Final Results
for Sudden Cardiac Mortality
Ecanide/
Flecanide
Placebo
Z
1
22
10
1.77
2
33
9
3.70 vs. (3.22
for logrank)
Final
43
16
3.52
Interim Review
Z (normal approx. to Poisson) =
a-b
√ a+b
a= no. deaths on encainide/flecainide
b= no. deaths on placebo
Lesson Learned: CAST
• A 1-sided boundary is usually not appropriate
(DMC modified it)
• DSMBs need to be able to un-blind treatment
codes
• Timely review of data important (unblinded
statistician alerted DMC)
Example: CPCRA NuCombo Study, a
Trial in Which DSMB’s Shared Data
Unblinded
ddI Arm
Blinded
ddI
+
AZT
ddC Arm
Blinded
Placebo (ddI)
+
AZT
ddC
+
AZT
Placebo (ddC)
+
AZT
Saravolatz L, N Eng J Med, 1996
NuCombo Interim Results: Late 1993
Treatment Group
A
A-P
B
B-P
No. Patients
330
167
332
166
No. AIDS or
Death
74
55
80
42
29.2
46.4
31.2
30.6
Rate
(per 100)
HR (A-P/B-P) = 1.6;
95% CI:
1.1 to 2.4;
p=0.03
Armitage P, Cont Clin Trials, 1999
Lessons Learned: NuCombo
• In some situations (e.g., a major safety
concern), exchange of information between
DMCs can be helpful.
• In most situations, external data considered by
DMCs should be limited to published
information. (see Dixon D and Lagakos S, Cont
Clin Trials, 2000; 21:1-6)
Example: MRC/BHF Heart Protection Study, a
Trial Some Felt Should Have Been Stopped
Early
• 20, 536 men and women in the UK with CHD,
other occlusive arterial disease or diabetes
with cholesterol > 135 mg/dl
• Randomized equally to simvastatin or placebo
• Primary comparisons were of effects from all
cause mortality, CHD mortality and from all
other causes.
Heart Protection Study Collaborative Group, Lancet, 2002.
DMC Monitoring Guidelines
With consideration of the study data and the
results from other studies, the DMC was to
advise the investigators if…
• “Proof beyond reasonable doubt” that either
for all participants or for some specific type of
participant, use of statin therapy was clearly
indicated or contraindicated in terms of the net
difference in all-cause mortality.
• Evidence that might reasonably be expected to
influence materially the management of
patients by many clinicians.
Case Examples of Trials That Had to React to
External Information
• BEST (beta-blockers for heart failure)
N Engl J Med 2001
• Concorde (zidovudine for asymptomatic HIV)
Lancet 1994 and Cont Clin Trials1999
• CPCRA 023 (CMV prophylaxis)
AIDS 1998 and Cont Clin Trials 2003
General Requirements for
Informed Consent
Significant new findings developed during the
course of the research that may relate to the
subject’s willingness to continue participation
will be provided.
WHAT IF THERE ARE NEW FINDINGS?
You will be told about any new information
learned during the study that might cause you
to change your mind about staying in the study.
Code of U.S. Federal Regulations Part 46, Subpart A, Section 46.116
Types of New External Information
• A finding from a randomized study with clinical
outcomes in the same target population?
• A trial in a different target population?
• A non-randomized study?
• Changes in labeling due to adverse events
(e.g., a modification to RISKS and/or
DISCOMFORT section of consent)?
All New Information is Not Equal: A Hierarchy of
Evidence Should be Considered in Assessing
Significance
• Coherence of evidence from multiple sources
• Systematic review of well-designed, large
randomized trials
• Strong evidence from one large randomized trial
• Systematic review of small trials (e.g., surrogate
outcome studies)
• Systematic review of well-designed cohort studies
• Strong evidence from one cohort study
• Unsystematic observations (expert opinions)
Adapted from Devereaux PJ et al, Evidence-Based Cardiology, 2nd Edition BMJ
Books 2003.
Responsibilities of the Data Monitoring
Committee (DMC) Concerning
External Information
“A DMC may be asked to consider the impact of
external information on the study being monitored.
Release of results of a related study may have
implications for the design of the ongoing study, or
even its continuation.”
“The role of the DMC in considering interim changes
to a study protocol or other aspects of a study
conduct in response to external information raises
additional issues that merit consideration.”
FDA Guidance for Clinical Trial Sponsors. Establishment and Operation of Clinical
Trial Data Monitoring Committees.
Consideration of External Information by the
DMC: An Issue that Merits Consideration
• Possible unblinding of study participants,
investigators and sponsor.
• In most cases, the study team and
sponsor (if blinded):
– Should make the assessment since they are
blinded to treatment differences.
– Should notify the DMC.
Case Example: Beta-Blocker
Evaluation of Survival Trial (BEST)
Control (standard of care) arm:
Optimal medical therapy and placebo for betablocker
Setting:
Beta-blockers had been avoided due to effects on
heart rate and BP in heart failure patients, but data
emerges indicating they may be effective
treatment for heart failure.
Several ongoing studies sponsored by
pharmaceutical companies but most patients
enrolled outside the U.S.
Case Example: BEST
Randomization of Patients with NYHA
Class III or IV Heart Failure
Bucindolol Hydrochloride
+
Optimal Medical Management
(ace-inhibitor digitalis, diuretic)
(N=1354)
Placebo
+
Optimal Medical Management
(N=1354)
Case Example: BEST
Timeline of Events
• May 1995: enrollment initiated
• March 1998: Cardiac Insufficiency Bisprolol Study (CIBIS-II)
(N=2647) stopped early due to significant mortality benefit of
bisprolol (monitoring guidelines were changed in 1998 to a
nominal 0.05 for primary outcome of mortality)
• October 1998: MERIT-HF trial (N=3991) stopped early due
to significant benefit of metoprolol on survival and HF
hospitalization
• July 1999: DMC recommends early termination due
“information…from other studies of beta-blockers…and by a
concern about the equipoise of the trial”. Sponsors (NHLBI
and VA) concurred. (p-value at termination=0.16 at time of
recommendation)
Mortality Results of CIBIS-II,
MERIT-HF and BEST
Treatment
Placebo
CIBIS-II (Bisoprolol)
156/1327
228/1320
MERIT-HF (Metoprolol)
145/1990
217/2001
BEST (Bucindolol)
411/1354
449/1354
Case Study: BEST Aftermath
• Do benefits of beta-blockers diminish in
patients with advanced heart failure?
- Copernicus trial of patients with advanced
heart failure (N=2289) stopped early in March
2000 for mortality benefit of carvedilol as
compared to placebo (130 vs 190 deaths)
• Is bucindolol fundamentally different from
other beta-blockers?
• Does beta-blockade work in AfricanAmericans with HF? (23% black in BEST vs < 5%
in other studies)
Case Example: Concorde
(Lancet, 1994)
Control (standard of care) arm:
Deferred zidovudine (placebo): open-label
treatment provided after AIDS/ARC
Setting:
Several ongoing trials of zidovudine.
Case Example: Concorde
Design Schematic
Randomization of
Asymptomatic Patients with
HIV
Immediate Zidovudine
(N=877)
Deferred Zidovudine
(Placebo)
(N=872)
Open label treatment
following AIDS/ARC
October 1989 Amendment:
•
Open-label treatment after 2 consecutive CD4+
counts <500
•
Primary prophylaxis for PCP
Case Example:
Concorde DMC Deliberations-1
(Armitage P, Cont Clinical Trials, 1999)
•
March 1988, enrollment begins.
•
September 1989: (987 participants randomized) Concorde DMC informed by
NIAID of early termination ACTG 019 (<500 CD4+)
- DMC recommends trial should continue as planned
•
October 1989, ACTG senior investigators meet with Concorde DMC and
coordinating committee
- Concorde DMC again recommends trial should continue as planned
- Protocol amendment allowing open-label treatment after two
consecutive CD4+ counts <500
- Letter sent to trial participants describing ACTG 019 results and why it was
important to continue Concorde
•
October 1991 (1715 participants), 1st formal interim analysis
- DMC recommends trial continue as planned and that recruitment end
Case Example:
Concorde DMC Deliberations-2
(Armitage P, Cont Clinical Trials, 1999)
• February 1992: DMC asked to consider another
trial, European-Australian Collaborative Group
Study, (CD4+ >400) which was terminated early
based on CD4+ differences
- DMC recommends continue as planned.
• June 1992: (1749 participants), 2nd formal interim
analysis
- DMC recommends follow-up end in December
1992 due to growing lack of equipoise among
investigators and growing use of open-label treatment
Case Example: Concorde
ACTG 019 and Interim Concorde Results
in September 1989
Concorde
Interim Data
September 1989++
ACTG 019+
500
mg
ZDV
1500
mg ZDV
AIDS, ARC or
death
17
AIDS
Death
No. patients
Avg followup:
+N
Engl J Med, 1990
++ Cont Clin Trials, 1999
Placebo
IMM ZDV
(1000mg)
DEF
ZDV
19
38
7
13
11
14
33
4
7
1
3
4
0
0
453
457
428
51-61 weeks
494
493
23 weeks
Final Concorde Study Results
Immediate
ZDV
Deferred ZDV
Deaths
95
76
AIDS or death
175
171
ARC, AIDS or death
263
284
CD4 difference over 3 years of follow-up
(immediate – deferred) = 30 cells (p<0.0001)
Lancet, 1994
Overview of Trials of ZDV vs. Placebo
(Immediate vs. Deferred)
Immediate
ZDV
Deferred ZDV
(placebo)
Risk
Ratio
Deaths
734
917
1.04
AIDS/death
1026
882
0.96
No. patients
4431
3291
Lancet 353: 2014-2025, 1999.
Case Example: CPCRA 023
(CMV Prophylaxis)
Control (standard of care) arm:
Placebo (no prophylaxis for CMV)
Setting:
Ganciclovir and foscarnet approved for
treatment, but not prophylaxis
A similar, but not identical, trial sponsored by
(Syntex) pharmaceutical sponsor is also
conducted. CPCRA study is confirmatory.
Case Example: CPCRA 023
(CMV Prophylaxis)
Randomization of HIV Patients
with CD4+ ≤100
Oral Ganciclovir
(3g daily)
(N=662)
Placebo
(N=332)
Background treatment: ART
and prophylaxis for other
infections
September 1994 amendment:
Open label treatment allowed
Case Example: CPCRA 023
(CMV Timeline of Events Prophylaxis)
(Hillman & Louis, Cont. Clin Trials, 2003)
•
•
•
•
November 1992: Syntex trial initiated
March 1993: CPCRA trial initiated
June 1994: Early termination of Syntex trial
July 1994: CPCRA provided results of Syntex study; DMC is
notified and holds special meeting
- Interim results shared with study co-chairs, and the DMC recommended
continuing CPCRA trial
• September 1994:
- Protocol amendment
- Patients informed of Syntex study results and DMC recommendation
- Patients also informed that CPCRA study did not show that CMV disease or
mortality differed (no numeric results given)
• Patients given 3 options:
1) Continue blinded treatment
2) Stop blinded treatment and receive open-label
3) Stop blinded treatment and no not take open-label
Case Example: CPCRA 023
(CMV Prophlyaxis)
Syntex and Interim CPCRA 023 Results in July 1994
Syntex Trial+
CPCRA 023
Interim Data
July, 1994++
No. Patients
725
994
No. with CMV
148
62
HR (95% CI) CMV
0.51 ( 0.36-0.73)
0.87 (0.52-1.46)
HR (95% CI)
Death
0.81 (0.61-1.07)
1.27 (0.78-2.07)
+ N Engl J Med, 1996
++ Cont Clin Trials, 2003
Final CPCRA 023 Results
(AIDS, 1998)
CMV
Ganciclovir
(N=662)
Placebo
(N=332)
HR
(95%CI)
101
55
0.92
(0.65-1.27)
Death
222
132
0.84
(0.67-1.04)
Open-label
ganciclovir (%)
39%
37%
Median months
on ganciclovir
9.2
2.0
Summary - 1
• Plan for interim looks in advance and state monitoring
guidelines in protocol.
• Recognize that every study is different and no single principle
(except assurance of patient safety) can be identified to guide
decisions – need to be flexible
• Consider all of the internal evidence (multiple outcomes and
subgroups), data quality and timeliness before recommending
early termination.
• Be cautious of trends and subgroups even though this may be
agonizing – in the end want a definitive answer.
• Timely data collection and summaries are critical.
Summary - 2
• If new information arises, even from a randomized
trial, it may be important to continue the ongoing
study.
– Different target population
– Longer follow-up
– Replication
• Treatments used lifelong require long-term studies to
assess risk/benefit; external information is more likely
to arise in long-term studies. Implications:
– Study designs need to anticipate new information
that might arise.
– Study investigators and DMCs must be prepared
to respond.