Transcript ISSUES THAT PLAGUE NONINFERIORITY TRIALS PAST AND FUTURE RALPH B. D’AGOSTINO, SR. BOSTON UNIVERSITY HARVARD CLINICAL RESEARCH INSTITUTE.

ISSUES THAT PLAGUE NONINFERIORITY TRIALS
PAST AND FUTURE
RALPH B. D’AGOSTINO, SR.
BOSTON UNIVERSITY
HARVARD CLINICAL RESEARCH
INSTITUTE
OBJECTIVES
1. REVIEW ISSUES: PAST, PRESENT
AND FUTURE IN NON-INFERIORITY
(NI) STUDIES
2. PRESENT/ DISCUSS EXAMPLES
3. MAKE SOME COMMENTS FOR
IMPROVEMENTS
4. PRESENT A PERSONAL VIEW
OUTLINE
1.
2.
3.
4.
5.
6.
7.
Early Objectives and Issues
Approaches to Non-inferiority Trials
Examples (Here are some Problems)
Non-Inferiority AND/OR Superiority
All is Non-Inferiority
Intent-to-Treat vs. Per Protocol
New Major Issues
EARLY OBJECTIVES AND
ISSUES: EQUIVALENCY
•
•
•
•
American Dental Association (ADA 1980s)
CREST equivalent to COLGATE?
Ho: A-B>= 10% or A-B<= 10%
What does the 10% mean?
– DFMS or DFMT for 2 years, 3 years?
• Study done on differences and ratio used as
descriptive measure of “effect”
– 5.0 vs 5.4 becomes (5.4-5.0)/5.0 = .4/5.= 8%
EARLY OBJECTIVES
• M = 10% CAME FROM NOWHERE, BUT
WE KNEW WHAT IT WAS, That is, 10%
• TREATMENT DIFFERENCES
CONCERNED DIFFERENCES (RATIOS)
BETWEEN ACTIVE TREATMENTS
• WE WERE LOST BUT WE BELIEVED
WE HAD A “SENSE” ABOUT IT
APPROACHES TO NI TESTS
•
•
MUST DO BETTER THAN PLACEBO
But you cannot use a Placebo (P)
1. Putative Placebo Approach
2. Test Treatment (T) vs Positive Control (C)
directly with given Margin M (Assay
Sensitivity approach)
APPROACH 1 (Putative Placebo)
Stellar Example from the Past
• CAPRIE Study. Hasselblad and Kong
(2001) present this as their major example
for using meta-analyses for dealing with
estimating assay sensitivity (T vs. P)
• Want T vs. C, C vs. P, T vs. P
21
CAPRIE STUDY (cont)
• Can we obtain effect of
Clopidogrel vs. Aspirin
• Yes, if we can locate Asprin vs.
Placebo
• Do we believe what we get?
For Aspirin vs. Placebo
Antiplatelet Trialists’ Collaboration
Meta-Analysis
• Meta-analysis of all published and
unpublished unconfounded randomized trials
available March 1990
• Trials identified by literature search, trial
registry and inquiry of investigators and
pharmaceutical manufacturers
• Clear definitions of endpoints
• Well defined statistical methodology
APPROACH
• T vs. C (from Caprie trial)
• C vs. P (from Meta-analysis)
• Obtain T vs. P (from multiplication)
• (T/C) (C/P) = (T/P)
Clopidogrel Vs. Synthetic Placebo Control
Odds Ratios and 95% Confidence Intervals
Overall Patient Population
CAPRIE: Clopidogrel Vs. Aspirin
Meta-Analysis: Aspirin Vs. Placebo
Endpoint
Estimated: Clopidogrel Vs. Placebo
All Strokes, MIs,
or Vascular Deaths
p < 0.000001
All Strokes, MIs
or Death from
Any Cause
p < 0.000001
Vascular
Deaths
p < 0.0016
All Cause
Deaths
p < 0.0045
0.4
0.6
First Drug Better
0.8
1.0
1.2
1.4
1.6
Second Drug Better
• Meta-analysis studies contain very old studies (only
up to 1990), many prior to all of the elaborate medical
interventions (procedures) now routinely provided
• Many of the studies did not have MI, IS or vascular
death as their outcomes (the meta-analysis went back
to original investigators who in turn, had to generate
data). Ever try to get data on something you did not
collect?
• None of the studies used for Clopidogrel with aspirin
comparison had PAD as an entry criteria (PAD
represented 1/3 of Clopidogrel Study)
EFFECT SIZE: Relative Risk Reduction by
Qualifying Condition (ASA vs Clopidogrel)
IS n = 6431
MI n = 6302
PAD n = 6452
Total n =19185
30
20
10
Clopidogrel Better
0
10
20
Aspirin Better
Problems With Historical Controls
• Biases
– Time Biases
• Change in recognition or diagnosis of disease
• Changing disease process
• Change in usual therapy
(Myocardial Infarctions MI, Dx, Tx)
– Selection Biases
• Patients/Health care systems
• Are we really seeing the same patients in historical studies
as those in active control trial?
Problems With Meta-Analyses
So What Is Sponsor to DO?
If we plan to use placebo controlled trials, what
should we require of the historical placebo
trials?
• Same Disease/Conditions?
• Same Population
• Same Dose and Administration Levels of Active
Control C?
• Same Outcomes?
• Combine “All” or “Some (good)” Placebo
Controlled Studies
Still Other Problems With
Meta-Analyses
• What if previous studies had multiple arms? How
to put correctly into meta-analysis?
• What if none of the individual studies achieved
significance?
• What are we to believe from meta-analyses?
• Do we believe the p-levels of the meta-analysis? (I
do not think we should.)
APPROACH 2
NON-INFERIORITY STUDIES
ACTIVE CONTROL STUDIES
NON-INFERIORITY TEST
H0: T-C >= M vs. H1: T-C < M
(Say data are event rates)
T is new treatment
C is positive control
M IS NON-INFERIORITY MARGIN
NON-INFERIORITY STUDIES
APPROACH 2
• SELECT A VALUE OF M THAT MAKES
SENSE
• WANT ASSURANCE THAT ASSAY
SENSITIVITY IS PRESENT (Placebo is
working)
• WANT CONSISTENCY WITH PAST
NON-INFERIORITY STUDIES
Statistical Approach
1. Need Active Control C vs. Placebo P data from
Historical data (C vs. P)
2. Need to test effectiveness of T vs. C
3. Need estimate of fraction of C-P preserved by T
(e.g., (T-P)/(C-P) = M) M=0.5 (C-P)
METHODS EXIST THAT ALLOW TEST TO
BUILD IN NEW AND HISTORICAL DATA
(STATISTICS IN MEDICINE, 2002)
WHAT IS NEEDED FOR 2
• CONFIDENCE INTERVAL IS OFTEN
USED. WANT M=1.11 (SAY) OUTSIDE
UPPER LIMIT OF CONFIDENCE
INTERVAL (M is relative risk)
• FDA ODAC 8/04 (non-small cell lung cancer)
1.0
1.11= M
SOME REALITIES
• Sounds nice
• What happens
Anti-infective Product
No placebo data
• Historical data is not Placebo, but C
• VRE (vancomycin resistant enterococcal)
High dose
Low dose
• MITT 60.0 % (N=65) vs. 46.2 % (N=52)
• Bacteremic
55.6 (N=18) vs. 25.0 (N=16)
• What is M? One trial OK? Any superiority?
ANOTHER EXAMPLE
Respiratory Distress
• Respiratory Distress Syndrome in
Premature Infants
– Treatments
• New Drug
• Comparator
– Outcome
• Survival at 28 Day
Respiratory Distress (cont)
• Survanta versus Sham (two studies one
positive, other negative) All Cause mortality
• Study 1: 8% vs. 23% Study2: 17% vs. 14%
• What is M? .23-.08? .180-.125?
CONSISTENCY
Example Control rate different
from historical
•
•
•
•
•
•
•
Historical Data says C=0.5 and P=0.6
Want T<=0.55
P-C=0.10, M=0.5(0.10) = 0.05
(T-C)/C = 0.05/0.50 = 10%
Data is C=0.30 and T=0.33, T-C=0.03
(T-C)/C = 0.03/0.30 = 10%
IS STUDY A SUCCESS? USE RATIOS?
ANSWER TO CONSISTENCY
• There was consistency
• Differences related to birth weight
Non-Inferiority and Inferiority at the
same time
• Sponsor falls apart
0
M
Non-Inferiority and Superiority
• Sponsor jumps for joy (Sequential test)
0
M
Switching trial design
(Cardiac Stent Trials)
• (1) New drug coated stents, we can do noninferiority study with margin set (15%)
• (2) We can do superiority study with noncoated stent as control
• With first option we have to worry about
evaluating Ms, Effect size and CREEP
• With superiority trial “clean” results
Respiratory Distress
• Compare new surfaxin to another “not so
great” one, but still used in practice
Switching from Superiority to nonInferiority
• HOW CAN WE SWITCH FROM A
SUPERIORITY TEST TO NONINFERIORITY ?
• This is a question thrown at me constantly
Assessing Efficacy NonInferiority and Safety Superiority
• Carotid artery Magnetic Resonance Imaging agent
• Imaging Agents
– Agent N (New) Agent C (Comparator)
• Non-inferiority” Outcome
– Endpoint: agent’s ability to classify correctly patients
with > 25% stenosis (sensitivity)
– Sensitivity of Comparator is .80 or 80%
– Non-inferiority margin M set to 0.10
Assessing Efficacy Non-Inferiority
and Safety Superiority (Cont’d)
• There is a specific adverse event that is
hypothesized to occur less often with New
than with Comparator
– Do we want to make the specific adverse event
rate an additional primary endpoint? WHY
NOT?
Non US STUDIES
• Forced off shore (ethical and other reasons)
The BLOB EFFECT
• Everything is suddenly Non-Inferiority
ALLHAT STUDY
• COMPARISON OF ANTIHYPERTENSIVE MEDICATIONS
(MULTIPLE ARMS)
• NOT A NON-INFERIORITY STUDY
Safety Studies
• Safety studies have become carefully
designed and executed studies
• Should they be non-inferiority studies?
SAFETY STUDIES (PHASE 4)
HISTORICAL APPROACH: NEW RATE > OLD
H01: T-C <= 0 vs. H11: T-C > 0
H02: RR=T/C <= 1 vs. H12: RR=T/C > 1
STUDY POWERED TO REJECT T/C >1.5 (SAY)
SHIFT IS TO MAKING THESE NONINFERIORITY STUDIES
• H0: T-C >= M vs. H1: T-C < M
H0: RR=T/C >= M vs. H1: RR=T/C < M
Safety Studies
• OLD
• NEW
1
M
•
•
•
•
SAFETY STUDY TO NONINFERIORITY STUDY
(QT LONGATION)
Safety issue: drug may cause QT problem
Ho: A/B = 1.0 vs H1: R = A/B > 1.0
Study powered for R > 1.0
When interest in risk fades can we suddenly
say this should be a non-inferiority study?
• Ho:R >= 1.5 vs. H1:R < 1.5 was not
original objective
• If we do not reject Ho is that enough?
Form of Interest and Sample Size
• Ho: p1-p2 >= M
• Ho: p1-p2>=Rp2
• Ho: p1/p2 >= R
• Best Choice does depend on p2 (control rates)
Intent-to-Treat vs. Per-Protocol
• In superiority trials, the primary analysis is
often on intent-to-treat (ITT) population
• Per Protocol (PP) “bigger” differences of
treatments
• In non-inferiority should we use PP?
Intent-to-Treat vs. Per-Protocol
(Cont’d)
• PP as primary not always accepted
– “the ITT analysis is as important as the PP analysis”
– “need to reconcile differences between ITT and PP
analysis”
– Perform “sensitivity” analyses. Results should be similar
in both populations (ROBUSTNESS).
– The Committee on Proprietary Medicinal Products draft
Points to Consider: “…similar conclusions from both the
ITT and PP are required in a noninferiority trial”.
• We ask sponsor to do both (ITT and PP) and
expect to achiev the sam significant result
on both.
• What is the true alpha associated with this?
NEW MAJOR ISSUES
• Missing Data
• Noncompliance
• Interim Analysis
• OUR USUAL LOGIC INCREASES
CHANCE OF ACCEPTANCE OF noninferiority
MORE NEW ISSUES
• Multiple endpoints
• Multiple groups
• Repeated Measures
WHERE ARE WE?
• NON-INFERIORITY TRIALS HAVE
MADE A BIG IMPACT
• They have brought many new problems and
challenges with them