Transcript Document 7442739

Issues of Simultaneous Tests for
Non-Inferiority and Superiority
Tie-Hua Ng*, Ph. D.
U.S. Food and Drug Administration
[email protected]
Presented at
MCP 2002
August 5-7, 2002
Bethesda, Maryland
_______
* The views expressed in this presentation are not
necessarily of the U.S. Food and Drug Administration.
Simultaneous Tests for
Non-Inferiority and Superiority
• Multiplicity adjustment is not necessary
– Intersection-union principle (IU)
• Dunnett and Gent (1996)
– Closed testing procedure (CTP)
• Morikawa and Yoshida (1995)
• Indisputable
2
A Big Question
Is Multiplicity Adjustment Necessary?
3
Is
Multiplicity
Adjustment
Necessary?
4
Outline
• Assumptions and Notations
• Switching between Superiority and
Non-Inferiority
• Is Simultaneous Testing Acceptable?
• Use of Confidence Interval in
Hypothesis Testing --- Pitfall
• Problems of Simultaneous Testing
• Conclusion
5
Assumptions/Notations
•
•
•
•
•
Normality and larger is better
T: Test/Experimental treatment (t)
S: Standard therapy/Active control (s)
: Non-Inferiority Margin (> 0)
For a given d (real number), define
– Null: H0(d): T  S - d
– Alternative: H1(d): T > S - d
• Non-Inferiority: d = 
• Superiority: d = 0
6
Non-Inferiority (d = )
H0(): T  S -  against H1(): T > S - 
H0()
H1()
°•

S
T
Worse
Boundary
Better
Mean Response
7
Superiority (d = 0)
H0 (0): T  S against H1 (0): T > S
H0(0)
°•
H1(0)
S
T
Worse
Boundary
Better
Mean Response
8
Switching between
Superiority and Non-Inferiority
CPMP
(Committee
for
Proprietary
Medicinal Products), European Agency
for the Evaluation of Medicinal Products
Points to Consider on Switching Between
Superiority and Non-Inferiority, 2000.
http://www.emea.eu.int/htms/human/ewp/ewpptc.htm
9
Switching between
Superiority and Non-Inferiority (2)
• Non-Inferiority Trial
–
–
If H0() is rejected, proceed to test H0(0)
No multiplicity issue, closed testing procedure
• Superiority Trial
–
–
Fail to reject H0(0), proceed to test H0()
No multiplicity issue
–
Post hoc specification of 
10
Switching between
Superiority and Non-Inferiority (3)
• Non-inferiority Trial
–
–
Intention-to-treat (ITT)
Per protocol (PP)
• Superiority Trial
–
–
•
Primary: Intention-to-treat (ITT)
Supportive: Per protocol (PP)
Assume ITT = PP
11
Simultaneous Testing
One-sided 100(1 - )% lower Confidence Interval
for T - S
Superiority
Non-inferiority
Neither
-
Test is worse
0
Test is better
Mean Difference (T – S)
12
Simultaneous Testing (2)
• Multiplicity adjustment is not necessary
– Dunnett and Gent (1996)
• Intersection-Union (IU):
Superiority: Both H0() and H0(0) are rejected
– Morikawa and Yoshida (1995)
• Closed Testing Procedure (CTP):
Test H0(0) when H0()H0(0) is rejected
13
Simultaneous Testing (3)
• Discussion Forum (October 1998)
– London
– PSI (Statisticians in Pharmaceutical Industry)
• Is Simultaneous Testing of Equivalence [NonInferiority] and Superiority Acceptable?
– Superiority trial:
• Fail to reject H0 (0)
• No equivalence/non-inferiority claim
– Ok: Morikawa and Yoshida (1995)
• Ref: Phillips et al (2000), DIJ
14
Is
Simultaneous
Testing
Acceptable?
15
Use of Confidence Interval
in Hypothesis Testing
H0(d): T  S - d (at significance level )
One-sided 100(1-)% lower CI for T-S
Reject H0(d) if and only if the CI excludes -d
Reject H0(d)
Do not reject H0(d)
Test is worse
-d
Test is better
Mean Difference (T – S)
16
Use of Confidence Interval
in Hypothesis Testing (2)
• If CI = (L, ), then H0(d) will be
rejected for all -d < L.
• A Tricky Question
– Suppose CI = (-1.999, ), L = -1.999
• H0(2): T  S - 2 is rejected (d=2) since -d < L
• Can we conclude that T > S - 2?
• Yes, if H0(2) is prespecified.
• No, otherwise.
17
Use of Confidence Interval
in Hypothesis Testing (3)
Post hoc specification of
H0(d)
is a
No No
18
Simultaneous Testing: Problems
and H0(d2), for d1 > d2
One-sided (1 - )100% lower CI for T - S
H0(d1)
Reject H0(d2)
Reject H0(d1)
Neither
-d1
Test is worse
-d2
Test is better
Mean Difference (T – S)
19
Simultaneous Testing: Problems (2)
H0(d1), H0(d2)
and H0(d3), for d1 > d2 > d3
One-sided (1 - )100% lower CI for T - S
Reject H0(d3)
Reject H0(d2)
Reject H0(d1)
None
-d1
Test is worse
-d2
-d3
Test is better
Mean Difference (T – S)
20
Simultaneous Testing: Problems (3)
H0(d1), H0(d2),…, H0(dk), for d1 > d2 > … > dk
One-sided (1 - )100% lower CI for T - S
Reject H0(dk)
.
.
Reject H0(d2)
Reject H0(d1)
None
.
…
-d1 -d2 -d3
Test is worse
… -d
k
Test is better
Mean Difference (T – S)
21
Simultaneous Testing: Problems (4)
• Choose k large enough
 Pr[-d1 < Lower limit < -dk] close to 1
 Max |dk - dk-1| < a given small number
 Simultaneous testing of H0(di), i = 1,…, k
 Post hoc specification of H0(d)
22
Simultaneous Testing: Problems (5)
Confirmatory
(one H0(d))
1
2
3
Simultaneous
H0() and H0(0)
4
………….
Exploratory
(many H0(d))
k
…………
Number of Nested hypotheses
23
Simultaneous Testing: Problems (6)
•
•
•
•
What is wrong with IU and CTP?
Nothing
Pr[Rejecting at least one true null]  
What kind of problems?
24
Simultaneous Testing: Problems (7)
• Post hoc specification of H0(d)
 Let -d0 = 100(1 - )% lower limit - 
 Reject H0(d0), since -d0 < lower limit
 Repeat the same trial independently
 Pr[Rejecting H0(d0)] = 0.5 +
25
Simultaneous Testing: Problems (8)
• Simultaneous testing of many H0(d)
– Repeat the same trial independently
– Low probability of confirming the finding
• 1st trial: Reject H0(dj) but not H0(dj+1)
• 2nd trial: Pr[Rejecting H0(dj)] is low (e.g., 0.5+)
26
Simultaneous Testing: Problems (9)
• Simultaneous testing of H0() and H0(0)?
• Confirm the finding
=2
 Known variance
 Let   T - S
 Significance level  = 0.025
 80% power for H0() (at  = 0)
27
Simultaneous Testing: Problems (10)
f() = Pr[Rejecting H0() | ]
f0() = Pr[Rejecting H0(0) | ]
28
Simultaneous Testing: Problems (11)
• Test one null hypothesis H0()
• Suppose that H0() is rejected
• Repeat the same trial independently
• Pr[Rejecting H0() again] = f()
29
Simultaneous Testing: Problems (12)
• Test H0() and H0(0) simultaneously
• Suppose that H0() or H0(0) is rejected
• Repeat the same trial independently
• Pr[Rejecting the same null hypothesis again]
= [1 - w()] · f() + w() · f0()
= f() - f0() [1 – f0()/f()],
where w() = f0()/f()
30
Simultaneous Testing: Problems (13)
Simultaneous tests
in the 2nd trial
[1 - w()] · f() + w() · f0()
where w() = f0()/f()
31
Simultaneous Testing: Problems (14)
• Ratio: 1 – [f0()/f()] [1 – f0()/f()]
• Ratio may be as low as 0.75
32
Conclusion
•
•
•
•
•
•
Many H0(d): Problematic
Not type I error rate
H0() and H0(0): Acceptable?
If “zero tolerance policy”: No
If 25% reduction cannot be tolerated: No
If 25% reduction can be tolerated: Yes
33
Is
Simultaneous Testing of
H0() and H0(0)
Acceptable?
34
You be the judge
35
References
• Dunnett and Gent (1976), Statistics in Medicine, 15,
1729-1738.
• Committee for Proprietary Medicinal Products
(CPMP; 2002). Points to Consider on Switching
Between
Superiority
and
Non-Inferiority.
http://www.emea.eu.int/htms/human/ewp/ewpptc.htm
• Morikawa T, Yoshida M. (1995), Journal of
Biopharmaceutical Statistics, 5:297-306.
• Phillips et al., (2000), Drug Information Journal,
34:337-348.
36
37