Transcript q - Ferran Torres

Simulation methods for calculating the
conditional power in interim analysis:
The case of an interim result opposite to the
initial hypothesis in a life-threatening disease.
Somatostin plus Isosorbide-5-Mononitrate
vs Somatostatin in the control of acute
gastro-oesophageal variceal bleeding:
a double blind, randomized, placebocontrolled clinical trial.
Junquera F, et al.
GUT 2000; 46 (1) 127-132.
Design
• Disease
– Acute variceal bleeding in cirrhotic patients
• Objective
– To test whether the addition of oral Isosorbide
5-Mononitrate (Is-5-Mn) improved the efficacy
of Somatostatine (SMS) alone in the control of
bleeding.
Design
• Treatments
– Group 1: SMS + PLB
(Control)
– Group 2: SMS + Is-5-Mn
(Experimental)
• Working hypothesis
– The rate of success would increase from 60% to
90%.
Sample size: Pre-determination
n=n per group
s2 = variance
q = effect size
f(a , b) = function of type I and II errors
n=
s2 / q2
* f(a , b)
Statistical errors: f(a,b)
f(a, b) = (Ua + Ub)2
a (1 tail)
0,050
0,025
0,005
a (2 tails)
0,100
0,050
0,010
0,200
6,183
7,849
11,679
0,100
8,564
10,507
14,879
0,050
10,822
12,995
17,814
b
Fixed sample size
ALPHA
=
0.05
POWER
=
0.90
P1
=
0.90
P0
=
0.60
Case sample size for uncorrected chi-squared test:
42
Introduction: interim analyses
• Often ethical concerns on these situations,
specially in life-threatening diseases.
• Sometimes, pre-defined working hypothesis
may not adjust to reality.
– Treatments may be better than expected
– Treatments may be worse than expected
(safety and/or efficacy)
• Long studies or big sample sizes make
advisable some kind of interim control.
Introduction
• At some fixed times, cumulated data can be
analysed and decisions may be taken in base
to the findings.
• Multiple analysis can lead to statistical
errors and mistaken clinical decisions.
• Several methods deal with multiplicity
issues.
Design
• For ethical reasons the design allows an interim
analysis, when half of the sample size is recruited.
• Pocock’s group sequential method (1977)
a = 0.05
b = 0.1
(power 90%)
p0= 60%, p1=90%
Group Sequential Methods
K
1
2
O'Brien & Fleming
z
a'
2.782
0.005
1.967
0.049
Peto
Pocock
z
2.576
1.969
a'
0.010
0.049
z
2.178
2.178
a'
0.029
0.029
1
2
3
3.438
2.431
1.985
0.001
0.015
0.047
2.576
2.576
1.969
0.010
0.010
0.049
2.289
2.289
2.289
0.022
0.022
0.022
1
2
3
4
4.084
2.888
2.358
2.042
0.000
0.004
0.018
0.041
3.291
3.291
3.291
1.969
0.001
0.001
0.001
0.049
2.361
2.361
2.361
2.361
0.018
0.018
0.018
0.018
1
2
3
4
5
4.555
3.221
2.630
2.277
2.037
0.000
0.001
0.009
0.023
0.042
3.291
3.291
3.291
3.291
1.969
0.001
0.001
0.001
0.001
0.049
2.413
2.413
2.413
2.413
2.413
0.016
0.016
0.016
0.016
0.016
a adjusted sample size
ALPHA
=
0.029
POWER
=
0.90
P1
=
0.90
P0
=
0.60
Case sample size for uncorrected chi-squared test:
48
Internal Participants
Monitoring Comittee
• Digestive System
Research Unit
• Pharmacist
• Liver Unit
• Clinical Pharmacologist
• Statistician
50% Sample size with evaluated outcome
Data for Interim analysis
Statistical analysis:
50 patients finalised
Interim analysis
Control
Exp
Sucess
21
87.5%
18
69.2%
Failure
3
12.5%
8
30.8%
24
100.0%
26
100.0%
Chi-square=2.427, p-value=0.119
OR1 (observed): 3.11 (0.72 –13.51)
ORr (design):
0.17
Problem statement
• Evidence from interim analysis against
working hypothesis
• Although no statistical evidence supporting
study termination, clinical criteria suggested
so.
• Search for objective support to clinical
intuition.
50% Sample size with evaluated outcome
Data for Interim analysis
Statistical analysis:
50 patients finalised
Recruitment:
10 patients
Possible solutions
1) Group sequential methods
2) Alpha spending function approach
3) Repeated confidence intervals
4) Stochastic curtailing methods
5) Bayesian methods
6) Boundaries approach (likelihood function)
Conditional power
• Negative results:
– CAST (I-II) study. NEJM (1989 & 1992)
• Group sequential testing using permutation
distribution & stochastic curtailment methods
– HPMPC trial, Ann Intern Med 1997
– ACTG Study 243. NEJM 1998
Conditional power
• Positive results:
– CRYO-ROP Arch Ophthalmology,1988
– Grable el al. Am J Obstet Gynecol, 1996
• Extension of trial:
– Proschan MA, Biometrics, 1995
Stochastic curtailment
Lan, Simon y Halperin
(1982)
Stop if in i inspection:
 q0, P(reject H0 | q) is high at the end
 q0, P(reject H0 | q) is small at the end
Application to real data
• design:
p(ctr) = 60%
p(exp) = 90%
• 1st Inspection (50 patients):
p(ctr) = 87.5%
p(exp) = 69.2%
• Probability of proving the working
hypothesis at the end (100 patients) projecting
the results from this inspection
Methods:
• OR design: 0.17
=>
qr = log(OR) = -1.792
• Simulations:
– Fortran 90
– 1,000,000 studies =>precision < 0.01%
– 15 possibilities ranging from –1.5xqr to +1.5xqr
Effect Size
Design
Observed
-1.5
x
qr
-0.63 x qr
0
q/qr
+1 x qr
ORr design: 0.17 qr = log(OR) = -1.79
+1.5
x
qr
Obs
H0
H1
absolute
p(Exp)
p(Ctr)
OR
q
q / qR
1
90.0%
99.25%
14.70
2.688
-1.50
diff.
9.2%
2
90.0%
98.83%
9.39
2.240
-1.25
8.8%
3
90.0%
98.18%
6.00
1.792
-1.00
8.2%
4
90.0%
97.18%
3.83
1.344
-0.75
7.2%
5
90.0%
96.55%
3.11
1.135
-0.63
6.6%
6
90.0%
95.66%
2.45
0.896
-0.50
5.7%
7
90.0%
93.37%
1.57
0.448
-0.25
3.4%
8
90.0%
90.00%
1.00
0.000
0.00
0.0%
9
90.0%
85.19%
0.64
-0.448
0.25
-4.8%
10
90.0%
78.61%
0.41
-0.896
0.50
-11.4%
11
90.0%
74.31%
0.32
-1.135
0.63
-15.7%
12
90.0%
70.13%
0.26
-1.344
0.75
-19.9%
13 90.0%
60.00%
0.17
-1.792
1.00
-30.0%
14
90.0%
48.94%
0.11
-2.240
1.25
-41.1%
15
90.0%
37.98%
0.07
-2.688
1.50
-52.0%
Conditional power calculation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
OR
q / qr
14.70
9.39
6.00
3.83
3.11
2.45
1.57
1.00
0.64
0.41
0.32
0.26
0.17
0.11
0.07
-1.50
-1.25
-1.00
-0.75
-0.63
-0.50
-0.25
0.00
0.25
0.50
0.63
0.75
1.00
1.25
1.50
q
% Sig.
Studies
% Sig.
Studies Exp.
2.69
2.24
1.79
1.34
1.13
0.90
0.45
0.00
-0.45
-0.90
-1.13
-1.34
-1.79
-2.24
-2.69
65.402
62.304
57.597
50.526
46.298
40.516
28.147
15.609
6.042
1.417
0.543
0.327
2.663
17.072
48.374
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.003
0.030
0.158
2.655
17.071
48.374
Conditional power calculation
100
Power (%)
80
q1 (1st inspection)
60
qr
40
(design)
20
0
-1.50
-1.00
-0.50
0.00
q/qr
0.50
1.00
1.50
P(q < q1 | q/qr= 1.00) = 53/1,000,000
P(q < q1 | q/qr= 1.25) = 2/1,000,000
P(q < q1 | q/qr= 1.50) = 0/1,000,000
Interim analysis after completion
of 10 more patients
Control
Exp
Success
21
87.5%
18
69.2%
Failure
3
12.5%
8
30.8%
24
100.0%
26
100.0%
Chi-square=4.794, p-value=0.029
OR1’ (observed):
4.00
ORr (design):
0.17
Final Interpretation
• The study was interrupted not based in the
sequential pre-defined rule.
• The clinical intuition was confirmed by the
conditional power calculation.
• The study was finished due to:
– The low likeliness of the working hypothesis
– The high probability of a worse outcome with
the experimental treatment
Conclusions
• Simulations may be a very useful
tool in some design and analysis
situations, as it has been shown in
this case of the conditional power
calculation.