Transcript Robust Analysis of Incomplete Longitudinal Data in

Robust Analysis of Incomplete
Longitudinal Data in Clinical Trials
Robin Mogg* and Devan V. Mehrotra
Merck Research Laboratories
ICSA Applied Statistics Symposium
Raleigh, NC
June 4, 2007
* [email protected]
Outline
 The need for an HIV Vaccine
 Motivating trials:
» Two Phase IIa HIV Vaccine Trials
 Numerical example
 Statistical methods
 Simulation results
 Concluding remarks
 Interactions with CBER
June 4, 2007
2007 ICSA Applied Statistics Symposium
2
The Need for an HIV Vaccine
 UNAIDS 2005 Estimates:
» 38.6 million people living with HIV worldwide;
4.1 million people newly infected (>11,000/day);
2.8 million people died
 Antiretroviral therapy (ART):
» Dramatically decreased morbidity and mortality in
developed countries; treatment regimens complex and
costly; globally reaches only 1 of 5 in need.
 A safe and effective HIV Vaccine is the best hope
for controlling/ending the AIDS epidemic.
» Ideal vaccine candidate would be 100% effective in
preventing infection among those uninfected.
June 4, 2007
2007 ICSA Applied Statistics Symposium
3
Humoral and Cellular Immunity
 Immune responses of preventative vaccines are
designed to mimic those from natural exposure.
» Humoral immunity: mediated by virus-neutralizing
antibodies, prevents virus from infecting cells.
» Cellular immunity: mediated by T-lymphocytes, target
and kill already infected cells.
 The immune system “remembers” each encounter;
basis of vaccination against infectious diseases.
 In natural HIV infection:
» Humoral response is not completely effective in
preventing virus from infecting cells.
» Success of cellular response varies, “better” responses
result in lower virus and better clinical outcomes.
June 4, 2007
2007 ICSA Applied Statistics Symposium
4
HIV Infection Markers: CD4 count and Viral load
June 4, 2007
2007 ICSA Applied Statistics Symposium
5
HIV Vaccination
 Merck’s HIV Vaccine is designed to induce a cellmediated immune (CMI) response.
 Prophylactic vaccination:
» Goal is to induce broad cellular immune responses in
HIV uninfected individuals that provides either
protection from infection (sterilizing immunity) or
protection from disease (low viral load setpoint, slow
disease progression).
 Therapeutic vaccination:
» Goal is to induce broad cellular immune responses in
HIV infected individuals that provides protection
from disease (low viral load without ART, slow disease
progression).
June 4, 2007
2007 ICSA Applied Statistics Symposium
6
Motiviating Trial #1:
Prophylactic Proof of Concept (POC) Efficacy Trial
 Design: randomized, double-blind, placebo-controlled
study in a population at high risk of HIV infection.
 Two co-primary endpoints:
» Infection
» Viral load setpoint (among those infected)
– vRNA measured at time of diagnosis and at 2, 8, and
12 weeks after diagnosis.
– Viral load setpoint = mean of log10(vRNA) at
Weeks 8 and 12.
 Hypothesis: HIV vaccination will lead to a lower
incidence of HIV and/or lower viral load setpoints
among infected subjects.
June 4, 2007
2007 ICSA Applied Statistics Symposium
7
Motiviating Trial #2: ACTG A5197
Therapeutic POC Efficacy Trial
 Design: randomized, double-blind, placebo-controlled
study in an HIV-infected population with prolonged
(>2 yrs) ART-based suppression of viral load.
» After immunization phase, interrupt ART for everyone.
 Primary endpoint:
» Viral load setpoint
– vRNA measured at 1, 2, 4, 6, 8, 12, and 16 weeks
after interruption of therapy.
– Viral load setpoint = mean of log10(vRNA) at Weeks
12 and 16.
 Hypothesis: Therapeutic HIV vaccination will lead to
better control of viral replication during ART
interruption.
June 4, 2007
2007 ICSA Applied Statistics Symposium
8
Motiviating Trial #2 (cont.): ACTG A5197
Therapeutic nPOC
Trial
n (p(p) )==#Efficacy
#(proportion)
(proportion)who
whointerrupt
interruptART
ARTon
onvaccine
vaccine
randomizedtotovaccine
vaccine
NNv v==##randomized
randomizedtotoplacebo
placebo
NNp p==##randomized
v
v
v
v
(proportion)who
whointerrupt
interruptART
ARTon
onplacebo
placebo
nnp p(p(pp)p)==##(proportion)
RANDOMIZATION
RANDOMIZATION
ImmunizationPhase
Phase
Immunization
STOP
RESUME
STOP
RESUME
ART
ART*
ART
ART*
Treatment
Treatment
Interruption
Interruption
Phase
Follow-upPhase
Phase
Phase
Follow-up
Immunizationsatatweeks
weeks0,0,4,4,and
and26
26
Immunizations
SS 00 44
26
26
38 39
39
38
Week
Week
54 55
55
54
needed/desired
* *IfIfneeded/desired
84
84
TreatmentInterruption
InterruptionPhase
Phase
Treatment
11 22
(Wk39)
39)
(Wk
44
66
88
12
12
Week
Week
16
16
(Wk54)
54)
(Wk
Viral
Setpoint (VLS):
VLSLoad
i = Y i = mean of log10(vRNA)
VLS i = Y i = mean
of log10(vRNA)
at Weeks
12 and 16. at
Weeks 12 and 16 for subject i.
June 4, 2007
2007 ICSA Applied Statistics Symposium
9
Challenge: Missing vRNA Data Due to “Drop-Outs”
Start ART (viral failure)
100,000
90,000
HIV Viral Load (RNA copies/ml)
80,000
70,000
60,000
50,000
40,000
Complete Data
30,000
Lost to Follow-up
20,000
10,000
0
0
1
2
4
6
8
12
16
Weeks Post-Diagnosis or Post-Treatment Interruption
June 4, 2007
2007 ICSA Applied Statistics Symposium
10
Numerical Example: Hypothetical Data
based on Therapeutic POC Efficacy Trial
Trt
Group
Vaccine
Placebo
Patient
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
log10 viral load during the ART interruption phase
Wk 1 Wk 2 Wk 4
Wk 6 Wk 8 Wk 12 Wk 16
2.3*
3.8
4.8
4.5
3.5
2.3*
2.3*
2.3*
2.3*
3.6
4.1
4.4
4.3
4.2
2.3*
2.3*
4.7
4.4
3.8
3.4
2.6
3.9
5.5
5.8
.
.
.
.
2.3*
2.3*
4.1
4.7
4.3
5.4
4.9
2.3*
2.3*
4.3
4.1
4.2
3.8
3.9
2.3*
2.3*
2.7
3.2
3.0
2.3*
2.8
2.9
4.6
3.9
3.7
.
.
.
2.3*
2.3*
3.3
5.2
6.2
6.5
6.4
2.3*
4.0
4.8
5.2
4.6
4.4
4.5
3.6
3.5
4.0
3.9
4.5
4.2
4.7
2.3*
4.0
4.5
5.7
6.0
.
.
2.3*
2.3*
4.6
4.7
4.5
4.4
4.4
2.9
4.2
4.9
4.9
4.5
4.5
4.3
2.3*
3.7
.
.
.
.
.
2.3*
2.3*
2.9
3.4
4.1
2.3*
3.4
2.3*
4.4
4.7
4.5
4.8
4.4
3.7
2.3*
5.0
6.4
6.2
5.1
5.1
4.8
2.3*
4.0
4.9
5.1
5.1
5.0
4.7
2.3*
2.3*
4.7
4.8
4.9
3.7
4.2
VLS
2.3
4.3
3.0
?
5.2
3.8
2.5
?
6.4
4.5
4.4
?
4.4
4.4
?
2.9
4.1
5.0
4.9
4.0
Subject restarted ART; Subject was lost to follow-up
* log10 viral loads < log10(400) = 2.6 were replaced with log10(200)=2.3
June 4, 2007
2007 ICSA Applied Statistics Symposium
11
Numerical Example (cont.): Hypothetical Data
Vaccine
1000000
1000000
HIV Viral Load (RNA copies/ml)
HIV Viral Load (RNA copies/ml)
Placebo
100000
100000
10000
Median VLS = 4.06
10000
1000
1000
100
100
1
2
4
6
8
12
16
Weeks Post-ART Interruption
Completers:
Lost to Follow-up:
Restart ART:
June 4, 2007
Median VLS = 4.36
1
2
4
6
8
12
16
Weeks Post-ART Interruption
8/10 (80%)
1/10 (10%)
1/10 (10%)
2007 ICSA Applied Statistics Symposium
8/10 (80%)
1/10 (10%)
1/10 (10%)
12
“Standard”, but Ad hoc Statistical Methods
 LOCF
» Use LOCF to impute missing values after dropout.
» Calculate VLS, then use a t-test.
 Tied Worst Rank
» Assign VLS = 10^10 to all “drop-outs”.
» Use Wilcoxon Rank Sum (WRS) test.
 Untied Worst Rank
» Assign VLS = 10^10 – tlast to all “drop-outs”, where
tlast = time of dropout (penalizes earlier dropouts).
» Use Wilcoxon Rank Sum (WRS) test.
June 4, 2007
2007 ICSA Applied Statistics Symposium
13
Numerical Example: Hypothetical Data
using LOCF
Trt
Group
Vaccine
Placebo
Patient
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
log10 viral load during the ART interruption phase
Wk 1 Wk 2 Wk 4
Wk 6 Wk 8 Wk 12 Wk 16
2.3*
3.8
4.8
4.5
3.5
2.3*
2.3*
2.3*
2.3*
3.6
4.1
4.4
4.3
4.2
2.3*
2.3*
4.7
4.4
3.8
3.4
2.6
3.9
5.5
5.8
5.8
5.8
5.8
5.8
2.3*
2.3*
4.1
4.7
4.3
5.4
4.9
2.3*
2.3*
4.3
4.1
4.2
3.8
3.9
2.3*
2.3*
2.7
3.2
3.0
2.3*
2.8
2.9
4.6
3.9
3.7
3.7
3.7
3.7
2.3*
2.3*
3.3
5.2
6.2
6.5
6.4
2.3*
4.0
4.8
5.2
4.6
4.4
4.5
3.6
3.5
4.0
3.9
4.5
4.2
4.7
2.3*
4.0
4.5
5.7
6.0
6.0
6.0
2.3*
2.3*
4.6
4.7
4.5
4.4
4.4
2.9
4.2
4.9
4.9
4.5
4.5
4.3
2.3*
3.7
3.7
3.7
3.7
3.7
3.7
2.3*
2.3*
2.9
3.4
4.1
2.3*
3.4
2.3*
4.4
4.7
4.5
4.8
4.4
3.7
2.3*
5.0
6.4
6.2
5.1
5.1
4.8
2.3*
4.0
4.9
5.1
5.1
5.0
4.7
2.3*
2.3*
4.7
4.8
4.9
3.7
4.2
VLS
2.3
4.3
3.0
5.8
5.2
3.8
2.5
3.7
6.4
4.5
4.4
6.0
4.4
4.4
3.7
2.9
4.1
5.0
4.9
4.0
Subject restarted ART; Subject was lost to follow-up
* log10 viral loads < log10(400) = 2.6 were replaced with log10(200)=2.3
June 4, 2007
2007 ICSA Applied Statistics Symposium
14
Numerical Example: Hypothetical Data
using Tied Worst Rank
Trt
Group
Vaccine
Placebo
Patient
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
log10 viral load during the ART interruption phase
Wk 1 Wk 2 Wk 4
Wk 6 Wk 8 Wk 12 Wk 16
2.3*
3.8
4.8
4.5
3.5
2.3*
2.3*
2.3*
2.3*
3.6
4.1
4.4
4.3
4.2
2.3*
2.3*
4.7
4.4
3.8
3.4
2.6
3.9
5.5
5.8
.
.
.
.
2.3*
2.3*
4.1
4.7
4.3
5.4
4.9
2.3*
2.3*
4.3
4.1
4.2
3.8
3.9
2.3*
2.3*
2.7
3.2
3.0
2.3*
2.8
2.9
4.6
3.9
3.7
.
.
.
2.3*
2.3*
3.3
5.2
6.2
6.5
6.4
2.3*
4.0
4.8
5.2
4.6
4.4
4.5
3.6
3.5
4.0
3.9
4.5
4.2
4.7
2.3*
4.0
4.5
5.7
6.0
.
.
2.3*
2.3*
4.6
4.7
4.5
4.4
4.4
2.9
4.2
4.9
4.9
4.5
4.5
4.3
2.3*
3.7
.
.
.
.
.
2.3*
2.3*
2.9
3.4
4.1
2.3*
3.4
2.3*
4.4
4.7
4.5
4.8
4.4
3.7
2.3*
5.0
6.4
6.2
5.1
5.1
4.8
2.3*
4.0
4.9
5.1
5.1
5.0
4.7
2.3*
2.3*
4.7
4.8
4.9
3.7
4.2
VLS
2.3
4.3
3.0
1010
5.2
3.8
2.5
1010
6.4
4.5
4.4
1010
4.4
4.4
1010
2.9
4.1
5.0
4.9
4.0
Subject restarted ART; Subject was lost to follow-up
* log10 viral loads < log10(400) = 2.6 were replaced with log10(200)=2.3
June 4, 2007
2007 ICSA Applied Statistics Symposium
15
Numerical Example: Hypothetical Data
using Untied Worst Rank
Trt
Group
Vaccine
Placebo
Patient
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
log10 viral load during the ART interruption phase
Wk 1 Wk 2 Wk 4
Wk 6 Wk 8 Wk 12 Wk 16
2.3*
3.8
4.8
4.5
3.5
2.3*
2.3*
2.3*
2.3*
3.6
4.1
4.4
4.3
4.2
2.3*
2.3*
4.7
4.4
3.8
3.4
2.6
3.9
5.5
5.8
.
.
.
.
2.3*
2.3*
4.1
4.7
4.3
5.4
4.9
2.3*
2.3*
4.3
4.1
4.2
3.8
3.9
2.3*
2.3*
2.7
3.2
3.0
2.3*
2.8
2.9
4.6
3.9
3.7
.
.
.
2.3*
2.3*
3.3
5.2
6.2
6.5
6.4
2.3*
4.0
4.8
5.2
4.6
4.4
4.5
3.6
3.5
4.0
3.9
4.5
4.2
4.7
2.3*
4.0
4.5
5.7
6.0
.
.
2.3*
2.3*
4.6
4.7
4.5
4.4
4.4
2.9
4.2
4.9
4.9
4.5
4.5
4.3
2.3*
3.7
.
.
.
.
.
2.3*
2.3*
2.9
3.4
4.1
2.3*
3.4
2.3*
4.4
4.7
4.5
4.8
4.4
3.7
2.3*
5.0
6.4
6.2
5.1
5.1
4.8
2.3*
4.0
4.9
5.1
5.1
5.0
4.7
2.3*
2.3*
4.7
4.8
4.9
3.7
4.2
VLS
2.3
4.3
3.0
1010-4
5.2
3.8
2.5
1010-6
6.4
4.5
4.4
1010-8
4.4
4.4
1010-2
2.9
4.1
5.0
4.9
4.0
Subject restarted ART; Subject was lost to follow-up
* log10 viral loads < log10(400) = 2.6 were replaced with log10(200)=2.3
June 4, 2007
2007 ICSA Applied Statistics Symposium
16
Other “Standard” Statistical Methods
 REML: Parametric repeated measures analysis
(PROC MIXED default).
» Assumptions include: multivariate normality,
properly modeled covariance matrix, and missing
values (if any) are missing at random (MAR).
 Weighted GEE: Extension of semiparametric
repeated measures analysis (generalized estimating
equations) to accommodate non-normality and MAR
data.
» Assumptions include: correct modeling of the dropout mechanism to estimate weights (inverse
probability of response) and missing values are MAR.
June 4, 2007
2007 ICSA Applied Statistics Symposium
17
Numerical Example: Hypothetical Data
using Weighted GEE
Trt
Group
Patient
Vaccine
1
2
3
4
5
6
7
8
9
10
Placebo
1
2
3
4
5
6
7
8
9
10
Wk 1
2.3* (1.0)
2.3* (1.0)
2.3* (1.0)
3.9 (1.0)
2.3* (1.0)
2.3* (1.0)
2.3* (1.0)
2.9 (1.0)
2.3* (1.0)
2.3* (1.0)
3.6 (1.0)
2.3* (1.0)
2.3* (1.0)
2.9 (1.0)
2.3* (1.0)
2.3* (1.0)
2.3* (1.0)
2.3* (1.0)
2.3* (1.0)
2.3* (1.0)
log10 viral load during the ART interruption phase
Wk 2
Wk 4
Wk 6
Wk 8
Wk 12
3.8 (1.0)
4.8 (1.1)
4.5 (1.2)
3.5 (1.4)
2.3* (1.6)
2.3* (1.0)
3.6 (1.0)
4.1 (1.1)
4.4 (1.3)
4.3 (1.6)
2.3* (1.0)
4.7 (1.0)
4.4 (1.2)
3.8 (1.3)
3.4 (1.6)
5.5 (1.0)
5.8 (1.1)
.
.
.
2.3* (1.0)
4.1 (1.0)
4.7 (1.1)
4.3 (1.3)
5.4 (1.6)
2.3* (1.0)
4.3 (1.0)
4.1 (1.1)
4.2 (1.3)
3.8 (1.6)
2.3* (1.0)
2.7 (1.0)
3.2 (1.1)
3.0 (1.2)
2.3* (1.4)
4.6 (1.0)
3.9 (1.1)
3.7 (1.2)
.
.
2.3* (1.0)
3.3 (1.0)
5.2 (1.1)
6.2 (1.3)
6.5 (1.9)
4.0 (1.0)
4.8 (1.1)
5.2 (1.2)
4.6 (1.5)
4.4 (1.8)
3.5 (1.0)
4.0 (1.1)
3.9 (1.1)
4.5 (1.3)
4.2 (1.6)
4.0 (1.0)
4.5 (1.1)
5.7 (1.2)
6.0 (1.5)
.
2.3* (1.0)
4.6 (1.0)
4.7 (1.2)
4.5 (1.4)
4.4 (1.7)
4.2 (1.0)
4.9 (1.1)
4.9 (1.2)
4.5 (1.4)
4.5 (1.8)
3.7 (1.0)
.
.
.
.
2.3* (1.0)
2.9 (1.0)
3.4 (1.1)
4.1 (1.2)
2.3* (1.5)
4.4 (1.0)
4.7 (1.1)
4.5 (1.2)
4.8 (1.4)
4.4 (1.8)
5.0 (1.0)
6.4 (1.1)
6.2 (1.3)
5.1 (1.8)
5.1 (2.3)
4.0 (1.0)
4.9 (1.1)
5.1 (1.2)
5.1 (1.5)
5.0 (1.9)
2.3* (1.0)
4.7 (1.0)
4.8 (1.2)
4.9 (1.4)
3.7 (1.8)
Wk 16
2.3* (1.8)
4.2 (1.9)
2.6 (1.9)
.
4.9 (2.2)
3.9 (1.9)
2.8 (1.5)
.
6.4 (3.0)
4.5 (2.3)
4.7 (2.0)
.
4.4 (2.1)
4.3 (2.2)
.
3.4 (1.6)
3.7 (2.2)
4.8 (3.0)
4.7 (2.5)
4.2 (2.1)
Subject restarted ART; Subject was lost to follow-up
* log10 viral loads < log10(400) = 2.6 were replaced with log10(200)=2.3
June 4, 2007
2007 ICSA Applied Statistics Symposium
18
A New Method: Two-step Approach
 A rank-based analysis after multiple imputation
(Mogg and Mehrotra, 2007).
 Step 1: Impute missing values (Rubin, 1987)
» Create M (= 20) complete data sets using SAS PROC MI
to impute.
– Assumptions include multivariate normality and MAR.
 Step 2: Rank-based analysis
» Calculate the numerator and denominator of a rankbased test statistic for each complete data set.
» Combine the M results to get a single p-value for
inference.
– Valid inference when assumptions above are violated as
long as imputations are rank preserving.
June 4, 2007
2007 ICSA Applied Statistics Symposium
19
A New Method: Two-step Approach (cont.)
 Two options for rank-based test after imputation:
1) WRS test applied to the VLS values. [MI  WRS]
2) Separate WRS tests at last two time points, combined
with equal weight. (MI-based extension of Wei-Lachin,
1984.) [MI  WL]
– Mann & Whitney (1947) proposed a rank test equivalent to the
WRS test: θ  [Pr(Yv  Yp ) - Pr(Yv  Yp )]  p  - p 
– In the multivariate setting, Wei and Lachin (1984) present a Tvariate generalization of this test:
# (Yvt  Ypt )# (Ypt  Yvt )
ˆ
θt 
 pˆ t  - pˆ t , (1  t  T)
nvtnpt
– The vectors θˆ and W'θˆ, where W is a vector of weights, are
ˆ (θ) and
asymptotically normal with covariance matrices Σ
W'Σˆ ( θ)W.
June 4, 2007
2007 ICSA Applied Statistics Symposium
20
Numerical Example (cont.)
Method
MI  WL
VLS
p-value
.174
MI  WRS
.198
REML
.198
WGEE
.410
Tied Worst Rank
.354
Untied Worst Rank
.354
LOCF
.328
MI = multiple imputation, WL = Wei-Lachin, WRS = Wilcoxon rank sum test
SAS PROC MIXED used for REML, SAS PROC GENMOD used for WGEE
June 4, 2007
2007 ICSA Applied Statistics Symposium
21
Details of Simulation Study
(based on Therapeutic POC Efficacy Trial)

2 groups (P=Placebo, V=Vaccine); 7 time points;
Total N = 120 (80 vaccine, 40 placebo)

Three data generating distributions:
1. MVN(,)
2. SCN = 0.9MVN(,) + 0.1MVN(,16) [stochastic mix]
3. MVT() with 3 d.f.
P = V under H0 std dev. = 0.65, Toeplitz corr. (0.8)
Under HA std dev. vaccine = 0.75

Under H0, VLS = 4.5 for P and V

Under HA, VLS = 4.5 for P and VLS = 4.0 for V

10,000 simulations, nominal  = 2.5% (1-tailed)
June 4, 2007
2007 ICSA Applied Statistics Symposium
22
Details of Simulation Study (cont.)


Combination of two monotone missing data mechanisms:
»
MAR: Data for a subject was set to missing (subject
went back on ART) with 90% probability if 2 consecutive
vRNA measurements > 150,000 copies/ml.
»
MCAR: On average, 10% of subjects in each treatment
group drop-out at a random time point (lost to follow-up).
% Missing Data by Study Week
Week
1
2
4
6
8
12
16
June 4, 2007
MVN
Placebo
Vaccine
0%
0%
3%
3%
4%
4%
9%
9%
23%
18%
29%
22%
33%
24%
MVT
Placebo
Vaccine
0%
0%
3%
3%
6%
6%
13%
13%
26%
22%
32%
26%
36%
29%
SCN
Placebo
Vaccine
0%
0%
3%
3%
6%
6%
11%
11%
25%
21%
31%
25%
36%
27%
2007 ICSA Applied Statistics Symposium
23
Simulation Results
Type I Error Rate (=2.5%)
VLS
Method
MVN
SCN
MVT
MI  WL
2.2
2.0
2.0
MI  WRS
1.7
1.6
1.6
REML
2.4
2.3
2.7
WGEE
(4.5)
(8.7)
(9.4)
Tied Worst Rank
2.1
2.8
2.8
Untied Worst Rank
2.0
2.7
2.5
LOCF
1.8
2.7
2.6
Result in parentheses if > 2.97% (> 3 std. errors above 2.5%); 10,000 simulations
MVN = Multivariate Normality; MVT = Multivariate t3;SCN = Symmetric Contaminated Normal
For WGEE, weights estimated using logistic regression with categorical time and previous
log10(vRNA) as covariates.
June 4, 2007
2007 ICSA Applied Statistics Symposium
24
Simulation Results (cont.)
 Severely inflated type I error for WGEE.
 Agrees with other published reports (Demirtas,
2004 and Preisser et al., 2002).
» WGEE performs poorly when drop-out model is not
correctly specified.
» Even with “reasonable” model for drop-out, WGEE
method breaks down here.
» Virtually impossible in practice to properly specify
missing data model!
June 4, 2007
2007 ICSA Applied Statistics Symposium
25
Simulation Results
Power
VLS
Method
MVN
SCN
MVT
MI  WL
92
80
73
MI  WRS
90
76
69
REML
95
71
65
WGEE
(81)
(54)
(49)
Tied Worst Rank
84
72
66
Untied Worst Rank
84
72
66
LOCF
88
58
53
Result in parentheses if > 2.97% (> 3 std. errors above 2.5%); 10,000 simulations
MVN = Multivariate Normality; MVT = Multivariate t3;SCN = Symmetric Contaminated Normal
For WGEE, weights estimated using logistic regression with categorical time and previous
log10(vRNA) as covariates.
June 4, 2007
2007 ICSA Applied Statistics Symposium
26
Simulation Results when Specifying
Correct Drop-out Model for WGEE
VLS
Method
MVN
SCN
MVT
Type I error rate (=2.5%)
MI  WL
2.6
2.4
2.1
WGEE
3.0
3.6
2.6
Power
MI  WL
96
85
81
WGEE
91
62
58
1,000 simulations; generated and modeled drop-out using logistic regression with categorical time
and previous log10(vRNA) as covariates.
MVN = Multivariate Normality; MVT = Multivariate t3;SCN = Symmetric Contaminated Normal
June 4, 2007
2007 ICSA Applied Statistics Symposium
27
Concluding Remarks
 WGEE: type I error can be severely inflated when
drop-out model is not correctly specified.
 Multiple imputation followed by a rank-based
analysis is robust and efficient. We recommend
MI  WL, especially for proof-of-concept clinical
trials in a variety of therapeutic areas.
 LOCF and “worst rank” single imputation methods
are (unfortunately) popular, but inefficient!
 REML: no imputation is required, but analysis is
inefficient with non-normal and censored data.
 SAS macro is available upon request.
June 4, 2007
2007 ICSA Applied Statistics Symposium
28
Motiviating Trial #2: ACTG A5197
Therapeutic POC Efficacy Trial
 Proposed analysis in Statistical Analysis Plan:
Untied Worst Rank
» Assign VLS = 10^10 – tlast to all “drop-outs”, where
tlast = time of dropout (penalizes earlier dropouts).
» Use Wilcoxon Rank Sum (WRS) test.
 No comments on analysis from CBER.
 Analysis targeted to be performed later this
year.
June 4, 2007
2007 ICSA Applied Statistics Symposium
29
Motiviating Trial #1:
Prophylactic POC Efficacy Trial
 Proposed analysis in Statistical Analysis Plan:
MI  WRS
» Create M (= 20) complete data sets using SAS PROC MI to
impute.
» Calculate the numerator and denominator of Wilcoxon rank sum
test for each complete data set.
» Combine the M results to get a single p-value for inference.
 CBER accepted the proposed strategy for the
primary analysis!
» Requested a sensitivity analysis using the Worst Rank
method: subjects who initiate ART considered “failures”.
 First interim analysis targeted to be performed later
this year.
June 4, 2007
2007 ICSA Applied Statistics Symposium
30
References
[1]
Emini E and Koff W (2004). Developing an AIDS Vaccine: Need, Uncertainty, Hope, Science, 304,
1913-1914.
[2]
Demirtas, H (2004). Assessment of Relative Improvement Due to Weights Within Generalized
Estimating Equations Framework for Incomplete Clinical Trials Data, Journal of Biopharmaceutical
Statistics, 14, 1085-1098.
[3]
Hogan et al. (2004). Handling drop-out in longitudinal studies, Statistics in Medicine, 23, 1455-1497.
[4]
Johnston M and Fauci AS (2007). An HIV Vaccine – Evolving Concepts, New England Journal of
Medicine, 356, 2073-2081.
[5]
Liang and Zeger (1986). Longitudinal data analysis using generalized linear models, Biometrika, 73,
13-22.
[6]
Preisser et al. (2002). Performance of weighted estimating equations for longitudinal binary data
with drop-outs missing at random, Statistics in Medicine, 21, 3035-3054.
[7]
Mogg R and Mehrotra DV (2007). Analysis of antiretroviral immunotherapy trials with potentially
non-normal and incomplete longitudinal data, Statistics in Medicine, 26, 484-497.
[8]
Robins et al. (1995). Analysis of Semiparametric Regression Models for Repeated Outcomes in the
Presence of Missing Data, Journal of the American Statistical Association, 90, 106-121.
[9]
Rubin, DB (1987). Multiple Imputation for Non-Response in Surveys, New York: John Wiley and Sons
Inc.
[10]
Thall, PF and Lachin, JM (1988). Analysis of recurrent events: nonparametric methods for random
interval count data, Journal of the American Statistical Association, 83, 339-347.
[11]
Wei, LJ and Lachin, JM (1984). Two-sample Asymptotically Distribution-Free Tests for Incomplete
Multivariate Observations, Journal of the American Statistical Association, 79, 653-661.
June 4, 2007
2007 ICSA Applied Statistics Symposium
31