Transcript Bayesian Subgroup Analysis Gene Pennello, Ph.D. Division of Biostatistics, CDRH, FDA Disclaimer: No official support or endorsement of this presentation by the Food &

Bayesian Subgroup Analysis
Gene Pennello, Ph.D.
Division of Biostatistics, CDRH, FDA
Disclaimer: No official support or endorsement of this presentation
by the Food & Drug Administration is intended or should be inferred.
FIW 2006
September 28, 2006
1
Outline
Frequentist Approaches
Bayesian Hierarchical Model Approach
Bayesian Critical Boundaries
Directional Error Rate
Power
Summary
2
Frequentist Approaches
Strong control of FWE
Weak control of FWE
Gatekeeper: test subgroups (controlling
FWE) only if overall effect is significant
Confirmatory Study: confirm with a new
study in which only patients in the subgroup
are enrolled.
3
Concerns with Frequentist
Approaches
Limited power of FWE procedures
Powerlessness of gatekeeper if overall
effect is insignificant
Discourages multiple hypothesis testing,
thereby impeding progress.
Confirmation of findings, one at a time,
impedes progress.
4
“No aphorism is more frequently repeated in
connection with field trials, than that we
must ask Nature few questions, or, ideally,
one question at a time. The writer is
convinced that this view is wholly mistaken.
Nature, he suggests, will best respond to a
logical and carefully thought out
questionnaire …”
Fisher RA, 1926, The arrangement of field
experiments, Journal of the Ministry of
Agriculture, 33, 503-513.
5
A Bayesian Approach
Adjust subgroup inference for its
consistency with related results.
Choices
Build prior on subgroup
relationships.
Invoke relatedness by modeling a
priori exchangeability of effects.
6
Prior Exchangeability Model
Subgroups: Labels do not inform on
magnitude or direction of main
subgroup effects.
Treatments: Labels do not inform for
main treatment effects.
Subgroup by Treatment Interactions:
Labels do not inform for treatment
effects within subgroups.
7
Prior Exchangeability Model
Exchangeability modeled with
random effects models.
Key Result:
Result for a subgroup is related to
results in other subgroups
because effects are iid draws from
random effect distribution.
8
Bayesian Two-Way Normal
Random Effects Model
y ij ~ N ( ij , / n), i  1,
a, j  1,
2
b
fs /  ~  ( f ), f  ab(r  1)
2
2
2
ij     i   j   ij
 i ~ N (0,  ),  j ~ N (0,  ),  ij ~ N (0,  )
2
2
2
Jeffreys prior on (  , ,  ,  ,  )
2
2
2
2
9
Bayesian Two-Way Normal
Random Effects Model
Note: In prior distribution,
Pr(zero effect) = 0
That is, only directional
(Type III) errors can be
made here.
10
Known Variances Inference
Subgroup Problem:
Posterior
12, j  1 j  2 j
12, j | y,  2 ~
2
N (S A d12  SC dC , (S A  (b 1)SC ) d
d12  y1  y2 ,
/ b)),
dC  d12, j  d12
S A  1 1/  A ,  A   A2 /  2 ,  A2   C2  br2
SC  1 1/ C , C   C2 /  2 ,  C2  r 2   2
11
Bayes Decision Rule
Declare difference > 0 if

Pr 12, j  0 | y, 
Let
2
  1 / 2
z12, j  d12, j /  2 / r , d12, j  y1 j  y 2 j
z12,  d12, /  2 / br , d12,  y1  y 2
12
Bayes Critical z Value

Pr 12, j  0 | y, 
2
  0.975
1/ 2
z12, j
z1 / 2  S A b  1 




b 
SC  bSC
if
z12,  S A 

  1
b  SC 
Linear dependence on standardized
marginal treatment effect z12,
↑ with ↓ interaction (↑ S A / SC )
↓with ↑ # subgroups b.
13
Bayes Critical z Value
Full Interaction Case: S A  SC
Critical z value
z12, j 
z1 / 2
SC
↑ with ↓ true F ratio C measuring
heterogeneity of interaction effects.
14
Bayes Critical z Value
No Interaction Case:
Critical z value
z12, 
SC  0
z1 / 2
SA
Power can be > than for unadjusted
5% level z test for subgroup if true F
ratio  A measuring heterogeneity
of treatment effects is large.
15
16
17
18
Full Bayes Critical t Boundaries
19
Directional Error Control
Directional FDR controlled at A under 0-1A loss function for correct decision, incorrect
decision, and no decision (Lewis and
Thayer, 2004).
Weak control of FW directional error rate,
loosely speaking, because of dependence
on F ratio for interaction.
20
Comparisons of Sample Size to
Achieve Same Power
ULSD = 5% level unadjusted z test
Bonf = Bonferonni 5% level z test
HM = EB hierarchical model test
0  rULSD / rHM  b
0  rBonf / rULSD 
0  rBonf / rHM  b
z1 / 2b  z1 
z1 / 2  z1 
z1 / 2b  z1 
z1 / 2  z1 
21
EX. Beta-blocker for Hypertension
Losartan versus atenolol randomized trial
Endpoint: composite of Stroke/ MI/ CV Death
N=9193
losartan (4605),
atenolol (4588)
# Events
losartan (508),
atenolol (588)
80% European Caucasians 55-80 years old.
http://www.fda.gov/cder/foi/label/2003/020386s032lbl.pdf
22
EX. Beta-blocker for Hypertension
Cox Analysis
N
Overall
logHR SE
9193
Race Subgroups
Non-Black
8660 -.19
Black
533 .51
HR (95% CI)
p val
.87 ( .77, .98) 0.021
.06
.24
.83 ( .73, .94) 0.003
1.67 (1.04,2.66) 0.033
Is Finding Among Blacks Real or a
Directional Error?
23
EX. Beta-blocker for Hypertension
Bayesian HM Analysis
logHR
se/sd HR (95%CI)
p val Pr>0
-.19
-.18
.06
.06
0.83 ( .73 .94)
0.84 ( .74, .95)
0.003 0.001
0.003
.51
.38
.24
.27
1.67 (1.04, 2.67) 0.033 0.983
1.47 (0.87, 2.44)
0.914
non-black
frequentist
Bayesian
black
frequentist
Bayesian
Bayesian analysis cast doubt on finding, but
is predicated on not expecting a smaller
effect in blacks a priori.
24
Suggested Strategy
Planned subgroup analysis
Bayesian adjustment using above HM or
similar model
Pennello,1997, JASA
Simon, 2002, Stat. Med.
Dixon and Simon, 1991, Biometrics
25
Suggested Strategy
Unplanned subgroup analysis
Ask for confirmatory trial of subgroup.
Posterior for treatment effect in the
subgroup given by HM is prior for
confirmatory trial.
Prior information could reduce size of
confirmatory trial.
26
Summary
Bayesian approach presented here
considers trial as a whole, adjusts for
consistency in finding over subgroups.
Error rate is not rigidly pre-assigned
Can vary from conservative to liberal
depending on interaction F ratio and
marginal treatment effect.
Power gain can be substantial.
Control for directional error rate is made only
when warranted.
27
References
Dixon DO and Simon R (1991), Bayesian subset analysis,
Biometrics, 47, 871-881.
Lewis C and Thayer DT (2004), A loss function related to
the FDR for random effects multiple comparisons,
Journal of Statistical Planning and Inference 125, 49-58.
Pennello GA (1997), The k-ratio multiple comparisons
Bayes rule for the balanced two-way design, J. Amer.
Stat. Assoc., 92, 675-684
Simon R (2002), Bayesian subset analysis: appliation to
studying treatment-by-gender interactions, Statist. Med.,
21, 2909-2916.
Sleight P (2000), Subgroup analyses in clinical trials: fun to
look at but don’t believe them!, Curr Control Trials
28
Cardiovasc Med, 1, 25-27.
Other Notable References
Berry DA, 1990, Subgroup Analysis
(correspondence) Biometrics, 46, 12271230.
Gonen M, Westfall P, Johnson WO (2003),
Bayesian multiple testing for two-sample
multivariate endpoints, Biometrics, 59, 7682.
Westfall PH, Johnson WO, and Utts JM
(1997), A Bayesian perspective on the
Bonferroni adjustment, 84, 419-427
29