Transcript phrma_adapt_fda_sept27_09122006

Adaptive Clinical Trials
Short Course Presenters:
Michael Krams,
M.D.,
Vladimir Dragalin, Ph.D.,
Jeff Maca,
Ph.D.,
Keaven Anderson, Ph.D.,
Paul Gallo,
Ph.D.,
FDA/Industry
Statistics Workshop
Wyeth Research
Wyeth Research
Novartis Pharmaceuticals
Merck Research Laboratories
Novartis Pharmaceuticals
Adaptive Designs Working Group
September 27, 2006
Washington, D.C.
Short Course Outline



Topics
General Introduction
Terminology and Classification
Adaptive Seamless Design
Break



Sample Size Reestimation
and Group Sequential Design
Logistic, Operational and
Regulatory Issues
General Discussion
Time
Presenter
10 min Mike Krams
40 min Vlad Dragalin
40 min Jeff Maca
20 min
40 min
Keaven Anderson
40 min
20 min
Paul Gallo
Adaptive Designs Working Group
2
Adaptive Designs
General Introduction
Michael Krams
Wyeth Research
On behalf of the PhRMA Adaptive Designs Working Group
FDA/Industry
Statistics Workshop
Adaptive Designs Working Group
September 27, 2006
Washington, D.C.
PhRMA Adaptive Designs Working Group

Co-Chairs:
Michael Krams
Brenda Gaydos

Authors:
Keaven Anderson
Suman Bhattacharya
Alun Bedding
Don Berry
Frank Bretz
Christy Chuang-Stein
Vlad Dragalin
Paul Gallo
Brenda Gaydos
Michael Krams
Qing Liu
Jeff Maca
Inna Perevozskaya
Jose Pinheiro
Judith Quinlan

Members:
Carl-Fredrik Burman
David DeBrota
Jonathan Denne
Greg Enas
Richard Entsuah
Andy Grieve
David Henry
Tony Ho
Telba Irony
Larry Lesko
Gary Littman
Cyrus Mehta
Allan Pallay
Michael Poole
Rick Sax
Jerry Schindler
Michael D Smith
Marc Walton
Sue-Jane Wang
Gernot Wassmer
Pauline Williams
Adaptive Designs Working Group
4
Vision


To establish a dialogue between statisticians, clinicians,
regulators and other lines within the Pharmaceutical
Industry, Health Authorities and Academia,
with a goal to contribute to developing a consensus
position on when and how to consider the use of
adaptive designs in clinical drug development.
Adaptive Designs Working Group
5
Mission

To facilitate the implementation adaptive designs, but only where
appropriate

To contribute to standardizing the terminology and classification in the
rapidly evolving field of adaptive designs

To contribute to educational and information sharing efforts on adaptive
designs

To interact with experts within Health Authorities (FDA, EMEA, and others)
and Academia to sharpen our thinking on defining the scope of adaptive
designs

To support our colleagues in health authorities in their work towards the
formulation of regulatory draft guidance documents on the topic of
adaptive designs.
Adaptive Designs Working Group
6
Executive Summary of White Paper
Adaptive Designs Working Group
7
Full White Paper - to appear in DIJ in Nov 2006
Adaptive Designs Working Group
8
PhRMA/FDA conference on Adaptive Design:
Opportunities, Challenges and Scope in Drug Development

Nov 13/14th, 2006


Marriott Bethesda North Hotel & Conference Center
North Bethesda, MD 20852
Program committee
Dennis Erb
Brenda Gaydos
Michael Krams
Walt Offen
Frank Shen
Luc Truyen
Merck
Eli Lilly
Wyeth, Co-Chair
Eli Lilly, Co-Chair
BMS
J&J
Greg Campbell FDA CDRH
Shirley Murphy FDA CDER
Robert O’Neill
FDA CDER
Robert Powell
FDA CDER
Marc Walton
FDA CDER
Sue Jane Wang FDA CDER
Adaptive Designs Working Group
9
Adaptive Designs Working Group
10
Adaptive Designs
Terminology and Classification
Vlad Dragalin
Biomedical Data Sciences
GlaxoSmithKline
(currently with Wyeth Research)
FDA/Industry
Statistics Workshop
Adaptive Designs Working Group
September 27, 2006
Washington, D.C.
Primary PhRMA references

PhRMA White Paper section:

Dragalin V. Adaptive Designs: Terminology and
Classification. Drug Information Journal. 2006; 40(4):
425-436.
Adaptive Designs Working Group
12
Outline



Definition and general structure of adaptive
designs
Classification of adaptive designs in drug
development
Achieving the goals
Adaptive Designs Working Group
13
What are Adaptive Designs?
Flexible
Multi-stage
Response-driven
Dynamic
Sequential


Self-designing
ADAPTIVE
Novel
An adaptive design should be adaptive by "design" not
an adhoc change of the trial conduct and analysis
Adaptation is a prospective design feature, not a
remedy for poor planning
Adaptive Designs Working Group
14
What are Adaptive Designs?
Adaptive Plan
… not Adaptive Plane
Adaptive Designs Working Group
15
Definition
Adaptive Design

uses accumulating data
to decide on how to
modify aspects of the
study

without undermining
the validity and integrity
of the trial
Validity means

providing correct statistical
inference (such as adjusted pvalues, unbiased estimates and
adjusted confidence intervals, etc)

assuring consistency between
different stages of the study

minimizing operational bias
Integrity means

providing convincing results to a
broader scientific community

preplanning, as much as possible,
based on intended adaptations

maintaining confidentiality of data
Adaptive Designs Working Group
16
General Structure

An adaptive design requires the trial to be conducted in several
stages with access to the accumulated data

An adaptive design may have one or more rules:


Allocation Rule: how subjects will be allocated to available arms

Sampling Rule: how many subjects will be sampled at next stage

Stopping Rule: when to stop the trial (for efficacy, harm, futility)

Decision Rule: the terminal decision rule and interim decisions
pertaining to design change not covered by the previous three
rules
At any stage, the data may be analyzed and next stages redesigned
taking into account all available data
Adaptive Designs Working Group
17
Examples




Group Sequential Designs: only Stopping Rule
Response Adaptive Allocation: only Allocation Rule
Sample Size Re-assessment: only Sampling Rule
Flexible Designs:




Adaptive AR: changing the randomization ratio
Adaptive SaR: the timing of the next IA
Stopping Rule
Adaptive DR: changing the target treatment difference;
changing the primary endpoint; varying the form of the primary
analysis; modifying the patient population; etc
Adaptive Designs Working Group
18
Allocation Rules

Fixed (static) AR:




Randomization used to achieve balance in all prognostic
factors at baseline
Complete randomization uses equal allocation probabilities
Stratification improves the randomization
Adaptive (dynamic) AR:


Response-adaptive randomization uses interim data to
unbalance the allocation probabilities in favor of the “better”
treatment(s): urn models, RPW, doubly adaptive biased coin
design
Bayesian AR alters the allocation probabilities based on
posterior probabilities of each treatment arm being the “best”
Adaptive Designs Working Group
19
Sampling Rules

Sample size re-estimation (SSR)



Traditional Group Sequential Designs


Variable sample sizes per stage (but do not depend on
observations)
Sequentially Planned Decision Procedures


Fixed sample sizes per stage
Error Spending Approach


Restricted sampling rule
Blinded SSR or Unblinded SSR based on estimate of nuisance
parameter
Future stage sample size depends on the current value of test
statistic
Flexible SSR uses also the estimated treatment effect
Adaptive Designs Working Group
20
Stopping Rules

Early Stopping based on Boundary Crossing




Stochastic Curtailment



Superiority
Harm
Futility
Conditional power
Predictive power
Bayesian Stopping Rules


Based on posterior probabilities of hypotheses
Complemented by making predictions of the possible
consequences of continuing
Adaptive Designs Working Group
21
Decision Rules

Changing the test statistics




Redesigning multiple endpoints





Adaptive scores in trend test or under non proportional hazards
Adaptive weight in location-scale test
Including a covariate that shows variance reduction
Changing their pre-assigned hierarchical order in multiple testing
Updating their correlation in reverse multiplicity situation
Switching from superiority to non-inferiority
Changing the hierarchical order of hypotheses
Changing the patient population

going forward either with the full population or with a prespecified subpopulation
Adaptive Designs Working Group
22
Classification
Target to
tractable
hit
to
candidat
SINGLE ARM TRIALS e
Disease
selection
Target Family
selection
Candida
te
selection
to FTIM
Compound Progression Stages
FTIM to Commit
to PoC/Phase II
Phase II to
Commit
to Phase III
Phase III to launch
Lifecycl
e
Manage
-ment
Two-stage Designs
Screening Designs
TWO-ARM TRIALS
Group Sequential Designs
Information Based Designs
Adaptive GSD (Flexible Designs)
MULTI-ARM TRIALS
Bayesian Designs
Group Sequential Designs
Flexible Designs
DOSE-FINDING STUDIES
Dose-escalation designs
Dose-finding designs (Flexible Designs)
Adaptive Model-based Dose-finding
SEAMLESS DESIGNS
Dose-escalation based on efficacy/toxicity
Learning/Confirming in Phase II/III
Adaptive Designs Working Group
23
Two-Stage Designs





Objective: single-arm studies using short-term
endpoints; hypothesis testing about some minimal
acceptable probability of response
Gehan design: early stopping for futility; sample size of
the 2nd stage gives a specified precision for response
rate
Group sequential designs: Fleming (1982), Simon (1989)
Adaptive two-stage design: Banerjee&Tsiatis (2006)
Bayesian designs: Thall&Simon (1994)
Adaptive Designs Working Group
24
Screening Designs

Objective: adaptive design for the entire screening
program


Minimize the shortest time to identify the “promising” compound
Subject to the given constraints on type I and type II risks for
the entire screening program





type I risk = Pr(screening procedures stops with a FP compound)
type II risk= Pr(any of the rejected compounds is a FN compound)
Two-stage design (Yao&Venkatraman, 1998)
Adaptive screening designs (Stout and Hardwick, 2002)
Bayesian screening designs (Berry, 2001)
Adaptive Designs Working Group
25
Classification
Target to
tractable
hit
to
candidat
SINGLE ARM TRIALS e
Disease
selection
Target Family
selection
Candida
te
selection
to FTIM
Compound Progression Stages
FTIM to Commit
to PoC/Phase II
Phase II to
Commit
to Phase III
Phase III to launch
Lifecycl
e
Manage
-ment
Two-stage Designs
Screening Designs
TWO-ARM TRIALS
Group Sequential Designs
Information Based Designs
Adaptive GSD (Flexible Designs)
MULTI-ARM TRIALS
Bayesian Designs
Group Sequential Designs
Flexible Designs
DOSE-FINDING STUDIES
Dose-escalation designs
Dose-finding designs (Flexible Designs)
Adaptive Model-based Dose-finding
SEAMLESS DESIGNS
Dose-escalation based on efficacy/toxicity
Learning/Confirming in Phase II/III
Adaptive Designs Working Group
26
Fully Sequential Designs

Objective: testing two hypotheses with given
significance level and power at the prespecified
alternative





AR: fixed randomization
SaR: after each observation
StR: boundary crossing (e.g. SPRT, repeated significance test,
triangular test)
DR: final decision - to accept or reject the null hypothesis
References: Siegmund (1985); Jennison&Turnbull
(2000)
Adaptive Designs Working Group
27
Group Sequential Designs

Objective: testing two hypotheses with given significance level and
power at the specified alternative, prefixed maximum sample size


AR: fixed randomization
SaR: after a fixed number (a group) of observations,



StR: boundary crossing






or using error-spending function,
or using “Christmas-tree” adjustment
Haybittle, Pocock, O’Brien-Fleming type
linear boundaries
error-spending families
conditional power, stochastic curtailment
DR: final decision - to accept or reject the null hypothesis
References: Jennison&Turnbull (2000); Whitehead (1997)
Adaptive Designs Working Group
28
Information Based Designs

Objective: testing two hypotheses with given
significance level and power at the specified alternative,
prefixed maximum information





AR: fixed randomization
SaR: after fixed increments of information
StR: boundary crossing as for Group Sequential Designs
DR: adjust maximum sample size based on interim information
about nuisance parameters
References: Mehta&Tsiatis (2001); East (2005)
Adaptive Designs Working Group
29
Adaptive GSD (Flexible Designs)

Objective: testing two hypotheses with given
significance level and power at the specified alternative
or adaptively changing the alternative at which a
specified power is to be attained





AR: fixed or adaptive randomization
SaR: sample size of the next stage depends on results at the
time of interim analysis
StR: p-value combination, conditional error, variance-spending
DR: adapting alternative hypothesis, primary endpoint, test
statistics, inserting or skipping IAs
References: Bauer; Brannath et al; Müller&Schäfer; Fisher
Adaptive Designs Working Group
30
Classification
Target to
tractable
hit
to
candidat
SINGLE ARM TRIALS e
Disease
selection
Target Family
selection
Candida
te
selection
to FTIM
Compound Progression Stages
FTIM to Commit
to PoC/Phase II
Phase II to
Commit
to Phase III
Phase III to launch
Lifecycl
e
Manage
-ment
Two-stage Designs
Screening Designs
TWO-ARM TRIALS
Group Sequential Designs
Information Based Designs
Adaptive GSD (Flexible Designs)
MULTI-ARM TRIALS
Bayesian Designs
Group Sequential Designs
Flexible Designs
DOSE-FINDING STUDIES
Dose-escalation designs
Dose-finding designs (Flexible Designs)
Adaptive Model-based Dose-finding
SEAMLESS DESIGNS
Dose-escalation based on efficacy/toxicity
Learning/Confirming in Phase II/III
Adaptive Designs Working Group
31
Bayesian Designs

Objective: to use the posterior probabilities of hypotheses of
interest as a basis for interim decisions (Proper Bayesian) or to
explicitly assess the losses associated with consequences of
stopping or continuing the study (Decision-theoretic Bayesian)





AR: equal randomization or play-the-winner (next patient is allocated
to the currently superior treatment) or bandit designs (minimizing the
number of patients allocated to the inferior treatment)
SaR: not specified
StR: not formally pre-specified stopping criterion, or using a skeptical
prior for stopping for efficacy and an enthusiastic prior for stopping for
futility, or using backwards induction
DR: update the posterior distribution; formal incorporation of external
evidence; inference not affected by the number and timing of IAs
References: Berry (2001, 2004); Berry et al. (2001); Spiegelhalter
et al. (2004).
Adaptive Designs Working Group
32
Pairwise comparisons with GSD

Objective: compare multiple treatments with a control;
focus on type I error rate rather than power




A simple Bonferroni approximation is only slightly conservative
Treatments may be dropped in the course of the trial if they are
significantly inferior to others
“Step-down” procedures allow critical values for remaining
comparisons to be reduced after some treatments have been
discarded
References: Follmann et al (1994)
Adaptive Designs Working Group
33
p-value combination tests



Objective: compare multiple treatments with a control
in a two-stage design allowing integration of data from
both stages in a confirmatory trial
Focus: control of multiple (familywise) Type I error level
Great flexibility:





General distributional assumptions for the endpoints
General stopping rules and selection criteria
Early termination of the trial
Early elimination of treatments due to lack of efficacy or to
safety issues or for ethical/economic reasons
References: Bauer&Kieser (1994); Liu&Pledger (2005)
Adaptive Designs Working Group
34
Classification
Target to
tractable
hit
to
candidat
SINGLE ARM TRIALS e
Disease
selection
Target Family
selection
Candida
te
selection
to FTIM
Compound Progression Stages
FTIM to Commit
to PoC/Phase II
Phase II to
Commit
to Phase III
Phase III to launch
Lifecycl
e
Manage
-ment
Two-stage Designs
Screening Designs
TWO-ARM TRIALS
Group Sequential Designs
Information Based Designs
Adaptive GSD (Flexible Designs)
MULTI-ARM TRIALS
Bayesian Designs
Group Sequential Designs
Flexible Designs
DOSE-FINDING STUDIES
Dose-escalation designs
Dose-finding designs (Flexible Designs)
Adaptive Model-based Dose-finding
SEAMLESS DESIGNS
Dose-escalation based on efficacy/toxicity
Learning/Confirming in Phase II/III
Adaptive Designs Working Group
35
Dose-escalation designs

Objective: target the MTD (Phase I) or the best safe
dose (Phase I/II) or find the therapeutic window

AR: non-parametric (3+3 rule, up-and-down)









or model-based (Continual Reassessment Methods)
or Escalation With Overdose Control (EWOC)
or Bayesian Decision Design
or Bayesian Optimal Design
or Penalized Adaptive D-optimal Design
SaR: cohorts of fixed size or in two stages (Storer design)
StR: no early stopping or stopping by design (e.g. 3+3 rule)
DR: update model parameters (for model-based AR)
References: O’Quigley et al.; Babb et al.; Edler; O’Quigley
Adaptive Designs Working Group
36
Adaptive Model-based Dose-finding

Objective: find the optimal dose; working model for the
dose-response; dose sequence identified in advance





AR: Bayesian (based on predictive probabilities: smallest
average posterior variance) or frequentist (based on optimal
experimental design: maximum information per cost)
SaR: cohorts of fixed size or after each observation
StR: stopping for futility or when the optimal dose for
confirmatory stage is sufficiently well known (estimation!)
DR: update model parameters, Bayesian predictions of longterm endpoint using a longitudinal model
References: Berry et al. (2001); Dragalin&Fedorov;
Fedorov&Leonov
Adaptive Designs Working Group
37
Adaptive Dose-finding (Flexible Designs)


Objective: establishing a dose-response relationship or
combining Phase II/III using p-value combination tests

AR: drop or add doses

SaR: sample size reassessment for the next stage

StR: early stopping for futility or early termination of some
inferior doses

DR: adapting hypotheses, primary endpoint, test statistics,
inserting or skipping IAs
References: Bauer&Kohne; Lehmacher et al
Adaptive Designs Working Group
38
Classification
Target to
tractable
hit
to
candidat
SINGLE ARM TRIALS e
Disease
selection
Target Family
selection
Candida
te
selection
to FTIM
Compound Progression Stages
FTIM to Commit
to PoC/Phase II
Phase II to
Commit
to Phase III
Phase III to launch
Lifecycl
e
Manage
-ment
Two-stage Designs
Screening Designs
TWO-ARM TRIALS
Group Sequential Designs
Information Based Designs
Adaptive GSD (Flexible Designs)
MULTI-ARM TRIALS
Bayesian Designs
Group Sequential Designs
Flexible Designs
DOSE-FINDING STUDIES
Dose-escalation designs
Dose-finding designs (Flexible Designs)
Adaptive Model-based Dose-finding
SEAMLESS DESIGNS
Dose-escalation based on efficacy/toxicity
Learning/Confirming in Phase II/III
Adaptive Designs Working Group
39
Seamless Designs

Two-stage adaptive designs




1st Stage: treatment (dose) selection – “learning”
2nd Stage: comparison with control – “confirming”
Treatment selection may be based on a short-term
endpoint (surrogate), while confirmation stage uses a
long-term (clinical) endpoint
2nd Stage data and the relevant groups from 1st Stage
data are combined in a way that


Guarantees the Type I error rate for the comparison with
control
Produces efficient unbiased estimates and confidence intervals
with correct coverage probability
Adaptive Designs Working Group
40
Selection and testing


Objective: to select the “best” treatment in the 1st stage
and proceed to the 2nd stage to compare with control
Focus:




Method includes:



overall type I error rate is maintained (TSE)
trial power is also achieved (ST)
selection is based on surrogate (or short-term) endpoint (TS)
early termination of the whole trial
early elimination of inferior treatments
References: Thall,Simon&Ellenberg; Stallard&Todd;
Todd&Stallard
Adaptive Designs Working Group
41
Bayesian model-based designs



Objective: adaptive dose ranging within a confirmatory
trial
Focus: efficient learning, effective treatment of patients
in the trial
Method includes:





AR: to maximize information about dose response
SaR: Frequent analysis of the data as it accumulates
Seamless switch to confirmatory stage without stopping
enrollment in a double-blind fashion
Use of longitudinal model for prediction of the clinical endpoint
References: Berry et al; Inoue et al
Adaptive Designs Working Group
42
Classification
Target to
tractable
hit
to
candidat
SINGLE ARM TRIALS e
Disease
selection
Target Family
selection
Candida
te
selection
to FTIM
Compound Progression Stages
FTIM to Commit
to PoC/Phase II
Phase II to
Commit
to Phase III
Phase III to launch
Lifecycl
e
Manage
-ment
Two-stage Designs
Screening Designs
TWO-ARM TRIALS
Group Sequential Designs
Information Based Designs
Adaptive GSD (Flexible Designs)
MULTI-ARM TRIALS
Bayesian Designs
Group Sequential Designs
Flexible Designs
DOSE-FINDING STUDIES
Dose-escalation designs
Dose-finding designs (Flexible Designs)
Adaptive Model-based Dose-finding
SEAMLESS DESIGNS
Dose-escalation based on efficacy/toxicity
Learning/Confirming in Phase II/III
Adaptive Designs Working Group
43
Achieving the goals

The objective of a clinical trial may be either




to target the MTD or MED or to find the therapeutic range
or to determine the OSD (Optimal Safe Dose) to be
recommended for confirmation
or to confirm efficacy over control in Phase III clinical trial
This clinical goal is usually determined by




the clinicians from the pharmaceutical industry
practicing physicians
key opinion leaders in the field, and
the regulatory agency
Adaptive Designs Working Group
44
Achieving the goals

Once agreement has been reached on the objective, it
is the statistician's responsibility to provide the
appropriate design and statistical inferential structure
required to achieve that goal
Adaptive Designs Working Group
45
Achieving the goals



There are plenty of
available designs on
statistician’s shelf
The greatest challenge
is their implementation
Adaptive designs have
much more to offer than
the rigid conventional
parallel group designs in
clinical trials
Adaptive Designs Working Group
46
References















Babb J, Rogatko A, Zacks S. (1998). Cancer phase I clinical trials: efficient dose escalation with overdose
control. Stat. Med., 17, 1103-1120.
Bauer P. Statistical methodology relevant to the overall drug development program. Drug Inf J. 2003;37: 81-89.
Bauer P, Kieser M. Combining different phases in the development of medical treatments within a single trial.
Statistics in Medicine 1999;18: 1833-1848.
Bauer P, Köhne K. Evaluation of experiments with adaptive interim analyses. Biometrics 1994;50: 1029-1041.
Bauer P, Röhmel J. An adaptive method for establishing a dose-response relationship. Statistics in Medicine
1995;14: 1595-1607.
Berry D. Adaptive trials and Bayesian statistics in drug development. Biopharmaceutical Report 2001; 9:1-11.
(with comments).
Berry D. Bayesian statistics and the efficiency and ethics of clinical trials. Statistical Science 2004; 19:175-187.
Berry D., Mueller P., Grieve A.P., Smith M., Parke T., Blazek R. Mitchard N., Krams M. Adaptive Bayesian
designs for dose-ranging drug trials. 2002; 162:99-181. In. Gatsonis C, Carlin B, Carriquiry A (Eds) "Case
Studies in Bayesian Statistics V", New-York: Springer.
Brannath W, Posch M, Bauer P. Recursive combination tests. JASA 2002;97: 236-244.
Dragalin V, Fedorov V. Adaptive model-based designs for dose-finding studies. Journal of Statistical Planning
and Inference, 2006; 136:1800-1823.
East. Software for the design and interim monitoring of group sequential clinical trials, 2005. Cytel Software
Corporation.
Edler L. Overview of Phase I Trials. 2001:1-34. In J. Crowley (Ed) “Handbook of Statistics in Clinical Oncology”.
Marcel Dekker, NY
Fedorov V, Leonov S. Response driven designs in drug development. 2005, In: Wong, W.K., Berger, M. (eds.),
"Applied Optimal Designs", Wiley.
Follman DA, Proschan MA, Geller NL. Monitoring pairwise comparisons in multi-armed clinical trials. Biometrics
1994; 50: 325-336.
Gould L. Sample-size re-estimation: recent developments and practical considerations. Statistics in Medicine
2001; 20:2625-2643.
Adaptive Designs Working Group
47
References
















Inoue LYT, Thall PF, Berry D. Seamlessly expanding a randomized Phase II trial to Phase III. Biometrics. 2002; 58:
823-831.
Jennison C, Turnbull BW. Group Sequential Methods with Applications to Clinical Trials. Chapman & Hall, Boca
Raton, London, New York, Washington, D.C., 2000.
Lehmacher W, Kieser M, Hothorn L. Sequential and multiple testing for dose-response analysis. Drug Inf. J.
2000;34: 591-597.
Liu Q, Pledger GW. Phase 2 and 3 combination designs to accelerate drug development. JASA 2005; 100:493-502
Mehta CR, Tsiatis AS. Flexible sample size considerations using information based interim monitoring . Drug Inf. J.
2001;35: 1095-1112.
Müller HH, Schäfer H. Adaptive group sequential designs for clinical trials: Combining the advantages of adaptive
and of classical group sequential approaches. Biometrics 2001; 57: 886-891.
O'Quigley J, Pepe M, Fisher L. (1990). Continual reassessment method: a practical design for phase I clinical trials
in cancer. Biometrics 46: 33—48
O'Quigley J. Dose-finding designs using continual reassessment method. 2001:35-72. In J. Crowley (Ed)
“Handbook of Statistics in Clinical Oncology”. Marcel Dekker, NY
Proschan M. Two-stage sample size re-estimation based on nuisance parameter: a review. JBS 2005; 15: 559-574
Thall PF, Simon R, Ellenberg S. A two-stage design for choosing among several experimental treatments and a
control in clinical trials. Biometrics 1989; 45: 537-547.
Todd S, Stallard N. A new clinical trial design combining Phase 2 and 3: sequential designs with treatment selection
and a change of endpoint. Drug Inf J. 2005; 39:109-118.
Rosenberger W.F, Lachin J.M. Randomization in Clinical Trials: Theory and Practice. 2002, Wiley
Schwartz TA, Denne JS. Common threads between sample size recalculation and group sequential procedures.
Pharmaceut. Statist. 2003; 2: 263-271.
Siegmund D. Sequential Analysis. Tests and Confidence Intervals. Springer, New York, 1985.
Spiegelhalter D.J., Abrams K.R., Myles J.P. Bayesian Approaches to Clinical Trials and Health-Care Evaluation.
Wiley, 2004.
Whitehead J. The Design and Analysis of Sequential Clinical Trials. 2nd ed. Wiley, New York, 1997.
Adaptive Designs Working Group
48
Adaptive Designs Working Group
49
Adaptive Seamless Designs for
Phase IIb/III Clinical Trials
Jeff Maca, Ph.D.
Assoc. Director, Biostatistics
Novartis Pharmaceuticals
FDA/Industry
Statistics Workshop
Adaptive Designs Working Group
September 27, 2006
Washington, D.C.
Primary PhRMA references

PhRMA White Paper sections:

Maca J, Bhattacharya S, Dragalin V, Gallo P, and
Krams M. Adaptive Seamless Phase II/III Designs
– Background, Operational Aspects, and
Examples. Drug Information Journal. 2006; 40(4):
463-473.

Gallo P. Confidentiality and trial integrity issues for
adaptive designs. Drug Information Journal. 2006;
40(4): 445-450.
Adaptive Designs Working Group
51
Outline




Introduction and motivation of adaptive
seamless designs (ASD)
Statistical methodology for seamless designs
Considerations for adaptive design
implementation
Simulations and comparisons of statistical
methods
Adaptive Designs Working Group
52
Introduction and Motivation
Reducing time to market is/has/will be a top
priority in pharmaceutical development


Brings valuable medicines to patients sooner
Increases the value of the drug to the parent
company
Adaptive seamless designs can help reduce this
development time
Adaptive Designs Working Group
53
Definitions
Seamless design

A clinical trial design which combines into a single
trial objectives which are traditionally addressed in
separate trials
Adaptive Seamless design

A seamless trial in which the final analysis will use
data from patients enrolled before and after the
adaptation (inferentially seamless)
Adaptive Designs Working Group
54
Adaptive Seamless Designs
Primary objective – combine “dose selection”
and “confirmation” into one trial




Although dose is most common phase IIb objective,
other choices could be made, e.g. population
After dose selection, only change is to new enrollments
(patients are generally not re-randomized)
Patients on terminated treatment groups could be
followed
All data from the chosen group and comparator is used
in the final analysis. Appropriate statistical methods
must be used
Adaptive Designs Working Group
55
Adaptive Seamless Designs
Dose
A
Dose B
Dose C
Placebo
Phase II

< white space >
Phase III
Time
Stage A (learning)
Phase B (confirming)
Dose
A
Dose B
Dose C
Placebo
Adaptive Designs Working Group
56
Statistical methodology
Statistical methodology for Adaptive Seamless
Designs must account for potential biases
and statistical issues



Selection bias (multiplicity)
Multiple looks at the data (interim analysis)
Combination of data from independent stages
Adaptive Designs Working Group
57
Statistical methodology - Bonferronni
Simple Bonferonni adjustment
Test final hypothesis at α / ntrt



Accounts for selection bias: multiplicity adjustment
Multiple looks at the data: not considered
Combination of data from stages by simple pooling



In some sense, ignores that there was an interim analysis at all
Most conservative approach, simple to implement
No other adjustments (i.e., sample size) can be made
Adaptive Designs Working Group
58
Statistical Methodology – Closed Testing
And alternative and more powerful approach is
a closed testing approach, and combination
of p-values with inverse normal method
Methodology combines:



Closed testing of hypothesis
Simes adjustment of p-values for multiplicity
Combines data (p-values) from stages via the inverse
normal method (or Fisher’s combination)
Adaptive Designs Working Group
59
Statistical Methodology – Closed Testing
Closed test procedure



n null hypotheses H1, …, Hn
Closed test procedure
considers all intersection
hypotheses.
Hi is rejected at global level α
if all hypotheses HI formed by
intersection with Hi are
rejected at local level α
H1 can only be rejected
at α=.05 if H12 is also
rejected at α=.05
Adaptive Designs Working Group
60
Statistical Methodology – Closed Testing


A typical study with 3 doses  3 pairwise hypotheses.
Multiplicity can be handled by adjusting p-values from
each stage using Simes procedure
q S  min
i S
S
i
p i 
S is number of elements in Hypothesis,
p(i) is the ordered P-values
Adaptive Designs Working Group
61
Statistical Methodology – Closed Testing
Inverse Normal Method
If p1 and p2 are generated from independent data, then
C (p1 , p 2 ) 
t0 
1
(1  p 1 )  1  t 0  
1
(1  p 2 )
will yield a Z test statistic
Note: For adaptive designs, typical value for t0 is n1/ (n1+n2)
Adaptive Designs Working Group
62
Statistical Methodology – Example
Example: Dose finding with 3 doses + control


Stage sample sizes: n1 = 75, n2 =75
Unadjusted pairwise p-values from the first stage:



p1,1= 0.23, p1,2 = 0.18, p1,3 = 0.08
Dose 3 selected at interim
Unadjusted p-value from second stage: p2,3 = .01
Adaptive Designs Working Group
63
Statistical Methodology – Example
Three-way test:

q1,123 = min( 3*.08, 1.5*.18, 1*. 23)= .23
q2,123 = p2,3 = .01

C(q1,123, q2,123) = 2.17  P.value = .015

Adaptive Designs Working Group
64
Statistical Methodology – Example
Two -way tests:




q1,13 = min( 2*.08, 1*. 23)= .16
q1,23 = min(2*.08,1*.18) = .16
q2,13 = q2,23 = p2,3 = .01
C(q1,13, q2,13) = C(q1,23, q2,23) = 2.35  P.value = .0094
Adaptive Designs Working Group
65
Statistical Methodology – Example
Final test:




q1,3 = p1,3 = .08
q2,3 = p2,3 = .01
C(q1,13, q2,13) = C(q1,23, q2,23) = 2.64  P.value = .0042
Conclusion: Dose 3 is effective
Adaptive Designs Working Group
66
Statistical Methodology – Sample sizes
Choosing sample sizes




There are two sample sizes to consider for a seamless
design, n1, n2
If t is the number of treatments, the total same size N is:
N = t*n1 + 2*n2
The larger n1, the better job of choosing the “right” dose.
However, this makes the total much larger.
Power can be determined by simulation, and is also a
function of the (unknown) dose response
Adaptive Designs Working Group
67
Statistical Methodology – Power
Simulation for power comparison
To compare the two methods for analyzing an adaptive
seamless designs, the following parameters where
used:





Sample sizes were n1= n2 = 75
Primary endpoint is normal, with σ = 12
One dose was selected for continuation
Various dose responses were assumed
20,000 reps used for simulations (error = ±.5%)
Adaptive Designs Working Group
68
Statistical Methodology – Power
Simulation for power comparison
Selecting 1 treatment group from 2 possible treatments
Dose Response
(Δ placebo )
Power Bonferronni
Power Closed Test
0 , 4.5
83.1%
83.2%
4.5, 4.5
91.0%
92.2%
Adaptive Designs Working Group
69
Statistical Methodology – Power
Simulation for power comparison
Selecting 1 treatment group from 3 possible treatments
Dose Response
(Δ placebo )
Power
Bonferronni
Power Closed
Test
0 ,0, 4.5
79.4%
78.9%
4.5, 4.5, 4.5
90.8%
92.7%
Adaptive Designs Working Group
70
Considerations for Seamless Designs
With the added flexibility of seamless designs,
comes added complexity.




Careful consideration should be given to the feasibility
for a seamless design for the project.
Not all projects can use seamless development
Even if two programs can use seamless development,
one might be better suited than the other
Many characteristics add or subtract to the feasibility
Adaptive Designs Working Group
71
Considerations for Seamless Designs
Enrollment vs. Endpoint

The length of time needed to make a decision relative
to the time of enrollment must be small


Endpoint must be well known and accepted


Otherwise enrollment must be paused
If the goal of Phase II is to determine the endpoint for
registration, seamless development would be difficult
If surrogate marker will be used for dose selection, it
must be accepted, validated and well understood
Adaptive Designs Working Group
72
Considerations for Seamless Designs
Clinical Development Time


There will usually be two pivotal trials for registration
Entire program must be completed in shorter timelines,
not just the adaptive trial
Adaptive Designs Working Group
73
Considerations for Seamless Designs
Logistical considerations


Helpful if final product is available for adaptive trial
(otherwise bioequivalence study is needed)
Decision process, and personnel must be carefully
planned and pre-specified
Adaptive Designs Working Group
74
Considerations for Seamless Designs
Novel drug or indication



Decision process which will be overly complicated could
be an issue with an external board
If there are a lot of unknown issues with the indication
or drug, a separate phase II trial would be better
However, getting a novel drug to patients sooner
increases the benefit of seamless development
Adaptive Designs Working Group
75
Conclusions




Adaptive seamless designs have an ability to improve
the development process by reducing timelines for
approval
Statistical methods are available to account for adaptive
trial designs
Extra planning is necessary to implement an adaptive
seamless design protocol
Benefits should be carefully weighed against the
challenges of such designs before implementation
Adaptive Designs Working Group
76
References









Dragalin V. Adaptive designs: terminology and classification. Drug Inf J. 2006 (to
appear).
Quinlan JA, Krams M. Implementing adaptive designs: logistical and operational
considerations. Drug Inf J. 2006 (to appear).
Bechhofer RE, Kiefer J, Sobel M. Sequential Identification and Ranking Problems.
Chicago: University of Chicago Press;1968.
Paulson E. A selection procedure for selecting the population with the largest mean
from k normal populations. Ann Math Stat. 1964;35:174-180.
Thall PF, Simon R, Ellenberg SS. A two-stage design for choosing among several
experimental treatments and a control in clinical trials. Biometrics 1989;45:537-547.
Schaid DJ, Wieand S, Therneau TM. Optimal two stage screening designs for
survival comparisons. Biometrika 1990;77:659-663.
Stallard N, Todd S. Sequential designs for phase III clinical trials incorporating
treatment selection. Stat Med. 2003;22:689-703.
Follman DA, Proschan MA, Geller NL. Monitoring pairwise comparisons in multiarmed clinical trials. Biometrics 1994;50:325-336.
Hellmich M. Monitoring clinical trials with multiple arms. Biometrics 2001;57:892898.
Adaptive Designs Working Group
77
References









Bischoff W, Miller F. Adaptive two-stage test procedures to find the best treatment in
clinical trials. Biometrika 2005;92:197-212.
Todd S, Stallard N. A new clinical trial design combining Phases 2 and 3: sequential
designs with treatment selection and a change of endpoint. Drug Inf J. 2005;39:109118.
Bauer P, Köhne K. Evaluation of experiments with adaptive interim analyses.
Biometrics 1994;50:1029-1041.
Bauer P, Kieser M. Combining different phases in the development of medical
treatments within a single trial. Stat Med. 1999;18:1833-1848.
Brannath W, Posch M, Bauer P. Recursive combination tests. J Am Stat Assoc.
2002;97:236-244.
Müller HH, Schäfer H. Adaptive group sequential designs for clinical trials:
combining the advantages of adaptive and classical group sequential approaches.
Biometrics 2001;57:886-819.
Liu Q, Pledger GW. Phase 2 and 3 combination designs to accelerate drug
development. J Am Stat Assoc. 2005;100:493-502.
Posch M, Koenig F, Brannath W, Dunger-Baldauf C, Bauer P. Testing and estimation
in flexible group sequential designs with adaptive treatment selection. Stat Med.
2005;24:3697-3714.
Bauer P, Einfalt J. Application of adaptive designs – a review. Biometrical J.
2006;48:1:14.
Adaptive Designs Working Group
78
References








Inoue LYT, Thall PF, Berry DA. Seamlessly expanding a randomized phase II trial to
phase III. Biometrics 2002;58:823-831.
Berry, DA, Müller P, Grieve AP, Smith M, Parke T, Blazek R, Mitchard N, Krams M.
Adaptive Bayesian designs for dose-ranging drug trials. In Case Studies in Bayesian
Statistics V. Lecture Notes in Statist. Springer: New York;2002;162:99-181.
Coburger S, Wassmer G. Sample size reassessment in adaptive clinical trials using
a bias corrected estimate. Biometrical J. 2003;45:812-825.
Brannath W, König F, Bauer P. Improved repeated confidence bounds in trials with a
maximal goal. Biometrical J. 2003;45:311-324.
Sampson AR, Sill MW. Drop-the-losers design: normal case. Biometrical J.
2005;47:257-281.
Stallard N, Todd S. Point estimates and confidence regions for sequential trials
involving selection. J. Statist. Plan. Inference 2005;135:402-419.
US Food and Drug Administration. Guidance for Clinical Trial Sponsors.
Establishment and Operation of Clinical Trial Data Monitoring Committees. 2006;
Rockville MD: FDA. http://www.fda.gov/cber/qdlns/clintrialdmc.htm.
US Food and Drug Administration. Challenge and Opportunity on the Critical Path to
New Medicinal Products. 2006.
http://www.fda.gov/oc/initiatives/criticalpath/whitepaper.html.
Adaptive Designs Working Group
79
Adaptive Designs Working Group
80
Adaptive Designs
Sample size re-estimation:
A review and recommendations
Keaven M. Anderson
Clinical Biostatistics and Research Decision Sciences
Merck Research Laboratories
FDA/Industry
Statistics Workshop
Adaptive Designs Working Group
September 27, 2006
Washington, D.C.
Outline


Introduction/background
Methods


Fully sequential and group sequential designs
Adaptive sample size re-estimation






Background
Nuisance parameter estimation/internal pilot studies

Blinded sample size re-estimation

Unblinded sample size re-estimation
Conditional power and related methods
Discussion and recommendations
Case studies
References
Adaptive Designs Working Group
82
Background

Origin



PhRMA Adaptive Design Working Group
Chuang-Stein C, Anderson K, Gallo P and Collins S,
Sample size re-estimation: a review and recommendations.
Drug Information Journal, 2006; 40(4):475-484
Focus




Late-stage (Phase III, IV) sample size re-estimation
Frequentist methods
Control of Type I error
Potential for bias is critical in these ‘confirmatory’ trials

Implications for logistical issues
Adaptive Designs Working Group
83
Introduction
Adaptive designs
allow design specifications to be changed based on
accumulating data (and/or information external to the
trial)
Extensive literature exists on adapting through sample size
re-estimation, the topic of this talk
Since sample size in group sequential and fully sequential
trials are data-dependent, we consider these to be
included in a broad definition of adaptive design/sample
size re-estimation
Adaptive Designs Working Group
84
Introduction
Why consider sample size re-estimation?


Minimize number of patients exposed to inferior or highly
toxic treatment
Right-size the trial to demonstrate efficacy



Reduce or increase sample size
Stop the trial for futility if insufficient benefit
Incorporate new internal or external information into a
trial design during the course of the trial
Adaptive Designs Working Group
85
Introduction

Reasons for Unplanned Adaptation

Information that could not have been anticipated prior to
trial start has become available.



Adaptation, not planned at the design stage, is used to
possibly to ‘bail out’ a trial



Regulators change the primary efficacy endpoint from mean
change in viral load to % of patients with viral load below the
detectable limit in HIV patients.
Regulators decide that the new treatment needs to win on more
than the one efficacy endpoint used to size the trial
A competing therapy removed from market, allowing a lesser
treatment benefit to be viable.
Sponsor ‘changes mind’ about minimal treatment effect of
interest
This topic will receive minimal discussion here
Adaptive Designs Working Group
86
The Problem

In order to appropriately power a trial, you need to
know:


The true effect size you wish to detect
Nuisance parameters such as




Variability of a continuous endpoint
Population event rate for a binary outcome or time to event
Other ancillary information (e.g., correlation between coprimary endpoints needed to evaluate study-level power)
Inappropriate assumptions about any of these
factors can lead to an underpowered trial
Adaptive Designs Working Group
87
Consequences of incorrect planning for treatment difference 
and/or standard deviation  (=0.05, planned Power=90%)
N planned/
N required
Power
Over-estimate  or under-estimate  by 50%
0.44
58%
Under-estimate  or over-estimate  by 50%
2.25
99.8%
Over-estimate  AND under-estimate  by 50%
0.20
30%
Under-estimate  AND over-estimate  by 50%
5.06
>99.9%
1
90%
Under-estimate  AND under-estimate  by
50%
Adaptive Designs Working Group
88
Solutions to the problem

Plan a fixed trial conservatively



Use group sequential design and plan conservatively



Pro: trial should be well-powered
Cons: Can lead to lengthy, over-powered, expensive trial
Pro: can power trial well and stop at appropriate, early interim
analysis if your assumptions are too conservative
Con: over-enrollment occurs past definitive interim analysis
because it takes time to collect, clean and analyze data
Use adaptive design


Pro: can decide to alter trial size based on partial data or new,
external information
Cons: methods used to adapt must be carefully chosen,
regulatory scrutiny over methods and ‘partial unblinding,’ may
not improve efficiency over group sequential design
Adaptive Designs Working Group
89
Outline


Introduction/background
Methods


Fully sequential and group sequential designs
Adaptive sample size re-estimation






Background
Nuisance parameter estimation/internal pilot studies

Blinded sample size re-estimation

Unblinded sample size re-estimation
Conditional power and related methods
Discussion and recommendations
Case studies
References
Adaptive Designs Working Group
90
Fully sequential design


Not commonly used due to continuous monitoring
May be useful to continuously monitor a rare serious
adverse effect




Intracranial hemorrhage in a thrombolytic/anti-platelet trial
Intussusception in rotavirus vaccine trial
Unblinded analysis suggests need for an independent
monitor or monitoring committee
References

Wald (1947), Sequential Analysis


Sequential probability ratio test (SPRT)
Siegmund (1985), Sequential Analysis: Tests and Confidence
Intervals
Adaptive Designs Working Group
91
Group sequential design

Classic





Variations




Fixed sample sizes for interim and final analyses
Pre-defined cutoffs for superiority and futility/inferiority at each analysis
Trial stops (adapts) if sufficient evidence available to decide early
Independent data monitoring committee often used to review unblinded
interim analyses
Adjustment of interim analysis times (spending functions)
Adjustment of total sample size or follow-up based on, for example,
number of events (information-based designs)
Properties well understood and design is generally well-accepted by
regulators
See: Jennison and Turnbull (2000): Group Sequential Methods with
Applications to Clinical Trials
Adaptive Designs Working Group
92
Outline


Introduction/background
Methods


Fully sequential and group sequential designs
Adaptive sample size re-estimation






Background
Nuisance parameter estimation/internal pilot studies

Blinded sample size re-estimation

Unblinded sample size re-estimation
Conditional power and related methods
Discussion and recommendations
Case studies
Evolving issues
Adaptive Designs Working Group
93
The Opportunity

Size the study appropriately to reach study objectives
in an efficient manner based on interim data that
offers more accurate information on

Nuisance parameter




Within-group variability (continuous data)
Event rate for the control group (binary data)
# of subjects and amount of exposure needed to capture
adequate occurrences of time-to-event endpoint

Treatment effect

Other ancillary information (e.g., correlation between coprimary endpoints needed to evaluate study-level power)
Ensure that we will have collected enough exposure
data for safety evaluation by the end of the study
Adaptive Designs Working Group
94
SSR Strategies

Update sample size to ensure power as desired based
on interim results

Internal pilot studies: Adjust for nuisance parameter estimates
only




Blinded estimation
Unblinded estimation
Testing strategy: no adjustment from usual test statistics
Adjusting for interim test statistic/treatment effect



All methods adjust based on unblinded treatment difference
Adjust sample size to retain power based on interim test statistic

Assume observed treatment effect at interim

Assume original treatment effect
Testing strategy: adjust stage 2 critical value based on interim test
statistic
Adaptive Designs Working Group
95
SSR - Issues

Planned vs Unplanned (at the design stage)

Control of Type I error rate and power

If we have a choice, do we do it blinded or unblinded?

If we do it unblinded, how do we maintain confidentiality?

Who will know the exact SSR rule?

Who will do it, a third party?

Who will make the recommendation, a DMC?

How will the results be shared?

Who will know the results, the sponsors, investigators?

When is a good time to do SSR?

Regulatory acceptance
Adaptive Designs Working Group
96
SSR reviews

These all concern what might be considered ‘internal
pilot’ studies

Friede and Kieser, Statistics in Medicine, 2001; 20:3861-73




Also Biometrical Journal, 2006; 48:537-555
Gould, Statistics in Medicine, 2001; 20:2625-43
Jennison and Turnbull, 2000, Chapter 14
Zucker, Wittes, Schabenberger, Brittain, 1999; Statistics in
Medicine, 18:3493-3509
Adaptive Designs Working Group
97
Blinded SSR



When SSR is based on nuisance parameters

Overall variability (continuous data)

Overall rate (binary data)
Advantage

No need to break the blind.

In-house personnel can do it.

Minimal implication for Type I error rate.
Disadvantage

The estimate of the nuisance parameter could be wrong,
leading to incorrect readjustment.
Adaptive Designs Working Group
98
Blinded SSR


Internal pilot studies to estimate nuisance parameter without adjustment of
final test statistic/critical value
Gould and Shih (1992)





Friede and Kieser (2001)



Assume treatment difference known (no EM algorithm required)
Adjust within group sum of squares using this constant
Type I error and power appear good


Uses EM algorithm to estimate individual group means or event rates
Estimates variance (continuous case)
Updates estimate of sample size required for adequate power
Software: Wang, 1999
Some controversy over appropriateness of EM (Friede and Kieser, 2002?;
Gould and Shih, 2005?)
Question to ask:



How well will this work if treatment effect is different than you have assumed for
the EM procedure?
Will it be under- or over-powered?
Group sequential version (Gould and Shih, 1998) may bail you out of this
Adaptive Designs Working Group
99
Blinded SSR gone wrong?
4000
3500
n per arm
3000
17.8% vs. 14.2%
n=1346
2500
2000
1500
Assuming
20%
placebo
event rate
Assuming
25%
reduction
1000
500
0
12.5%
14.5%
16.5%
18.5%
combined event rate
20%Observed
vs. 12%:
N=436
90% power, 2-sided Type I error 5%
Adaptive Designs Working Group
100
Unblinded SSR

Advantage


Could provide more accurate sample-size estimate.
Disadvantages

Re-estimate sample size in a continuous fashion can reveal
interim difference.

There could be concerns over bias resulting from knowledge of
interim observed treatment effect.

Typically require an external group to conduct SSR for
registration trials.

Interim treatment differences can be misleading

Due to random variation or

If trial conditions change
Adaptive Designs Working Group
101
Internal Pilot Design: Continuous Data

Adjusts sample size using only nuisance parameter estimate



Question to ask: does updated sample size reveal observed treatment effect?
Use some fraction of the planned observations to estimate error variance
for continuous data, modify final sample size, allow observations used to
estimate the variance in the final analysis.
Plug the new estimate into the SS formula and obtain a new SS. If the
SS re-estimation involves at least 40 patients per group, simulations have
shown (Wittes et al, SIM 1999,18:3481-3491; Zucker et al, SIM
1999,18:3493-3509)



The type I error rate of the unadjusted (naïve) test is at about the desirable level If we do
not allow SS to go down
The unadjusted test could lead to non-trivial bias in the type I error rate If we allow the SS
to go down
Power OK

Coffey and Muller (Biometrics, 2001, 57:625-631) investigated ways to
control the type I error rate (including different ways to do SSR).

Denne and Jennison, (Biometrika, 1999) provide a group sequential
version
Adaptive Designs Working Group
102
Internal pilot design: binary data


See, e.g., Herson and Wittes (1993), Jennison
and Turnbull (2000)
Estimate control group event rate at interim


Type I error OK if interim n large enough
Options (see Jennison and Turnbull, 2000 for
power study)

Assume p1-p2 fixed


Power appears OK
Assume p1/p2 fixed

Can be underpowered
Adaptive Designs Working Group
103
Combination tests

Methods for controlling Type I error

The invariance principle – calculate separate standardized test
statistics from different stages and combine them in a predefined
way to make decisions.

Weighting of a stage does not increase if sample size for that stage
is increased, meaning that individual observations for that stage are
down-weighted in the final test statistic


Efficiency issue (Tsiatis and Mehta, 2003)
Many methods available, including




Fisher’s combination test (Bauer, 1989)
Conditional error functions (Proschan and Hunsberger, 1995; Liu and
Chi, 2001)
Inverse normal method (Lehmacher and Wassmer, 1999)
Variance spending (Fisher, 1998)
Adaptive Designs Working Group
104
Combination tests

Apply combination test method to determine the critical value for
the second stage based on the observed data from the first stage.

Make assumption on treatment effect; options include:


Observed effect (highly variable)

External estimate

Original treatment effect used for sample size planning
Compute next stage sample size based on critical value, set
conditional power to originally desired power given interim test
statistic and assumed second stage treatment effect

Generally, will only raise sample size – not lower
Adaptive Designs Working Group
105
Outline


Introduction/background
Methods


Fully sequential and group sequential designs
Adaptive sample size re-estimation






Background
Nuisance parameter estimation/internal pilot studies

Blinded sample size re-estimation

Unblinded sample size re-estimation
Conditional power and related methods
Discussion and recommendations
Case studies
References
Adaptive Designs Working Group
106
Blinded vs Unblinded SSR

For SSR due to improved estimate on variance (continuous
data), Friede and Kieser (Stat in Med, 2001) conclude that
there is not much gain in conducting SSR unblinded.

They only studied a constant treatment effect

Statistical approaches to control Type I error rate particularly
important when adjusting sample size to power for observed
treatment difference

Decisions related to SSR because of inaccurate assumption
on the nuisance parameters can differ significantly from those
due to inaccurate assumption on the treatment effect.
Adaptive Designs Working Group
107
Relative efficiency of SSR methods

Internal estimates of treatment effect lead to very inefficient trials
(Jennison and Turnbull, 2003) due to the variability of the
estimates.

External or pre-determined minimal treatment effect assumptions
can yield comparable efficiency to group sequential (Liu and Chi,
2001, Anderson et. al, 2004)


Adding in a maximum sample size adjustment limit can improve over
group sequential (Posch et al, 2003)

Based on comparison of optimal group sequential and adaptive
designs, improvement of adaptive designs over group sequential is
minimal (Jennison and Turnbull, SIM 2006; see also Anderson, 2006)
Use of sufficient statistic design rather than weighted combination
test improves efficiency (Lokhnygina, 2004)
Adaptive Designs Working Group
108
Group Sequential vs SSR Debate


Efficiency

The adaptive designs for SSR using combination tests with
fixed weights are generally inefficient.

Efficient adaptive designs for SSR have little to offer over
efficient group sequential designs in terms of sample size.
However, the latter might require more interim analyses and
offer minimum gain. In addition, the comparisons were
made as if we knew the truth.
Flexibility and upfront resource commitment

SSR offers flexibility and reduces upfront resource
commitment. The flip side is the need to renegotiate budget
and request additional drug supply when an increase in SS
is necessary.

SSR addresses uncertainty at the design stage.
Adaptive Designs Working Group
109
Group Sequential vs SSR Debate

SSR is fluid and can respond to changing
environment both in terms of medical care and
the primary endpoint to assess treatment effect.


The above is important for trials lasting 3-5 years when
environmental changes are expected.
Need to ascertain treatment effect in major
subgroups even though the subgroups are
not the primary analysis populations


Xigris for disease severity groups
Cozaar for race groups
Adaptive Designs Working Group
110
Recommendation #1

Before considering adaptive sample-size re-estimation,
evaluate whether or not group sequential design is
adequate

Pros:




Regulatory acceptance
Well-understood methods allow substantial flexibility
Experienced monitoring committee members available
Cons:

May not work well in some situations when trial cannot be
stopped promptly (long follow-up, slow data collection, cleaning or
analysis)
Adaptive Designs Working Group
111
Recommendation #2

Anticipate as much as possible at the planning stage the
need to do SSR to incorporate information that will
accumulate during the trial




Treatment effect size
Nuisance parameters
The effect of environmental changes on the design assumptions
Do not use SSR to


Avoid up-front decisions about planning
As a ‘bait-and-switch’ technique where a low initial budget can
be presented with a later upward sample size adjustment.
Adaptive Designs Working Group
112
Recommendation #3

For SSR based on variance, consider using blinded SSR


However, when there is much uncertainty about the treatment effect,
consider using unblinded SSR.
For a binary outcome, one can either do blinded SSR based
on the overall event rate or an unblinded SSR based on the
event rate of the control group. There is no clear preference,
choice dependant on several factors.

If there is much uncertainty about treatment effect, unblinded SSR
using conditional power methods (see next slides).

If SSR is blinded, consider conducting interim analysis to capture
higher than expected treatment effect early.
Adaptive Designs Working Group
113
Recommendation #4

To help maintain confidentiality of the interim results, we
recommend

Do not reveal exact method for adjusting sample size.

Make the outcome of SSR discrete with only 2-3 options.

Under the first approach, details on SSR methodology will not be
described in the protocol, but documented in a stand-alone
statistical analysis plan for SSR not available to study personnel.

For SSR based on observed treat effect (continuous case), it will be
beneficial to base SSR on both variability and effect.

We recommend that the protocol include the maximum sample size
allowed to minimize the need to go back to the IRB.
Adaptive Designs Working Group
114
Recommendation #5

For unblinded SSR

Invite a third party to do the calculations following a prespecified rule.

If possible, combine SSR with a group sequential design
where SSR will be conducted at the same time with an
interim analysis.


Convene a DMC (or preferably an IDMC) to review the SSR
recommendation from the third party. If an IDMC is used, the
IDMC statistician can carry out the SSR.
Assuming Recommendation #4 is followed, the new
sample size will be communicated to the sponsor. The
investigators will be told to continue enrollment.
Adaptive Designs Working Group
115
Recommendation #6

Carefully consider the number of times to do SSR.


E.g., for variance estimation, is once enough?
Timing of the SSR should be based on multiple considerations such
as



available info at the design stage,
disease,
logistics



Method


delay from enrollment until follow-up complete and data available
enrollment rate,
whether the SSR will be based on variance or treatment effect
Gould and Shih (1992) recommend early update as soon as variance
estimate stable due to administrative considerations, while Sandvik et
al. (1996) recommend as late as possible to get accurate variance
estimate
Adaptive Designs Working Group
116
Recommendation #7

Acceptance of SSR by regulators varies, depending on the reasons
for SSR. In general, blinded SSR based on a nuisance parameter
is acceptable.

When proposing unblinded SSR, should include


The objective for SSR

Statistical methodology including the control of Type I error

When to do the SSR

How to implement (e.g., DMC, third party)

How to maintain confidentiality

How will the results be shared

Efficiency (power/sample size) considerations
Discuss the plan with regulatory agencies in advance.
Adaptive Designs Working Group
117
Outline


Introduction/background
Methods


Fully sequential and group sequential designs
Adaptive sample size re-estimation






Background
Nuisance parameter estimation/internal pilot studies

Blinded sample size re-estimation

Unblinded sample size re-estimation
Conditional power and related methods
Discussion and recommendations
Case studies
References
Adaptive Designs Working Group
118
Case Study #1: Blinded SSR Based on Variance

Drug X low-dose, high-dose and placebo

Main efficacy endpoint - percent change in continuous primary
outcome

N=270 provides 90% power to detect a 10% difference versus
placebo



estimate of the variability obtained from study performed in a different setting (SD
estimated at 20%)
seasonal disease: interim analysis performed during long pause in enrollment
Recruitment was anticipated to be difficult,



specified in protocol that a blinded estimate of the variability of primary outcome
would be computed when the sample size is 100
If the variability is less than anticipated (eg., SD 15%) then the final sample size
could be reduced
If the variability is greater than anticipated (e.g., > 25%), the main comparison would
be the pooled Drug X groups (low and high-dose) vs. placebo
Adaptive Designs Working Group
119
Case Study: REST Study Design
Sample size:
Minimum of 60,000 (1V:1P)
Age:
6 to12 weeks at enrollment
Dose regimen:
3 oral doses of Rotavirus every 4-10 wks
Formulation:
Refrigerated liquid buffer/stabilizer intended
for licensure
Potency:
Study Period:
Release range intended for licensure
2001 to 2005
Adaptive Designs Working Group
120
Primary Safety Hypothesis


Oral RotaTeq™ will not increase the risk of
intussusception relative to placebo within 42 days after
any dose
To satisfy the primary safety hypothesis, 2 criteria must
be met:
1.
During the study, the vaccine/placebo case ratio does not
reach predefined unsafe boundaries being monitored by the
DSMB


2.
1 to 42 days following any dose
1 to 7 days following any dose
At the end of the study, the upper bound of the 95% CI
estimate of the relative risk of intussusception must be 10
Adaptive Designs Working Group
121
Safety Monitoring for Intussusception (IT)
IT Surveillance
at Study Sites
Safety Endpoint
Data and Safety
Adjudication Committee Monitoring Board (DSMB)
Active surveillance
-contacts on day 7
14, and 42
Pediatric surgeon,
radiologist, & emergency
department specialists
Passive surveillance
-parent education
Use specific case
definition
Intense surveillance
during 6 weeks after
each dose
Individual & collaborative adjudications
Potential
IT Case
Unblind each case
as it occurs and make
recommendations
about continuing
Review all safety data
every 6 months
Positively
Adjudicated
IT Case
Adaptive Designs Working Group
122
Safety Monitoring for Intussusception


Trial utilizes two predefined
stopping boundary graphs for
the 1 to 7 and 1 to 42 day
ranges after each dose
Stopping boundaries were
developed to ensure that the
trial will be stopped if there is
an increased risk of
intussusception within these
day ranges
Stopping Boundary
(For 1 to 42 Day Period
After Any Dose)
V a ccin e In tu ssu sce p tio n C a se s

16
14
12
10
8
6
4
2
0
DSMB plots intussusception
0
1
2
3
4
5
6
7
8
cases on graphs and makes
P la c e b o In tu s s u s c e p tio n C a s e s
recommendations about
123
Adaptive Designs Working Group
continuing the study
V 2 6 0 -6 P D 1 -4 2 fe b 0 3 F e b . 2 6 , 2 0 0 3
Intussusception 42 days post-dose
Unsafe
Boundary
14 (LB on 95%
Vaccine Intussusception Cases
16
CI >1.0)
12
10
8
Acceptabl
Safety
Profile
e
6
(UB on 95% CI <10)
4
2
0
2
4
6
8
10
12
14
16
Placebo Intussusception Cases
Adaptive Designs Working Group
124
REST Group Sequential Study Design
Enroll subjects
Monitor continuously for
intussusception (IT)
Evaluate statistical criteria
with 60,000 subjects
Stop trial early if detect
increased risk of IT
Monitor continuously and
stop early if detect increased
risk of IT
Primary hypothesis
Data inconclusive:
satisfied: Stop Enroll 10,000 more infants
Evaluate statistical criteria
with 70,000 subjects
Adaptive Designs Working Group
125
Comments on REST Study Design


The goal of the REST study design and the extensive
safety monitoring was to provide:
i.
High probability that a safe vaccine would meet the end of
study criteria; and simultaneously
ii.
High probability that a vaccine with increased
intussusception risk would stop early due to ongoing safety
monitoring
The statistical operating characteristics of REST were
estimated using Monte Carlo simulation
Adaptive Designs Working Group
126
Statistical Operating Characteristics of REST*
Risk Scenario
Probability of reaching Probability of meeting
unsafe monitoring
end of study safety
boundary
criteria
Safe Vaccine (RR=1)
~6%
~94%
RRV-TV Risk Profile**
Case-control study
Case-series study
~91%
~85%
~9%
~15%
* Assumes background intussusception rate of 1/2000 infant years and102
days of safety follow-up over three doses.
** RRV-TV = rhesus rotavirus tetravalent vaccine; Murphy et al, New Engl
J Med. 344(2001): 564-572.
Adaptive Designs Working Group
127
References (Blinded SSR)














Gould, Shih (1992) Comm in Stat (A), 21:2833-2853.
Gould (1992) Stat in Medicine, 11:55-66.
Shih (1993) Drug Information Journal, 27:761-764.
Shih, Gould (1995) Stat in Medicine, 14:2239-2248.
Shih, Zhao (1997) Stat in Medicine, 16:1913-1923.
Shih, Long (1998) Comm in Stat (A), 27:395-408.
Shih, Zhao (1997) Stat in Medicine, 16:1913-1923.
Gould, Shih (1998) Stat in Medicine, 17:89-100.
Kieser, Friede (2000) Drug Info Journal, 34:455-460.
Friede, Kieser (2002) Stat in Medicine, 21:165-176.
Friede, Kieser (2003) Stat in Medicine, 22:995-1007.
Friede, Kieser (2006) Biometrical Journal, 2006; 48:537-555
Wust, Kieser (2003) Biometrical J, 45:915-930.
Wust, Kieser (2005) Comp Stat & Data Analysis, 49:835-855
Adaptive Designs Working Group
128
References (Others)
















Bauer, P. (1989) Biometrie und Informatik in Medizin und Biologie 20:130–
148.
Wittes, Brittain (1990) Stat in Medicine, 9:65-72
Bristol (1993) J of Biopharm Stat, 3:159-166
Herson, Wittes (1993) Drug Info Journal, 27:753-760
Birkett, Day (1994) Stat in Medicine, 13:2455-2463
Proschan, Hunsberger (1995) Biometrics, 51:1315–1324
Shih, Zhao (1997) Stat in Medicine, 16:1913-1923
Fisher (1998) Stat in Medicine,17:1551-1562
Cui, Hung, Wang (1999) Biometrics, 55:853-857
Wittes et al (1999) Stat in Med, 18:3481-3491
Zucker, Wittes et al (1999) Stat in Medicine,18:3493-3509
Lechmacher, Wassmer (1999) Biometrics, 55:1286-1290
Denne, Jennison (1999) Stat in Medicine, 18:1575-1585
Denne (2000) J of Biopharm Stat, 10:131-144
Kieser, Friede (2000) Stat in Medicine, 19:901-911
Shun, Yuan, Brady, Hsu (2001) Stat in Medicine, 20:497-513
Adaptive Designs Working Group
129
References (Others)

Muller, Schafer (2001) Biometrics, 57:886-891
Coffee, Muller (2001) Biometrics, 57:625-631
Mehta, Tsiatis (2001) Drug Info Journal, 35:1095-1112
Liu, Chi (2001) Biometrics, 57:172–177
Jennison, Turnbull (2003) Stat in Medicine, 22:971-993
Jennison, Turnbull (2006) Biometrika, 93:1-21
Jennison, Turnbull (2006) Stat in Medicine: 25:917-932
Tsiatis, Mehta (2003) Biometrika, 90:367–378
Schwartz, Denne (2003) Pharm Stat, 2:263-27
Brannath, Konig, Bauer (2003) Biometrical J, 45:311-324
Coburger, Wassmer (2003) Biometrical J, 45:812-825
Posch, Bauer, Brannath (2003) Stat in Med, 22:953-969
Friede, Kieser (2004) Pharm Stat, 3:269-279
Cheng, Shen (2004) Biometrics, 60:910-918
Jennison, Turnbull (2004) Technical Reports, U of Bath, UK

Anderson et al (2004) JSM Proceedings, ASA.














Adaptive Designs Working Group
130
References (Others)




Lokhnygina (2004) NC State stat dissertation (with Anastasio Tsiatis as the
advisor).
Sandvik, Erikssen, Mowinckel, Rodland (1995), Statistics in Medicine, 15:158790
Denne and Jennison (2000), Biometrika, 87:125-134
Anderson (2006) Biometrical Journal, to appear
Adaptive Designs Working Group
131
Adaptive Designs Working Group
132
Adaptive Designs
Logistic, Operational and
Regulatory Issues
Paul Gallo
Biostatistics and Statistical Reporting
Novartis
FDA/Industry
Statistics Workshop
Adaptive Designs Working Group
September 27, 2006
Washington, D.C.
Primary PhRMA references

PhRMA White Paper sections:

Quinlan JA and Krams M. Implementing adaptive
designs: logistical and operational considerations.
Drug Information Journal 2006; 40(4): 437-444.

Gallo P. Confidentiality and trial integrity issues for
adaptive designs. Drug Information Journal 2006;
40(4): 445-450.
Adaptive Designs Working Group
134
Outline

Motivations and opportunities

Cautions and challenges

Logistic and feasibility issues

Interim monitoring and confidentiality issues

review of current conventions

monitoring processes for adaptive designs

information conveyed by adaptive designs
Adaptive Designs Working Group
135
General motivation

The greater flexibility offered within the adaptive
design framework has the potential to translate into
more ethical treatment of patients within trials
(possibly including the use of fewer patients), more
efficient drug development, and better focusing of
available resources.

The potential appeal of adaptive designs is
understandable, and motivates the current high
level of interest in this topic.
Adaptive Designs Working Group
136
Cautions

But, being too eager, and proceeding without all
relevant issues being fully thought out, is not
advisable either.

The question should be:


“What is the most appropriate (e.g., ethical, efficient)
means at hand to address the research questions of
importance?”
rather than:

“How can adaptive designs be integrated into our
program at all costs?”
Adaptive Designs Working Group
137
Challenges

Clearly, there will be many challenges to be
addressed or overcome before adaptive
designs become more widely utilized.

Statistical

Logistic

Procedural / regulatory
Adaptive Designs Working Group
138
General considerations




Like any new technology with challenges, some
resistance is to be expected.
Closer scrutiny is natural, and constructive.
But we should not make “the perfect be the
enemy of the good ”.
Can we address the challenges to a sufficient
extent so that in particular situations the
advantages outweigh the drawbacks?
Adaptive Designs Working Group
139
Planning


Adaptive designs are not a substitute for poor
planning, and in fact will generally require more
planning.
They are part of a rational strategy to achieve
research objectives more efficiently and
ethically:


by utilizing knowledge gained from the study
in a manner which maintains the validity and
interpretability of the results.
Adaptive Designs Working Group
140
Feasibility issues


Endpoint follow-up time vs recruitment speed

Shorter read-out time is generally favorable to
adaptive designs.

Surrogates / early predictors can have a role.
Timely data collection is important, as well as
efficient analysis and decision-making
processes.

Electronic Data Capture should be helpful.
Adaptive Designs Working Group
141
Data quality


All else being equal, cleaner is better
But the usual trade-off exists:



cleaner takes longer, and results in less data being
available for decisions
lack of data is a source of noise also!
There is no requirement that data must be fully
cleaned for adaptive designs.

Details of data quality requirements should be
considered on a case-by-case basis.
Adaptive Designs Working Group
142
Opportunities

Early-phase trials may in the short term be the
most favorable arena for wider-scale
implementation of adaptive designs.



More uncertainties, and thus more opportunity for
considering adaptation
Lesser regulatory concerns
Lower-risk opportunities to gain experience with ADs
to learn, solve operational problems, and set the
stage for more important applications.
Adaptive Designs Working Group
143
Simulation

Simulation will play an important role in
planning of adaptive trials.


Detailed simulation scenarios should be of broad
interest in evaluating adaptive design proposals
(e.g., health authorities).
Main simulation results may be included in the
protocol or analysis plan.
Adaptive Designs Working Group
144
Monitoring / confidentiality issues
Issues relating to

monitoring of accruing data

restriction of knowledge of interim results

and the processes of data review, decisionmaking and implementation
are likely to be critical in determining the extent
and shaping the nature of adaptive design
utilization in clinical trials.
Adaptive Designs Working Group
145
Current monitoring conventions
Monitoring of accruing data is of course a common
feature in clinical trials. Most frequently for:



safety monitoring

formal group sequential plan allowing stopping for
efficacy

lack of effect / futility judgments.
Current procedures and conventions governing
monitoring are a sensible starting point for
addressing similar issues in trials with adaptive
designs.
Adaptive Designs Working Group
146
Current monitoring conventions

As described in the FDA DMC guidance (2006):
Comparative interim results and access to
unblinded data should not be accessible to trial
personnel, sponsor, investigators.

Access to interim results diminishes the ability of
trial personnel to manage the trial in a manner
which is (and which will be seen by interested
parties to be) completely objective.
Adaptive Designs Working Group
147
Current monitoring conventions

Knowledge of interim results could introduce subtle,
unknown biases into the trial, perhaps causing slight
changes in characteristics of patients recruited,
administration of the intervention, endpoint
assessments, etc.

Changes in “investigator enthusiasm”?

The equipoise argument: knowledge of interim
results violates equipoise.
Adaptive Designs Working Group
148
Current conventions - sponsor

FDA (2006): “Sponsor exposure to unblinded
interim data . . . can present substantial risk to the
integrity of the trial.”

Risks include lack of objectivity in trial management;
further unblinding, even if inadvertent; SEC
requirements and fiduciary responsibilities, etc.

Sponsor is thus usually not involved in monitoring of
confirmatory trials.
Adaptive Designs Working Group
149
Issues for adaptive designs
I.
Adaptive designs will certainly require review of
accruing data.




Who will be involved in the analysis, review, and
decision-making processes?
Will operational models differ from those we’ve
become familiar with?
Will sponsor perspective and input be relevant or
necessary for some types of adaptations?
Will sponsors accept and trust decisions made
confidentially by external DMCs in long-term trials /
projects with important business implications (e.g.,
seamless Phase II / III)?
Adaptive Designs Working Group
150
Issues for adaptive designs
II. An important distinction versus common monitoring
situations: the results will be used to implement
adaptation(s) which will govern some aspect of
the conduct of the remainder of the trial.

Can observers infer from viewing the actions taken
information about the results which might be perceived
to rise to an unacceptable level?
Adaptive Designs Working Group
151
Analysis / review / decision process

Concerns about confidentiality to ensure objective
trial management, and potential bias from broad
knowledge of interim results, should be no less
relevant for adaptive designs than in other settings.

The key principles to adhere to would seem to be:

separation / independence of the DMC from other
trial activities

limitation of knowledge about interim treatment
effects.
Adaptive Designs Working Group
152
Analysis / review / decision process

Adaptive design trials may utilize a single monitoring
board for adaptations and other responsibilities
(e.g., safety); or else a separate board may be
considered for the adaptation decisions.

DMCs in adaptive design trials may require
additional expertise not traditionally represented on
DMCs; perhaps to monitor the adaptation algorithm,
or to make the type of decision called for in the
adaptation plan (e.g., dose selection).
Adaptive Designs Working Group
153
Sponsor participation
Sponsor participation and knowledge of interim
results in confirmatory adaptive trials may be a
hard sell.


The “objective trial management” issue - sponsor
can have some influence on trial management
activities, even for individuals not directly
participating in the trial.

There seems to be an assumption that information,
once within the sponsor organization, may not be
controlled, whether inadvertently or otherwise.
Adaptive Designs Working Group
154
Sponsor participation
Proposal - There should be potential for sponsor
involvement in certain types of decisions if:





a strong rationale can be described whereby these
individuals are needed for the best decision
the individuals are not involved in trial operations
all involved clearly understand the issues and risks to
the trial, and adequate firewalls are in place
sponsor exposure to results is “minimal” for the needed
decision, i.e., only at the adaptation point, only the
relevant data (e.g., unlike a DMC with whom they may
be working, which may have a broader ongoing role).
Adaptive Designs Working Group
155
Information apparent to observers

Adaptive designs may lead to changes in a trial
which will be apparent to some extent - sample
size, randomization allocation, population, dosage,
treatment arm selection, etc., etc. - and can thus be
viewed as providing some information to observers
about the results which led to those changes.

Considering the concerns which are the basis for
the confidentiality conventions: can we distinguish
between types and amounts of information, and
how risky they would be in this regard?
Adaptive Designs Working Group
156
Information apparent to observers

Note: conventional monitoring is not immune
from this issue.

It has never been the case that no information
can be inferred from monitoring; i.e., all
monitoring has some potential action
thresholds, and lack of action usually implies
that such thresholds have not been reached.
Adaptive Designs Working Group
157
Example – Triangular test
Design:


Normal data, 2 group comparison

Study designed to detect Δ = 0.15

4 equally-spaced analyses

will require about 2276 patients.
Adaptive Designs Working Group
158
Example – Triangular test
‘Christmas tree’ boundary
Z 80
60
40
20
V
0
0
100
200
300
400
500
600
700
800
-20
-40
Adaptive Designs Working Group
159
Example – Triangular test
^

Δ=Z/V

Continuation beyond the 3rd look would imply
(barring over-ruling of the boundary) that the point
estimate is between 0.076 and 0.106.

Doesn’t that convey quite a bit of information about
the interim results?

In conventional GS design practice, this issue
seems not to be perceived to compromise trials nor
to discourage monitoring.
Adaptive Designs Working Group
160
Information apparent to observers

Presumably, in GS practice it’s viewed that
reasonable balance is struck between the objectives
and benefits of the monitoring and any slight
potential for risk to the trial, with appropriate and
feasible safeguards in place to minimize that risk.

The same type of standard should make sense for
adaptive designs.
Adaptive Designs Working Group
161
Information apparent to observers

In some cases we may have opportunities to lessen
this concern by withholding certain details of the
strategy from the protocol, and placing them in
another document of more limited circulation.

For example, if some type of selection is to be made
based upon predictive probabilities, do full details
and thresholds need to be described in the
protocol?
Adaptive Designs Working Group
162
Information apparent to observers
A proposal:


Selection decisions (choice of dose, subgroup,
etc. for continuation) generally do NOT give away
an amount of information that would be
considered to compromise or influence the trial,
as long as the specific numerical results on which
the decisions were based remain confidential.
Adaptive Designs Working Group
163
Information apparent to observers
Consider the alternative 
In a seamless Phase II / III design, we might instead
have run a conventional separate-phase program.

Phase II results would be widely known (what about
equipoise ??)

In this sense, maybe the adaptive design offers a
further advantage relative to the traditional
paradigm?
Adaptive Designs Working Group
164
Algorithmic changes

More problematic - changes based in an algorithmic
manner on interim treatment effect estimates in
effect provide knowledge of those estimates to
anyone who knows the algorithm and the change.

Most typical example - certain approaches to
sample size re-estimation:

SSnew = f (interim treatment effect estimate)

=> estimate = f -1 (SSnew)
Adaptive Designs Working Group
165
Mitigating the concerns
Perhaps the adaptation can be made based upon a
combination of factors in order to mask the observed
treatment effect.




e.g., SS re-estimation using the treatment effect, the
observed variance, and external information.
If possible, “discretize” the potential actions, i.e., a
small number of potential actions correspond to
ranges of the treatment effects.
We may at times try to quantify that the knowledge
which can be inferred is comparable to that of
accepted group sequential plans.
Adaptive Designs Working Group
166
Summary





Adaptive designs suggest real benefits for the clinical
development process.
Achieving this promise will require full investigation
and understanding of the relevant issues, trade-offs,
and challenges.
Advantages should be considered in balance against
any perceived risks or complexities.
This should be expected to require more planning,
not less.
We can expect that adaptive designs will inevitably
be scrutinized closely because of their novelty.
Adaptive Designs Working Group
167
Summary

We should not aim to broadly undo established
monitoring conventions, but rather to fine-tune them
to achieve their sound underlying principles.

To justify sponsor participation in monitoring, provide
convincing rationale and “minimize” this involvement,
and enforce strict control of information.

Some types of adaptations convey limited information
for which it seems difficult to envision how the trial
might be compromised.

Others convey more information, but perhaps we can
implement extra steps to mask this.
Adaptive Designs Working Group
168
References

Committee for Medicinal Products for Human Use. Guideline on Data
Monitoring Committees. London: EMEA; 2006.

Committee for Medicinal Products for Human Use. Reflection paper on
methodological issues in confirmatory clinical trials with flexible design and
analysis plans (draft). London: EMEA; 2006.

DeMets DL, Furberg CD, Friedman LM (eds.). Data Monitoring in Clinical
Trials: A Case Studies Approach. Springer; 2006.

Ellenberg SE, Fleming TR, DeMets DL. Data Monitoring Committees in
Clinical Trials: A Practical Perspective. Chichester: Wiley; 2002.

US Food and Drug Administration. Guidance for Clinical Trial Sponsors on
the Establishment and Operation of Data Monitoring Committees.
Rockville MD: FDA; 2006.
Adaptive Designs Working Group
169