Transcript John Stevens

Bayesian Sample Size
Determination in the Real World
John Stevens
AstraZeneca R&D Charnwood
Tony O’Hagan
University of Sheffield
Department
Author
Reference:
Bayesian Assessment of Sample Size for Clinical Trials of
Cost-Effectiveness
O’Hagan A, Stevens JW
Medical Decision Making 2001;21:219-230
Department
Author
Contents
• The Sample Size Problem
• The Utility Function
• Study Objectives
• The Prior Distribution
• An Example
• Discussion
Department
Author
The (Real World) Sample Size Problem
• A study is to be designed to compare the efficacy of two
treatments, Treatment 2 (the experimental treatment) and
Treatment 1 (the control treatment).
• Consideration is to be given to
•
the gain (i.e. profit) achieved from sales of the new treatment if it is
successfully marketed,
•
the loss (i.e. costs) associated with setting up the study and the
cost per patient in the study, allowing for any delay in coming to
market.
• It will be assumed that all gains and losses are associated with
the conduct of a single study.
• What is the optimal sample size?
Department
Author
The Utility Function
• In order to determine sample size using decision theory it is
necessary to define a Utility Function.
• The Utility Function can be written as :
U(n1, n2, x) = u . L(x) - c . (n1 + n2) -c0
where,
n1 and n2 are the sample sizes in the two treatment arms,
x denotes the data obtained in the study,
L(x) takes values one if the data, x, are convincing enough to the
regulator to approve the drug and zero otherwise,
u is the profit to the company if the drug is approved,
c is the cost per patient in the study (allowing for delays in
marketing),
c0 is a set-up cost for the study.
Department
Author
Sample Size - The Two Stages
• There are two stages in the conduct of a study:
•
Design stage. Plan the study to maximise the expected utility
associated with the desired outcome.
•
Analysis stage. Analyse the data from the study to see whether
we can report the desired outcome.
• These two stages are reflected in
•
setting the objectives,
•
how we do the sample size calculations.
Department
Author
Study Objectives
Analysis Objective:
We will regard the outcome of the study as positive if the data
obtained are such that there is a probability of at least ω that
Treatment 2 is more efficacious than Treatment 1.
Design Objective:
We wish to maximise the expected utility, U(n1, n2), associated
with the desired outcome.
Department
Author
Analysis Objective
• Frequentist formulation
•
We wish to reject the null hypothesis that μ2 - μ1 = 0, at the
100(1 - w)% level of significance.
e.g. w = 0.95 corresponds to usual 5% significance test in a
one-sided test
• Bayesian formulation
•
We wish to have at least a 100w% posterior probability that
μ2 - μ1 > 0.
Department
Author
Design Objective
• Bayesian formulation
•
We want to choose sample sizes that maximise the expected
utility associated with achieving the desired posterior
probability that μ2 - μ1 > 0.
Department
Author
Expected Utility
• L(x) is a function of the sample size in each treatment group.
• The expected value of L(x) is P(n1, n2), and is the probability of
obtaining data to convince the regulator.
• The expected Utility is then,
U(n1, n2) = u . P(n1, n2) - c . (n1 + n2).
Department
Author
Understanding the Bayesian Formulation
• At the Analysis stage:
•
The Bayesian and frequentist formulations are similar
(particularly if we employ weak prior information in the
analysis of the data).
•
The Bayesian statement is often how the p-value is
interpreted anyway.
• At the Design stage:
•
The frequentist and Bayesian formulations are different.
•
The frequentist approach fixes the parameters at (more or
less) arbitrary values.
•
The Bayesian formulation defines prior distributions for the
parameters.
Department
Author
The Prior Distribution
• When advocating a Bayesian approach, the usual question arises.
What about the prior distribution?
• Various options are available:
•
Realistic beliefs of the wider community
•
The company’s genuine prior beliefs
•
Non-informative priors
•
Sceptical priors
Department
Author
Which Prior Distribution?
Analysis
Design
Company
Y
Y
Community
?
N
Noninformative
N
Y
Sceptical
N
?
Department
Author
The Two Prior Distributions
• We will allow different prior distributions at the two stages of
design and analysis.
• Analysis prior distribution.
•
This will typically be non-informative, sceptical or some kind
of consensus.
• Design prior distribution.
•
This should generally be based on all information available to
the company.
Department
Author
Example
Modified from Briggs and Tambour (1998).
• Bayesian formulation,
•
analysis and design priors informative and identical,
•
prior means μ1 = 5, μ2 = 6.5;
prior variance = 4; prior co-variance = 3,
•
ω = 0.975 (1-sided, equates to 5% 2-sided test).
• Profit : £5bn
• Cost per patient : £1000; £10,000; £100,000; £1,000,000
• (Ratio : £5,000,000; £500,000; £50,000, £5,000)
Department
Author
Department
Author
Department
Author
Example - Conclusions
• We have a prior probability of 85.55% that Treatment 2 is more
efficacious than Treatment 1, which we approach rapidly at sample
sizes above 2000 per treatment group.
• The maximum utility (return on investment) depends on the ratio of
the profit to the cost per patient.
• Unnecessarily large sample sizes reduce the return on investment
with no major increase in the probability that the regulator will be
convinced to approve the registration of the treatment.
Department
Author
Discussion
• The Bayesian approach allows considerable flexibility to represent
the (real world) problem.
• The Bayesian approach encourages the assessment of the
genuine beliefs regarding the true treatment means.
• Unequal sample sizes could be used is there is a prior belief that
the variances are different between treatment groups.
• The design objective allows the incorporation of prior information
in determining the optimal sample size to maximise the return on
investment.
• Using the decision-based approach could provide an informed
basis for management to use when allocating limited clinical
resources to different clinical projects.
Department
Author