Transcript Practical aspects of Modelling (I) Fitting distributions

What role should
probabilistic sensitivity analysis
play in SMC decision making?
Andrew Briggs, DPhil
University of Oxford
What probabilistic modelling offers
• Generating the appropriate (expected) costeffectiveness
• Reflects combined implications of parameter uncertainty
on the outcome(s) of interest (cost-efectiveness)
• Can make probability statements about costeffectiveness results – error probability under decision
maker’s control
• Offers a means of to calculate the value of collecting
additional information
Role of probabilistic sensitivity analysis
Overview
•
•
•
•
•
•
•
Data sources for parameter fitting
Distributions for common model parameters
Correlating parameters
Presenting simulation results
Using PSA for decision making
Continuing role of traditional sensitivity analysis
Micro simulation models
Data sources for parameter estimation
• Primary data
– Can ‘fit’ parameters using standard statistical methods
– Provides standard estimates of variance and correlation
• Secondary data
– With appropriate information reported can still fit parameters
– Meta analysis may be possible
• Expert opinion
– Usefulness of Delphi limited (focus on consensus!)
– Variability across estimates
– Individual estimates of dispersion
Distributions for common parameters
Probability parameters
• Probabilities are constrained on the interval zero-one
• Probabilities must sum to one
• Probabilities often estimated from proportions
– Data informing estimation are binomially distributed
– Use Beta distribution
• May estimate probabilities from rates
– E.G. from hazard rates in survival analysis
– Use (multivariate) normal on log scale
– Must make transformation from rates to probabilities
Distributions for common parameters
Cost parameters
• Costs are a mixture of resource counts and unit costs
• Could model counts individually as Poisson with
Gamma distributed mean (parameter)
• Costs are constrained to be zero or positive
• Can use Gamma distribution if cannot rely on the
Central Limit Theorem (if skewed)
• Popular alternative is log-normal, particular when using
regression models on log cost
Distributions for common parameters
Utility parameters
• Utilities are somewhat unusual with one representing
perfect health and zero representing death
• Can have states worse than death so constraints are
negative infinity up to one
• If far from zero, pragmatic approach is to fit beta
distribution
• If it is important to represent negative utilities consider
the transformation X = 1- U (utility decrement) and fit
Gamma or log normal distribution to X
Distributions for common parameters
Relative risk parameters
• Relative risks are ratios!
• Can log transform to make additive
• Variances and confidence intervals are estimated on the
log-scale then exponentiated
• Suggests the log-normal distribution
Relative risk from published meta-analysis
Example
• Suppose a published meta analysis quotes a relative
risk of 0.86 with 95%CI(0.71 to 1.05)
• Log transform these to give
-0.15 (-0.35 to 0.05) on log scale
• Calculate the SE on log scale:
(0.05 - -0.35)/(1.96*2) = 0.1
• Generate a normally distributed random variable with
mean –0.15 and SE 0.10
• Exponentiate the resulting variable
Correlating parameters
• PSA has sometimes been criticised for treating
parameters as independent
• In principle can correlate parameters if we have
information on covariance structure
– e.g. covariance matrix in regression
• Cholesky decomposition used for correlated normal
distributions
• Correlations among other distributional forms not
straightforward
Variability and nonlinearity
Even if we are interested only in expected values we need
to consider uncertainty when nonlinearities are involved:
E[ g(x) ]  g( E[x])
• Uncertainty needed to calculate expectation of
nonlinear parameters
• Uncertainty needed to calculate expectation of
nonlinear models
Point estimates and variability
Standard point estimate
0.6
0.7
0.8
Expected value
0.9
Relative risk
RR: 0.86 (95% CI: 0.71-1.05)
1
1.1
1
A model of Total Hip Replacement
Example: interpreting simulation results
cPrimary
Primary THR
omrPTHR
1- (omrPTHR)
cSuccess
Successful
Primary
mr[age]
Death
uSuccessP
RR[age,sex,time]
omrRTHR + mr[age]
mr[age]
cRevision
cSuccess
Revision THR
1- (omrRTHR + mr[age])
uRevision
Successful
Revision
uSuccessR
RRR
Example on the CE plane
Spectron versus Charnley Hip prosthesis
£400
£200
Additional Cost
£-
-£200
-£400
-£600
-£800
-£1,000
-0.05
0.00
0.05
0.10
0.15
0.20
QALYs gained
0.25
0.30
0.35
0.40
Corresponding CEAC
Spectron versus Charnley Hip prosthesis
1.00
Probability cost-effective
0.90
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
£-
£5,000
£10,000
Value of ceiling ratio
£15,000
£20,000
Multiple acceptability curves
Why and how?
• Two reasons for employing multiple acceptability curves
– Heterogeneity between patient groups
– Multiple treatment options
• Correspond to two situations in CEA
– Independent programmes
– Mutually exclusive options
• Lead to two very different representations!
Multiple CEACs: handling heterogeneity
Spectron versus Charnley (Males)
1
Probability cost-effective
0.9
0.8
Ages 40 & 50
Age 60
Age 70
0.7
0.6
Age 80
0.5
0.4
Age 90
0.3
0.2
0.1
0
£-
£5,000
£10,000
Value of ceiling ratio
£15,000
£20,000
Multiple CEACs: handling heterogeneity
Spectron versus Charnley (Females)
1
Age 40
Age 50
Probability cost-effective
0.9
0.8
0.7
Age 60
Age 70
0.6
Age 80
0.5
0.4
Age 90
0.3
0.2
0.1
0
£-
£5,000
£10,000
Value of ceiling ratio
£15,000
£20,000
Example: GERD management
Baseline results
1200
F
900
800
W
B
4/GF
1000
A: Intermittent PPI
B: Maintenance PPI
C: Maintenance H2RA
D: Step-down maintenance PA
E: Step-down maintenance H2RA
F: Step-down maintenance PPI
$26
Strategy cost
1100
D
$10/GFW
700
FW E
G
/
36
$
A
C
600
38.00
39.00
40.00
41.00
42.00
43.00
44.00
Weeks free of GERD
45.00
46.00
47.00
48.00
Example: GERD management
$1,200
Uncertainty on the CE plane
Strategy cost
$1,100
B
$1,000
F
$900
$800
D
E
$700
A
C
$600
38.00
39.00
40.00
41.00
42.00
43.00
44.00
45.00
Weeks free of GERD
46.00
47.00
48.00
Example: GERD management
Multiple CEACs
1
Probability cost-effective
0.9
0.8
E
C
0.7
B
0.6
0.5
0.4
A
0.3
0.2
0.1
F
0
1
10
100
Ceiling Ratio (Rc)
1000
Using probabilistic analysis for making
decisions?
Link with standard statistical methods
1. Use standard inference (link with frequentist methods)
2. Use cost-effectiveness acceptability curves to allow
decision maker to select own ‘threshold’ error
probability (more Bayesian)
3. Use PSA to establish the value of collecting additional
information to inform decision (fully Bayesian decision
theoretic approach)
Cost of uncertainty (value of information)
£3,500,000,000
Net benefit probability density
Implementation loss function
Non-implementation loss function
£2,500,000,000
£2,000,000,000
£1,500,000,000
£1,000,000,000
£500,000,000
-£30,000
-£20,000
-£10,000
£0
£10,000
Net monetary benefit
£20,000
£0
£30,000
Opportunity loss
£3,000,000,000
Micro-simulation models and PSA
• Microsimulation is an ‘individual’ (rather than ‘cohort’)
method of model evaluation
• Typically used to capture patient histories
• Calculation requires large number of individual
simulations
• PSA would require a second ‘layer’ of simulations
(increases computational time)
• Think carefully about whether a micro simulation is
necessary
• If it is, buy a fast machine, or use an approximate
solution
Is there any role for standard sensitivity
analysis?
• Probabilistic sensitivity analysis is important for
capturing parameter uncertainty
• Other forms of uncertainty relate to
–
–
–
–
Methodology
Structural uncertainty
Data sources
Heterogeneity
• Standard sensitivity analysis retains an important role
(in conjuction with PSA)
Critiquing a probabilistic CE model
• Are all parameters included in PSA?
• Were standard distributions specified?
– No triangular/uniform distributions
• Was the appropriate expected value calculated?
• Was standard sensitivity analysis employed to handle
non sampling uncertainty?
• Was heterogeneity handled separately?
• Was the effect of individual parameters explored?
Summary: the role of PSA
PSA has important role to play
• Calculating the correct expected value
• Calculating combined effect of uncertainty in all
parameters
• Opening the debate about appropriate error probability
• Required to calculate the value of information
• Continuing role for standard sensitivity analysis