kraemer 6 18
Download
Report
Transcript kraemer 6 18
Clinically Significant Treatment Effects:
Effect Sizes and Moderators
Helena Chmura Kraemer, Ph.D.
Stanford University (Emerita)
University of Pittsburgh
Randomized Clinical Trails
(RCTs)
RCT: the “gold standard” method to
evaluate the efficacy or effectiveness
of a treatment in a population.
RCT: Background
A patient’s response following
treatment combines:
• Statistical Regression to the mean
• Expectation Effects
• Spontaneous changes
• Secular trends
• PLUS: the actual effect of treatment
Thus the effect of treatment cannot
be assessed in pre-post studies.
RCT: The Effect of Treatment
Definition of “Effect of Treatment”:
• Response(T) compared to Response(not-T)
Requires some control/comparison condition:
“not-T”=C
Requires measurement of response that is not
influenced by T or C: “blindness”
Problem: Cannot observe any individual patient
both with T and not-T…. Thus cannot assess the
effect of treatment on an individual patient.
RCT: Randomization
Can assess the typical effect of treatment in a
population by:
• Sampling the population of interest
• Randomize patients to T or to C.
• Compare T vs C
Treatment effect on the “typical” patient on.
Analysis “by intention to treat”.
There is no assumption that the effect of
treatment is the same for all individuals in the
population.
RCT: Comparisons? (Classic)
“Statistical Significance” means that the
design (sample size) was good enough to
detect a non-random difference.
• Comment on design, not on effectiveness
of treatment
A treatment effect may be statistically
significant and clinically trivial.
• Absence of “statistical significance” means
Flawed conceptualization, inadequate
design, measurement, sample size
NOT absence of “clinical significance”.
NEEDED: A measure of effect size that allows
assessment of clinical significance.
Powering up a RCT
T<C
1.000
T=C
T>C
Prob significant result
0.900
0.800
0.700
0.600
0.500
0.400
0.300
0.200
0.100
0.000
0.0
0.2
0.4
0.6
Effect Size, AUC
0.8
1.0
The ”critical value” or “threshold
of clinical significance”?
Below this effect size, clinicians and
patients would consider the effect
clinical trivial.
The more above this value, the
greater the preference for T over C.
Obtained from previous research,
clinical experience, opinion of
experts.
Effect Sizes—Recommended
AUC=Prob(T>C)+.5Prob(T=C)
• Null value: .5; Extremes: 0,1.
SRD=Prob(T>C)-Prob(T<C)=2AUC-1
• Null value: 0; Extremes: -1, +1.
Number Needed to Treat: NNT=1/SRD.
How many patients would you have to
treat with T to get one more success than
if you had treated them with C?
• Null value: infinity: Extremes: -1, +1.
Effect Sizes—More Common
Cohen’s d= Standardized mean
difference between T and C.
• Meant to be used when the responses
have a normal distribution in both T and
C. Almost never exact!
• However, when reasonable:
AUC=normsdist(d/√2)
Odds Ratio—not recommended.
• NNT> (√OR+1)/(√OR-1)
Effect Size—Standards
d
AUC
SRD
NNT
0 Null
.50
0
Infinity
.2 Small
.56
.11
8.9
.5 Med
.64
.28
3.6
.8 Large
.71
.43
2.3
1.0
.76
.52
1.9
Possible RCT Outcomes T1 vs T2: c the critical
effect size: 95% Confidence Intervals for the SRD
*T1 is clinically superior to T2
*T1 is non-inferior to T2
*T1 and T2 clinically equivalent.
T1 and T2 clinically equivalent.
A failed RCT
-1
-c
T1<T2
0
T1=T2
+c
Effect Size (SRD)
+1
T1>T2
Moderators and Mediators of
Treatment in a RCT
Moderator: M moderates the effect of T in
a RCT on O if
• M is a baseline variable (hence precedes T and
O and is uncorrelated with T)
• The effect size of T on O differs depending on
what M is.
Mediator: M mediates the effect of T in a
RCT on O if
• M is an event or change that happens during
treatment (hence follows T but precedes O).
• M is correlated with T
• The effect size of T on O is explained wholly or
in part by the effect of T on M.
Examples: Moderators
Gene (5HTT) moderates the effect of
drug treatment on outcome for
depressed patients
(pharmacognetics).
Baseline depression moderates the
effect of psychotherapy for treatment
of anorexia .
Level of addiction moderates the
effect of smoking cessation treatment
on abstinence.
Example 2: Exploration after a RCT.
How???
Exploration
Secondary
Data analysis
RCT Hypothesis
RCT Design
Publication
Validation
Pilot Study
RCT