Transcript Implementation and Analysis of PROs in Clinical Trials Jeff A. Sloan, Ph.D. DIA Meeting, D.C., June 26, 2005 CM923712-1

Implementation and Analysis of
PROs in Clinical Trials
Jeff A. Sloan, Ph.D.
DIA Meeting, D.C., June 26, 2005
CM923712-1
Why is it difficult to deal with PROs?
• Relatively recent acceptance
• 25 years ago physicians were the sole
raters of patient pain
• JCAHO 2000 guideline: every patient’s
pain to be assessed upon intake on a 0-10
scale
• Time and experience alleviates novelty and
skepticism
CM923712-2
Checklist for designing, conducting and reporting
HRQL - PRO in clinical trials
Patient Reported Outcomes (PRO) and Regulatory Issues : A European Guidance Document for the
improved integration of health-related quality of life assessment in the drug regulatory process. Chassany
O et ERIQA Working Group. Drug Information Journal 2002.
HRQL / PRO objectives
Statistical analysis plan
• Added value of HRQL / PRO
• Choice of the questionnaires
• Hypotheses of HRQL / PRO changes
• Primary or secondary endpoint
Study design
• Superiority or equivalence trial
• Sample size
• ITT, type I error, missing data
• Basic principles of RCT fulfilled ?
• Timing and frequency of assessment
• Mode and site of administration...
Reporting of results
HRQL / PRO measure
Interpreting the results
• Description of the measure (items, domains…)
• Evidence of validity
• Evidence of cultural adaptation
CM923712-3
• Participation rate, data completeness
• Distribution of HRQL / PRO scores
• Effect size,
• Minimal Clinically Important Difference
• Comparison with other criteria / scores
• Number needed to treat…
Take home messages:
there is good news
• There are problems with using PROs as
indicators of efficacy in clinical trials.
• There are scientifically sound solutions
to these problems. The problems have
been disseminated widely and
consistently. The solutions have not.
CM923712-4
It takes a certain amount of bravery to work with PRO’s
CM923712-5
Primary goal: advance the state of the science
to help patients soar
CM923712-6
How do you analyze PRO data?
CM923712-7
Science is a candle in the dark
- Carl Sagan
We will use the candle of science to improve the QOL of cancer patients
CM923712-8
… by answering
scientific questions
• What is the value added of PROs to
treatment trials?
• How do you deal with multiple endpoints?
• How do you handle missing data?
• What is the clinical significance of PRO
assessments?
CM923712-9
What is the value added
of additional questions?
CM923712-10
CM923712-11
Single-Item or Multiple-Item PRO?
Situations where a single item may
suffice
Phase II study attempting to assess
whether a treatment has any impact on
QOL
A stratification factor for the presence
or absence of depressive issues
Need to assess fatigue/pain as a
correlate of toxicity (brief fatigue/pain
inventory)
Identifying patients who have need of
further QOL assessment (e.g., score of
6 or less on a single item)
A clinical setting wherein a basic idea
of which domains of QOL (mental,
physical, social) may be affected by a
particular treatment or situation
CM923712-12
Situations where a multi-item index
may be needed
A Phase III study where it is known that
QOL is impacted and more delineation
of which QOL components are affected
is needed
A screen to identify the presence or
absence of clinical depression
Need to assess the impact of
fatigue/pain on the activities of daily
living (ADL items for pain/fatigue)
Detailing the QOL-related issues once
a cut off score on a single item has
been obtained
A clinical setting wherein precise
indications of the way in which the
different domains of QOL may be
affected by a particular treatment or
situation
Guidelines for endpoint determination
• Several good references (Beitz, 1996;
Chassany, 2002; Fayers, 1999; Sloan, 2002)
• Reliability and validity data available
• Pilot/focus groups to establish R/V
• What aspects of PROs are likely to
•
CM923712-13
change?
Can one expect an overall change in
well-being, health status or QOL?
Recipe for endpoint determination
• List PRO aspects likely to change.
• Operationalize each item from a tool.
• Survey clinicians/patients if unsure.
• Keep the total number of items under 25.
• Mock up tables with “perfect world” data,
•
labels with “perfect” results.
Link sample size to a priori clinical
significance.
CM923712-14
How do you deal
with multiple endpoints?
CM923712-15
An example of combined symptoms:
Gemzar (gemcitabine)
• Indication: Advanced pancreatic cancer
• Instrument or Method:
• Negotiated PRO outcome, “clinical benefit
•
•
response”
PRO Domains Assessed:
• Pain, analgesic consumption, performance
status, weight
Results:
• Clinical benefit response was experienced by
24% of patients receiving Gemzar versus 5% of
patients receiving 5FU, p=0.002
CM923712-16
Gemzar-specific clinical benefit response
Analgesic
Consumption
Pain
Intensity
Performance
Scales
Pain
Responder
Improvement in both
parameters
Stable in one parameter,
improvement in another
parameter
Stable
Nonresponder
In both parameters
Worsening in either
parameter
Weight
CM923712-17
Responder
Nonresponder
21% Increase in body weight
Stable or decreased weight
A patient was considered a clinical
benefit responder to Gemzar if ….
• The patient showed >=50% reduction in pain
•
intensity or analgesic consumption, or a 20+
point improvement in performance status (for at
least 4 weeks with no worsening of other
parameters)
• Memorial Pain Assessment Card and
Karnofsky Performance Scale
The patient was stable on all of the parameters
mentioned and showed a marked, sustained
weight gain not due to fluid accumulation ( >7%
increase maintained for 4 weeks)
CM923712-18
O’Brien Global Test
for Multiple Outcomes
• Example: Venlafaxine for Hot Flashes
• Hot flash frequency per day
• Hot flash average severity per day
• none, mild, moderate, severe, very severe
• scored 0, 1, 2, 3, 4
• Hot flash score (severity times frequency)
• Uniscale QOL
• Hot flash affect on QOL
• Toxicity incidence on 11 variables
CM923712-19
O’Brien p-values
Endpoints Included
Hot Flash Frequency
Hot Flash Average Severity
Hot Flash Score
Uniscale QOL
Hot Flash Affects QOL
Toxicity
CM923712-20
p-value
0.0071
0.0050
0.7528
How do you handle the
problem of missing data?
CM923712-21
Impact of hydrazine sulfate on colorectal cancer patient QOL
Impact of different imputation methods for missing data
CM923712-22
Effect of imputation method on treatment comparison
CM923712-23
The data are usually trying to tell you something….
…you just have to pay attention
CM923712-24
What is the clinical significance
of PRO assessments?
CM923712-25
Two general methods
for clinical significance
• Anchor-based methods requirements
• independent interpretable
measure (the anchor) which has
appreciable correlation between
anchor and target
• Distribution-based methods
• rely on expression of magnitude
CM923712-26
of effect in terms of measure of
variability of results (effect size)
The MID method in one slide
CM923712-27
The ERES Approach
• QOL tool range = 6 standard Deviations
• SD Estimate = 100 percent / 6
= 16.7% of theoretical range
• Two-sample t-test effect sizes (Cohen):
small, moderate, large effect (0.2, 0.5, 0.8 SD
shift)
• S,M,L effects = 3%, 8%, 12% of range
CM923712-28
Assessing Clinical Significance
• 1) Methods used to date
• 2) Group versus individual differences
• 3) Single item versus multi-item
• 4) Patient, clinician, population perspectives
• 5) Changes over time
• 6) Practical considerations for specific
audiences
• MCP, April, May, June 2002
CM923712-29
The solutions found for tumor response
cutoffs may provide guidance
• We call a reduction of 50% a response.
• Have reductions of 49% all the time, but
do not worry about misclassification.
• Moertel (1976) basis for 50% cutoff
• Find a cutoff and stick to it?
CM923712-30
The Good News
• Statistical, Philosophical, Empirical, Clinical,
Historical, Practical approaches to defining a
clinically significant effect for symptom
assessments are all in the same ballpark
• A 10 point difference on a 100-point scale (1/2 SD)
is almost always going to be clinically significant
• Smaller differences may also be meaningful (data)
• Applies to groups or individuals (just different SD)
Norman GR, Sloan JA, Wyrwich KW. Expert Review of Pharmacoeconomics and Outcomes
Research Sept 2004; 4(5): 515 – 519
Sloan JA, Cella D, Hays R. J Clin Epidemiol (in press).
CM923712-31
What’s next?
CM923712-32
A Mayo/NCCTG meeting on
FDA guidances on patient-reported outcomes (PRO)
Discussion, Education, and Operationalization
• FDA to release guidances for assessing PRO’s in all
clinical trials (3rd quarter 2005?)
• Meeting co-sponsored with FDA to:
• provide a focused process to facilitate discussion among all
stakeholders
• educate stakeholders on background, content, and concerns
• provide an opportunity for input
• delineate ways to best operationalize the guidance into clinical
trials
• February 23-25, 2006, DC (Westfields Marriott, Chantilly,
VA, 7 miles from Dulles)
• Seeking stakeholders involvement
CM923712-33
New ideas have enabled us to make advances in PRO science
With your help, there will be more to come
CM923712-34
Thank you
References: [email protected]
CM923712-35