Transcript Document

Is PROMIS Promising for
Neurotherapeutics Research?
David Cella, PhD
Northwestern University
ASENT – February 24, 2011
Dynamic Tools to Measure Health Outcomes from the Patient Perspective
®
PROMIS
Measures
Science
Software
Challenges
•
•
•
•
•
•
•
Many measures of same
health concept
Widely varying quality
Difficult to compare and
combine data
. . . across studies
. . . across conditions
Complex
Long
“The clinical outcomes research
enterprise would be enhanced
greatly by the availability of a
psychometrically validated,
dynamic system to measure
PROs efficiently in study
participants with a wide range of
chronic diseases and
demographic characteristics.”
National Institutes of Health, 2003
PROMIS is a 6 year old, 15-site
Cooperative Group
More than 30 research protocols aligned with
evolving PROMIS standards
Nearly 40,000 people have contributed data
> 1,500 in qualitative research
> 35,000 in quantitative research
More than 9,000 children
More than 2,000 adult proxies for children
More than 25,000 adults on their own behalf
…including more than 4,000 Spanish-speaking adults and children
Early Engagement with
Clinical Researchers
Early Engagement with
Clinical Researchers
Identified most
common needs
via “use cases”
2005 PROMIS I 20092010 PROMIS II 2013
6 sites
Create
measures
12 sites
February 24, 2011
Assessment
Center
New measures
Additional
Validity studies invalidity testing
clinical
populations
Comparability
Chemotherapy
trial
Stroke trial
Heart failure
trial
Reliability and
Validity
Flexibility
Value of PROMIS
1. Comparability
2. Reliability and Validity
3. Flexibility
PROMIS is
Domain-specific;
not Disease-specific
“Domain”
“Item bank”
“Domain”
“Item bank”
A domain is the specific
feeling, function, or
perception you want to
measure.
Cuts across different diseases
PROMIS Domain Framework
Symptoms
Physical Health
Function
Affect
Self-Reported
Health
Mental Health
Behavior
Cognition
Social
Health
Relationships
Function
“Domain”
“Item bank”
An item bank is a large
collection of items measuring a
single domain.
Any and all items can be used to
provide a score for that domain.
Cycle of Development and
Validation
Qualitative
Research
and Item
Writing
Testing
Item
Bank
Analysis
Interpretation
Refining
General Population
Clinical Samples
PROMIS Current Physical Health Banks
Adult
Pediatric
Pain Behavior
Physical
Health
Pain Interference
Pain Interference
Fatigue
Fatigue
Physical Function
Upper Extremity
Function
Sleep Disturbance
Mobility
Sleep-related
Impairment
Asthma Impact
Sexual Function
PROMIS Current Mental Health Banks
Adult
Mental
Health
Pediatric
Anxiety
Anxiety
Depression
Depression
Anger
Anger
Illness Impact
Negative
Illness Impact Positive
Applied Cognition Concerns
Applied Cognition Abilities
PROMIS Current Social Health Banks
Adult
Ability to Participate
in Roles & Activities
Social
Health
Satisfaction with Roles
& Activities
Companionship
Emotional Support
Informational
Support
Instrumental Support
Social Isolation
Pediatric
Peer Relationships
These are the Domains;
What is the Metric?
•
•
T Score
 Mean = 50
 SD = 10
Referenced to the US General
Population
0
50
100
Physical Functioning Item Bank
Item
1
Item
2
Item
3
Item
4
Item
5
Item
6
Item
7
Item
8
Are you able to get in and out of bed?
Are you able to stand without losing your balance for 1 minute?
Are you able to walk from one room to another?
Are you able to walk a block on flat ground?
Are you able to run or jog for two miles?
Are you able to run five miles?
Item
9
Item
n
Value of PROMIS
1. Comparability
2. Reliability and Validity
3. Flexibility
Comparability
Qualitative Research and
Modern Measurement
Enable us to Build a
Common PRO Language
The Beginning:
2004-2005
• Building a Health Outcome Framework of
Common PRO Domains and Definitions
• Informing Consensus with Literature,
Analysis and Qualitative Research
PROMIS Predates but is Aligned
with FDA PRO Guidance
Defining Common Domains of
Self-Reported Health
Defining Common Domains of
Self-Reported Health
Literature review
Defining Common Domains of
Self-Reported Health
Literature review
79 focus groups with
patients from different
disease populations
Dealing with Multiple
Measures of Same Domain
Physical
Function
Fatigue
Depression,
Anger,
Anxiety
Items identified
1,860
1,066
2,187
Items after review
and revision
247
135
299
PROMIS Basic Tools
Derived from Item Banks
•
Computerized Adaptive Testing (CAT)
•
Fixed Length Forms
 Dynamic testing averaging 6 items per domain
 By individual domain (8-10 items)
 By health profile (-29, -43, -57)
Global Health Index
•
Global-10
Demonstration of
Computerized
Adaptive Testing
(CAT)
Time 1
Fatigue, Pain Interference,
Depression
Moderate Symptoms
Time 1 Scores
Time 2
Fatigue, Pain Interference,
Depression
Normal to Mild Symptoms
Comparing Time 1 and Time 2
PROMIS Profile Short Forms
4
Mental
Physical
Social
Anxiety
29
Depression
28
Fatigue
95
Pain Interference
41
Sleep Disturbance
27
Physical Function
112
Social Role
14
6
8
Common measurement
domains and metrics
across conditions
What does it mean for
clinical research and policy?
We can compare conditions
on individual PROMIS
domains.
Fatigue Item Bank
Chemotherapy
trial
Osteoarthritis
trial
Parkinson’s
Disease trial
Items 1-10
CAT
Items 6-12
Diabetes trial
Items 2, 4, 9, 13
Epilepsy trial
Items 1-5
Same metric, same meaning
Reliability and
Validity
Reliability (Precision)
Relative Precision of PROs
Error
High
(Physical Function)
SF-36
HAQ
HAQ
(10 items)
items) CAT
(20(20
items)
(10 items)
Full Item Bank
(126 items)
Low
Rheumatoid
Arthritis Patients
Worse
Representative Sample
Physical Functioning
(T-Score; Mean=50, SD=10)
Better
Relative Precision of PROs
Error
High
(Physical Function)
Low
Rheumatoid
Arthritis Patients
Worse
Representative Sample
Physical Functioning
(T-Score; Mean=50, SD=10)
Better
Validity
Scores on PROMIS measures
should correlate with accepted
measures of the same domain
(Concurrent Validity)
60
60
Depression
50
40
0
10
20
30
CESD
30
20
10
0
0
100
250
-2
-1
0
1
2
3
4
2
3
4
Depression
0 80
CESD
40
50
r =0.84
-2
-1
0
1
PROMIS
Depression
Depression
When people experience clinical
benefit or decline, their PROMIS
scores should also change
(Responsiveness)
Clinical Validation:
Outpatient Depressive Disorder
•
Sample Size
 127
• Population
 Adults who started treatment for Major
Depressive Disorder in the last 4 months at
Western Psychiatric Institute and Clinic
• Assessment Times
 Baseline, 1 month, 3 month
Responsiveness in Depressive Disorder:
PROMIS Anger
Responsiveness in Depressive Disorder:
PROMIS Anxiety
Responsiveness in Depressive Disorder:
PROMIS Depression
Clinical Validation:
Back Pain with Sciatica
•
Sample Size
 226
• Population
 Adults with at least 6 weeks of back pain with
sciatica scheduled for epidural steroid
injection
• Assessment Times
 Baseline, 1 month, 3 month
Responsiveness in Back Pain:
PROMIS Pain Interference
Responsiveness in Back Pain:
PROMIS Pain Behavior
Flexibility
Assessment Center
www.nihpromis.org
PROMIS Instruments Available
on Assessment Center
•
•
•
All Item banks using CAT
All short forms
Profiles

29, 43, 57 item versions
CAT
Graph
Multiple ways to
administer
measures
Do different modes of
administration produce
differences in how people
respond to items?
Mode of Administration Study
N = 921
Measures
community adults
Fatigue
Depression
and arthritis
Physical Functioning
patients
=
=
=
No meaningful differences
found between modes of
administration
< 1.5 points on 100-point scale
The Promise of PROMIS
1. Comparability
2. Reliability and Validity
3. Flexibility
www.nihpromis.org