Transcript CaPS PROMIS SC 2009 - Welcome to GIM & HSR

A Comprehensive Approach to the
Measurement of Health Outcomes
Ron D. Hays, Ph.D
UCLA Division of General Internal Medicine &
Health Services Research
K30 Module 2
November 16, 2010 (9:00-10:30 am)
1st floor Conference Room 1357, UCLA
Patient-Reported Outcomes Measurement
Information System (PROMIS)
• A nine-year $70 million commitment of
NIH to improve and standardize
measurement of patient-reported
outcomes (PROs)
– Self-reported health
• An answer to the PRO “Tower of Babel”
The Tower of Babel (Brueghel, 1563)
3
PROMIS-1 Network:
2004-2009
● University of Washington
Northwestern ♥
● Stanford
● StoneyBrook
● University of Pittsburgh
● NIH
UNC –Chapel Hill ●● Duke University*
♥Coordinating Center
Patient
Focus
Groups
Literature
Review
Expert
Input and
Consensus
Existing
Items
Newly
Written
Items
Item Pool
Expert
Review
 


Questionnaire
administered to large
representative sample
Secondary
Data Analysis



2.5
2.0
0.8
Psychometric
Testing
0.6
0.4
0.2
Information
Probability of Response
1.0
Cognitive
Testing
Translation
1.5
1.0
0.5
0.0
0.0
-3
-2
-1
0
1
2
3
-3
Item Bank
(IRT-calibrated items)
Short Form
Instruments
-2
-1
0
1
2
Theta
Theta
CAT
3
The Life Story of a PROMIS Item
Focus
groups
Binning
and
winnowing
Domain
Framework
Literature
review
Archival
data
analysis
Large-scale
testing
Cognitive
interviews
Expert item
revision
Translation
review
Literacy
level
analysis
Statistical
analysis
Intellectual
property
Calibration
decisions
Short form
Validation
studies
Expert
review/
consensus
CAT
   
  

50
Physical Functioning Item Bank
Item
Item
Item
Item
Item
Item
Item
Item
Item
Item
1
2
3
4
5
6
7
8
9
n
•Are you able to get in and out of bed?
•Are you able to stand without losing your balance for 1 minute?
•Are you able to walk from one room to another?
•Are you able to walk a block on flat ground?
•Are you able to run or jog for two miles?
•Are you able to run five miles?
Interpretation
Person Fatigue Score
Low







 
 
 
 
High
Likely Q Q QQ QQQQQQ QQQQQ Q Q Q Q Q Q Q Q Unlikely
Q Q QQ Q Q Q
“I get tired
Q Q QQ Q Q QQ Q Q Q
“I get tired
Q Q Q Q Q Q Q
when I run
when I get
Q Q Q
a marathon”
out of a
chair”
Item Location
Interpretation Aids
PRO Bank Person Score
Low







 
 
 
 
30
40
50
60
M = 50, SD = 10
T = (z * 10) + 50
70
High
Example of high fatigue
Fatigue Score=60
Low







 
 
 
 
30
40
50
60
High
70
This patient’s fatigue score is 60, significantly worse than average (50). People
who score 60 on fatigue tend to answer questions as follows:
…”I have been too tired to climb one flight of stairs: VERY MUCH
…”I have had enough energy to go out with my family: A LITTLE BIT
Example of low fatigue
Fatigue Score=40
Low







 
 
 
 
30
40
50
60
High
70
This patient’s fatigue score is 40, significantly better than average (50). People who
score 40 on fatigue tend to answer questions as follows:
…”I have been too tired to climb one flight of stairs: SOMEWHAT
…”I have had enough energy to go out with my family: VERY MUCH
Computerized Adaptive Testing (CAT)
• Select questions based on a person’s response
to previously administered questions.
• Iteratively estimate a person’s standing on a
domain (e.g., fatigue, depressive symptoms)
• Administer most informative items
• Desired level of precision can be obtained using
the minimal possible number of questions.
Best Item-I felt depressed
Beginning of CAT
1.0
Item 15
0.6
0.4
0.2
Probability
0.8
1
2
3
4
5
SE = 10
0.0
T-Score = 50
20
30
40
50
60
70
80
70
80
T-Score
Item 15
Max at T-Score=56
Items: 15
5
80
4
70
3
Information
50
2
T-Score
Posterior Distribution
60
1
40
20
20
30
40
50
T-Score
60
70
80
T-Score: 50 SEM: 10
0
1
2
3
4
5
Items Administered
|
0
30
6
7
8
20
30
40
50
T-Score
60
Next Best Item-I felt like a failure
I felt depressed
Never
Rarely
Sometimes
Often
Always
1.0
Item 10
0.4
0.6
0.8
1
2
3
4
5
0.2
Probability
1.
2.
3.
4.
5.
SE = 4
0.0
T-Score = 52
20
30
40
50
60
70
80
70
80
T-Score
Item 10
Items: 15
Max at T-Score=57
5
80
Posterior Distribution
4
70
3
Information
50
2
T-Score
60
1
40
20
20
30
40
50
T-Score
60
70
80
T-Score: 52 SEM: 4
0
1
2
3
4
5
Items Administered
|
0
30
6
7
8
20
30
40
50
T-Score
60
Next Best Item-I felt worthless
I felt like a failure
Never
Rarely
Sometimes
Often
Always
1.0
Item 1
0.4
0.6
0.8
1
2
3
4
5
0.2
Probability
1.
2.
3.
4.
5.
SE = 3
0.0
T-Score = 53
20
30
40
50
60
70
80
70
80
T-Score
Item 1
Items: 15,10
Max at T-Score=59
5
80
Posterior Distribution
4
70
3
Information
50
2
T-Score
60
1
40
20
20
30
40
50
T-Score
60
70
80
T-Score: 53 SEM: 3
0
1
2
3
4
5
Items Administered
|
0
30
6
7
8
20
30
40
50
T-Score
60
Next Best Item-I felt helpless
I felt worthless
Never
Rarely
Sometimes
Often
Always
1.0
Item 3
0.4
0.6
0.8
1
2
3
4
5
0.2
Probability
1.
2.
3.
4.
5.
SE = 2
0.0
T-Score = 55
20
30
40
50
60
70
80
70
80
T-Score
Item 3
Items: 15,10,1
Max at T-Score=58
5
80
Posterior Distribution
4
70
3
Information
50
2
T-Score
60
1
40
20
20
30
40
50
T-Score
60
70
80
T-Score: 55 SEM: 2
0
1
2
3
4
5
Items Administered
|
0
30
6
7
8
20
30
40
50
T-Score
60
I felt helpless
Never
Rarely
Sometimes
Often
Always
SE =
2
Items: 15,10,1,3,21,2,5
80
70
Posterior Distribution
60
T-Score
1.
2.
3.
4.
5.
T-Score
= 55
50
40
30
20
20
30
40
50
T-Score
60
70
80
T-Score: 55 SEM: 2
0
1
2
3
4
5
Items Administered
6
7
8
CAT assessments can achieve higher
precision than fixed forms
measurement precision (standard error)
0.6
5
0.5
SF-12 items
4
0.4
SF-36 items
SE = 3.2
rel = 0.90
3
0.3
HAQ items
SE = 2.2
rel = 0.95
2
0.2
CAT 10 items
Full Item Bank
1
0.1
rheumatoid arthritis
patients
representative
sample
0
0
10
10
20
20
30
30
40
40
50
50
60
60
70
70
normed theta values
Rose et al, J Clin Epidemiol 2007 (accepted)
US-Representative
Sample
80
PROMIS Domains in AC, 2010
# Items
Adult Bank
29
# Items Adult
Short
Forms
8
# Items
Peds
Bank
# Items
Peds Short
Form
6
29
4, 6, 7, 8
15
8
28
4, 6, 8a, 8b
14
8
95
4, 6, 7, 8
23
10
Pain – Behavior
39
7
Pain – Interference
41
4, 6a, 6b, 8
13
8
Physical Function
124
4, 6, 8, 10, 20
-- Mobility
23
8
-- Upper Extremity
29
8
Asthma Impact
17
8
Domains
Emotional Distress –
Anger
Emotional Distress –
Anxiety
Emotional Distress –
Depression
Fatigue
PROMIS Domains in AC, 2010
Domains
Satisfaction with
Discretionary
Social Activities
Satisfaction with
Social Roles
Peer Relationships
# Items
Adult Bank
12
# Items Adult
Short
Forms
7
14
4, 6, 7, 8
Sleep Disturbance
27
4, 6, 8a, 8b
Sleep-Related
Impairment
Global Health
16
8
10
# Items
Peds
Bank
# Items
Peds Short
Forms
15
8
2010 PROMIS Profile Instruments
Domains
Emotional Distress –
Anxiety
Emotional Distress –
Depression
Fatigue
PROMIS-29
4
PROMIS-43
6
PROMIS-57
8
4
6
8
4
6
8
Pain – Interference
4
6
8
Pain – Intensity
1
1
1
Satisfaction with Social
Role
Sleep Disturbance
4
6
8
4
6
8
Physical Function
4
6
8
8
6
4
Anxiety
Mental
Depression
Fatigue
Pain Interference
Physical
Sleep Disturbance
Physical Function
Social
Social Role
Thank You
www.nihpromis.org
Reliability and SEM
• For z-scores (mean = 0 and SD = 1):
– Reliability = 1 – SEM2
= 0.91 (when SEM = 0.30)
= 0.90 (when SEM = 0.32)
• With 0.90 reliability
– 95% Confidence Interval
• z-score:
- 0.62  0.62
• T-score = (z-score * 10) + 50
• T-score:
44  56
35