Working Paper No.7 22 November 2005 STATISTICAL COMMISSION and UN ECONOMIC COMMISSION FOR EUROPE STATISTICAL OFFICE OF THE EUROPEAN COMMUNITIES (EUROSTAT) CONFERENCE OF EUROPEAN STATISTICIANS WORLD HEALTH ORGANIZATION (WHO) Joint UNECE/WHO/Eurostat Meeting on.
Download ReportTranscript Working Paper No.7 22 November 2005 STATISTICAL COMMISSION and UN ECONOMIC COMMISSION FOR EUROPE STATISTICAL OFFICE OF THE EUROPEAN COMMUNITIES (EUROSTAT) CONFERENCE OF EUROPEAN STATISTICIANS WORLD HEALTH ORGANIZATION (WHO) Joint UNECE/WHO/Eurostat Meeting on.
STATISTICAL COMMISSION and UN ECONOMIC COMMISSION FOR EUROPE CONFERENCE OF EUROPEAN STATISTICIANS
Joint UNECE/WHO/Eurostat Meeting on the Measurement of Health Status (Budapest, Hungary, 14-16 November 2005)
Session 2 – Invited paper
Working Paper No.7
22 November 2005
STATISTICAL OFFICE OF THE EUROPEAN COMMUNITIES (EUROSTAT) WORLD HEALTH ORGANIZATION (WHO) Overview of PROMIS Network
University of Washington
PROMIS
This work was funded by the National Institutes of Health through the NIH Roadmap for Medical Research. Principal Investigator Deborah N. Ader, Ph.D.
Director, Behavioral and Prevention Research Program National Institute of Arthritis and Musculoskeletal and Skin Diseases
Goals, Network, Tasks
Presented by:
Dagmar Amtmann, Ph.D.
PI University of Washington Center for Outcomes in Rehabilitation Research
PROMIS
P atient R eported O utcomes M easurement I nformation S ystem
The PROMIS Outcome…
To improve assessment of self reported symptoms and other health related quality of life measures across disabilities.
Primary uses:
• Enhance research • Improve clinical decision-making • Facilitate policy-making by health plan and systems and public programs
The PROMIS is on “the map” …
The PROMIS is one of the key projects under the “Re-engineering the Clinical Research Enterprises” branch of the trans-NIH Roadmap Initiative. All NIH institutes contribute funds to the initiative.
PROMIS will develop:
A publicly available, adaptable and sustainable Internet-based system that will: 1.
Administer individually “tailored” questionnaires (using Computer Adaptive Testing (CAT) technology) to measure health status 2. Collect and analyze responses
The Structure of the PROMIS Network
PRS PRS PRS S C C SC NIH Science Officers SAB
The PROMIS
Network
●
University of Washington
●
Stanford University
● ●
Evanston Northwestern Healthcare
♦ ● ● ●
University of Pittsburgh Stony Brook
●
University
● ●
▲ NIH UNC –Chapel Hill
●●
Duke University
Goals of PROMIS
• Item banks for specific domains • Relevant for a variety of chronic illnesses • Relevant for a variety of literacy levels • English, Spanish, and then other languages if funding can be acquired • Ability to “cross-walk” between PROMIS score and selected Legacy Measures
Items from Instrument A Items from Instrument B Items from Instrument C New Items Item Pool Content Expert Review Focus Groups Cognitive Testing Secondary Data Analysis 1.0
0.8
0.6
0.4
0.2
0.0
-3 -2
-1 0 Theta 1
2
3 Questionnaire administered to large representative sample
2.5
2.0
Item Response Theory (IRT) 1.5
1.0
0.5
0.0
-3 -2 -1 0 Theta 1 Item Bank (IRT-calibrated items reviewed for reliability, validity, and sensitivity)
CAT
2 Short Form Instruments 3
PROMIS Data Repository
Assessment Module
Laptop PDA Phone Web
Web Laptop PDA IVR Import Module Paper and Pencil Central Database Item Banks Study Protocols Users Patients Patient Reported Data Report Module Patients Clinicians Study Coordinators
What is the PROMIS Time Line?
2005-06: 2007:
Choose specific domains Identify, review instruments and items
April 2006:
Collect pilot response data
2008:
Continue data collection Create alpha version of CAT Conduct final calibration process, Put CAT into final form
2009:
Carry out feasibility testing Build sustaining partnerships
PROMIS PRS’s
• Each PROMIS
primary research sites (PRS)
has its own Independent Project, in addition to contributing to the Network goal of developing an item bank and CAT module for measuring patient reported outcomes in adults.
PROMIS Independent Projects and
• • •
PIs
Duke University:
Kevin Weinfurt, PhD
Challenges for Using IRT-Based Assessments in Multi-center Clinical Trials
UNC, Chapel Hill:
Harry A. Guess, MD, PhD
Pediatric Reported Outcomes Assessment Using CAT ("Kitty")
University of Pittsburgh:
Paul A. Pilkonis, PhD
Psychiatric Symptoms and Social Functioning: IRT and DIF
PROMIS Independent Projects and PIs
Continued •
Stanford University:
James F. Fries, MD
Improved Outcome Assessment in Arthritis and Aging
•
Stony Brook University:
Arthur A. Stone, PhD
Ecological Validity in Patient-Reported Chronic Disease Outcomes
•
University of Washington:
Dagmar Amtmann, PhD
Improving Measurement of Pain and Fatigue and Increasing the Scientific Understanding of Pain and Fatigue in Children and Adults with Disabilities
Health PRO
PROMIS Domain Framework
General Health Physical Health Satisfaction Function/Disability Symptoms Mental Health Satisfaction Emotional Distress Cognitive Function Positive Psychological Functioning Social Health Satisfaction Role Participation Social Support Upper Extremities (ADL): grip, buttons, etc Lower Extremities (ADL): walking, arising, etc.
Central (ADL): neck & back (twisting, bending) Activities: IADL (e.g. errands) Pain Fatigue Other Anxiety Depression Anger/Aggression Alcohol & Substance Use Negative Impacts of illness Subjective Well-Being (positive effect) Meaning and Coherence (spirituality) Mastery and Control (self-efficacy) Positive Impacts of Illness Performance Satisfaction
Domain Definition: Physical Function
Physical Function is defined as
one's ability to carry out various activities, ranging from self-care (activities of daily living) to more challenging and vigorous activities that require increasing degrees of mobility, strength or endurance
(Stewart & Kamberg, 1992; Haley, Coster & Binda-Sundberg, 1994; Haley, McHorney & Ware, 1994; Wilson & Cleary, 1995)*. Physical Function items, when considered as an outcome endpoint for clinical research in chronic illness, have a “capability” stem and a corresponding “capability” set of response items (e.g., “Are you able to…normally, with some difficulty, with moderate difficulty, with great difficulty, unable to do.”). This specifically excludes some items that may have great utility in other settings, as with “performance” items with the "Do you?" type of stem, which get at social or psychological issues, and “satisfaction” items (e.g., “How satisfied are you with your disability?”), which get at coping, stress, anxiety, etc. Also, since Physical Function/Disability is a much more fixed latent trait in chronic disease than Pain and some other domains, the response options will seldom be best expressed as "frequency" (“A little bit of the time”, etc.). Further, since many persons with a chronic disease will have more than one chronic disease and cannot distinguish the fraction of a problem attributable to each disease, Physical Function items attempt to quantitate the sum of these disease effects, leaving the teasing out of relative contributions to the analysis stage.
Domain Definition: Pain
Pain is an
unpleasant sensory and emotional experience associated with actual or potential tissue damage, or described in terms of such damage
(Sherbourne, 1992a; Merskey & Bogduk, 1994; Chang, 1999; Meuser, et al, 2001). Pain is what the patient says it is (that is, the "gold standard" of pain assessment is self-report).
Domain Definition: Fatigue
Fatigue is defined as an
overwhelming, debilitating and sustained sense of exhaustion that decreases one’s ability to carry out daily activities, including the ability to work effectively and to function at one’s usual level in family or social roles
(Stewart, Hays & Ware, 1992; North American Nursing Diagnosis Association, 1996; Glaus, 1998).
Domain Definition: Emotional Distress
Emotional distress refers commonly to
unpleasant emotions or cognitions that may interfere with the ability to cope with a disease, its physical symptoms, and its treatment
. It covers a wide range of feelings, including worry, powerlessness, sadness, fear, depression, anxiety and panic (Schag et al, 1994; Lawton, Parmellee, Katz & Nesselroade, 1996; van’t Spijker, et al, 1997; Bottomley, 1998a; 1998b; Stark, et al, 2002)*. Problems in mental health may also be manifested, however, in maladaptive behaviors less commonly associated with subjective distress. For example, research in psychiatric nosology (Krueger, 1999) suggests that the “structure of common mental disorders” (excluding psychosis) can be captured at a general (second-order) level by two dimensions of psychopathology: an internalizing dimension reflected in unipolar depression and anxiety disorders and an externalizing dimension reflected in alcohol and substance use and antisocial behavior.
The PROMIS Core Domain Definition: Social Role Participation
Social Role Participation
: Role participation refers to involvement in, and satisfaction with, one’s usual social roles, including marital relationships, parental responsibilities, work abilities and social activities (Sherbourne, 1992b; McDowell & Newell, 1996; Dijkers, Whiteneck & El-Jaroudi, 2000)*. This has also been referred to as social adjustment (McDowell & Newell, 1996).
*References provided in the slide notes
Current State of PROMIS Items
• Items mostly from classical test theory • Created for a specific scale • Large variety of response options • Large variety of recall periods • Various methodologies in the development and validation of items
Qualitative Item Review Process
• Identification of extant items • “Binning and Winnowing” • Legacy scales • Expert item revision • Focus groups on domain coverage • Cognitive interviews for individual items • Integrate quantitative analysis from archival data • Final revision
Expert Item Revision
• Put item in the PROMIS format • 7 day recall period • Preferred response options • Stand alone on one screen
Revision Example
Original Due to your feet How much difficulty did you have climbing up or down curbs 101 point scale VAS (0 = No difficulty to 100 = unable) Revised Over the last 7 days: Could you climb up or down one step?
0 = Without difficulty 1 = With some difficulty 2 = With much difficulty 3 = Unable to do
Final Revisions
• Revise based on focus group feedback, cognitive interview results • Use Lexile framework to rate readability • Send on to field testing (April 2006)
PROMIS Website http://www.nihPROMIS.org/
Contact Information:
Shani Rolle, M.S.
NIH Coordinator [email protected]