Transcript Outcome Measures

Armando Miciano, M.D.
Nevada Rehabilitation Institute, Las Vegas NV
2014 AAPMR Annual Assembly, San Diego CA,
2014 Nov 14


1. Understand the steps in determination of
outcome measures of interest
2. Determine the applicability of the CPG by its:
• identification of criteria to assess guideline
implementation or adherence to
recommendations
• criteria for assessing impact of
implementing the recommendations
• advice on the frequency and interval of
measurement
• descriptions or operational definitions of
how the criteria should be measured

Process measures, which assess the extent to
which physicians effectively implement
clinical practices (or treatments) that have
been shown to result in high-quality or
efficient care.
◦ e.g. PQRS measure #148-151, LBP measure group

Outcome measures, which track the results of
physician care, such as mortality, infections,
and how patients experience that care.
◦ e.g. Patient-reported outcomes (PRO)
◦ e.g. Performance-based assessment (PBA)
GAO. Clinical Data Registries. 2013. Accessed
10/20/14 at:
http://www.gao.gov/assets/660/659701.pdf
PRO
 = Patient-Reported Outcomes
 – from the “patient”
PBA
 = Performance Based Assessment
 medical search term used also as outcome
measures
 clinician-derived objective tests
ENDORSED BY AAPMR:
 Diagnosis and Treatment of Degenerative Spondylolisthesis
◦ developed by the North American Spine Society (NASS).

Diagnosis and Treatment of Degenerative Lumbar Spinal Stenosis

Treatment of Symptomatic Osteoporotic Spinal Compression
Fractures
◦ developed by NASS.
◦ developed by the American Academy of Orthopaedic Surgeons (AAOS)
AFFIRMED BY AAPMR:
 Low back pain: clinical practice guidelines linked to the
International Classification of Functioning, Disability, and Health
◦ developed by the Orthopaedic Section of the American Physical Therapy
Association.
Accessed 10/20/14 at:
http://www.aapmr.org/research/practiceguidelines/Pages/Applicable-PMRGuidelines.aspx


Ghogawala Z, et al. Guideline update for the
performance of fusion procedures for
degenerative disease of the lumbar spine. Part 2:
assessment of functional outcome following
lumbar fusion. J Neurosurg Spine. 2014
Jul;21(1):7-13.
North American Spine Society. Diagnosis and
treatment of lumbar disc herniation with
radiculopathy. Burr Ridge (IL): North American
Spine Society; 2012. 100 p.
Spondylolisthesis Spinal Stenosis
Compression Fx
APTA LBP CPG
Reviewed by a group
experienced in
performance measure
development
to identify those
recommendations
rigorous enough for
measure development.
Reviewed by a group
experienced in
performance measure
development to identify
those
recommendations
rigorous enough for
measure development.
Patient-oriented
outcomes are included
wherever possible.
Not provided
All relevant medical
specialties involved in
the guideline
development and at the
Consortium
will be invited to
collaborate in the
development
of evidence-based
performance measures
related to
spine care.
All relevant medical
specialties involved in
the guideline
development
and at the Consortium
will be invited to
collaborate in
the development of
evidence-based
performance measures
related to spine care.
If patient-oriented
outcomes were not
available
surrogate/intermediate
outcomes were
considered. Surrogate
outcome measures are laboratory
measurements or another
physical sign used as substitutes
for a clinically meaningful end
point that measures directly how
a patient feels, functions, or
survives. Radiographic results are
an example of a surrogate
outcome.
The results using
validated instruments
are considered the best
available evidence.
All outcomes we
reported were validated
in a spine patient
population.
GRADE A
recommendation
GRADE B rec.
INSUFFICIENT
EVIDENCE
•The Zurich
Claudication
Questionnaire
(ZCQ)/Swiss Spinal
Stenosis Questionnaire
(SSS),
•The Japanese
Orthopedic
Association (JOA)
Score
•The Shuttle Walking
Test (SWT),
•Oswestry Disability
Index (ODI),
•Likert Five-Point Pain
Scale
•36-Item Short Form
Health Survey (SF-36)
•the calculated
Recovery Rate
•Oxford Claudication
Score (OCS),
• Low Back Pain
Bothersome Index
•Stenosis Bothersome
Index

“The North American Spine Society has a
publication entitled Compendium of Outcome
Instruments for Assessment and Research of
Spinal Disorders.” To purchase a copy of the
Compendium, visit
https://webportal.spine.org/Purchase/ProductDe
tail.aspx?Product_
“For additional information about the
Compendium, please contact the NASS Research
Department at [email protected].”
Accessed 10/20/14 at:
https://www.spine.org/Documents/ResearchClinicalCare/
Guidelines/LumbarStenosis.pdf
Major Outcomes Considered:







Sensitivity and specificity of diagnostic tests
Visual analog scale (VAS) score
Quality of life/basic activities of daily living
Walking distance
Patient satisfaction
Postoperative complication rate
Age-related outcomes
Accessed 10/20/14 at:
http://www.guideline.gov/content.aspx?id=3
4839

Clinical studies often report many different outcomes. For this guideline,
patient-oriented outcomes are included wherever possible.
◦ If patient-oriented outcomes were not available surrogate/intermediate outcomes
were considered.
◦ Surrogate outcome measures are laboratory measurements or another physical sign
used as substitutes for a clinically meaningful end point that measures directly how a
patient feels, functions, or survives.
◦ Radiographic results are an example of a surrogate outcome.

For outcomes measured using “paper and pencil” instruments (e.g. the
visual analogue scale), the results using validated instruments are
considered the best available evidence.
◦ In the absence of results using validated instruments, results using non-validated
instruments are considered as the best available evidence and the strength of the
recommendation is lowered.

For this guideline, all outcomes we reported were validated in a spine
patient population.
Accessed 10/20/14 at: http://www.aaos.org/research/guidelines/SCFguideline.pdf


Considered the effects of treatments in terms of the minimal
clinically important improvement (MCII) in addition to whether
their effects were statistically significant.
The MCII is the smallest clinical change that is important to
patients,
◦ recognizes the fact that there are some treatment-induced statistically
significant improvements that are too small to matter to patients.

The values used for MCIIs are derived from a published study
investigating the Visual Analogue Scale, the Numerical Rating Scale,
the Oswestry Disability Index, and the Roland Disability
Questionnaire; a study investigating the Physical Component
Summary of the SF-36; a study investigating the Assessment of
Quality of Life instrument (AQoL); and a study investigating the EQ5D instrument.
Accessed 10/20/14 at:
http://www.aaos.org/research/guidelines/SCFguideline.pdf
Outcome Measure
MCII (points)
Pain – VAS (0-100)
15
Pain – NRS (0-10)
2
Oswestry Disability Index
10
Roland-Morris Disability
Questionnaire
5
SF-36 Physical Component
Summary
4.9
AQoL
0.06
EQ-5D
0.074
VAS – Visual Analog Scale; NRS – Numerical Rating Scale;
AQOL – Assessment of Quality of Life;
Accessed 10/20/14 at:
http://www.aaos.org/research/guidelines/SCFguidel
ine.pdf
Major Outcomes Considered:

Diagnostic accuracy
◦ (sensitivity, specificity, positive and negative likelihood
ratios)







Effectiveness of interventions in terms of pain
relief, improvement of functioning, quality of life
Pain
Perceived disability
Back-specific function
Mobility
Generic health status
Patient satisfaction
Accessed 10/20/14 at:
http://www.orthopt.org/uploads/content_files/ICF/Up
dated_Guidelines/Low_Back_Pain_Clinical_Practice_Gui
delines___JOSPT_2012.pdf
EXAMINATION – OUTCOME MEASURES: Clinicians should use validated
self-report questionnaires, such as the Oswestry Disability Index
and the Roland-Morris Disability Questionnaire.
These tools are useful for identifying a patient’s baseline status relative to pain,
function, and disability and for monitoring a change in a patient’s status throughout the
course of treatment.
(Recommendation based on strong evidence.)
EXAMINATION – ACTIVITY LIMITATION AND PARTICIPATION
RESTRICTION MEASURES: Clinicians should routinely assess activity
limitation and participation restriction through validated
performance-based measures. Changes in the patient’s level of
activity limitation and participation restriction should be monitored
with these same measures over the course of treatment.
(Recommendation based on expert opinion.)
Accessed 10/20/14 at:
http://www.orthopt.org/uploads/content_files/ICF/Updated_Guidel
ines/Low_Back_Pain_Clinical_Practice_Guidelines___JOSPT_2012.pdf

Measuring the application of guideline
recommendations can facilitate their ongoing use.

This requires clearly defined criteria that are derived
from the key recommendations in the guideline.
◦ The criteria may include
 process measures,
 behavioral measures,
 clinical or health outcome measures. [1]

Examples of monitoring and audit criteria are: % of
LBP assessed for NRS, functional disability,
psychological factors
[1] AGREE Next Steps Consortium (2009). Applicability – Monitoring
and/or auditing criteria (no. 21). In: The AGREE II Instrument
[Electronic version]. Retrieved 05/18/14, from
http://www.agreetrust.org . P. 36

Item content includes the following CRITERIA:
• identification of criteria to assess guideline
implementation or adherence to recommendations
• criteria for assessing impact of implementing the
recommendations
• advice on the frequency and interval of measurement
• descriptions or operational definitions of how the
criteria should be measured [1]
[1] AGREE Next Steps Consortium (2009). Applicability –
Monitoring and/or auditing criteria (no. 21). In: The AGREE II
Instrument [Electronic version]. Retrieved 05/18/14, from
http://www.agreetrust.org . P. 36


Measure Description – LBP Initial Visit:
Percentage of patients at least 18 years of age and
younger than 80 with a diagnosis of back pain who
have medical record documentation of all of the
following on the date of the initial visit to the
physician:
◦ 1. Pain assessment
◦ 2. Functional status
◦ 3. Patient history, including notation of presence or
absence of “red flags”
◦ 4. Assessment of prior treatment and response, and
◦ 5. Employment status
National Quality Forum – Measure no. 322: Back Pain Initial Visit.
Accessed 05/18/14 at:
http://www.aapmr.org/research/performance/pmrmeasures/Pages/Low
-Back-Pain-Measures.aspx


Measure Description – LBP Patient
Reassessment:
Percentage of patients at least 18 years of
age and younger than 80 with back pain with
documentation that the physician conducted
reassessment of both of the following within
four to six weeks of their initial back pain
visit or of a surgical procedure date:
◦ 1) Pain AND
◦ 2) Functional status
National Quality Forum – Measure no. 306: Back Pain Patient Reassessment.
Accessed 05/18/14 at:
http://www.aapmr.org/research/performance/pmrmeasures/Pages/LowBack-Pain-Measures.aspx

Adult acute and subacute low back pain:
percentage of patients with low back pain
diagnosis who have a reassessment at each
follow-up visit that includes:
◦ pain assessment using the Visual Analog Scale, pain
diagram or other assessment tool;
◦ functional status using the Oswestry Disability
Questionnaire or other assessment tool;
◦ clinician's objective assessment; and
◦ psychosocial screening that includes depression
and chemical dependency screening.
Accessed 05/18/14 at:
http://www.qualitymeasures.ahrq.gov/summary_redirect.aspx?t
ype=replaced&objectID=34201

Not addressed in most CPGs

EXAMINATION – OUTCOME MEASURES:
Clinicians should use validated self-report
questionnaires such as:
◦ the Oswestry Disability Index and
◦ the Roland-Morris Disability Questionnaire.
Delitto A, et al. J Orthop Sports Phys Ther. 2012 Apr;42(4):A1-57.
NASS:
NASS:
Spondylolisthesis Spinal Stenosis
AAOS:
Compression FX
Ortho PT:
LBP
•Pain – VAS (0-100)
•Oswestry Disability Index
•Pain – NRS (0-10)
•Roland-Morris Disability
Questionnaire.
Rec. Grade A
•The Zurich Claudication
Questionnaire (ZCQ)/Swiss
Spinal Stenosis
Questionnaire (SSS),
•Oswestry Disability Index
(ODI),
•Likert Five-Point Pain Scale
•36-Item Short Form Health
Survey (SF-36)
“Refer to a publication
entitled Compendium of
Outcome Instruments for
Assessment and Research
of
Spinal Disorders.”
•Oswestry Disability Index
•Roland-Morris Disability
Questionnaire
•SF-36 Physical
Component Summary
•AQoL
•EQ-5D
CPG
Aggregate score (out of 7)
NASS Spondylolisthesis
2.75
NASS Spinal Stenosis
3.40
AAOS Osteoporotic Compression
Fracture
2.33
APTA: LBP
2.20
General Conclusion – weak in applicability due to:
• weak identification of criteria to assess guideline implementation
or adherence to recommendations
• no provided criteria for assessing impact of implementing the
recommendations
• no specific advice on the frequency and interval of measurement
• weak descriptions or operational definitions of how the criteria
should be measured





Symptom quality: NRS
Functional Status: ODI, RMDQ
Quality of life: SF-36 physical component
Behavioral Measure: ?
Process Measure: PQRS #148-151
Based on critically analysis of 4 CPGs: NASS
Spondylolisthesis, NASS Spinal Stenosis, AAOS
Compression Fracture, APTA LBP

Contact Info:
Armando Miciano, M.D.
Nevada Rehabilitation Institute, Las Vegas NV
www.springmountainrehab.com
702-869-4401
[email protected]