Transcript Document
Standards for
Developing Trustworthy
Clinical Practice Guidelines
Institute of Medicine
January 11, 2010
Sandra Zelman Lewis, PhD
Asst VP, Health & Science Policy
American College of Chest Physicians
Most Important Challenge: Definition
1. Need for a universally accepted definition of evidence-based
guidelines (EBGs)
The problem
Consensus statements and “even less structured”
documents are often designated as “guidelines”
Physicians are misled into thinking that these are
evidence-based and methodologically rigorous
Patient care can be adversely impacted
Most Important Challenge: Definition
What action should this committee take?
Set the bar high for methodological rigor
Require a minimal threshold of rigor or at least provide
a rating scale that EBG users can understand
Do not allow consensus statements (and less
evidence-based documents) to be titled “guidelines”
Most Important Challenge: Definition
Challenges
Within EBGs, should consensus-based recs be allowed?
• Reduce to areas of identified need for guidance
• Allow only when the evidence is weak,
inconsistent, or nonexistent
• Downgrade appropriately
• Ensure panel: expertise, multidisciplinary, vetted for
COIs
• Appropriate review and balanced viewpoints
Some guideline developers must adjust their processes
• Set a timetable to meet the new definition and
standards
• Provide instruction on new standards and methods
Another Important Challenge: Funding
2. Funding guidelines is the biggest challenge that guidelines
developers face.
The problem
EBGs are very expensive if done correctly
Little funding available, difficult to obtain
Charge to this committee
Identify sources for funding guidelines produced
according to the established standards
When Evidence is Nonexistent or Poor Quality
o What does ACCP do when the scientific evidence is absent
or poor?
• Sets minimal threshold for evidence: Must be
published in a peer-reviewed journal
• Downgrade recs when poor quality evidence
• Allow consideration of indirect evidence if justification
(described in text) acceptable but downgrade
appropriately
• If evidence is not sufficient, then discuss in text but do
not provide recommendation
Disagreements in Interpretations of Evidence
o How do you reconcile disagreements in evidence
interpretation among guidelines?
•
Assess rigor of methodology/adherence to the evidence
•
Request invited associations to review and comment on
our guideline recommendations
•
Offer competing guideline organizations opportunity to
appoint a member to our next edition or update panel
(providing he/she passes COI vetting and approval
process)
Accommodating Subgroups
o How do guidelines accommodate subgroups (ex. older
populations or persons with multi-morbidities) whose
treatment outcomes may differ from the average patient?
• If data exist for subgroups, use it.
• If not, use indirect evidence from similar groups and
downgrade appropriately
• Always describe patient population in each
recommendation based on the patient population in
the original studies
• A major challenge is multiple co-morbidities
Setting Standards
o What topics and/or processes do you think the committee
should consider in deriving quality standards for clinical
practice guidelines?
• Establish a definition of evidence-based guidelines,
possibly with rating scales
• Standard setting topics and criteria
. PCPI (Physician Consortium for Performance
Improvement) criteria for guidelines permitted as
basis of performance measures
. AGREE instrument (AGREE III is in development)
. COGS (Conference on Guideline Standardization)
. GLIA – for implementation purposes
Panel Composition
o What should the composition of CPG development panels,
in particular the balance of methodologists, topical experts,
and consumers, look like?
•
Depends on structure of panel but should have
methodologists reviewing evidence and developing
evidence tables or profiles
•
Consumers must be educated in EBM
•
Content experts provide credibility but all must go
through rigorous review, including COIs
•
Other considerations: health economists, frontline
clinicians, patient preference consultants
Grading System
o Is there an available assessment tool that adequately rates
both the level of the scientific evidence and strength of
clinical recommendations that should be used as standard
practice in guideline development?
•
The ACCP Grading System
. A modification of GRADE (major difference: restricts
evidence to approved threshold)
.
Based on (1) assessment of the quality of the original
studies and (2) a balance of the risks to benefits
.
Has been adopted by other guideline developers
Grading System
Grade of
rec
Benefit versus risk
and burdens
Methodologic quality of supporting
evidence
1A
Benefits clearly outweigh
risks/burdens, or vice versa
(negative recs)
Consistent evidence from randomized controlled trials
without important limitations, or exceptionally strong
evidence from observational studies
1B
Benefits clearly outweigh
risks/burdens, or vice versa
(negative recs)
Evidence from randomized controlled trials with important
limitations (inconsistent results, methodologic flaws,
indirect or imprecise), or very strong evidence from
observational studies
1C
Benefits clearly outweigh
risks/burdens, or vice versa
(negative recs)
Evidence for at least one critical outcome from
observational studies, case series, or from randomized,
controlled trials with serious flaws or indirect evidence
2A
Benefits closely balanced with
risks/burdens
Consistent evidence from randomized controlled trials
without important limitations, or exceptionally strong
evidence from observational studies
2B
Benefits closely balanced with
risks/burdens
Evidence from randomized, controlled trials with
important limitations (inconsistent results, methodologic
flaws, indirect or imprecise), or very strong evidence from
observational studies
2C
Benefits closely balanced with
risks/burdens
Evidence for at least one critical outcome from
observational studies, case series, or from randomized,
controlled trials with serious flaws or indirect evidence
Recommendations for PMs
o What methods might be developed for determining which
recommendations among those in a guideline should be
applied to quality measures or electronic medical record
decision prompts?
•
Evidence should dictate direction and strength of recs
•
Suggest some 1A and 1B recs for PMs (although not all)
•
1C, 2A, 2B, and 2C recs should generally not be used
•
However, all should be evaluated based on feasibility,
usability, scientific importance, practicality, and
applicability
QI and Harmonization
o What administrative (eg, accreditation) or legal approaches
might improve the quality of clinical practice guidelines?
• Published guideline quality rating scale
• Listed on NGC Web site
• Currency rating also listed in NGC
• PMs & CMS policies should be based on highly rated
EBGs
o What explicit approaches might harmonize guideline
developers and increase guidelines convergence?
• Funding requires multiple societies to collaborate
• Require compliance with evidence-based standards,
rating scale
EBG Promotion
o What types of strategies might promote greater utilization
of guidelines?
•
Requirements for implementation into:
. EMRs
. Registries
. PMs, including PQRI
•
Education to allow developers to learn about new
techniques and processes (see last slide)
•
Healthcare providers need to know how to find good
guidelines and good guidelines should address the
needs of providers (ask frontline clinicians what they
need!)
Other Considerations
o Are there other characteristics of guideline standards you
think are important for the committee to consider?
•
•
Most important: address funding
Also should move the field toward incorporating
resource considerations into the EBG recs
Promotion of IOM Standards
The IOM report and new standards should be presented
to appropriate audiences:
Guideline developers will be attending the
Guidelines International Network conference and
guideline methodology course
Guidelines International Network
2010 Conference - Chicago
August 26-28, 2010 – Conference Dates
August 25, 2010 – Pre-meeting Course in Guideline
Methodology
Chicago, Illinois, USA
www.GIN2010.org
Host: American College of Chest Physicians
(G-I-N)