Transcript Critical Appraisal of Clinical Practice Guidelines

Critical Appraisal of
Clinical Practice Guidelines
Manit Srisurapanont, MD
Department of Psychiatry
Chiang Mai University
For Evidence-Based Medicine Workshop,
Chiang Mai, Nov 17-18, 2011
Outline
• Practice Guidelines
• Assessing Recommendations (Gayatte et al. Users’
Guide to the Medical Literature: a Manual for
Evidence-Based Medicine, 2008)
• Appraisal of Guidelines for Research & Evaluation II
(AGREE II, 2009)
2
Practice Guidelines
• Are systematically developed statements to assist
practitioner and patient decisions about
appropriate health care for specific clinical
circumstances
• Play an important role in health policy formation
• Practice guidelines are evidence-based if they
undertake a review of the literature and link their
concluding recommendations to the evidentiary
base identified through the literature search
3
From evidence to practice
+ clinical
expertise**
*Haynes RB, Haines A. BMJ 1998;317:273-6.
**Haynes RB, et al. EBM 2002;7:36-8.
4
Assessing Recommendations: 1
1. Do the recommendations consider all relevant
patient groups, management options, and possible
outcomes?
• All relevant patient groups, eg, low risk/high risk,
more/less susceptible to adverse effects
• All relevant management options, eg,
surgical/medical , no-treatment option
• All patient-important outcomes, eg, morbidity and
mortality, quality of life, toxicity/adverse effects,
cost to the patient or to society
Guyatt G, et al. User’s Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. 2008
5
Assessing Recommendations: 2
2. Are there systematic reviews of evidence that
estimate the relative effect of management options on
relevant outcomes?
• For example, hormone replacement therapy: the
incidence of hip fracture, breast cancer,
endometrial caner, myocardial infarction, stroke,
and dementia , as well as quality of life
• The guideline developers’ systematic review must
summarize the quality of the evidence on which
they base their recommendations
6
Assessing Recommendations: 3
3. Is there an appropriate specification of values and
preferences associated with outcomes?
• It may be that the greater participation by
methodologists, frontline clinicians, and members
of the general public would lead to guidelines more
in keeping with the public interest
• It is important that authors of guidelines report the
principal sources of judgments and the method of
seeking consensus on values and preferences
7
Assessing Recommendations: 4
4. Do the authors indicate the strength of their
recommendations?
• The grades of recommendation, assessment,
development, and evaluation (GRADE) system:
• Strong vs. weak recommendation
• High-, moderate-, low- and very-low-quality
evidence
• For example:
• Strong recommendation, high-quality evidence:
Benefits clearly outweigh risk and burden, or vice
versa, RCTs without important limitations or
overwhelming evidence from observational studies
8
AGREE II
• The Appraisal of Guidelines for Research &
Evaluation (AGREE) Instrument was developed to
address the issue of variability in guideline quality
• The AGREE instrument is a tool that assesses the
methodological rigor and transparency in which a
guideline is developed
• The original AGREE instrument (2001) has been
refined, which has resulted in the new AGREE II
(2010) and includes a new User’s Manual
• AGREE II comprises 23 items in 6 domains, rated
from 1 strongly disagree to 7 strongly agree
The AGREE Next Steps Consortium. Appraisal of Guidelines for Research & Evaluation II, 2009.
9
AGREE II Domain 1:
Scope and Purpose
1. The overall objective(s) of the guideline is (are)
specifically described
• Criteria: health intent(s) (ie, prevention,
treatment, etc.), expected benefit or outcome,
and target(s) (eg, patients population)
2. The health question(s) covered by the guideline
is (are) specifically described
3. The population (patients, public, etc.) to whom
the guideline is meant to apply is specifically
described
10
AGREE II Domain 2:
Stakeholder involvement
4. The guideline development group includes
individuals from all relevant professional groups
5. The views and preferences of the target
population (patients, public, etc.) have been sought
6. The target users of the guideline are clearly
defined
11
AGREE II Domain 3:
Rigor of Development
7. Systematic methods were used to search for
evidence
8. The criteria for selecting the evidence are clearly
described
9. The strengths and limitations of the body of
evidence are clearly described
10. The methods for formulating the
recommendations are clearly described (eg, steps
used in modified Delphi technique, voting
procedures that were considered)
12
AGREE II Domain 3:
Rigor of Development (cont.)
11. The health benefits, side effects, and risks have
been considered in formulating the
recommendations
12. There is an explicit link between the
recommendations and the supporting evidence
13. The guideline has been externally reviewed by
experts prior to its publication
14. A procedure for updating the guideline is
provided
13
AGREE II Domain 4:
Clarity of Presentation
15. The recommendations are specific and
unambiguous
16. The different options for management of the
condition or health issue are clearly presented (eg,
description of population or clinical situation most
appropriate to each option)
17. Key recommendations are easily identifiable
14
AGREE II Domain 5:
Applicability
18. The guideline describes facilitators and barriers to
its application
19. The guideline provides advice and/or tools on how
the recommendations can be put into practice
20. The potential resource implications of applying the
recommendations have been considered (eg,
presenting cost information)
21. The guideline presents monitoring and/or auditing
criteria (eg, criteria for assessing guideline
implementation, adherence to recommendations, and
impact)
15
AGREE II Domain 6:
Editorial Independence
22. The views of the funding body have not
influenced the content of the guideline
23. Competing interests of guideline development
group members have been recorded and addressed
16
AGREE II Overall Guideline Assessment
1. Rate the overall quality of this guideline
– from 1 lowest possible quality to
– 7 highest possible quality
2. I would recommend this guideline for use
– Yes
– Yes, with modifications
– No
17