Transcript Methodology for Guideline Development for the 7th ACCP

The Science of Guidelines
The 7th ACCP Conference on Antithrombotic and
Thrombolytic Therapy: Evidence-Based Guidelines
Holger Schünemann, MD, PhD
Italian National Cancer Institute, Rome, Italy
McMaster University, Hamilton, Canada
University at Buffalo, NY, USA
Topics for this talk
 What makes guidelines evidence based
in 2005?
 High- vs low-quality evidence
 Strong vs weak recommendations
 Example recommendation
 Example of the influence of values,
preferences, and cost
 Grading system
What makes guidelines evidence
based in 2005?
 Evidence – recommendation:
transparent link
 Explicit inclusion criteria
 Comprehensive search
 Standard consideration of
study quality
 Conduct/use meta-analysis
 Grade recommendations
 Acknowledge values and
preferences underlying
recommendations
Schünemann J, et
al. Chest. 2004;126
Suppl 3:688S-696S.
Background
 First ACCP guidelines in 1986 (J. Hirsh; J. Dalen)
 Initially aimed at consensus
 Group of experts and methodologists formally
convening every 2 to 3 years
 ~260,000 copies in 2001
 7th conference held in 2003
 87 panel members
 22 chapters
 Across subspecialities
 Over 500 recommendations; 230 new
 Evidence-based recommendations
ACCP = American College of Chest Physicians.
Schünemann HJ, et al. Chest. 2004;126 Suppl 3:174S-178S.
Schünemann HJ, et al. Chest. 2004;126 Suppl 3:174S-178S.
The clinical question
 Transparent link: from evidence to recommendations
 Explicit inclusion criteria
Table 1
Eligibility criteria
Section
Inclusion criteria
Population
Intervention(s)
or exposure
Outcome
Methodology
…
…
…
…
4.1.
Patients with unstable
angina, MI, TIA, and
non-acute stroke
Any antiplatelet drug
compared with placebo
or one or more other
antiplatelet drug(s)
4.2
Patients with
cardioembolic stroke
Oral anticoaluation
…
…
…
…
MI = myocardial infarction; RCTs = randomized
controlled trials; TIA = transient ischaemic attack.
 Death
 Stroke or recurrent
stroke
 Other vascular
events
 Death
 Stroke or recurrent
stroke
…
RCTs
RCTs
…
Albers GW, et al. Chest. 2004;126 Suppl 3:483S-512S.
Comprehensive search
for evidence
 Use questions to develop search strategy
– e.g. identify all search terms (MeSH and keywords) for
antiplatelet drugs or MI
 Search
– Cochrane Database of Systematic Reviews
– Database of Abstracts of Reviews of Effectiveness
– Cochrane Central Register of Controlled Trials
– MEDLINE and EMBASE (1966 to December 2002)
– ACP Journal Club
 Provide search results
– use EndNote® software
– e.g. 490 citations on thrombolysis in acute stroke
ACP = American College of Physicians; MeSH = Medical Subject Headings.
Schunemann
HJ, et al. Chest.
Schünemann
HJ et
2004;126 Suppl
3:174S-178S
al. Chest 2004
The ACCP grading system:
GRADE* approach
Clear separation of 2 issues:
 Evidence: very low, low, moderate, or high
quality?
– methodological quality of evidence
– likelihood of bias
 Recommendation: weak or strong?
– trade-off between benefits and downsides
– patient values and preferences
GRADE = Grading of Recommendations Assessment,
Development and Evaluation.
*www.GradeWorking-Group.org
GRADE Working Group. BMJ. 2004;328:1490-9.
Why grade recommendations?
 People draw conclusions about the
– quality of evidence and strength of recommendations
 Systematic and explicit approaches can help
– protect against errors, resolve disagreements
– communicate information
 Change practitioner behaviour
 Strong: apply uniformly
– just do it
 Weak: think about it
– examine evidence yourself, consider patient circumstances
very carefully and explore with the patient
 However, wide variation in approaches (GRADE)
GRADE Working Group. BMJ. 2004;328:1490-9.
Grades of recommendation:
methodological quality
 High (A): consistent results from RCTs or
observational studies with very strong
association and secure generalization
 Moderate (B): inconsistent results from RCTs
or RCTs with methodological limitations
 Low (C): unbiased observational studies
(e.g. well-executed cohort studies)
 Very low (D): other observational studies
(e.g. case series)
GRADE Working Group. BMJ. 2004;328:1490-9.
RCT starts high –
what moves quality down?
 Flawed design and execution
 Inconsistency
 Indirectness
 Imprecision
 Reporting bias
GRADE Working Group. BMJ. 2004;328:1490-9.
Design and execution
 Concealment
 Intention-to-treat principle observed
 Blinding
 Completeness of follow-up
 Early stopping
GRADE Working Group. BMJ. 2004;328:1490-9.
Moving quality up:
observational studies – high or moderate quality?
 Strong association
– strong association: RR > 2 or RR < 0.5
– very strong association: RR > 5 or RR < 0.2
 Dose–response relationship
– bleeding risk associated with increasing INR
(blood thinning with warfarin)
 Plausible confounders would have reduced
the effect
INR = International Normalized Ratio;
RR = relative risk.
GRADE Working Group. BMJ. 2004;328:1490-9.
Grades of recommendation:
strength of recommendations
 Stronger recommendations (we recommend)
– high-quality methods with large, precise effect
– benefits much greater than downsides, or downsides much
greater than benefits
– do it or don’t do it – we recommend
– Grade 1
 Weak recommendations (we suggest)
– lower-quality methods with imprecise estimate
– benefits not clearly greater or smaller than downsides
– values and preferences very important
– probably do it or probably don’t do it – we suggest
– Grade 2
Example: stroke prevention
In patients with history of non-cardioembolic stroke or
TIA…, we recommend treatment with an antiplatelet
agent (Grade 1A). Aspirin, aspirin + XR dipyridamole, or
clopidogrel are all acceptable options for initial therapy.
Clopidogrel: higher cost
If we had to make a choice between aspirin and
clopidogrel, what would that choice be?
XR =extended release.
Albers GW, et al. Chest. 2004;126 Suppl 3:483S-512S.
CAPRIE trial
 Aspirin vs clopidogrel in patients at risk for
cardiovascular event
 19,185 patients, 3 subgroups with > 6,300
patients each (TIA/stroke; MI; peripheral
arterial occlusive disease)
 Mean duration of follow-up: 1.9 years
 Primary outcome: ischaemic stroke, MI,
or vascular death
CAPRIE = Clopidogrel versus Aspirin in Patients at Risk of Ischaemic Events.
CAPRIE Steering Committee.
Lancet. 1996;348:1329-39.
CAPRIE trial results:
relative risk reduction
40
Clopidogrel
better
30
23.8
20
Relative risk 10
reduction
0
%
Relative risk
increase -10
(Aspirin
better)
8.7
7.3
-3.7
-20
-30
STROKE
p=
PAOD = peripheral arterial occlusive disease.
0.26
MI
PAOD
Total
0.66
0.0028
0.043
CAPRIE Steering Committee. Lancet. 1996;348:1329-39.
CAPRIE trial results:
absolute risk
NNT 200
10
8
*p < 0.05
Absolute risk 6
%
4
*p < 0.05
2
0
Strok e
MI
PAOD
Total
Clopidogrel
7.15
5.03
3.71
5.32
Aspirin
7.71
4.84
4.86
5.83
NNT = number needed to treat.
CAPRIE Steering Committee. Lancet. 1996;348:1329-39.
Which of the following
recommendations should be given?
1. Aspirin over clopidogrel in patients with prior
history of TIA/stroke?
– OPTION 1
2. Clopidogrel over aspirin in patients with prior
history of TIA/stroke?
– OPTION 2
Audience at a prior thrombosis
meeting
Preferred recommendation
100%
57%
43%
80%
60%
40%
20%
0%
Aspirin
Clopidogrel
Values and preferences
 Underlying values and preferences
always present
 Sometimes crucial
 Important to make explicit
Judgements about
recommendations
1. Benefit and downside evaluation
Benefits <<
downsides
Benefits ?
downsides
Benefits ?
downsides
Benefits >>
downsides

?
?

2. Recommendation (wording)
STRONG
Recommend
don’t do it / should
not do it
WEAK
Suggest
probably don’t do
it / might not do it
WEAK
Suggest
probably do it
/ might do it
STRONG
Recommend
do it / should do it
Example: stroke prevention
 In patients with history of non-cardioembolic stroke or
TIA…
 …we recommend treatment with an antiplatelet agent
(Grade 1A). Aspirin, aspirin + XR dipyridamole, or
clopidogrel are all acceptable options for initial therapy
 …, we suggest use of clopidogrel over aspirin (Grade 2B)
Underlying values and preferences:
 This recommendation places a relatively high value on a
small absolute risk reduction in stroke rates, and a
relatively low value on minimizing drug expenditures
Albers GW, et al. Chest. 2004;126 Suppl 3:483S-512S.
Judgement:
benefits vs downsides*
 (Quality of evidence)
 Relative importance of the outcomes
(benefits, harms, and burden)
 Baseline risk of outcomes
 Magnitude of the effect (RR)
 Absolute benefit and harm
 Precision of the estimates
 Cost
*Downsides include harm, burden, and cost
Grade of Recommendation
Clarity of risk/benefit
Quality of supporting evidence
Strong recommendation
High-quality evidence
1A
Benefits clearly outweigh
risk and burdens, or vice
versa
Strong recommendation
Moderate-quality evidence
1B
Benefits clearly outweigh
risk and burdens, or vice
versa
Strong recommendation
Low-quality evidence
1C
Strong recommendation
Very low-quality evidence
1D
Weak recommendation
High-quality evidence
2A
Benefits appear to
outweigh risk and
burdens, or vice versa
Benefits possibly outweigh
risk and burdens, or vice
versa
Benefits closely balanced
with risks and burdens
Consistent evidence from well-performed
RCTs or overwhelming evidence of some
other form. Further research is unlikely to
change our confidence in the estimate of
benefit and risk.
Evidence from RCTs with important limitations
(inconsistent results, methodological flaws,
indirect or imprecise), or very strong evidence
of some other research design.
Evidence from observational studies, or from
RCTs with serious flaws. Any estimate of
effect is uncertain.
Evidence from unsystematic clinical
observations
Weak recommendation
Moderate-quality evidence
2B
Benefits closely balanced
with risks and burdens,
some uncertainly in the
estimates of benefits, risks
and burdens
Weak recommendation
Low-quality evidence
2C
Uncertainty in the
estimates of benefits,
risks, and burdens;
Evidence from observational studies, or from
RCTs with serious flaws. Any estimate of
effect is uncertain.
Very weak recommendation
Very low-quality of evidence
2D
Major uncertainty in the
estimates of benefits,
risks, and burdens;
Evidence from unsystematic clinical
observations,
Consistent evidence from well-performed
RCTs or overwhelming evidence of some
other form. Further research is unlikely to
change our confidence in the estimate of
benefit and risk.
Evidence from RCTs with important limitations
(inconsistent results, methodological flaws,
indirect or imprecise), or very strong evidence
of some other research design.
Guyatt G, et al. Chest. 2004;126 Suppl 3:179S-187S.
Summary
 Guidelines require evidence-based methods
 GRADE approach to grading
 Integration of values and preferences
 Grade 1: strong recommendation
 Grade 2: weaker recommendation/suggestion
 High transparency between evidence and
recommendations
End