Transcript Methodology for Guideline Development for the 7th ACCP

Using evidence-based clinical
practice guidelines:
Examples from the ACCP Antithrombotic and
Thrombolytic Therapy Conference
Holger Schünemann, MD, PhD
Deborah Cook, MD, MSc
Roman Jaeschke, MD, MSc
Janek Brozek, MD
Gordon Guyatt, MD, MSc
Where would you prefer to live?
← Option 1
Option 2 →
← Option 1
(pink card)
Option 2 →
(green card)
Today’s talk
Intro: Clinical practice guidelines
What makes guidelines evidence based in 2005?
Strong vs. weak recommendation
High vs. low quality evidence
Grading system
1
Intro: Clinical Practice Guidelines
Clinical Practice Guidelines
Systematically developed statements to assist
practitioner and patient decisions about appropriate
health care for specific clinical circumstances
Users’ Guide to the Medical Literature, 2002
Why do clinicians need
guidelines?
• Rising Healthcare Cost
• Increasing demand for care
• More expensive technologies
• Variations in service delivery among:
Providers, hospitals and geographical regions*
Assumption that this variation is a result of
inappropriate (too much/too little) use of services
*BMJ 1999;318: 527
Clinical Practice Guidelines
…are a result of the desire:
• of healthcare workers to offer and of patients
to receive the best possible care
• to make care more efficient and consistent by
bridging the gap between what clinicians do
and what the evidence shows
The leaky pipeline from research to practice
Aware Accept Target Doable Recall Agree Done
Valid
Research
If 80% achieved at each stage then
0.8 x 0.8 x 0.8 x 0.8 x 0.8 x 0.8 x 0.8 = 0.21
Glasziou and Haynes, ACP JC; 2005: 7-9
Do you use guidelines in
your practice?
Where do you get your
guidelines from?
www.guidelines.gov
“Practice guidelines … have been
demonstrated to improve patient
outcomes and lower cost”
…be based on sound scientific
evidence and implemented in
an effective manner
S. Weingarten. Hospital Medicine 2005
2
What makes Guidelines
Evidence-Based in 2005?
Background: ACCP Antithrombotic
and Thrombolytic Therapy Guidelines
First issue of ACCP guidelines in 1986 (CHEST)
Initially aimed at consensus
Methodologists involved since beginning
Now formally convening every 2 to 3 years
~200.000 copies in 2001
Seventh conference held in 2003
87 panel members, 22 chapters
Across subspecialties
565 recommendations, 230 new
Translated: Polish, Spanish, Italian, French
What makes guidelines evidence
based in 2005?
 Evidence – recommendation:
transparent link
 Explicit inclusion criteria
 Comprehensive search
 Standardized consideration
of study quality
 Conduct/use meta-analysis
 Grade recommendations
 Acknowledge values and
preferences underlying
recommendations
Schünemann et al.
Chest 2004
A bit more practice using
the voting instrument….
Remember
← Option 1
(pink card)
Option 2 →
(green card)
You are hiking.
Which of the following animals
would you prefer to encounter?
← Option 1
(pink card)
Option 2 →
(green card)
You are buying an ice cream.
Which flavor do you prefer?
Strawberry
← Option 1
(pink card)
Chocolate
Option 2 →
(green card)
You are buying a new car.
Which one would you buy?
Red Ferrari
Option 2 →
(green card)
← Option 1
(pink card)
Yellow fox
What determines your
choices?
• pleasure
• social responsibilities
•Risk taking
•Life crisis
•Resources
•Safety
•Past experiences
•Expectations
•Ongoing cost/inconvenience
• impuls control/politics
Case scenario and clinical
question
75 year old men with history of hypertension presents to
the ED with right upper extremity weakness and slurred
speech for approximately two hours earlier in the day.
Workup is negative. The symptoms are now resolved.
Antihypertensive therapy is initiated.
In elderly men with TIA and hypertension, do
antiplatelet agents compared to no antiplatelet
agents reduce recurrent strokes?
Which antithrombotic treatment would you
recommend?
ACCP Example: Stroke prevention
In patients with history of non-cardioembolic stroke or
TIA…, we recommend treatment with an antiplatelet
agent (Grade 1A). Aspirin, aspirin + XR dipyridamole
or clopidogrel are all acceptable options for initial
therapy.
Clopidogrel: Higher cost
If we had to make a choice between aspirin and
clopidogrel, what would that choice be?
Albers et al. Chest 2004
Transparent link between evidence
and recommendations
&
Explicit inclusion criteria
Table 1
Eligibility Criteria
Section
…
Inclusion Criteria
Population
Intervention(s)
or Exposure
Outcome
Methodology
…
…
…
…
4.1.
Patients with unstable
angina, MI, TIA and
non-acute stroke
4.2
Patients with
cardioembolic stroke
…
…
 Death
Any antiplatelet agent
 Stroke or recurrent
compared with placebo
stroke
or one or more other
 Other vascular
antiplatelet agents (s);
events
 Death
 Stroke or recurrent
Oral anticoaluation
stroke
…
…
Randomized
controlled trials
Randomized
controlled trials
…
Albers et al. Chest 2004
CAPRIE Trial
Aspirin vs clopidogrel in patients at risk for
cardiovascular event
19,185 patients, 3 subgroups with > 6,300 patients
each (TIA/Stroke; myocardial infarction;
peripheral arterial occlusive disease)
Mean duration of follow-up: 1.9 years
Primary outcome: ischemic stroke, myocardial
infarction, or vascular death
Clopidogrel versus aspirin in patients at risk of ischaemic events. Lancet 1996
CAPRIE* trial results
Absolute risk
NNT 200
10
8
*
Absolute risk 6
%
4
*
2
0
Strok e
MI
PAOD
Total
Clopidogrel
7.15
5.03
3.71
5.32
Aspirin
7.71
4.84
4.86
5.83
Clopidogrel versus aspirin in patients at risk of ischaemic events. Lancet 1996
CAPRIE* trial results
Relative risk reduction
40
Clopidogrel
better
30
23.8
20
Relative risk 10
reduction
%
0
Relative risk
Increase -10
(Aspirin
better)
8.7
7.3
-3.7
-20
-30
STROKE
p=
0.26
MI
PAOD
Total
0.66
0.0028
0.043
Clopidogrel versus aspirin in patients at risk of ischaemic events. Lancet 1996
Which of the following
recommendations should one give?
1. Aspirin over clopidogrel in patients with
prior history of TIA/Stroke?
OPTION 1 (pink)
2. Clopidogrel over aspirin in patients with
prior history of TIA/Stroke?
OPTION 2 (green)
Audience at a prior thrombosis
meeting
Preferred recommendation
100%
57%
43%
80%
60%
40%
20%
0%
Aspirin
Clopidogre l
3
Strong vs. weak recommendation
ACCP Recommendations?
Stronger recommendations
strong methods
large precise effect
benefits much greater than downsides, or downsides much
greater than benefits
one size fits all
expect uniform clinician and patient behavior
Grade 1
Weaker recommendations
weaker methods imprecise estimate
small effect
benefits not clearly greater or smaller than downsides
expect action to vary
Grade 2
Case scenario
65 year old female with history of
hypertension and DM type 2 complaining of
chest pain. Diagnosed as unstable angina.
Who would recommend aspirin for our
patient?
YES (pink)
No (green)
4
Strong vs. weak recommendation
Evidence weak or strong?
Study design
basic
detailed design and execution
Consistency
Directness
secure generalization?
populations (VKA for patients with A. fib and
mitral valve stenosis)
interventions (Aspirin the same as
clopidogrel?; LMWH)
outcomes (important versus surrogate
outcomes; cholesterol)
comparison (A - C versus A - B & C - B)
Grades of recommendation
Methodological quality
Grade A: consistent results from RCTs
Grade B: inconsistent results from RCTs or RCTs with
methodological limitations
Grade C: observational studies
Grade C+: observational studies with very strong
effects or secure generalization from RCTs
Example: Stroke prevention
In patients with history of non-cardioembolic stroke or
TIA…:
we recommend treatment with an antiplatelet agent (Grade
1A). Aspirin, aspirin and XR dipyridamole or clopidogrel
are all acceptable options for initial therapy.
…, we suggest use of clopidogrel over aspirin (Grade
2B).
Underlying values and preferences:
This recommendation places a relatively high
value on a small absolute risk reduction in
stroke rates, and a relatively low value on
minimizing drug expenditures
Albers et al. Chest 2004
Example: Acute coronary
syndrome
For all patients presenting with NSTE ACS,
without a clear allergy to aspirin, we
recommend immediate aspirin, 75 to 325 mg
po, and then daily, 75 to 162 mg po (Grade 1A).
What makes guidelines evidence
based in 2005?
 Evidence – recommendation:
transparent link
 Explicit inclusion criteria
 Comprehensive search
 Standardized consideration
of study quality
 Conduct/use meta-analysis
 Grade recommendations
 Acknowledge values and
preferences underlying
recommendations
Schünemann et al.
Chest 2004
The ACCP Antithrombotic
Therapy grading system
Clear separation of two issues:
Evidence: weak or strong?
methodological quality of evidence
likelihood of bias
Recommendation: weak or strong?
trade-off between benefits and downsides
Values and preferences
If available, they are integrated into recommendations and
described by guideline developers
If unavailable, adequate representation of patients’ or
society’s interests is assumed
To increase the likelihood of adequate representation, the
process included review of recommendations by research
methodologists, practicing generalists and specialists
Grading system
Grade of
recommendation
Clarity of risk/benefit
Strength of supporting evidence
1A
Benefits clearly outweigh risk
and burdens, or vice versa
1B
Benefits clearly outweigh risk
and burdens, or vice versa
1C
Benefits appear to outweigh risk
and burdens, or vice versa
2A
Benefits closely balanced with
risks and burdens
2B
Benefits closely balanced with
risks and burdens; some
uncertainty in the estimates of
benefits, risks, and burdens
2C
Uncertainty in the estimates of
benefits, risks, and burdens;
benefits may be closely
balanced with risks and burdens
Consistent evidence from well performed
randomized, controlled trials or
overwhelming evidence of some other form.
Further research is unlikely to change our
confidence in the estimate of benefit and
risk.
Evidence from randomized, controlled trials
with important limitations (inconsistent
results, methodologic flaws, indirect or
imprecise), or very strong evidence of some
other form. Further research (if performed) is
likely to have an impact on our confidence in
the estimate of benefit and risk and may
change the estimate.
Evidence from observational studes,
unsystematic clinical experience, or from
randomized, controlled trials with serious
flaws. Any estimate of effect is uncertain.
Consistent evidence from well performed
randomized, controlled trials or
overwhelming evidence of some other form.
Further research is unlikely to change our
confidence in the estimate of benefit and
risk.
Evidence from randomized, controlled trials
with important limitations (inconsistent
results, methodologic flaws, indirect or
imprecise), or very strong evidence of some
other form. Further research (if performed) is
likely to have an impact on our confidence in
the estimate of benefit and risk and may
change the estimate.
Evidence from observational studies,
unsystematic clinical experience, or from
randomized, controlled trials with serious
flaws. Any estimate of effect is uncertain.
GRADE
Grades of Recommendation
Assessment, Development and
Evaluation
System adopted by:
ACCP
UpToDate
Urology associations
Endocrine Society
*Grade Working Group. CMAJ 2003, BMJ 2004, BMC 2004, BMC 2005
Summary
Integration of values and preferences is
challenging but critical for clinical practice
guideline development and application
High transparency between evidence and
recommendations required
GRADE approach to grading quality of evidence
and strength of recommendations is gaining
acceptance and application
QUESTIONS?
End
Evidence alone does not make decisions
Expert opinion is not evidence – expert
opinion is an interpretation of the
evidence
Finalization and harmonization of
the guidelines
Preliminary versions formulated by authors and
presented before and during conference
Controversial recommendations were presented
during conference
Editors harmonized the chapters and facilitated
discussion of contested recommendations
Limitations of guidelines
Possibility that some authors followed this
methodology more closely than others
Possibility of missing relevant studies
No centralization of the methodological evaluation of
all studies
Few meta-analysis conducted
Sparse data on patients’ values and preferences and
resources utilization
Future directions of ACCP
Guidelines
Tackle limitations mentioned above
Perform additional evaluations,
supervised and coordinated centrally, of
the quality of included trials
Formed “Cost” and “Grading” task forces
Merge with GRADE* approach
*Grading Recommendations Assessment, Development and Evaluation Working Group.
BMJ 2004

Evidence –
recommendation:
transparent link
 Explicit inclusion criteria
 Comprehensive search
 Standard consideration of
study quality
( Conduct/use meta-analysis)
 Grade recommendations
 Acknowledge values and
preferences underlying
recommendations
What we have achieved
1.1. Patient group/condition, outcome, intervention
1.1. Discussion of eligible evidence answering the question
1.1. Statement of values and preferences if not obvious or
particularly pertinent to the recommendation
1.1. Recommendation: Based on (quality) evidence, statement of
recommendation with wording related to strength (GRADE
STRENGTH/EVIDENCE QUALITY).
Summary of Recommendations
Long Distance Travel
For long-distance travelers with other risk factors for
VTE, we recommend the general strategies listed
above. If active prophylaxis is considered, because of
perceived increased risk of venous thrombosis, we
suggest … single prophylactic dose of LMWH, injected
prior to departure (Grade 2B).
Geerts et al. Chest 2004
Chronic limb ischemia
We recommend clopidogrel in comparison to no
antiplatelet therapy (Grade 
1C+),
but suggest
that aspirin
Evidence
– recommendation:
be used instead of clopidogreltransparent
(Grade 2A).link
 Explicit inclusion criteria
 Comprehensive
search
Underlying values and preferences:
This recommendation
 on
Standard
consideration
of
places a relatively high value
avoiding
large
qualityin vascular
expenditures to achieve small study
reductions
events.
( Conduct/use meta-analysis)
 Grade recommendations
Acknowledge values and
We recommend clopidogrelover
ticlopidine (Grade 1C+)
preferences underlying
recommendations
Knee Arthroscopy
For patients undergoing arthroscopic knee surgery we
recommend against routine thromboprophylaxis, other than
early mobilization (Grade 2B).
For patients undergoing arthroscopic knee surgery and who
are at higher than usual risk, based on pre-existing VTE risk
factors or following a prolonged or complicated procedure,
we suggest thromboprophylaxis with LMWH (Grade 2B).
Geerts et al. Chest 2004
Why Grade Recommendations?
Strong recommendation
one size fits all
expect uniform clinician and patient behavior
Weaker recommendation
expect action to vary
Factors that influence the strength of
the recommendation
Issue
Example
Evidence for less serious event than one hopes to prevent
Preventing post-phlebitic syndrome with thrombolytic therapy in
DVT rather than death from PE.
Smaller Treatment Effect
Clopidogrel versus aspirin leads to a smaller stroke reduction in
TIA (8.7%% RRR) than anticoagulation versus placebo in AF (68%
RRR)
Imprecise Estimate of Treatment Effect
ASA versus placebo in AF has a wider confidence interval than
ASA for stroke prevention in patients with TIA
Lower Risk of Target Event
Some surgical patients are at very low risk of post-operative DVT
and PE while others surgical patients have considerably higher
rates of DVT and PE
Higher Risk of Therapy
Quality of evidence
The extent to which one can be confident that an estimate of effect
or association is correct. This depends on the:
study design (e.g. RCT, cohort study, case series)
study quality (protection against bias; e.g. concealment of
allocation,blinding, follow-up)
consistency of results
directness of the evidence including the
populations (those of interest versus similar; for example, older,
sicker or more co-morbidity)
interventions (those of interest versus similar; for example, drugs
within the same class)
outcomes (important versus surrogate outcomes)
comparison (A - C versus A - B & C - B)
Factors that influence the
strength of the recommendation
• Evidence for less serious event than one
hopes to prevent
• Smaller Treatment Effect
• Imprecise Estimate of Treatment Effect
• Low Risk of Target Event
• Higher Risk of Therapy
• Higher Costs
• Varying Values
• Higher Burden of Therapy
Factors that influence the
strength of the recommendation
Issue
Evidence for less serious event than one
hopes to prevent
Example
Preventing post-phlebitic syndrome with
thrombolytic therapy in DVT rather than death
from PE.
Smaller Treatment Effect
Clopidogrel versus aspirin leads to a smaller
stroke reduction in TIA (8.7%% RRR2) than
anticoagulation versus placebo in AF (68% RRR)
ASA versus placebo in AF has a wider confidence
interval than ASA for stroke prevention in patients
with TIA
Some surgical patients are at very low risk of
post-operative DVT and PE while others surgical
patients have considerably higher rates of DVT
and PE
ASA and clopidogrel in acute coronary syndromes
have a higher risk for bleeding than ASA alone
TPA has much higher cost than streptokinase in
acute MI
Most young, healthy people will put a high value
on prolonging their lives (and thus incur suffering
to do so); the elderly and infirm are likely to vary
in the value they place on prolonging their lives
(and may vary in the suffering they are ready to
experience to do so).
Taking adjusted-dose warfarin is associated with a
higher burden than taking aspirin; warfarin
requires monitoring the intensity of
anticoagulation and a relatively constant dietary
vitamin K intake
Imprecise Estimate of Treatment Effect
LowHigher Risk of Target Event
Higher Risk of Therapy
Higher Costs
Varying Values
Higher Burden of Therapy
Peripheral arterial occlusive
disease
We recommend lifelong aspirin therapy (75 - 162 mg/d)
in comparison to no antiplatelet therapy in both
patients with clinically manifest coronary or
cerebrovascular disease (Grade 1A) and those
without clinically manifest coronary or
cerebrovascular disease (Grade 1C+).
We recommend clopidogrel in comparison to no
antiplatelet therapy (Grade 1C+).
Which of the following options
would you recommend?
1. Aspirin over clopidogrel in patients with
PAOD?
OPTION 1
2. Clopidogrel over aspirin in patients with
PAOD?
OPTION 2
PAOD
In patients with PAOD we suggest that aspirin be
used instead of clopidogrel (Grade 2A).
Underlying values and preferences:
This recommendation places a relatively high
value on avoiding large expenditures to achieve
small reductions in vascular events.
Evidence weak or strong?
study design
basic
detailed design and execution
consistency
directness
secure generalization?
populations (VKA for patients with A.fib and mitral valve
stenosis)
interventions (Aspirin the same as clopidogrel; LMWH)
outcomes (important versus surrogate outcomes;
cholesterol)
comparison (A - C versus A - B & C - B)
Why Grade Recommendations?
Strong recommendations
strong methods
large precise effect
few downsides of therapy
Weak recommendations
weak methods
imprecise estimate
small effect
substantial downsides
Why Grade Recommendations?
Strong recommendations
strong methods
large precise effect
few downsides of therapy
one size fits all
expect uniform clinician and patient behavior
Weak recommendations
weak methods
imprecise estimate
small effect
substantial downsides
Why Grade Recommendations?
Strong recommendations
strong methods
large precise effect
few downsides of therapy
one size fits all
expect uniform clinician and patient behavior
Weak recommendations
weak methods
imprecise estimate
small effect
substantial downsides
expect action to vary
Chapter authors
Develop the Clinical Question
Organize by patient groups or conditions
Examples from chapter on Ischemic Stroke
Previous
Now
Stroke Prevention
Antiplatelet agents
Non-cardioembolic stroke
Cardioembolic stroke
Oral Anticoagulation
Cardioembolic stroke
Non-cardioembolic stroke
Stroke Prevention
Non-cardioembolic stroke
Antiplatelet agents
Oral Anticoagulation
Cardioembolic stroke
Oral Anticoagulation
Antiplatelet agents
Explicit eligibility criteria
Example: Thrombolysis compared with no
thrombolysis for acute stroke
Patients: Patients presenting with acute thrombotic
stroke
Intervention: any thrombolytic regimen
Outcome: death, or validated functional status
instrument
Methodology: randomized trials
Trombolisi confrontata con non
trombolisi per stroke acuto
Questo quesito clinico orienta verso diverse raccomandazione:
 Evidence – recommendation:
1.1. tPA per via intra-venosa in caso di stroke ischemico acuto
transparent
caratterizzato dalla presenza di sintomi
per < 3 link
ore
Explicit
inclusion
criteria
1.2. tPA per via intra-venosa per 
stroke
ischemico
acuto
caratterizzato
dalla presenza di sintomi dalle 3 alle 6 ore
 Comprehensive search
1.3. Streptokinase intravenoso in caso di stroke ischemico acuto
 sintomi
Standard
consideration of
caratterizzato dalla presenza di
< 3 ore
1.4. Streptokinase intravenoso per study
stroke quality
ischemico acuto
caratterizzato dalla presenza di
dalle 3 alle
6 ore
 sintomi
Conduct/use
meta-analysis
 Grade recommendations
 Acknowledge values and
preferences underlying
recommendations
Role of librarians
Use questions to develop search strategy
e.g. identify all search terms (MESH and keywords) for antiplatelet
agents or myocardial infarction
Search:
Cochrane database of systematic reviews
Database of Abstracts of Reviews of Effectiveness
Cochrane Register of Controlled Trial
MEDLINE and Embase (1966 - Dec 2002)
ACP Journal Club
Provide search results
Used Endnote ® software
e.g. 490 citations on thrombolysis in acute stroke
Chapter authors
Identifying the Clinical Question:
Prior experience
Prior recommendations
What matters in clinical practice
The questions:
Identify patients, interventions, and
outcomes, but also methodological
criteria
Methodological quality
Criteria for baseline risk studies in
specific populations:
Cohort studies reporting of at least 200 participants
Control groups of RCTs reporting 200 participants
Focus in similar populations
Sufficient length of follow-up
Less than 20% loss to follow-up
Case scenario
A 67 year old engineer is brought to the ER with
tachyarrhythmia and near syncope. An EKG
reveals atrial fibrillation. Other workup is negative,
but the patients states that he – on and off – felt
his heart racing for several days. Together with
your team you diagnose the patient with lone atrial
fibrillation.
Schünemann et al.
Chest 2004
Schünemann HJ et
al. Chest 2004