Transcript Systematic Reviews & Guidelines

Searching for Evidence
Task 1
Find a full text document from a reference
The ALLHAT Officers and Coordinators for the
ALLHAT Collaborative Research Group .
Major outcomes in moderately
hypercholesterolemic, hypertensive patients
randomized to pravastatin vs usual care: the
antihypertensive and lipid- lowering
treatment to prevent heart attack trial
(ALLHAT-LLT). JAMA. 2002;288:2998-3007.
Task 2
Find the best evidence concerning the use of
aspirin for the primary prevention of
cardiovascular disease in people with diabetes
ATHENS ACCOUNT
• To get a NHS Athens account, username and
password. Register for free at:
https://register.athensams.net/nhs/nhseng/
Task 3
NNT - Revision
From the diagram on the next slide
Calculate:
• Relative Risk
• Absolute Risk Reduction
• Numbers Needed to Treat
Task 4
NNT Calculation
In the Statins Metaanalysis
over a mean 4.3 years:
Baseline risk of a major coronary event = 5.7%
Relative Risk Reduction is 29.2%
Calculate Absolute Risk Reduction and
Numbers Needed to Treat (for 4.3 years)
Task 5
From the NNT found in the previous task
Calculate the drug cost to prevent one coronary
event using Simvastatin (£1.37 for 28 tablets)
Task 6
The JUPITER trial of
Rosuvastatin 20mg vs placebo
(in 17802 healthy men & women LDL<3.4 mmol/l, high sensitivtiy CRP 2.0+ mg/l)
reported:
0.77 vs 1.36 events per 100 person-years
HR 0.56 (0.46-0.69) p<0.00001
Rosuvastatin 20mg costs £26.02 for 28 tabs
What was the drug cost to prevent 1 event in this trial?
Systematic Reviews & Guidelines
Chris Lewis Apr 2009
Types of “paper” research evidence
• Primary studies
–
–
–
–
Case studies
Experiments
Surveys
Clinical Trials
• Secondary studies
– Non-systematic reviews
– Systematic reviews
• Meta-analyses
• Guidelines
• Decision analyses
• Economic analyses
Types of evidence
Advantages
Summarises all relevant
research about all possible
interventions for a clinical
problem. Explores benefits
and harms.
Disadvantages
May become out-of-date
quickly.
Expert opinion often fills
gaps in evidence.
Systematic Review
Summarises all research
about an intervention.
Usually only one of several
possible interventions is
considered. May not
explore benefits vs harms.
Primary Study
Very specific information
Not comprehensive
Evidence-based
Guideline
Systematic Review
• A systematic review is a literature review
• focused on a single question
• that tries to identify, appraise, select and
synthesize
• all high quality research evidence relevant to that
question.
Meta-analysis
A systematic review in which the data from
the primary studies is sufficiently similar that
they can be analyzed as if they were the same
trial.
They provide higher statistical power to detect
an effect and a more powerful estimate of
true effect size than individual studies.
How to conduct a systematic review
Formulate Question
PICO
Title / Abstract
Find all relevant studies
Search +++
Methods
Assess studies
At least 2 appraisers
Methods
Synthesize, analyze & present results
Results / Conclusions
FAST critical appraisal
4 FAST Questions
for appraising systematic reviews:
• Finding: Did they find all relevant studies?
• Appraisal: Did they select good studies?
• Synthesis: When the studies are put together
what do they mean?
• Transferability: Are the conclusions applicable
to my patient?
Critical Appraisal Q1 – Formulate Question
Did the study address a focused clinical
question?
•
•
•
•
Population
Intervention
Comparator
Outcome
Should be found in the title, abstract or
introduction
Critical Appraisal Q2 – Find all relevant studies
Were the inclusion/exclusion criteria
appropriate?
• Clearly defined before the literature search
• Should specify PICO
• May specify type of study (eg RCT)
Should be found in the methods section
Critical Appraisal Q3 – Find all relevant studies
Is it unlikely that important relevant
studies were missed?
Search strategy should include:
• Major databases: Medline, EMBASE, Cochrane
• Reference lists from relevant studies
• Inquiry into unpublished studies
• Foreign language publications
• MESH terms and text words
Methods section should describe search strategy
Results section should state number of studies retrieved
and number of studies excluded with reasons
Literature review flowchart
Thavendiranathan, P. et al. Arch Intern Med 2006;166:2307-2313.
Copyright restrictions may apply.
Publication Bias
• ‘Negative’ studies less likely to be published
than ‘ positive’ studies
• In a follow-up of 737 studies submitted to the
ethics committee at the Johns Hopkins
hospital positive studies were 2.5 times more
likely to be SUBMITTED than negative studies
(Dickersin, JAMA, 1992)
• All trials registered at inception
• Unethical not to make results available
Critical Appraisal Q4 – Assess studies
Were the included studies valid?
• Should describe how quality of included studies was
assessed
• Criteria for quality assessment should be predetermined
• Criteria for quality assessment should be appropriate
to the question asked
Methods section should describe assessment process and
quality criteria
Results section should give information on quality of the
individual studies
Critical Appraisal Q5 – Assess studies
Were assessments of studies reproducible?
• Assessed independently by 2+ reviewers
• Level of agreement between reviewers
• Procedure for dealing with disagreement
Methods should describe how and by who
assessments were done
Results should show level of agreements
Critical Appraisal Q6 – Analysis/Results
Were the results similar from study to
study?
• Ideally results should be similar
(homogeneous)
• Significance of heterogeneity may be assessed
• Reasons for heterogeneity should be explored
Results section.
Forest plot should show chi2 test for heterogeneity
Meta-analysis (Forest) plot
1.
2.
3.
4.
5.
6.
How many studies are there?
How many studies favour treatment?
How many studies are statistically
significant?
Which is the largest study?
Which is the smallest study?
What is the combined result?
Assessing heterogeneity
“Eyeball” test:
In the Forest plot a vertical line running
through the combined OR should cross the
horizontal lines (95% CI) of all the individual
studies
Assessing heterogeneity
Chi2 test or Cochran Q:
• If chi2 is statistically significant (p<0.1) there is
definite heterogeneity
• If chi2 not statistically significant (p>0.1) and
Q/df >1 there is possible heterogeneity
• If chi2 not statistically significant (p>0.1) and
Q/df >1 heterogeneity is very unlikely
Explaining heterogeneity
•
•
•
•
Variation in population studied
Variation in intervention
Variation in outcome measures
Variation in study methods
Plotted relative risk ratios (RRs) (95% confidence intervals [CIs]) for major coronary events
Thavendiranathan, P. et al. Arch Intern Med 2006;166:2307-2313.
Copyright restrictions may apply.
Plotted relative risk ratios (RRs) (95% confidence intervals [CIs]) for major cerebrovascular
events
Thavendiranathan, P. et al. Arch Intern Med 2006;166:2307-2313.
Copyright restrictions may apply.
Critical appraisal – Question 7
Are conclusions supported by data?
Biased interpretation or emphasis of results is
surprisingly common
Best to form your own conclusions from the
data before reading authors conclusions
Then consider the reasons for any differences
between your conclusions and the authors’
Critical Appraisal Q8
Are the results clinically relevant?
Assuming we have a statistically significant
benefit from treatment:
• Is the size of the benefit worthwhile?
• Are the results applicable to my patient?
Funding Bias?
• Funding / employment of authors
• Drug company sponsorship
Results
Statins reduce risks by:
• Coronary events
29.2% (16.7-39.8%) p<.001
• Cerebrovascular events 14.4% (2.8-24.6%) p0.02
• Revascularisations
33.8% (19.6-45.5%) p<.001
• Cardiovascular death 22.6% (0.56-1.08) p.13
• All deaths
0.92(0.84-1.01) p.09
Inclusion Criteria / Baseline characteristics
Trial
Age
WOSCOPS 45 - 64
Mean Sex
GP / Hosp
age
% male
Chol mmol/l Other
55
100
GP
(W.Scotland)
F LDL >4.5
<6.0
CVD 16%
Smoke 44%
AFCAPS/T
exCAPS
M45-73 58
F55-73
85
Texas
?Hosp
TC 4.65-6.82
Trig<=4.52
102800 screened
6600 randomised
PROSPER
70-82
75
42
?Hosp
(Scot/Ire/Net
hrlnds)
TC 4.0-9.0
Smo/HT/Diab
ALLHATLLT
55+
66
51
GP
(N.America)
LDL >=3.1 <=4.9
Trig <3.95
Hypertension
+1 other RF
ASCOTLLA
40 - 79
63
81
GP
(UK/Scand)
TC <=6.5
Hypertension
+3 other RF’s
HPS
40 - 80
NA
NA
Hosp (UK)
F TC >=3.5
Diabetes
CARDS
40 - 75
61
68
GP & Hosp
(UK/Ireland)
LDL <=4.14
Diabetes
F Trig <=6.78
CARDS required at least 1 of retinopathy, albuminuria, smoking, hypertension
Influence of Diabetes
Trial
RR Major Coronary Events
% Diabetic
WOSCOPS 1995
0.70 (0.58-0.85)
1.0
AFCAPS/TexCAPS 1998
0.60 (0.43-0.83)
3.8
PROSPER 2002
0.91 (0.71-1.15)
12.2
ALLHAT-LLT 2002
0.91 (0.79-1.04)
34.4
ASCOT-LLA 2003
0.65 (0.50-0.83)
24.3
HPS 2003
0.57 (0.41-0.79)
100
CARDS 2004
0.53 (0.35-0.82)
100
Systematic Reviews
Advantages:
• Larger numbers with greater statistical power
• Robustness across differing trial populations,
drugs within a class
Disadvantages:
• Small, but consistent, biases may produce an
invalid conclusion of real effect
Resources
• Oxford Centre for Evidence-based Medicine
http://www.cebm.net/
• The Cochrane Collaboration
http://www.cochrane.org/
Formula for caluclating NNT from OR if Patient
Expected Event Rate is known
GUIDELINES
The AGREE Instrument
Appraisal of Guidelines for REsearch & Evaluation
The AGREE Collaboration – June 2001
www.agreecollaboration.org
The AGREE Instrument
23 items organised in six domains
Each item is rated on a 4-point score scale
1.
2.
3.
4.
Strongly disagree
Disagree
Agree
Strongly agree
Overall assessment
•
•
•
•
Strongly recommend
Recommend (with provisos/alterations)
Would not recommend
Unsure
Domain 1:
Scope & Purpose
1. Overall objective(s)
2. Clinical question(s)
3. Target population
should all be specifically described
Domain 2:
Stakeholder Involvement
4. Guideline development group includes
individuals from all relevant professional
groups
5. Patients’ views & preferences sought
6. Target users clearly defined
7. Piloted by end users
Domain 3:
Rigour of development
8. Systematic search for evidence
9. Criteria for selecting evidence described
10.Methods for formulating recommendations
described
11.Health benefits, unwanted effects and risks
considered in formulating recommendations
12.Explicit link between recommendations and
evidence
13.External review by experts before publication
14.Procedure for update provided
Domain 4:
Clarity & Presentation
15.Recommendations specific & unambiguous
16.Management options clearly presented
17.Key recommendations easily identifiable
18.Guideline supported by tools for application
Domain 5:
Applicability
19.Potential organisational barriers in applying
the recommendations discussed
20.Cost implications considered
21.Key review criteria for monitoring/audit
presented
Domain 6:
Editorial Independence
22.Editorially independent from funding body
23.Conflicts of interest of guideline
development members recorded