Transcript Evidence Based Medicine: An Overview

An introduction to
Evidence-Based Medicine
part-1
Akbar Soltani, MD
Tehran University of Medical Sciences
Endocrine and Metabolism Research Center
Evidence-Based Medicine Working Team
Shariati Hospital
www.soltaniebm.com
www.ebm.ir
www.avicennact.ir
Some assumptions
• You, the audience, between you know much
more than I do about this
• Lao Tzu said: “Those who know do not
speak/Those who speak do not know.”
• Kafka: What a silence had been established
in the world if every person talk correlated
with his/her knowledge
What is Critical Thinking?
• What is the best way
of walking?
• What is the best way
of thinking?
[email protected]>
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Definition:
• Science is devoted to formulating and
testing naturalistic explanations for
natural phenomena. It is a process for
systematically collecting and recording
data about the physical world, then
categorizing and studying the collected
data in an effort to infer the principles of
nature that best explain the observed
phenomena.
72 Nobel laureates. (From the Amicus Curiae presented in the US Supreme
Court Case of Edwards vs Agullard, 1986)
Observation
Systematic
Nonsystematic
Accurate,
precise
Not accurate/
precise
More scientific
More
pseudoscientific
Reference based medicine
Is conventional/reference
based medicine
systematic
in search, appraisal
extraction of data..?
Trace back to the development of
EBM.
• 1972:Archie Cochrane published an book,
and told about the important role of RCTs.
• 1980's: Dave Sackett at McMaster
University in Canada.
• 1990s :The term was generated and coined
by Gordon Guyatt and his colleagues from
McMaster University
Trace back to the development of
EBM
Current best
evidence
A model for evidence-based clinical decisions
Sackett et al, 2000
background knowledge + evidence= decision making
I.Individual Clinical Expertise:
• Experience: Relates to what we’ve done and
to knowledge.
1. Clinical skills
2. Clinical judgment
3. Vital for determining whether the evidence
applies to the individual patient at all and, if
so, how
Expertise for
Diagnosis,
Procedures
Helps
clinicians
Interventions
(diagnostic or
therapeutic)
Need
accurate/precise
information
Do not need
accurate/precise
information
Not adequate
Adequate
II. Best External Evidence:
• From real clinical research among
intact patients.
• Has a short doubling-time (10
years).
III. Patients’ Values &
Expectations
• Have always played a central role in
determining whether and which
interventions take place
Current best
evidence
A model for evidence-based clinical decisions
Sackett et al, 2000
Bayesian approach: background knowledge + evidence= decision making
Conventional medicine
Expertise
(intuition…)
Pathophysiology,
references, tradition…
Patient value
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Why Is It So Hard to Be Up-to-date?
• MEDLINE has approximately:
– 6 million references from
– 4.000 journals with about
– 400.000 new entries added each year.
•
Doubling time of biomedical
science is about 20 months in
2001
Increasing Knowledge
Number of articles on Hypertension cited in
Medline by Year
8000
6000
4000
Articles
2000
0
1966
1976
1986
1996
Thrombolytic Therapy & MI mortality
RCT
23 Patients
1960
Odds Ratio
0.5
2
1

Treatment
Control
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of meta-analyses of
randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8
Thrombolytic Therapy & MI mortality
Cumulative
Year
Pts
1960
1965
RCTs
1
2
3
4
7
Odds Ratio
0.5
23 
65 
149 
316
1793
1

2

Treatment
Control
Antman JAMA 92
Thrombolytic Therapy & MI mortality
Cumulative
Year
Pts
1960
1965
1970
1975
1980
1985
1990
RCTs
1
2
3
4
7
10
11
15
17
22
23
27
33
65
70
Odds Ratio
0.5
2
1
23 
65 
149 
316
1793
2544
2651
3311
3929
5452
5767
6125
6571
47185
48154













Treatment
p < 0.01
p < 0.001
p < 0.00001
Control
Antman JAMA 92
Thrombolytic Therapy & MI mortality
Cumulative
Year
Pts
1960
1965
1970
1975
1980
1985
1990
RCTs
1
2
3
4
7
10
11
15
17
22
23
27
33
65
70
Textbook
Odds Ratio
0.5
2
1
23 
65 
149 
316
1793
2544
2651
3311
3929
5452
5767
6125
6571
47185
48154
Recommendations
Rout Specif Exp NOT













Treatment
1
1
2
p < 0.01
p < 0.001
5
15
p < 0.00001 6
Control
1
1
1
2
8
1
8
7
2
21
5
10
2
8
7
8
12
4
3
1
1
Antman JAMA 92
Antman JAMA 92
Some parts of textbooks are out-of-date
• Fail to recommend Rx up to ten years after
it’s been shown to be efficacious.
• Continue to recommend therapy up to ten
years after it’s been shown to be useless.
• Different textbooks, different
recommendations.
• Textbooks are appraisable?
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of
meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8
Expert component
• Professor Paul Knipschild has described
how Nobel prize winning biochemist Linus
Pauling used selective quotes from the
medical literature to "prove" his theory that
vitamin C helps you live longer and feel
better.
• When Knipschild and his colleagues
searched the literature systematically for
evidence
They found that
Expert component
• One or two trials did strongly suggest that
vitamin C could prevent the onset of the
common cold
• There were far more studies which did not
show any beneficial effect.
Interesting Example
Dr Naserimoghaddam
 182 Health authorities selected
 2 Articles: 1 on cardiac rehabilitation
1 on breast Ca screening
 Results of each presented in 4 ways:
 RRR (Relative Risk Reduction)
 ARR (Absolute Risk Reduction)
 PEFP (Proportion of Event Free
Patients)
 NNT ( Number Needed to Treat)
Dr Naserimoghaddam
Evidence based purchasing: understanding results of clinical trials and systematic reviews
T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
 They were told that these were the
results of 4 articles on each topic
 Question: According to which set of
data you may choose to adopt the
method as part of your regional
practice policy?
Dr Naserimoghaddam
Evidence based purchasing: understanding results of clinical trials and systematic reviews
T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Interesting Results !
N=140
Mammography
Cardiac
Rehabilitation
RRR
79%
76%
ARR
38%
56%
PEFP
38%
53%
NNT
51%
62%
Dr Naserimoghaddam
Evidence based purchasing: understanding results of clinical trials and systematic reviews
T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
 Only 3 noted that all 4 sets
of data are the same!
 None were clinicians!
Dr Naserimoghaddam
Evidence based purchasing: understanding results of clinical trials and systematic reviews
T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Hypothesis?
Sample size
estimation
None!
Failure to detect
a difference
=
Equivalence?
Assume non-inferiority if
the lower limit of 95% CI is
less than –5%,
N=904 per group!
Percent correct answers for knowledge
questions
100
90
80
70
60
50
40
30
RCT
Hip
NNT
LR
Rule
In/Out
10
0
Sen90
20
Agenda
• Definitions
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Global judgment by experts
• A pervasive problem for
primary care physicians
attempting to appraise clinical
information is the conflicting
recommendations by experts.
Vote counting!
Variation in prostatectomy
Pathophysiologic approach
Resident : Do you recommend HRT fore high LDL
in postmenopausal patients?
Attending
:
YES because estrogen increase HDL and
decrease LDL, Lpa,and ………..
Evidence Based Fallacy
WHI: Coronary Heart Disease
years
6
1
2
3
4
5
Answering
question
logic, mathematics,
philosophy,
social science?
Empirical science
Medicine…
Analytic
thinking
Synthetic:
Re/search,
reading…
The Slippery Slope
100%
knowledge
of current 50%
best care
. ..
r = -0.54
p<0.001
. . ....
.
... ...
...
... ....
..
....
0%
Choudhry, Fletcher and Soumerai,
years since
Ann Intern Med 2005;142:260-73
graduation
-94% of 62 studies found decreasing competence for at least some tasks, with
increasing physician age.
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Heuristical errors
• Heuristic = rule of thumb; mental process used to
learn, recall, or understand knowledge
• Some examples:
– Recency
– Rarity
– “burned” by missing a case
– Regression towards the mean
–…
(Tversky& Kahneman, 1974)
Agenda
• Definitions
• Dimension of problems
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Probability estimates of various qualitative verbal
expressions
Certain
Likely
Possible
Probable
Low probability
Suggests
High probability
Unlikely
Moderate probability
Pathognomonic
classic
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
West Vs East – Language
Eloquent=expressive
American ways-A guide for foreigner
Why Evidence-based Medicine?
• Science is a process for systematically
collecting and recording data
• Time not available to find and assimilate
evidence into practice
• Doubling time of biomedical science is about
20 months in 2001
– Medical Journals: too voluminous
– Scientific chaos
Dr Naserimoghadam
Why Evidence-based Medicine?
• Traditional sources of info:
– Textbooks :partially (10-30%) outdated before publication
– Experts:
•
•
•
•
•
Pathophysiologic approach
Conflicting recommendations
Biased towards their own works & knowledge
Heuristic and errors
Problems of communication
– CME: ineffective
• Clinical judgment / diagnostic skills increase with
time, but up-to-date clinical knowledge declines
Dr Naserimoghadam
Recommendations
•
•
•
•
•
Think and search systematically!
Learn methodology
Learn errors of expertise
Find EBM recourses
…
An introduction to
Evidence-Based Medicine
part-2
Akbar Soltani, MD,MS.
Tehran University of Medical Sciences
Endocrine and Metabolism Research Center
Evidence-Based Medicine Working Team
Shariati Hospital
Esfand 87 (2009)
www.soltaniebm.com
www.ebm.ir
www.avicennact.ir
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Evidence
Based
Medicine
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
Evidence-Based Medicine: How to Practice and Teach EBM by David Sackett
Ask
• We need it twice for every 3 outpatients and
2 times for every inpatient
• Questions are most likely to be about
treatment
• Most of the questions generated in
consultations go unanswered.
EBM process
P: Among patients with NIDDM who
are having MI
I: does tight control of their blood
sugar
C: in comparison to conventional
methods
O: reduce their risk of dying?"
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
EBM process
2 Efficient track-down of the best
evidence
–Secondary (pre-appraised)
sources e.g.,
– Cochrane (systematic reviews)
– E-B Journals
–primary literature
Example of a search strategy
#1
RANDOMIZED-CONTROLLED-TRIAL in PT
#2
CONTROLLED-CLINICAL-TRIAL in PT
#3
RANDOMIZED-CONTROLLED-TRIALS
#4
RANDOM-ALLOCATION
#5
DOUBLE-BLIND-METHOD
#6
SINGLE-BLIND-METHOD
#7
#1 or #2 or #3 or #4 or #5 or #6
#8
TG=ANIMAL not (TG=HUMAN and
TG=ANIMAL)
#9
#7 not #8
#10
CLINICAL-TRIAL in PT
#11
explode CLINICAL-TRIALS
#12
(clin* near trial*) in TI
#13
(clin* near trial*) in AB
#14
(singl* or doubl* or trebl* or tripl*) near (blind*
or mask*)
#15
(#14 in TI) or (#14 in AB)
#16
PLACEBOS
#17
placebo* in TI
#18
placebo* in AB
#19
random* in TI
#20
random* in AB
#21
RESEARCH-DESIGN
#22
#10 or #11 or #12 or #13 or #15 or #16 or #17 or
#18 or #19 or #20 or #21
#23
TG=ANIMAL not (TG=HUMAN and
TG=ANIMAL)
#24
#22 not #23
#25
#24 not #9
#26
TG=COMPARATIVE-STUDY
#27
explode EVALUATION-STUDIES
#28
FOLLOW-UP-STUDIES
#29
PROSPECTIVE-STUDIES
#30
control* or prospectiv* or volunteer*
#31
(#30 in TI) or (#30 in AB)
#32
#26 or #27 or #28 or #29 or #31
#33
TG=ANIMAL not (TG=HUMAN and
TG=ANIMAL)
#34
#32 not #33
#35
#34 not (#9 or #25)
#36
#9 or #25 or #35
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
EBM process
3-Critical appraisal of the
evidence for its validity and
clinical applicability
Level of evidence for treatment
SR
A
B
C
D
E
Why do I have to bother?
Can’t I trust the editors?
Percent of articles meeting quality criteria
NEJM
Ann Int Med
JAMA
Lancet
BMJ
Arch Int Med
12.6
7.6
7.2
6.2
4.4
2.4
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
EBM process
4 Integration of that critical appraisal
with clinical expertise and the patient’s
unique biology and beliefs  action.
5 Evaluation: evaluating our
effectiveness and efficiency in
executing steps 1–4 and seeking ways
to improve them both for next time.
Even fully EB-trained clinicians
work in all 3 modes
“Searching & appraising” mode for the
problems I encounter daily.
“Searching only” mode among E-B
resources for problems I encounter once a
month.
“Replicating” the practice of experts mode
for problems I encounter once a decade .
Sackett et al, 2000
What Proportion of Healthcare is
Evidence-Based ?
• BMJ Editorial: about 15%
• Archie Cochrane: less than 10%
• NIH : Diagnostic technology 20 %
Smith R: Where is the wisdom...? The poverty of medical evidence. BMJ 1991;303:798-9.
Barriers to EBP
16.4%
Lack of skill accessing/assessing research
7%
Research Not Valued
41.8%
Lack of Time
11.1%
Unaware of Scope of
Autonomous Nursing Practice
23.6%
Lack of resources
N=1033
What are the Challenges
• time
•
•
•
•
•
•
access
skill in critical appraisal
language of research
sense of control over practice
environment/culture
applicability
Reference based medicine?
•
•
•
•
First, idea second, references
Vague questions
systematic search is not usual
systematic critical appraisal is not
usual
• Inadequate evaluation
It’s like pseudoscience, isn’t it?
Reference based medicine?
Conventional
medicine
Some times systematic
in search, appraisal
extraction of data..
Use
pathophysiologic
approach
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
• Summary
Why Evidence-based Medicine?
• Physicians have daily need for valid info
– 5 times for every inpatient
– 2 times for every 3 outpatients
• Traditional sources of info:
–
–
–
–
–
Textbooks :partially (10-30%) outdated before publication
Experts: biased towards their own works & knowledge
CME: ineffective
Medical Journals: too voluminous
Scientific chaos
• Clinical judgment / diagnostic skills increase with time, but upto-date clinical knowledge declines
• Time not available to find and assimilate evidence into practice
• Resources are limited
Dr Naserimoghadam
Practicing EBM:
New Developments
• New strategies for finding and evaluating evidence
• New tools:
• Meta-analyses
• Systematic reviews / the Cochrane Collaboration
• Evidence-based journals of secondary publication
• Information systems bring info in seconds
Dr Naserimoghadam
Evidence-Based Joke
SR
A
B
C
D
E
Evidence-Based Joke
Class0:Things I believe
Class0a:Things I believe despite the available data
Class1:Randomised controlled clinical trials that agree with what I believe
Class2:Other prospectively collected data that agree with what I believe
Class3:Expert opinion that agree with what I believe
Class4:Randomised controlled clinical trials that do not agree with what
I believe
Class5:What you believe that I do not
egocentrism/ narcissistic trait/personality
think about
• Imagine your life and the lives of your
friends and family placed in the hands of
juries and judges who let their biases and
stereotypes govern their decisions, who do
not attend to the evidence, who are not
interested in reasoned inquiry, who do not
know how to draw an inference or evaluate
one.
Critical Thinking:
What It Is and Why It Counts
Peter A. Facione
Dean of the College of Arts and Sciences
Santa Clara University
1998
Acknowledgment
• Dr Mortaz Hejri for her research and
contribution to understand our situation in
EBM in TUMS
• Dr Naserimoghadam , Dr Haynes RB, Dr
Sackett DL, Guyatt GH for using their
slides
• EMRC for financial support
References
• Cook DJ, Meade MO, Fink MP: How to keep up with the
critical care literature and avoid being buried alive. Crit Care
Med 24:1757-1768, 1996
• Evidence-Based Medicine: A Framework for Clinical Practice
by Friedland et. al
• Evidence-Based Medicine: How to Practice and Teach EBM by
David Sackett
• How to Read a Paper: The Basics of evidence based medicine
by Trisha Greenhalgh
• Studying a Study and Testing a Test by Richard Riegelman and
Robert Hirsch
• Smith R: Where is the wisdom...? The poverty of medical
evidence. BMJ 1991;303:798-9.
•
•
•
•
•
•
•
•
•
•
References
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of
results of meta-analyses of randomised control trials and recommendations of
clinical experts. JAMA 1992;268:240-8
Bero L, Rennie D. The Cochrane Collaboration. JAMA 1995;274:1935–8.
Villanueva EV, Burrows EA, Fennessy PA, Rajendran M, Anderson JN. Improving
question formulation for use in evidence appraisal in a tertiary care setting: a
randomised controlled trial. BMC Med Inform Decis Mak. 2001;1(1):4. Epub 2001
Nov 08.
Booth A, O'Rourke AJ, Ford NJ. Structuring the pre-search reference interview: a
useful technique for handling clinical questions. Bull Med Libr Assoc. 2000
Jul;88(3):239-46
Haynes RB. Clinical review articles. BMJ. 1992;304:330-1.
Oxman AD, Cook DJ, Guyatt GH. Users’ guides to the medical literature. VI. How
to use an overview. Evidence-Based Medicine Working Group. JAMA.
1994;272:1367-71.
Evidence-based Medicine Working Group. Evidence-based medicine: a new
approach to teaching the practice of medicine. JAMA 1992;268:2420-5
Guyatt GH, Rennie D. Users' guides to the medical literature. JAMA 1993;270:20967
Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. I. How
to get started. JAMA 1993;270:2093-5.
Evidence based purchasing: understanding results of clinical trials and systematic
reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Thank you