Transcript EBM 1-2

An introduction to
Evidence-Based Medicine
(critical thinking in medicine)
Akbar Soltani. MD,MSc
Tehran University of Medical Sciences (TUMS)
Shariati Hospital
www.soltaniebm.com
www.ebm.ir
www.avicennact.ir
Educational Activities
• Whole spectrum of the medical profession
• From 2000 to 2006 we had more than 200
lectures in EBM, MDM, Methodology,
• From 2006 to 2007 we had more than 50
lectures in CT
• More than 7000 slides have been prepared
• 10 books have been compiled
•
www.soltaniebm.com or www.ebm.ir and www.avicennact.ir
Some assumptions
• You, the audience, between you know much
more than I do about this
• Lao Tzu said: “Those who know do not
speak/Those who speak do not know.”
• Kafka: What a silence had been established
in the world if every person talk correlated
with his/her knowledge
Workshop objectives
• Problems of conventional medicine
• Definition and philosophy of EBM/IM
• Different concepts such as
– answerable question, systematic review,
NNT,NNH,…
• Search methods
• Most popular EBM data bases
• Critical appraisal skills
What is Critical Thinking?
• What is the best way
of walking?
• What is the best way
of thinking?
[email protected]
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Definition:
• Science is devoted to formulating and
testing naturalistic explanations for
natural phenomena. It is a process for
systematically collecting and recording
data about the physical world, then
categorizing and studying the collected
data in an effort to infer the principles of
nature that best explain the observed
phenomena.
72 Nobel laureates. (From the Amicus Curiae presented in the US Supreme
Court Case of Edwards vs Agullard, 1986)
Philosophers in science:
Trace back to the development of
EBM.
• 1972:Archie Cochrane told about the
role of randomized control trial in
scientific medicine.
• 1980's: Dave Sackett
• 1990s :The term was generated by
Gordon Guyatt from McMaster
University
Trace back to the development of
EBM
What evidence-based medicine is
• “The conscientious (careful), explicit (clear,
unambiguous) and judicious (sensible) , use of
current best evidence in making
clinical decisions about the care of
individual patients.”
Sackett et al, 2000
What evidence-based medicine is:
The practice of EBM is the integration of
• Individual clinical expertise
with the
• Best available external clinical evidence
from systematic research.
and
• Patient’s values and expectations
Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS: Evidence based medicine:
what it is and what it isn’t. BMJ 1996;312:71-2.
I.Individual Clinical Expertise:
•
•
Experience: Relates to what we’ve done and to
knowledge.
“An expert is a person who has made all the
mistakes that can be made in a very narrow field”
(Niels Bohr)
1. Clinical skills
2. Clinical judgment
3. Vital for determining whether the evidence applies
to the individual patient at all and, if so, how
Patient seen
in practice
Matches research
result to specific
patients
Clinical
judgment
?
Confirms or denies
hypothesis
Hypothesis
generating
Outcomes
research
Expertise for
Diagnosis,
Procedures
Helps
clinicians
Interventions
(diagnostic or
therapeutic)
Need
accurate/precise
information
Do not need
accurate/precise
information
Not adequate
Adequate
II. Best External Evidence:
• From real clinical research among
intact patients.
• Has a short doubling-time (10 years).
• Replaces currently accepted diagnostic tests
and treatments with new ones that are more
powerful, more accurate, more efficacious,
and safer.
III. Patients’ Values &
Expectations
• Have always played a central role in
determining whether and which
interventions take place
Current best
evidence
A model for evidence-based clinical decisions
Sackett et al, 2000
Bayesian approach: background knowledge + evidence= decision making
Model of Evidence-Based Medicine
Evidence
Clinical
Setting
Clinical Expertise
Patient’s
Preferences
Conventional medicine
Expertise
(intuition…)
Pathophysiology,
references, tradition…
Patient value
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Why Is It So Hard to Be
Up-to-date?
• The database of the National
Library of Medicine MEDLINE has
approximately 6 million references
from 4.000 journals with about
400.000 new entries added each year.
•
Doubling time of biomedical
science is about 20 months in
2001
Increasing Knowledge
Number of articles on Hypertension cited in
Medline by Year
8000
6000
4000
Articles
2000
0
1966
1976
1986
1996
How many original articles should a
specialist read each week to remain
up to date in his/her own field only ?
5
 10
 20
 40
 100
Dr.S.Naserimoghaddam
How many original articles should a
specialist read each week to remain
up to date in his/her own field only ?
5
 10
 20
 40
 100
The story is different for a generalist: 17 /day!
Dr.S.Naserimoghaddam
Thrombolytic Therapy & MI mortality
RCT
23 Patients
1960
Odds Ratio
0.5
2
1

Treatment
Control
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of meta-analyses of
randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8
Thrombolytic Therapy & MI mortality
Cumulative
Year
Pts
1960
1965
RCTs
1
2
3
4
7
Odds Ratio
0.5
23 
65 
149 
316
1793
1

2

Treatment
Control
Antman JAMA 92
Thrombolytic Therapy & MI mortality
Cumulative
Year
Pts
1960
1965
1970
1975
1980
1985
1990
RCTs
1
2
3
4
7
10
11
15
17
22
23
27
33
65
70
Odds Ratio
0.5
2
1
23 
65 
149 
316
1793
2544
2651
3311
3929
5452
5767
6125
6571
47185
48154













Treatment
p < 0.01
p < 0.001
p < 0.00001
Control
Antman JAMA 92
Thrombolytic Therapy & MI mortality
Cumulative
Year
Pts
1960
1965
1970
1975
1980
1985
1990
RCTs
1
2
3
4
7
10
11
15
17
22
23
27
33
65
70
Textbook
Odds Ratio
0.5
2
1
23 
65 
149 
316
1793
2544
2651
3311
3929
5452
5767
6125
6571
47185
48154
Recommendations
Rout Specif Exp NOT













Treatment
1
1
2
p < 0.01
p < 0.001
5
15
p < 0.00001 6
Control
1
1
1
2
8
1
8
7
2
21
5
10
2
8
7
8
12
4
3
1
1
Antman JAMA 92
Antman JAMA 92
Some parts of textbooks are out-of-date
• Fail to recommend Rx up to ten years after
it’s been shown to be efficacious.
• Continue to recommend therapy up to ten
years after it’s been shown to be useless.
• Different textbooks, different
recommendations.
• Textbooks are fact or opinion?
• Textbooks are appraisable?
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of results of
meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992;268:240-8
The Prognosis of Ignorance is Poor
Worse with “duration in practice”
Interesting Example
Dr Naserimoghaddam
 182 Health authorities selected
 2 Articles: 1 on cardiac rehabilitation
1 on breast Ca screening
 Results of each presented in 4 ways:
 RRR (Relative Risk Reduction)
 ARR (Absolute Risk Reduction)
 PEFP (Proportion of Event Free
Patients)
 NNT ( Number Needed to Treat)
Dr Naserimoghaddam
Evidence based purchasing: understanding results of clinical trials and systematic reviews
T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
 They were told that these were the
results of 4 articles on each topic
 Question: According to which set of
data you may choose to adopt the
method as part of your regional
practice policy?
Dr Naserimoghaddam
Evidence based purchasing: understanding results of clinical trials and systematic reviews
T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Interesting Results !
N=140
Mammography
Cardiac
Rehabilitation
RRR
79%
76%
ARR
38%
56%
PEFP
38%
53%
NNT
51%
62%
Dr Naserimoghaddam
Evidence based purchasing: understanding results of clinical trials and systematic reviews
T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
 Only 3 noted that all 4 sets
of data are the same!
 None were clinicians!
Dr Naserimoghaddam
Evidence based purchasing: understanding results of clinical trials and systematic reviews
T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Hypothesis?
Sample size
estimation
None!
Failure to detect
a difference
=
Equivalence?
Assume non-inferiority if
the lower limit of 95% CI is
less than –5%,
N=904 per group!
Percent correct answers for knowledge
questions
100
90
80
70
60
50
40
30
RCT
Hip
NNT
LR
Rule
In/Out
10
0
Sen90
20
Agenda
• Definitions
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Global judgment by experts
• A pervasive problem for
primary care physicians
attempting to appraise clinical
information is the conflicting
recommendations by experts.
Bloor M. Bishop Berkeley and the adenotonsillectomy enigma: an
exploration of the social construction of medical disposals. Sociology
1976; 10: 43–61.
EVIDENCE-BASED PRACTICE 2000
Variation in current practice
Pathophysiologic approach
• Resident: Do you recommend HRT fore
high LDL in postmenopausal patients?
• Attending: YES because estrogen increase
HDL and decrease LDL, Lpa,and ………..
Evidence Based Fallacy
Answering
question
logic, mathematics,
philosophy,
social science?
Empirical science
Medicine…
Analytic
thinking
Synthetic:
Re/search,
reading…
WHI: Coronary Heart Disease
years
6
1
2
3
4
5
Does CME Work?
– Traditional CME in a nice place
with pleasant after lecture
diversions is, unfortunately,
completely ineffective in
changing our behavior.
Davis DA, Thompson MA, Oxman AD, Haynes RB: Changing physician performance. A systematic review of the effect of continuing medical education
strategies. JAMA 1995;274:700-5.
The Slippery Slope
100%
knowledge
of current 50%
best care
. ..
r = -0.54
p<0.001
. . ....
.
... ...
...
... ....
..
....
0%
Choudhry, Fletcher and Soumerai,
years since
Ann Intern Med 2005;142:260-73
graduation
-94% of 62 studies found decreasing competence for at least some tasks, with
increasing physician age.
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
A quick assessment
Consider the following list of words:
goiter ,weight loss, sweating, hair loss,
proptosis, lid lag, dyspnea, wide pulse
pressure, weakness, hyperphagia, staring,
diarrhea ,anxiety
Write down as many as you can remember…
A quick assessment:
• Did you include tremor or palpitation
in the list of words you thought you
heard?
• Results : based on nonrandom sampling (N=600),
error proportion was 20% (unpublished!)
• What is the validity of the estimation
of frequency (or other measures) of the
clinical findings?
Heuristical errors
• Heuristic = rule of thumb; mental process used to
learn, recall, or understand knowledge
• Some examples:
– Recency
– Rarity
– “burned” by missing a case
– Regression towards the mean
–…
(Tversky& Kahneman, 1974)
Agenda
• Definitions
• Dimension of problems
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Probability estimates of various qualitative verbal
expressions
Certain
Likely
Possible
Probable
Low probability
Suggests
High probability
Unlikely
Moderate probability
Pathognomonic
classic
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
West Vs East – Language
Eloquent=expressive
American ways-A guide for foreigners
Why Evidence-based Medicine?
• Science is a process for systematically
collecting and recording data
• Time not available to find and assimilate
evidence into practice
• Doubling time of biomedical science is about
20 months in 2001
– Medical Journals: too voluminous
– Scientific chaos
Dr Naserimoghadam
Why Evidence-based Medicine?
• Traditional sources of info:
– Textbooks :partially (10-30%) outdated before publication
– Experts:
•
•
•
•
•
Pathophysiologic approach
Conflicting recommendations
Biased towards their own works & knowledge
Heuristic and errors
Problems of communication
– CME: ineffective
• Clinical judgment / diagnostic skills increase with
time, but up-to-date clinical knowledge declines
Dr Naserimoghadam
End of part
one
An introduction to
Evidence-Based Medicine
(critical thinking in medicine)
Akbar Soltani. MD,MSc
Tehran University of Medical Sciences (TUMS)
Shariati Hospital
www.soltaniebm.com
www.ebm.ir
www.avicennact.ir
Part-2
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
Evidence
Based
Medicine
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
Evidence-Based Medicine: How to Practice and Teach EBM by David Sackett
Ask
• We need it twice for every 3 outpatients and
2 times for every inpatient
• Questions are most likely to be about
treatment
• Most of the questions generated in
consultations go unanswered.
EBM process
P: Among patients with NIDDM who
are having MI
I: does tight control of their blood
sugar
C: in comparison to conventional
methods
O: reduce their risk of dying?"
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
EBM process
2 Efficient track-down of the best
evidence
–Secondary (pre-appraised)
sources e.g.,
– Cochrane (systematic reviews)
– E-B Journals
–primary literature
Example of a search strategy
#1
RANDOMIZED-CONTROLLED-TRIAL in PT
#2
CONTROLLED-CLINICAL-TRIAL in PT
#3
RANDOMIZED-CONTROLLED-TRIALS
#4
RANDOM-ALLOCATION
#5
DOUBLE-BLIND-METHOD
#6
SINGLE-BLIND-METHOD
#7
#1 or #2 or #3 or #4 or #5 or #6
#8
TG=ANIMAL not (TG=HUMAN and
TG=ANIMAL)
#9
#7 not #8
#10
CLINICAL-TRIAL in PT
#11
explode CLINICAL-TRIALS
#12
(clin* near trial*) in TI
#13
(clin* near trial*) in AB
#14
(singl* or doubl* or trebl* or tripl*) near (blind*
or mask*)
#15
(#14 in TI) or (#14 in AB)
#16
PLACEBOS
#17
placebo* in TI
#18
placebo* in AB
#19
random* in TI
#20
random* in AB
#21
RESEARCH-DESIGN
#22
#10 or #11 or #12 or #13 or #15 or #16 or #17 or
#18 or #19 or #20 or #21
#23
TG=ANIMAL not (TG=HUMAN and
TG=ANIMAL)
#24
#22 not #23
#25
#24 not #9
#26
TG=COMPARATIVE-STUDY
#27
explode EVALUATION-STUDIES
#28
FOLLOW-UP-STUDIES
#29
PROSPECTIVE-STUDIES
#30
control* or prospectiv* or volunteer*
#31
(#30 in TI) or (#30 in AB)
#32
#26 or #27 or #28 or #29 or #31
#33
TG=ANIMAL not (TG=HUMAN and
TG=ANIMAL)
#34
#32 not #33
#35
#34 not (#9 or #25)
#36
#9 or #25 or #35
Validity: Find the Best Valid Evidence
First
search for the best (prevalidated)
database information first.
Cochrane Library
Clinical Evidence
Clinical Inquiries
Specialty-specific
Usefulness
POEMs
Best Evidence
Textbooks, Up-toDate, 5-Minute
Clinical Consult
Medline
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
EBM process
3-Critical appraisal of the
evidence for its validity and
clinical applicability
Level of evidence for treatment
Why do I have to bother?
Can’t I trust the editors?
Percent of articles meeting quality criteria
NEJM
Ann Int Med
JAMA
Lancet
BMJ
Arch Int Med
12.6
7.6
7.2
6.2
4.4
2.4
EBM process
1. Ask
2. Search
3. Appraise
4. Apply
5. Evaluate
EBM process
4 Integration of that critical appraisal
with clinical expertise and the patient’s
unique biology and beliefs  apply.
5 Evaluation: evaluating our
effectiveness and efficiency in
executing steps 1–4
Which doctor do you want?
William Osler, 1900
Smart young doctor
Which doctor do you want?
Wise & experienced smart young doctor
What Proportion of Healthcare is
Evidence-Based ?
• BMJ Editorial: about 15%
• Archie Cochrane: less than 10%
• NIH : Diagnostic technology 20 %
Smith R: Where is the wisdom...? The poverty of medical evidence. BMJ 1991;303:798-9.
What are the Challenges
• time
•
•
•
•
•
•
access
skill in critical appraisal
language of research
sense of control over practice
environment/culture
applicability
Criticisms of Evidence-Based Medicine
•
•
•
EBM has been or might be used by
payers as an excuse to deny payment and
limit clinician autonomy.
Evidence-based treatment
recommendations tend towards the
nihilistic
EBM over-values randomized blinded
trials and denigrates other forms of
evidence, including clinical experience.
Reference based medicine?
•
•
•
•
First, idea second, references
Vague questions
systematic search is not usual
systematic critical appraisal is not
usual
• Inadequate evaluation
It’s like pseudoscience, isn’t it?
Agenda
• Definitions: Science and EBM
• Dimensions of the problem
1.
2.
3.
4.
Information management (mastery)
Limitations of current clinical practice
Heuristic and errors
Problems of communication
• EBM
• Summary
Practicing EBM:
New Developments
• New strategies for finding and evaluating
evidence
• New tools:
Meta-analyses
• Systematic reviews /Cochrane Collaboration
•
• Evidence-based journals of secondary
publication
• Information systems bring info in seconds
Dr Naserimoghadam
Conventional medicine
experiences
Pathophysiology,
references,…
Patient value
Current best
evidence
A model for evidence-based clinical decisions
Sackett et al, 2000
Evidence-Based Joke
Evidence-Based Joke
Class0:Things I believe
Class0a:Things I believe despite the available data
Class1:Randomised controlled clinical trials that agree with what I believe
Class2:Other prospectively collected data that agree with what I believe
Class3:Expert opinion that agree with what I believe
Class4:Randomised controlled clinical trials that do not agree with what
I believe
Class5:What you believe that I do not
egocentrism/ narcissistic trait/personality
think about
• Imagine your life and the lives of your
friends and family placed in the hands of
juries and judges who let their biases and
stereotypes govern their decisions, who do
not attend to the evidence, who are not
interested in reasoned inquiry, who do not
know how to draw an inference or evaluate
one.
Critical Thinking:
What It Is and Why It Counts
Peter A. Facione
Dean of the College of Arts and Sciences
Santa Clara University
1998
What are the alternatives to
EBM?
Isaacs, BMJ
References
• Cook DJ, Meade MO, Fink MP: How to keep up with the
critical care literature and avoid being buried alive. Crit Care
Med 24:1757-1768, 1996
• Evidence-Based Medicine: A Framework for Clinical Practice
by Friedland et. al
• Evidence-Based Medicine: How to Practice and Teach EBM by
David Sackett
• How to Read a Paper: The Basics of evidence based medicine
by Trisha Greenhalgh
• Studying a Study and Testing a Test by Richard Riegelman and
Robert Hirsch
• Smith R: Where is the wisdom...? The poverty of medical
evidence. BMJ 1991;303:798-9.
•
•
•
•
•
•
•
•
•
•
References
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC: A comparison of
results of meta-analyses of randomised control trials and recommendations of
clinical experts. JAMA 1992;268:240-8
Bero L, Rennie D. The Cochrane Collaboration. JAMA 1995;274:1935–8.
Villanueva EV, Burrows EA, Fennessy PA, Rajendran M, Anderson JN. Improving
question formulation for use in evidence appraisal in a tertiary care setting: a
randomised controlled trial. BMC Med Inform Decis Mak. 2001;1(1):4. Epub 2001
Nov 08.
Booth A, O'Rourke AJ, Ford NJ. Structuring the pre-search reference interview: a
useful technique for handling clinical questions. Bull Med Libr Assoc. 2000
Jul;88(3):239-46
Haynes RB. Clinical review articles. BMJ. 1992;304:330-1.
Oxman AD, Cook DJ, Guyatt GH. Users’ guides to the medical literature. VI. How
to use an overview. Evidence-Based Medicine Working Group. JAMA.
1994;272:1367-71.
Evidence-based Medicine Working Group. Evidence-based medicine: a new
approach to teaching the practice of medicine. JAMA 1992;268:2420-5
Guyatt GH, Rennie D. Users' guides to the medical literature. JAMA 1993;270:20967
Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. I. How
to get started. JAMA 1993;270:2093-5.
Evidence based purchasing: understanding results of clinical trials and systematic
reviews T Fahey, S Griffiths, T J Peters. BMJ 1995;311:1056-1059 (21 October)
Thank you
COMPARE TWO paradigm in
iran and weast
•
•
•
•
•
•
•
•
Intuition
Personal
Not transferable
applicable in religeon
and philosophy
Powerful
Rooye mien dar jang
Vague
Introspective/deep
• Fact as a sourse of
maarefat
• Interpersonal
• Apply to scince and
experiences
• Rooye mein nemireh
• Clear accurate
• Factual
• Probabilistic
• Negotiation is possible
West Vs East - Language
• In the West, logical
and semantic clarity
are among the most
celebrated of the
ideals of Reason.
• These ideas are
associated with
univocal definition
guaranteeing
unambiguous usage.
• In this sense, the
opposite of clarity is
confusion – a state of
• In classical Eastern
texts, allusive (indirect) and
connotatively rich
language is more highly
prized than clarity,
precision, and
argumentative rigor.
• We must attempt to
avoid ‘the Fallacy of the
Perfect Dictionary.’
Paradigm Shift
•
•
•
•
•
•
•
Changing between original and review
Evidence-based in clinical practice
Learning according to levels of evidence
Skills to make critical appraisal topics
From Sp / Sn to NNT / LR / OR
Resources on the internet
Review a clinical question through RCT’s