Transcript Evidence-based Medicine: What it is and what it isn’t

Limitations of Evidence Based Health Care
Sharon E. Straus MD MSc FRCPC
Li Ka Shing Knowledge Institute
St. Michael’s Hospital
University of Toronto
Competing Interests
 No Pharma $
 Associate Editor for ACP JC, EBM J, CMAJ
 Advisory Board for BMJ, BMJ Evidence Centre, Journal
of Clinical Epidemiology
Objectives
 To describe what EBHC is and how to practise it
 To review some of challenges to EBHC
We need information
 We need it twice for every 3 outpatients and 5 times
for every inpatient
 But, we rarely get what we need
We don’t get what we need
 Textbooks are out of date
 Searching bibliographic databases can be challenging
 The volume of clinical literature is increasing
exponentially
Median minutes/week spent
reading about my patients
Self-reports at 17 Grand Rounds
 Medical students
90 minutes
 House Officers (PGY1)
0
 SHOs (PGY2-4)
20
 Registrars
45
 Consultants
45
The inevitable consequence
 On average, the clinically-important knowledge of
physicians deteriorates rapidly after we complete our
training
 And, our performance deteriorates too
Our performance deteriorates
 In a study of 300 untreated, uncontrolled
hypertensives who were referred to local GPs,
investigators found that 6 months later, only 2/3 had
been started on treatment
 The physician’s year of graduation from medical
school was one of the most important determinants of
whether antihypertensive therapy was started
Care Gaps
 1/3 patients do not get treatments of proven
effectiveness
 1/4 patients get care that is not needed or potentially
harmful
 Up to 3/4 of patients don’t get the information they
need for decision making
 Up to 1/2 of clinicians don’t get the information they
need for decision making
Some solutions
1. Learn how to practise EBM ourselves
2. Seek and apply EBM summaries generated by others
3. Apply evidence-based strategies for changing our
behaviour
What evidence-based medicine is:
The practice of EBM is the integration of:
 individual clinical expertise
with the
 best available clinical evidence from systematic
research
with the
 patient’s values, expectations and circumstances
How to practise EBM
1.
2.
3.
4.
5.
Asking
Acquiring
Appraising
Applying
Assessing
Growth in EBHC
 Interest in EBHC has grown exponentially from 1
Medline citation in 1992 to over 46000 in 2009
 Professional organizations and training programs
have moved from whether to teach EBHC to how to
teach it
 Determinants of attendance at postgraduate journal
clubs include teaching of critical appraisal skills,
focusing on primary literature (and providing free
food!)
Criticisms of EBHC
 Misunderstandings and misperceptions of EBHC:
 It ignores patient values
 It promotes a cookbook approach
 Limitations of EBHC
 Limitations universal to science and medicine
 CMAJ 2000;163:837-41.
Limitations universal to science
and medicine
1.
2.
3.
Shortage of coherent, consistent clinical evidence
Difficulties in applying evidence
Barriers to the practice of high-quality medicine
1. Shortage of coherent, consistent
evidence
 ‘medicine seems to consist of a few things we know, a
few things we think we know (but probably don’t), and
lots of things we don’t know at all
 Clinical experience and reasoning ‘must be applied to
traverse the many grey zones of practice’
 Lancet 1995;345:840-2.
Inconsistent evidence
 Randomised trial comparing thrombolytic therapy
with PTCA in 3145 patients with acute MI reported an
ARR in mortality of 0.1% with PTCA
 Confidence interval around this ARR meant that it was
indeterminate
3 different interpretations!
 Investigators concluded there was no difference
 An accompanying editorial concluded that PTCA was
the superior treatment
 Another editorial in the same issue supported
thrombolysis
NEJM 1996;335:1253-60; NEJM 1996;335:1311-12; NEJM 1996;335:1313-6.
2. Difficulties in applying evidence
to individual patients
 Universal occurrence of biological variation
 Not unique to therapeutic trials
 Applied researchers carrying out subgroup analyses,
pragmatic trials, N-of-1 trials
 Need concise, intelligible methods for application of
evidence at the bedside/in the clinic
3. Barriers to the practice of highquality medicine
 Gap between demands for health care and resources
available to meet them
 Conflicts between interests of individual patients and
those of society at large
 Purchasers set priorities and ration services
Limitations unique to the practice
of EBHC
1.
2.
3.
The need to develop new skills
Limited time and resources
Paucity of evidence that EBHC works
1. The need to develop new skills
 Skills in literature searching, critical appraisal and
bedside/clinic application
 Are clinicians interested in acquiring these skills?
 Not everyone wants (or needs) to develop all of these
skills
For EBHC Practitioners
 Different modes of practising EBHC
 Doing, using, replicating
 EBHC skills can be acquired at any stage of training
 Members of clinical teams can collaborate and can
expand to include other disciplines
2. Limited time and resources
 Busy clinicians have little time to set aside to read
around their patients
 Even clinicians with the time may not have access to
the evidence
Awareness: Making evidence
available
 Need it within seconds if it is to be incorporated into
busy clinical rounds or within minutes if it’s to be used
between clinic patients
 It must be available in user friendly formats and must
be individualised
We need to make practising EBHC
doable for busy clinicians
 Evidence of care gaps
 We need efficient/effective EBHC practice tools
including:
 Improved query-answer systems
 Improved search filters
 Meta-search engines
 Voice recognition
 More robust evidence summary services
 Personalised evidence resources
 Resources seamlessly integrated with the EHR and online
prescribing tools
Examples t
Systems
Summaries
Computerized decision support
Evidence-based textbooks
(eg, ACP Med, CE, Dynamed, PIER, UTD)
Synopses of
Syntheses
Evidence-based journal abstracts
(eg, ACPJC, EBM, EBN, DARE)
Systematic reviews
Syntheses
(eg, ACPJC+. EvidenceUpdates, Cochrane)
Evidence-Based journal abstracts
Synopses of Studies
Studies
Original journal articles
(eg, ACPJC+, EvidenceUpdates)
 90% of searches were
successful
 52% confirmed
management
 23% led to changes in
management
 25% led to additional
decisions
 But cart was too big!

JAMA 1998;280:1336-8
Our approach and challenges

PalmPilot Xybernaut
Blackberry
iPAQ

Wireless server
Formatter
XML Server
Clinical
Evidence
Acute
Medicine
Gov’t
Guidelines

What kinds of info are most
useful to clinicians?
What is the most effective
way of querying evidencebased resources?
How do we format
answers?
Which search style would you use?
Personalising the evidence
 Different clinicians have different knowledge needs,
learning styles and preferences about format of
information
 Not all clinicians want or need to develop skills in
literature searching, critical appraisal and
bedside/clinic application
3. Paucity of Evidence that EBHC
works
 No evidence from RCTs
 Evidence from process studies
 Evidence from outcomes research
What’s the ‘E’ for EBHC?
 Are we asking the right question?
 Providing evidence from clinical research is
necessary but not sufficient for the provision of
optimal care
 Changing behaviour is a complex process requiring
comprehensive approaches directed towards
patients, physicians, managers and policy makers
 Provision of evidence is but one component
 BMJ 2003;327:33-5
But,
 Evidence is only 1 link in the chain
 Changing behaviour is a complex process requiring
systematic, comprehensive approaches directed
towards patients, clinicians, managers and policy
makers
 BMJ 2003;327:33-5
The challenges
 Aware
 Appraise and accept
 Apply (must be applicable and available)
 Adhere (by our patient and us)
Evidence based medicine should
be complemented by evidencebased implementation
Richard Grol
Some Resources
 JAMA Evidence
 Netting the Evidence
 www.cebm.utoronto.ca