Transcript Jonathan Underhill presentation

Making Decisions Better
…….Evidence-informed decision making
……how to feel comfortable with not knowing everything
….. working with our human nature, not against it
Jonathan Underhill
Associate Director, Medicines Evidence
NICE Medicines and Prescribing
Centre
www.nice.org.uk/mpc
[email protected]
1
What I want to talk about
How we make decisions
How to manage information overload
How to keep up to date
2
What I want to talk about
How we make decisions
How to manage information overload
How to keep up to date
3
Humans make decisions by……
Small number of variables
+
Allocate value to those variables
+
Time frame
=
DECISION
4
HOW?
Allocating value to those variables
•
Brief reading
• Talking to other people
5
6
7
Diagnosis
8
Rx
Can this approach let you down???
9
Steve is very shy and withdrawn, invariably helpful, but
with little interest in people. He has a need for order and
structure and a passion for detail
It is most likely that Steve is a ……
10
1.
Farmer
2.
Pharmacist
3.
Disc jockey
4.
Librarian
5.
Member of Parliament
How we acquire and use information
• Where did you get the information from to make that
decision about Noah and the sheep?
• If you had had time, what would you have done to make
sure you had the right answer?
11
Information and decision making
• Most decisions are based on what we think is the evidence, not what
we know is the evidence
• No one has time to appraise all of the evidence on everything, and
even if that were possible the human brain can’t recall and compute it,
and certainly not in a 10 minute primary care consultation
• We use brief reading and talking to other people as our information
sources
• We often use patterns to make a diagnosis
• We create mindlines ( = patterns) of what to do in common situations
12
13
How is knowledge managed in primary care?
Gabbay and le May BMJ 2004; 329: 1013 – 6.
• Not once was a guideline read
• Expert computer systems rarely used (never in real time)
• Shortcuts to evidence
– free magazines
– network of trusted colleagues (rarely if ever questioned)
– Pharma reps – considerable scepticism (but not without influence)
– Pharmaceutical adviser – highly trusted source.
“Clinicians rarely accessed, appraised, and used explicit evidence directly
from research or other formal sources; rare exceptions were where they
might consult such sources after dealing with a case that had particularly
challenged them.”
14
“Instead, they relied on what we have called "mindlines,"
collectively reinforced, internalised tacit guidelines, which
were informed by brief reading, but mainly by their
interactions with each other and with opinion leaders,
patients, and pharmaceutical representatives and by other
sources of largely tacit knowledge that built on their early
training and their own and their colleagues' experience.”
15
Information habits of doctors
Ely JW,et al. BMJ 1999; 319: 358-361
Covell DG et al. Ann Intern Med 1985; 103: 596-9)
• Eastern Iowa, 103 family doctors.
• If you ask doctors, they say they need information about once a week.
• But if you debrief them, they raise about 2 questions for every three
patients
• Answers to most questions were not immediately pursued.
• Doctors spent an average of less than 2 minutes pursuing an answer, and
they used readily available print and human resources.
• Only two questions (out of over 1100) led to a formal literature search.
16
16
Information and decision making
• Most decisions are based on what we think is the evidence, not what
we know is the evidence
• No one has time to appraise all of the evidence on everything, and
even if that were possible the human brain can’t recall and compute it,
and certainly not in a 10 minute primary care consultation
• We use brief reading and talking to other people as our information
sources
• We often use patterns to make a diagnosis
• We create mindlines ( = patterns) of what to do in common situations
17
52 cognitive biases
•
•
•
•
•
•
•
•
•
•
18
Anchoring bias – early salient feature
Ascertainment bias – thinking shaped by prior expectation
Availability bias – recent experience dominates evidence
Bandwagon effect – we do it this way here
Omission bias – natural disease progression preferred to
those occuring due to action of physician
Sutton’s slip – going for the obvious
Gambler’s fallacy – I’ve seen 3 recently; this can’t be a
fourth
Search satisfycing – found one thing, ignore others
Vertical line failure – routine repetitive tasks leading to
thinking in silo
Blind spot bias – other people are susceptible to these
biases but I am not
What I want to talk about
How we make decisions
How to manage information overload
How to keep up to date
19
Information Management
20
More reading?
• Potential journals
10,000
• Potential new articles per week
40,000
• Even if 97% are not relevant (no POOs)
• Time to read each article
1,200
15minutes
• 10h a day, 6 days a week = 240 articles.
• So at the end of the first week you are about 4 weeks behind in your
reading.
• At the end of the first month, you are 4 months behind in your reading.
• And at the end of the first year you are almost 5 years behind in your
reading.
21
Effect of Exercise on Pain in Knee OA
Roddy E, et al. Ann Rheum Dis 2005; 64: 544-8
22
How can we keep up?
Sackett D et al BMJ 1996;312:71-72
“The difficulties that clinicians face in keeping abreast of all the medical
advances reported in primary journals are obvious from a comparison
of the time required for reading
 for general medicine, enough to examine 19 articles per day, 365
days per year
with the time available
 well under an hour a week by British medical consultants, even on
self reports.”
How to best use use your Golden Hour?
23
What I want to talk about
How we make decisions
How to manage information overload
How to keep up to date
24
So is there a better way????
“Better is possible.
It does not take genius, it takes
diligence, it takes a clarity of
purpose, it takes ingenuity, it takes
a willingness to try.”
www.gawande.com/
25
What do we know about how people
make decisions?
• Behavioural economics and cognitive psychology:
– Bounded rationality (Herbert Simon 1978)
– Dual process theory (Dan Kahneman 2002)
– Most decisions are informed by brief reading and talking to other
people
26
How can work with this?
• We all need a system for keeping up to date:
– Hunting: find the best possible answer to a specific
question and recognise it as such, quickly and efficiently.
– Foraging: be alerted to new, important, relevant, valid
information that requires a change in practice
– Hot synching: update your brain once or twice a year on
the 30-40 conditions you see most frequently
27
Pre-digested sources of evidence from
trusted sources:
Public-sector ethos
Published methodology of how produced
Translation of evidence into practice
Context of the rest of the evidence
28
Finding the ‘best answer’, first time
Slawson DC and Shaughnessy AF
Cochrane Library
NICE etc
Clinical Evidence
InfoPOEMs, CKS
BestTreatments
Usefulness
EBM DTB MeReC
“Ivy League” journals
Textbooks
Medline,
Google scholar
29
Medicines
awareness
www.nice.org.uk/mpc
30
Medicines Awareness Service
31
Prescribing support inc.
• NICE CKS
• Awareness service:
– MAD, MAW, MECs
– Evidence summaries
32
• British National Formulary:
– Book
– Web
– Apps
• Good practice guidance
• Key therapeutic topics for QIPP
• NICE Evidence
• Pathways
• eLearning tools for NICE Guidance
BNF
www.nice.org.uk/mpc
• Monthly updates for digital
versions
• Book going to once a year
• User research
• Exploring ways to develop
content and integrate content
with NICE CKS, NICE
Guidance/pathways etc
33
Best use of a clinician’s ‘Golden Hour’?
• Give up on reading primary research:
– You don’t have time and you wont be able to do it
• Design your CPD based on these principles:
– Foraging:
• 2-3 bits of key new research, summarised for you and set in the
context of the rest of the evidence (MEC)
• Evidence awareness service e.g. NICE MPC Medicines Awareness
Daily Weekly
– Hot synching
• Be aware when new NICE/SIGN guidance comes out
• Take time to digest it, talk to others about it
• Ask yourself, “what are the important changes since I last updated my
brain?”
• ONLY about conditions YOU see commonly
34
– Hunting:
• Much more difficult
• Use the information pyramid (NHS Evidence/TRIP)
Cognitive Reflective Test
• The test distinguishes intuitive (system 1) from
analytical (system 2) processing….
• …….the ability to resist first response that comes to
mind
• Of 3,428 people tested only 17% got all 3 correct
• 33% answered all three incorrectly
Frederick 2002 (MIT)
35
Teaching “Think as well as blink”?
T
36
Better is possible
• Self-awareness (meta-cognition)
– “the right system at the right time”
• Information management
• Teaching “Think as well as blink”?
37
• Decision = Experiences + Evidence
system 1
system 2
• Usual practice:
– Decision =
Experiences +
Evidence
• Making decisions better:
– Decision = Experiences
38
+ Evidence
If you want more…..
www.npc.nhs.uk/evidence
39
40
www.NPC.nhs.uk
InnovAiT: Autumn / Winter 2009-2010