Transcript Is surgery an anachronism in an evidence based age?

Is surgery an
anachronism in an
evidence based age?
Richard Smith
Editor, BMJ
www.bmj.com/talks
What I want to talk about
• “The history of surgery: my
contribution”
• Surgery: historical and
literary reflections
• The state of evidence in
surgery
• Improving the evidence base
“The history of
surgery: my
contribution”
My role as emotional lightning conductor
--hanging onto the second retractor
• “Married are you, Richard?”
• “No.”
• “Good idea. The only point I can see in being married is
that it saves you having to find somebody to go on
holiday with.”
• “Do you have horses?”
• “No. I live in a flat.”
• “Good chap. Super weather, isn’t it?”
• And so it went until he cut something he shouldn’t have.
“Bloody hell. Will you pull on that retractor, Smith? You
are a bloody fool. Don’t you know anything about
surgery? For God’s sake, pull harder.”
My vasectomy
•
•
•
•
•
•
•
•
•
Twenty years later I had a similar conversation, only this time I was
under the knife. I was having a vasectomy. The surgeon, who did 20
vasectomies a day five days a week, had already cut my right vas
when he asked me what I did.
“I’m a sort of journalist.”
“What sort?”
“A medical journalist.”
“Oh. Where do you work?”
“At the BMJ.”
“What exactly do you do there?”
“I’m the editor.”
Suddenly the surgeon began to sweat. It had taken him two minutes
to find and cut one vas. He now took 20 minutes to find the second.
“Oh God,” he said, “I suppose I’m going to read about this in next
week’s journal.”
Surgery: historical
and literary
reflections
Hemicorporectomy
• Hemicorporectomy or translumbar amputation is
probably the most mutilating operation ever to be
described in surgical literature.
• Treatment for pelvic malignancy
• The procedure involves removal of the bony pelvis,
both lower limbs, the external genitalia, the
bladder, rectum and anus.
• First proposed in 1951 and performed in 1960
• Two of the first three patients died within days of
oedema; one survived 19 years
• A series of 10 cases reported from New York in
1982
Nemesis
• Nemesis is the goddess of divine
justice and vengeance. Her anger is
directed toward human
transgression of the natural, right
order of things and of the arrogance
causing it. Nemesis pursues the
insolent and the wicked with
inflexible vengeance.
Sir William Arbuthnot
Lane (1853-1946)
• Performed total colectomies for the
treatment of “auto-intoxication,”
something like chronic fatigue syndrome
• Operated on many of London’s
fashionable set
• Operation had a 10% mortality
• Dreadful side effects
• The placebo effect guarantees that some
will be “cured”
Ironic quote from
Arbuthnot Lane
• "If everyone believes a
thing it is probably
untrue!"
• Quoted by W. E. Tanner in Sir W. Arbuthnot
Lane, "Genesis"
Monologue by Sir
Patrick Ridgeon
•
I tell you, Colly, chloroform has done a lot of mischief. It's enabled
every fool to be a surgeon. I know your Cutler Walpoles and their like.
They've found out that a man's body is full of bits and scraps of old
organs he has no mortal use for. Thanks to chloroform, you can cut
half a dozen of them out without leaving him any the worse, except
for the illness and the guineas it costs him. I knew the Walpoles
fifteen years ago. The father used to snip off the ends of people's
uvulas for fifty guineas, and paint throats with caustic every day for
a year at two guineas a time. His brother-in-law extirpated tonsils for
two hundred guineas until he took up women's cases at double the
fees. Cutler himself worked hard at anatomy to find something fresh
to operate on; and at last he got hold of something he calls the
nuciform sac, which he's made quite the fashion. People pay him five
hundred guineas to cut it out. They might as well get their hair cut
for all the difference it makes; but I suppose they feel important after
it. You can't go out to dinner now without your neighbor bragging to
you of some useless operation or other.
Cutler Walpole’s
Nuciform Sac
• Cutler Walpole made his fortune
from removing the nuciform sac
from the fashionable in London
• “And have you had your nuciform
sac removed, Mr Walpole.”
• “I’m one of the lucky and unusual
people who was born without
one.”
GBS on operations
• The large range of operations which consist of
amputating limbs and extirpating organs
admits of no direct verification of their
necessity. There is a fashion in operations as
there is in sleeves and skirts. ... There are men
and women whom the operating table seems
to fascinate: half-alive people who through
vanity, or hypochondria, or a craving to be the
constant objects of anxious attention or what
not, lose such feeble sense as they ever had of
the value of their own organs and limbs.
GBS on doctors sticking
together
• The only evidence that can decide a
case of malpractice is expert
evidence: that is, the evidence of
other doctors; and every doctor will
allow a colleague to decimate a
whole countryside sooner that
violate the bond of professional
etiquette by giving him away.
GBS on EBM
• It does happen exceptionally that a
practising doctor makes a contribution to
science; but it happens much oftener that
he draws disastrous conclusions from his
clinical experience because he has no
conception of scientific method, and
believes, like any rustic, that the handling
of evidence and statistics needs no
expertness.
John Rowan Wilson on
surgeons
• Old Fred had always tended to discount
the surgeons. He visualised them as a
pack of blood and thunder, cut and thrust,
bombastic dunderheads, too lacking in
intelligence to constitute a danger to his
own system. In his eyes they were the
cavalry officers of medicine, dashing and
romantic, useful to impress the simple
hearted, but totally unimportant when
there was any serious business to be done.
He had outwitted the surgeons all his life
and regarded them with scant respect.
The state of
evidence in
surgery
Useless operations
• Ligation of the internal
mammary artery for angina
• Extracranial/intracranial artery
connection
• Radical mastectomy
• Sympathectomy for peripheral
vascular disease
• Spinal fusion
State of the evidence
• “Unless assessments of
surgical procedures are seen to
be unbiased, properly
randomised, and with objective
assessment of outcomes they
will continue to lack credibility.”
• A G Johnson and J Michael Dixon
Removing bias in surgical trials
BMJ, Mar 1997; 314: 916.
Large, randomised trials
are needed
• Benefits are often small--so large trials
are needed so as not to miss small but
still important effects. Large trials are
also better at detecting harms, which
are often rare.
• Randomisation is needed to avoid the
bias that is all pervasive
• The higher the quality of the trial the
smaller the beneficial effect
The safety and efficacy of
new interventional
procedures:
published evidence and
clinical guidance
Tom Dent, Sally Wortley,
Bruce Campbell
NICE review of
evidence: methods
• 245 procedures
• 84 are in the Safety and Efficacy Register
of New Interventional Procedures
(SERNIP)
• Authors randomly selected 25
procedures considered “safe and
efficacious enough for routine use” and
25 of uncertain safety and efficacy
• Two authors independently reviewed the
evidence
NICE review of
evidence: results
• 262 studies (an average of 5.2
per procedure)
• 178 (68%) were case series
• 43 (16%) non-randomised
comparisons
• 33 (13%) randomised trials
• 8 (3%) systematic reviews
NICE review of
evidence: results
• A fifth of studies supporting “safe
and efficacious” procedures by
randomised trials--in contrast to 3%
of the studies supporting uncertain
procedures
• Median number of patients in the
studies supporting each procedure
583 (range 3 to 2873)
• Follow up poorly reported
NICE review of
evidence: conclusions
• “Many interventional procedures
enter clinical practice with limited
published evaluation of their safety
and efficacy”
• “The nature of the available studies
means that the validity and
durability of benefit often cannot be
adequately assessed and important
risks cannot be excluded”
State of the evidence
• Few systematic reviews
• Few randomised trials
• Those that exist are often of poor
quality--which is true of all trials
• Very few placebo controlled trials
• Too many case series, which are
ultimately uninterpretable
A snapshot of the evidence
• RCTs declined from 14% of research articles in the
British Journal of Surgery in 1985 to 5% in 1992
• Treatments in general surgery are half as likely to be
based on RCT evidence as treatments in internal
medicine
• 75% of “surgical trials” are actually of medical
treatments in surgical patients
• Only a third of surgical trials have adequate blinding
• In a study of 10 international journals from 1988 to
1994, Hall et al found that, of the few randomised
controlled trials that were published, less than half
included objective methods for assessing outcome
Article in two issues of the
British Journal of Surgery
• Case series
• Animal studies
• Non-systematic reviews
• RCTs
• Audit
• Four other designs
29(51%)
7 (12%)
6 (11%)
4 (7%)
3 (5%)
8 (14%)
Analysis of general practice
studies from the BMJ
•
•
•
•
•
Questionnaire survey
RCT
Database study
Case control study
Five other designs
6
5
2
2
7
(27%)
(23%)
(9%)
(9%)
(42%)
The evidence patients and
doctors need on a new treatment
• A large, well conducted RCT
comparing the new treatment with
established evidence based
treatments
• To be “evidence based” a treatment
will at some time have had to be
tested against placebo
• An RCT against placebo only if no
evidence based treatment exists
A placebo controlled trial of
athroscopy for osteoarthritis
of the knee
• 185 patients randomise to arthroscopic
débridement, arthroscopic lavage, or placebo
surgery
• Patients in the placebo group received skin
incisions and underwent a simulated débridement
without insertion of the arthroscope
• Blind assessment of outcome
• Followed up for two years
• Several outcome measures
• 165 patients completed the trial
•
Moseley JB, O'Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall
DH, et al. A controlled trial of arthroscopic surgery for osteoarthritis of
the knee. N Engl J Med 2002; 347: 81-88[
Mean Values (and 95 Percent Confidence Intervals) on the Knee-Specific Pain Scale
Moseley, J. B. et al. N Engl J Med 2002;347:81-88
Causes of lack of
evidence and solutions
• Much of what I have to say comes from the
following article
• Randomised trials in surgery: problems
and possible solutions.
– Peter McCulloch, Irving Taylor, Mitsuru
Sasako, Bryony Lovett, and Damian
Griffin
BMJ 2002 324: 1448-1451.
Causes of lack of
evidence and solutions
• History: Many operations
developed before RCTs were
expected
• Solution: Comprehensively
review surgical evidence. Be
clear what we know and don’t
know.
Causes of lack of
evidence and solutions
• Commercial competition and personal
prestige: many financial drivers in
surgery
• Hard to test objectively “your operation”
• “Surgeons' eagerness to learn the
operation [laparascopic
cholecystectomy] seemed related more
to commercial concerns than to concern
for patients.”
Causes of lack of
evidence and solutions
• Solution: Move towards constant
collection of data on outcomes of
surgery and use statistical
systems of continuous quality
control to accumulate evidence of
effectiveness and harms and
decide when RCTs are necessary
Causes of lack of
evidence and solutions
• Surgeon’s equipoise: Surgeons are
(attractively) decisive and used to
acting in uncertain circumstances.
They may thus be less comfortable
with accepting the uncertainty and
recognising that they have the
“equipoise” necessary for a trial.
• Solution: include parallel, non-
randomised, preference arms alongside
RCTs.
Causes of lack of
evidence and solutions
• Lack of funding, infrastructure,
and experience of data
collection
• Solution: change to a culture of
cooperation rather than
competition. Form large groups
to perform specific trials
Causes of lack of
evidence and solutions
• Lack of education in clinical
epidemiology. Surgeons' knowledge
of clinical epidemiology remains
poor
• Solution: Include training in routine
surgical training. (Clinical
epidemiology is the basic science of
clinical research.) More advanced
training for some.
Causes of lack of
evidence and solutions
• The learning curve: Should RCTs be
done on new treatments or once
surgeons have learnt to do them?
Obvious dangers in comparing a new
with a familiar operation.
Solution: statistical techniques can
help. Determine whether the top of
the curve has been reached before
analysing data.
Causes of lack of
evidence and solutions
• The quality of the surgery: It
must be similar for all
interventions being tested
• Solution: quality control
must be part of the trial.
Video evidence might be
collected.
Causes of lack of
evidence and solutions
• Development versus research:
much innovation comes from
small sequential changes not
large leaps. When to do an RCT?
• Solution: Use statistical quality
improvement techniques. Ask
independent third parties to
decide when an RCT is necessary
Causes of lack of
evidence and solutions
• Patient equipoise: This equipoise is
really more important than the
surgeon’s equipoise--and allocation
of treatment by chance is especially
hard when it’s a medical versus a
surgical treatment
• Solution: Not easy. Decision analysis
techniques may help.
Causes of lack of
evidence and solutions
• Blinding: clearly very difficult in
surgical trials
• Solution: Blinded observers
should be used routinely for
assessing outcome
High quality databases can
supplement or even provide an
alternative to RCTs
• Must include individual data on all
consecutive cases
• Must use standard definitions of conditions
and outcomes
• Must ensure data are complete and
accurate
• Must include data on all known patient
characteristics that affect outcome
• Users must know how to risk adjust
NICE (National Institute of
Clinical Excellence) on
interventions
• All new interventions must be notified--by
government decree; always doubt on what is
a new procedure
• IPAC (Interventional Procedures Advisory
Committee) prepares an overview (not a
systematic review) of “the literature”
• IPAC also consults specialist advisers-people “held in high regard by their
colleagues; they will not necessarily be
enthusiasts, with the bias which can
accompany such enthusiasm”
NICE on interventions
• IPAC rules
– “Safe and efficacious”
– Uncertainty
– Unsafe
• Uncertain procedures can be undertaken
“judiciously” but patients must be informed
of the uncertainty; data must be gathered
• All subject to consultation
• Guidance is published
• Existing procedures have been reviewed
Comments
• This is progress
• Or is it bureaucracy?
• It looks like a compromise. Is this a
“practical, sensible way forward” or
a “cop out”? Such a system wouldn’t
be acceptable for drugs
• Can surgery ever be more evidence
based?
Conclusions
• Surgery has been far from evidence
based, which has led to some
horrible excesses
• The state of evidence in surgery is
poor
• It’s much harder to do RCTs in
surgery than in medicine
• But the problems can be overcome-and should be
Conclusions
• High quality databases should be
very helpful
• Various countries--including
Australia and Britain--are developing
ways of regulating new interventions
• These are very much at the
beginning and have a long way to
develop
Final thoughts
A good surgeon
• “Good surgeons know
how to operate, better
surgeons when to
operate, and the best
when not to operate.”
Is there anybody less
evidence based than
surgeons?
Yes
Politicians
Managers
Editors