Transcript informing policy, practice and research - Sir Iain

Systematic reviews –
informing policy, practice and
research
Iain Chalmers
Coordinator, James Initiative
www.jameslindlibrary.org
SysNet Annual Lecture, Cardiff University
23 April 2012
Questions relevant
to clinicians &
patients?
Low priority questions
addressed
Important outcomes
not assessed
Clinicians and
patients not involved
in setting research
agendas
Appropriate design
and methods?
Accessible
full publication?
Unbiased and
usable report?
Over 50% studies
designed without
reference to
systematic reviews of
existing evidence
Over 50% of studies
never published in full
Over 30% of trial
interventions not
sufficiently described
Biased underreporting of studies
with disappointing
results
Over 50% of planned
study outcomes not
reported
Over 50% of studies
fail to take adequate
steps to reduce
biases, e.g.
unconcealed
treatment allocation
50
%
Most new research
not interpreted in the
context of systematic
assessment of other
relevant evidence
50
%
85% Research waste = over $85 Billion / year
50
%
Questions relevant
to clinicians &
patients?
Low priority questions
addressed
Important outcomes
not assessed
Clinicians and
patients not involved
in setting research
agendas
Appropriate design
and methods?
Accessible
full publication?
Unbiased and
usable report?
Over 50% studies
designed without
reference to
systematic reviews of
existing evidence
Over 50% of studies
never published in full
Over 30% of trial
interventions not
sufficiently described
Biased underreporting of studies
with disappointing
results
Over 50% of planned
study outcomes not
reported
Over 50% of studies
fail to take adequate
steps to reduce
biases, e.g.
unconcealed
treatment allocation
50
%
Most new research
not interpreted in the
context of systematic
assessment of other
relevant evidence
50
%
85% Research waste = over $85 Billion / year
50
%
Does anyone here think that
researchers should NOT review
existing evidence systematically before
developing treatment guidelines and
planning new research?
Patients have suffered and died unnecessarily,
and resources for health care and health
research have been wasted, because
existing research evidence has not been
reviewed systematically.
JAMA 1992;268:240-248.
The human costs of failing to
cumulate evidence from research
scientifically
“Advice on some life-saving therapies
has been delayed for more than a
decade, while other treatments have
been recommended long after
controlled research has shown them
to be harmful.”
Antman et al. JAMA 1992;268:240-8.
Oxford Textbook of Medicine,
2nd edn, 1987
“The clinical benefits of thrombolysis
… remain to be established.”
Sometimes very important
advances in knowledge come from
systematic reviews of existing
evidence
"It is essential that existing sources of
evidence, especially systematic reviews, are
considered carefully prior to undertaking
research. Research which duplicates other
work unnecessarily, or which is not of
sufficient quality to contribute something
useful to existing knowledge, is in itself
unethical."
Department of Health. Research Governance
Framework for Health and Social Care, 2001,
para 2.3.1
TGN1412
TGN 1412
Cumulative estimate from trials of the effect of
aprotinin on perioperative blood transfusion, 1987-2002.
Aprotinin better
Aprotinin worse
New trials of aprotinin ignored
previous trials
But was it the right question?
Is tranexamic acid better than aprotinin?
A comparison of aprotinin and lysine analogues in high-risk cardiac surgery.
Ferguson et al. N Engl J Med. 2008 May 29;358(22):2319-31.
*
*actually, the non-use
Only 11 of 24 responding authors of trial reports
that had been added to existing systematic
reviews were even aware of the relevant reviews
when they designed their new studies.
STUDIES IN ANIMALS
20 animal studies: “The results of this review did not show
convincing evidence to substantiate the decision to perform
trials with nimodipine in large numbers of patients.
Stroke 2001;32:2433-8.
STUDIES IN HUMANS
Horn J, Limburg M.
Calcium antagonists for acute ischemic stroke.
The Cochrane Database of Systematic Reviews, 2000
“46 trials were identified of which 28 were included (7521 patients).
No effect of calcium antagonists on poor outcome at the end of followup (OR 1.07, 95% CI 0.97/1.18), or on death at end of follow-up (OR
1.10, 95% CI 0.98/1.24) was found.”
2005
“…there is a relative scarcity of systematic reviews…”
“…it would therefore be desirable to undertake further
systematic reviews and meta-analyses to evaluate
more fully the predictability and transferability of
animal models.”
Can this policy be defended on scientific, ethical,
or economic grounds?
No subsequent contact made.
Richard Hughes, personal communication,
30 Nov 2011
To reiterate
Patients have suffered and died unnecessarily,
and resources for health care and health
research have been wasted, because existing
research evidence has not been reviewed
systematically.
Austin Bradford
Hill, 1965
Four questions to which
readers want answers
when reading reports of
research.
1. Why did you start?
2. What did you do?
3. What answer
did you get?
4. And what does it
mean anyway?
Lord Rayleigh, 1842-1919
“One of the very few members of the higher nobility
who won fame as an outstanding scientist.”
“If, as is sometimes supposed, science
consisted in nothing but the laborious
accumulation of facts, it would soon come to
a standstill, crushed, as it were, under its
own weight…The work which deserves, but I
am afraid does not always receive, the most
credit is that in which discovery and
explanation go hand in hand, in which not
only are new facts presented, but their
relation to old ones is pointed out.”
Lord Rayleigh, 1884
Things that should be done:
Address uncertainties by reviewing existing
evidence systematically.
Ensure that new research begins and ends by
referring to systematic reviews of other
relevant evidence.
How can we expect patients and the
public to trust that we have their
interests at heart if we and our
professional institutions continue to
acquiesce in our failure to make
systematic use of the results of research
that they have funded?
Promote research on the effects of treatments…
Promote research on the effects of treatments…
…but only if it meets scientific and ethical principles
Sir Paul Nurse, President of the Royal
Society, urged research funders to
trust scientists to decide where
public funding should be directed.
In speaking about initiatives that arise from reviews to
decide priorities Sir Paul warned that “such initiatives
may attract less creative and effective scientists who
simply follow where resources are being made available.”
The case for greater interaction between scientists and
business was also made as being crucial to rebuilding an
innovation-based economy.
De Havilland Alert, 1 Dec 2011.
Finally, a challenge for you
We need to be more efficient in
preparing and updating
systematic reviews.
The decline in costs of computer
memory and gene sequencing
• Computer power:
costs halve every 18
Months
• Gene sequencing
faster!
• Why not systematic
reviews?
The steps in a Systematic Review
• What is current
time for each step?
• How can we make
each easier/faster?
– Standardize
– Streamline
– Computerize
More efficient searching
Steve
Pritchard
Alison
Weightman
Better abstract screening
• Highlight the search terms that
had been used - yellow in
example
• Highlight methodological terms
(from a supplementary list NOT
used in the search) – pink in
example.
• Allowing easy marking of
definite, possible, and
definitely not abstracts, and
comparison of these markings
across two reviewers.
[Sente 6]
BMC Fam Pract. 2010 Jul 29;11:54. Randomised controlled trial of local corticosteroid injections for
carpal tunnel syndrome in general practice. Peters-Veluthamaningal C, Winters JC, Groenier KH,
Meyboom-de Jong B.
BACKGROUND: Carpal tunnel syndrome is caused by entrapment of the median nerve and results in
pain, tingling and numbness in the wrist and hand. It is a common condition in general practice.
Effectiveness of treatment by intracarpal corticosteroid injection has never been investigated in
general practice. The objective of this study was to determine if corticosteroid injections for carpal
tunnel syndrome provided by general practitioners are effective.
METHODS: In this study 69 participants with a clinical diagnosis of carpal tunnel syndrome were
recruited from 20 general practices. Short-term outcomes were assessed in a randomised, placebocontrolled trial. Long-term results were assessed in a prospective cohort-study of steroid responders.
Participants were randomised to intracarpal injections of 1 ml triamcinolonacetonide 10 mg/ml (TCA)
or 1 ml NaCl (placebo). Non-responders to NaCl were treated with additional TCA injections. Main
outcomes were immediate treatment success, mean score of the Symptom Severity Scale (SSS) and
Functional Status Scale (FSS) of the Boston carpal tunnel questionnaire, subjective improvement and
proportion of participants with recurrences during follow-up. Duration of follow-up was twelve
months.
RESULTS: The TCA-group (36 participants) had better outcomes than the NaCl-group (33 participants)
during short-term assessment for outcome measures treatment response, mean improvement of
SSS-score (the mean difference in change score was 0.637 {95% CI: 0.320, 0.960; p < 0.001}) and FSSscore (the mean difference in change score was 0.588 {95% CI: 0.232, 0.944; p = 0.002}) and
perceived improvement (p = 0.01). The number to treat to achieve satisfactory partial treatment
response or complete resolution of symptoms and signs was 3 (95% CI:1.83, 9.72).49% of TCAresponders (17/35) had recurrences during follow-up. In the group of TCA-responders without
recurrences (51%, 18/35) outcomes for SSS-score and FSS-score deteriorated during the follow-up
period of 12 months (resp. p = 0.008 and p = 0.012).
CONCLUSIONS: Corticosteroid injections for CTS provided by general practitioners are effective
regarding short-term outcomes when compared to placebo injections. The short-term beneficial
treatment effects of steroid injections deteriorated during the follow-up period of twelve months
and half of the cohort of steroid-responders had recurrences. PMID: 20670438
Your ideas please!
For any review step, do you have tips
to:
•Standardize?
•Streamline?
•Automate?
Ideas, please, to:
[email protected] or
[email protected]