Transcript Slide 1

Reporting Guideline for Systematic
Reviews and Meta-Analyses:
the PRISMA Statement
Doug Altman
17.07.2015
STROBE Statement revision
1
QUOROM Statement
• Developed in 1996
– Following CONSORT model
• Published in 1999
– Checklist and flow diagram
• Since 1996 increased evidence base from methodological and
empirical research
– e.g. Cochrane Methodology Register
• 1000 entries in 1999
• 8255 entries in 2006
• Important deficiencies in QUOROM were recognised
2
Update/revision of QUOROM
• A 3-day meeting was held in Ottawa, Canada, in June 2005
– 29 participants: systematic reviewers, methodologists, editors
and a consumer
• Name changed to PRISMA
• Draft checklist - much iteration subsequently
• Revised Statement published in 2009 in 5 journals
– 27-item checklist
– four-phase flow diagram
• Identification, screening, eligibility, inclusion
Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for systematic
reviews and meta-analyses: the PRISMA Statement. 2009
• PLoS Med; Open Med; Ann Intern Med; BMJ; J Clin Epidemiol
3
Conceptual issues affecting the update
• Distinction between articles and studies
• Iterative nature of completing a systematic review
• Need to distinguish between conduct and reporting of
primary studies
• Quality assessment
– Key idea is “risk of bias”
– Both study level and outcome level assessment
• Need to consider risk of reporting bias (between and within
study)
• “Systematic review” or “meta-analysis”?
4
Rationale for checklist items
• Necessary to evaluate the study
• Evidence-based, whenever possible
• Minimum set of essential items
5
slide 6 of 56
Flow diagram
• Original QUOROM Statement:
“Provide a meta-analysis profile summarising trial flow”
• Revised Statement is more precise:
“Give numbers of studies screened, assessed for eligibility,
and included in the review, with reasons for exclusions at each
stage, ideally with a flow diagram”
7
24 August 2010
STROBE Statement revision
8
PRISMA – Item 8, search
• “Present full electronic search strategy for at least one
database, including any limits used, such that it could be
repeated”
Rationale for reporting search
• Allows interested readers to assess the comprehensiveness
and completeness of the search, and to replicate it
– Essential for updating (i.e., keeping systematic reviews up-todate)
slide 9 of 56
PRISMA – Item 12,
Risk of bias in individual studies
•
“Describe methods used for assessing risk of bias of individual studies (including
specification of whether this was done at the study or outcome level), and how
this information is to be used in any data synthesis.”
Rationale for reporting risk of bias
• The likelihood that the treatment effect reported in a systematic review
approximates the truth depends on the validity of the included studies,
as certain methodological characteristics may be associated with effect
sizes
– For example, trials without reported adequate allocation concealment
exaggerate treatment effects on average compared to those with adequate
concealment.
• Therefore, it is important for authors to describe any methods that they
used to gauge the risk of bias in the included studies and how that
information was used.
• Additionally, authors should provide a rationale if no assessment of risk
of bias was undertaken.
slide 10 of 56
Reporting risk of bias
• Authors should report how they assessed risk of bias
• Whether assessment was in a blind manner
• If assessments were completed by more than one person, and if
so, whether they were completed independently.
• Report any calibration exercises among review team members
that were done.
• Report how their assessments of risk of bias are used
subsequently in the data synthesis
slide 11 of 56
PRISMA – Item 17, study selection
• Give numbers of studies screened, assessed for eligibility, and
included in the review, with reasons for exclusions at each
stage, ideally with a flow diagram
slide 12 of 56
Caughey et al. Ann Intern Med 2009
Fuccio et al. Ann Intern Med 2007
www.prisma-statement.org
slide 14 of 56
PRISMA Explanation and Elaboration
(E&E)
• Published in 4 journals –
– BMJ, Ann Intern Med, PLoS Med, J Clin Epidemiol
17.07.2015
STROBE Statement revision
15
Evidence of selective reporting of
outcomes in RCTs
• Comparison of 102 trial protocols with 122 subsequent
publications
• Nearly two-thirds had a change in at least one primary
outcome between the protocol and publication
• Statistically significant outcomes had a higher likelihood of
being reported compared to non-significant ones
[Chan et al JAMA 2004]
• Present data on risk of bias of each study and, if available, any
outcome-level assessment (see Item 12).
• Present results of any assessment of risk of bias across studies
(see Item 15).
16
PRISMA is not specific
• Most of the checklist is not specific to RCTs or to studies of
interventions
• Can be the basis for extension to other study designs
Box 2:
• Finally, the type of study design(s) included in the review should be
reported. Some reviews only include reports of randomized trials
whereas others have broader design criteria and include
randomized trials and certain types of observational studies. Still
other reviews, such as those specifically answering questions
related to harms, may include a wide variety of designs ranging
from cohort studies to case reports. Whatever study designs are
included in the review, these should be reported.
17
PRISMA E&E
• “We developed the PRISMA Statement and this explanatory
document to help authors report a wide array of systematic
reviews to assess the benefits and harms of a health care
intervention. We consider most of the checklist items relevant
when reporting systematic reviews of nonrandomized studies
assessing the benefits and harms of interventions.
• However, we recognize that authors who address questions
relating to etiology, diagnosis, or prognosis, for example, and
who review epidemiological or diagnostic accuracy studies
may need to modify or incorporate additional items for their
systematic reviews.”
24 August 2010
STROBE Statement revision
18
PRISMA E&E
• “We believe that the issues we have highlighted in this paper
are relevant to ensure transparency and understanding of the
processes adopted and the limitations of the information
presented in systematic reviews of different types. We hope
that PRISMA can be the basis for more detailed guidance on
systematic reviews of other types of research, including
diagnostic accuracy and epidemiological studies.”
24 August 2010
STROBE Statement revision
19
24 August 2010
STROBE
revision
Ramakrishnan
et Statement
al, Lancet
Inf Dis 2010
20
24 August 2010
Lim
et al
PLoS revision
One, 2010
STROBE
Statement
21
24 August 2010
STROBE Statement revision
22