Transcript Document
Systematic Reviews: principles and processes
MED 264
Mary Linn Bergstrom
Nancy Stimson
“A systematic review is a review of a clearly formulated
question that uses systematic and explicit methods to
identify, select, and critically appraise relevant research, and
to collect and analyze data from the studies that are
included in the review.
Statistical methods (meta-analysis) may or may not be used
to analyze and summarize the results of the included
studies. Meta-analysis refers to the use of statistical
techniques in a systematic review to integrate the results of
included studies.”
PLOS Medicine v6 issue 7 July 1009 doi:10.1371/journal.pmed.1000097
Types of reviews
• narrative or integrative
• systematic
• Describes the methodology in detail; includes search strategy, study
selection criteria, assessment of study quality, and data synthesis
• Balancing sensitivity and specificity
• meta analysis
• analysis of combined data from quantitative studies with similar
methodologies
Sources
Cochrane Collaboration
www.cochrane.org
EPPI-Centre
http://eppi.ioe.ac.uk
Campbell Collaboration
www.campbellcollaboration.org
Cardiff University Library
Cochrane Archive
University Hospital Llandough
Standards
IOM
Institute of Medicine of the National Academies –
Standards for Systematic Reviews
http://www.iom.edu/Reports/2011/Finding-What-Works-in-Health-Care-Standards-forSystematic-Reviews/Standards.aspx
PRISMA
Preferred Reporting Items for Systematic reviews and Meta-Analyses
…an evidence-based minimum set of items for reporting in systematic
reviews and meta-analyses
http://www.prisma-statement.org/
Evidence Based Pyramid
http://healthinformatics.wikispaces.com/systematic_review
Formulate Question
• PICO: population, intervention, comparison, outcome
• Check for recent systematic reviews on topic
Inclusion & Exclusion Criteria
Criteria for considering studies for this review
Types of studies
All randomized controlled trials with parallel or cross-over design, blinded or open-label with
a duration of 24 weeks or longer. Reports of which no full publication exists were considered for
inclusion in this review only, if the information available would allow for a publication in accordance
with all criteria of the CONSORT statement.
Types of participants
People with type 2 diabetes mellitus.
Types of interventions
Comparison of long-acting insulin analogues (insulin glargine or insulin detemir) to NPH
insulin. In case of a combination therapy (long-acting insulin analogue combined with another
antihyperglycaemic drug) the additional antihyperglycaemic agent had to be part of each treatment
arm. Only studies reporting on insulin scheme with subcutaneous application were considered for
inclusion in this review.
Types of outcome measures
….
Long-acting insulin analogues versus NPH insulin (human isophane insulin) for type 2 diabetes mellitus
Cochrane Database of Systematic Reviews 2007
DOI: 10.1002/14651858.CD005613.pub3
Select Databases
Reduce bias and improve sensitivity by searching multiple
databases
PubMed, Cochrane Library, Scopus (for Embase content)
Consider other sources:
Grey literature
Clinical trial registers
Cited reference searching
Article bibliographies
Hand searching
Experts in the field
Develop Search Terms
Brainstorm
Review controlled / structured vocabulary in selected
databases
Harvest keywords and structured vocabulary indexing terms
from good database records on topic
Harvest keywords and structured vocabulary indexing terms
from good articles on topic
Use a harvest form (spreadsheet, Word doc, etc.)
PubMed Harvesting Process
For each concept, perform a separate “quick and dirty” search in PubMed
Enter a single concept( MeSH, textword, phrase) in the PubMed search box and run the search
Examine records:
Look for MeSH that reflect your major concept
Look in title and abstract for author-generated terms (i.e., text words, phrases) that reflect the concept.
Add any new MeSH and text words/phrases into harvesting form
Indicate MeSH with the tag [mh]
Indicate text word with the tag [tw]
title word or abstract word[tiab] [ti] [ab]
Use other tags as appropriate, e.g., [pa]
PubMed search field descriptions & tags:
http://www.ncbi.nlm.nih.gov/books/NBK3827/#pubmedhelp.Search_Field_Descrip
Create Search Strategy
Strategies will differ in different databases
Document search details & results for each database
Common errors affect results
conceptualization of research question
spelling errors
translation of search strategy to different databases
missed subject headings
missed natural language search terms
spelling variants and truncation
irrelevant subject headings
irrelevant natural language terms
search limits
Run Search
Create a selective test database
Run search
Store search results
Compare search results with test database citations
Revise, re-run
Select and Assess Studies
Select studies for inclusion into SR
Apply your established inclusion/exclusion criteria
Usually conducted in two passes
Review title/abstract
Review full-text
Use two independent reviewers with a third person available as ‘tie breaker’ for
conflicts
Keep a log of excluded studies with reasons for exclusion
Assess study quality
Each study meeting the inclusion criteria is assessed for quality
More studies may be excluded in this process
Document decisions
Analyze. Interpret, Report
Extract data from the included articles for qualitative or quantitative
(meta-analysis) analysis
Analyze, synthesize results
Interpret results
Comment on
Strength of the evidence
Applicability of the results
Benefits/costs/tradeoffs
Limitations
Implication for future research
Report
“Publish the final report in a manner that ensures free public access”
IOM Standards for Systematic Reviews STANDARD 5.3
Meta-analysis
PRISMA
Why document?
Ensure scientific integrity, replicability
Documentation is standard practice for any research project
Meet publication standards
Investigator must have documentation details for final paper
Utilize PRISMA checklist to structure report
Keep your sanity
Searching is a complex iterative process completed over an
extended period of time
Track internal decision-making affecting search construction
What to document?
The entire process:
•
•
•
•
•
•
•
•
•
Define question
Formulate search terms and strategies
Select databases & sources
Run searches
Store results in a test database
Revise, re-run
Collect & compare results
Manage data selection
Evaluate evidence, conduct qualitative
and/or quantitative analysis
• Prepare and publish final report
See PRISMA 2009 Checklist
http://www.prisma-statement.org/
How to document?
MED264 course website
https://piazza.com/ucsd/fall2014/med264/home
Confirm documentation practices with entire
research team
Citation format, etc.
Back up, back up, back up
Figure 1. Flow of information through the different phases of a
systematic review
Moher D, Liberati A, Tetzlaff J, Altman DG, et al. (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The
PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed.1000097
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.1000097
Mary Linn Bergstrom
[email protected]
Nancy Stimson
[email protected]
http://ucsd.libguides.com/med264_systematic_reviews