Transcript Systematic reviews - Welcome | Cochrane Oral Health Group

Cochrane Systematic Reviews
& Cochrane Oral Health Group
Luisa Fernandez
Anne-Marie Glenny
Cochrane Oral Health Group, University Dental Hospital of Manchester
Emma Tavender
Cochrane EPOC Satellite, Melbourne Australia
Aims
To discuss the role of Cochrane systematic
reviews and meta-analyses
Cochrane Oral Health Group – progress,
undertaking a systematic review with the
group & looking to the future
Cochrane Systematic Reviews
& Oral Health
Anne-Marie Glenny
Cochrane Oral Health Group, University Dental Hospital of Manchester
What are systematic reviews?
• The process of systematically locating,
appraising and synthesising evidence from
scientific studies in order to obtain a reliable
overview
• Aim to find all studies addressing the
review’s question using an objective and
transparent process
Why are they important ?
• Reduce large quantities of information into
manageable portions
• Formulate policy and develop guidelines
• Efficient use of resources
• Increased power/precision
• Limit bias and improve accuracy
A. Thrombolytic Therapy
Textbook/Review
Recommendations
Odds Ratio (Log Scale)
Cumulative 0.5
Year RCT
Pts I
1
23
2
85
3
149
I
I
I
I
1
I
I
I
I
I
2
I
1960
1965
21
4
138
7
1783
10
11
15
17
22
2544
2651
3311
3929
5452
23
5767
8
1970
1975
1 10
p<.01
1990
2
2
8
7
8
1
1980
1985
1
27
30
33
43
54
65
67
70
p<.001
6125
6346
6571
21050
22051
47185
47531
48154
p<.00001
1
8
4
1
7
3
5 2
2
1
15 8
6 1
Favours Treatment
Favours Control
12
1
Systematic review
•
Structured process involving several steps:
1.
2.
3.
4.
5.
Well formulated question
Comprehensive data search
Unbiased selection and abstraction process
Validity assessment of papers
Synthesis of data
1. Well formulated question
•
•
•
•
Participants
Interventions (Exposure)
Comparisons
Outcomes
1. Well formulated question
•
•
•
•
Participants
Interventions
Comparisons
Outcomes
Pit and fissure sealants versus placebo for the
prevention of dental caries in permanent teeth in
children and adolescents
Inclusion criteria
•
•
•
•
•
Participants
Interventions
Comparisons
Outcomes
Study design/methodological quality
What type of study design?
How effective is paracetamol at reducing pain?
Does smoking increase the risk of oral cancer?
STRONG
Experimental studies/ clinical trials
Randomised controlled trials
Non-randomised controlled trials
Observational studies
Cohorts
Case-controls
Cross-sectional surveys
Case series
Case reports
WEAK
Expert opinion, consensus
2. Search strategy
• Needs to be as comprehensive as possible
• Consider
– Electronic databases (Cochrane Controlled Trials
Register, Medline, Embase)
– Reference lists
– Handsearching
– English language/non-English language
– Sources of ongoing and/or unpublished studies
Reporting biases
• Statistically significant ‘positive’ results are;
- more likely to be published
- publication bias
- more likely to be published rapidly
- time lag bias
- more likely to be published in English
- language bias
- more likely to be cited by others
- citation bias
3. Unbiased selection and data
abstraction process
• Selection of relevant papers
3. Unbiased selection and data
abstraction process
• Selection of relevant papers
• Data abstraction/extraction
Data extraction is:
•
•
•
•
Time consuming
Often subjective
Prone to error
Often difficult
3. Unbiased selection and
abstraction process
• Predefined data abstraction form
• Independently and in duplicate
4. Validity assessment
• Can be used;
– As a threshold for inclusion of studies
– As a possible explanation for differences in
results between trials
– In sensitivity analyses
– As weights in statistical analysis of the results
Quality assessment tools
• Composite scales - assign numerical value
to individual items to provide overall
estimate of quality – problematic
• Component approach - assesses relevant
methodological aspects individually (e.g
randomisation, blinding, drop-outs) preferred
4. Validity assessment
• Process should be conducted independently
by at least two reviewers
• Results of the quality assessment should be
reflected in the analysis
5. Study synthesis
• Appropriate pooling
– qualitative (narrative)
– quantitative (meta-analysis)
– inappropriate when data are sparse or
heterogeneity exists
– Clear presentation of individual studies
included in the review
Meta-analysis
• The process of using statistical methods to
combine the results of different studies.
• The aim is to integrate the findings, pool the
data, and identify the overall trend of results
(Dictionary of Epidemiology, 1995)
What is a meta-analysis?
Optional part of a systematic review
Systematic reviews
Meta-analyses
When can/should you do
a meta-analysis?
• When more than one study has estimated an effect
• When there are no differences in the study
characteristics that are likely to substantially affect
outcome
• When the outcome has been measured in similar
ways
• When the data are available (beware when only
some data are available)
Summary statistic for each study
• Calculate a single summary statistic to
represent the effect found in each study
- usually displayed with 95% confidence
intervals (CI)
Weighting studies
• More weight to the studies which give us
more information
– More participants
– More events
– Lower variance
• Weight is proportional to inverse variance
Displaying results graphically
• Forest plots
– Commonly used
there’s a label to tell
you what the comparison
is and what the outcome
of interest is
Alderson 2002
At the bottom there’s
a horizontal line. This
is the scale measuring
the treatment effect.
Here the outcome is death
and towards the left the
scale is less than one,
meaning the treatment
has made death less
likely.
Take care to read what
the labels say – things to
the left do not always mean
the treatment is better than
the control.
Alderson 2002
The vertical line in the
middle is where the
treatment and control
have the same effect –
there is no difference
between the two
Alderson 2002
For each study
there is an id
The data for
each trial
are here, divided
into the experimental
and control groups
This is the % weight
given to this
study in the
pooled analysis
Alderson 2002
The data shown in
the graph are also
given numerically
The label above the graph
tells you what statistic
has been used
•Each study is given a blob, placed where the data measure the effect.
•The size of the blob is proportional to the % weight
•The horizontal line is called a confidence interval and is a measure of
how we think the result of this study might vary with the play of chance.
•The wider the horizontal line is, the less confident we are of the
observed effect.
Alderson 2002
The pooled analysis is given a diamond shape
where the widest bit in the middle
is located at the calculated
best guess (point estimate),
and the horizontal width is the
confidence interval
Note on interpretation
If the confidence interval crosses the line
of no effect, this is equivalent to saying that
we have found no statistically significant difference in
the effects of the two interventions
Alderson 2002
Heterogeneity
• Clinical heterogeneity – differences in trial
characteristics
• Statistical heterogeneity - the variability in the
reported effect sizes between studies
– how similar are the results?
– are the differences among the results of the trials
greater than could be expected by chance alone?
Heterogeneity
Odds ratio (95% CI)
1
Odds ratio (95% CI)
1
Heterogeneity
• Chi-squared test of heterogeneity
• P<0.1 demonstrates statistically significant
heterogeneity
– may not be appropriate to pool data
Subgroup analyses
• Where it is suspected in advance that certain
features may alter the effect of an
intervention
Example
– women
– a particular age group
– those with a specific disease subtype
Subgroup analysis
• Often misleading• Is there indirect evidence in support of a
difference?
• Did the hypothesis about the difference
precede rather than follow the analysis?
• Is the subgroup analysis one of a small
number of hypotheses tested?
Sensitivity analysis
• Does result change according to small
variations in the data and methods?
– Choice of treatment effects or method for
pooling
– Inclusion/exclusion of dubious data
– Inclusion/exclusion of trials
A common sensitivity analysis is to repeat the
analysis taking out lower quality trials
Useful websites;
http://www.shef.ac.uk/~scharr/ir/netting/
(comprehensive list of E.B.P. websites and links)
http://www.york.ac.uk/inst/crd
– CRD manual on how to conduct a systematic review
– DARE - database of abstracts of reviews of
effectiveness
http://www.cochrane.org
– reviewer’s handbook
– Cochrane Library (abstracts only)
Cochrane Oral Health Group –
looking to the future
Emma Tavender
Cochrane EPOC Satellite, Melbourne Australia
Aims
• Progress of the group
• Process of undertaking a systematic review
with the group & support available
• Challenges for the future
• Questions & answers
Cochrane Oral Health Group
Include all RCTs of oral health broadly conceived to include
the prevention, treatment and rehabilitation of oral, dental
and craniofacial diseases and disorders
Achievements
•
•
•
•
•
•
624 members from 40 countries
Panel of referees
Set up Specialised Register of Trials (21,000)
Handsearching programme
Annual Evidence Based Dental Practice Course
NIDCR funding for oral cancer reviews
21 Updates
20
-3
06
20 1
06
20 3
05
20 1
05
20 3
04
20 1
04
20 3
03
20 1
03
20 3
02
20 1
02
20 3
01
20 1
01
20 3
00
20 1
00
20 3
99
19 1
99
19 3
98
19 1
98
19
Cochrane Library Issue
54 Reviews
Reviews
50
Updates
60
68 Protocols
70
Protocols
40
30
Number of Publications
Oral Health Group Reviews
80
10
0
14
76
9
18
24
5
Growth of Oral Health Group Trials Register
January 1998 – January 2006
46
9
3
58
01
77
56
48
2
12
Reports of Trials
13
1
20
2
89
0
2
Jan-98
Jan-99
Jan-00
Jan-02
Jan-03
Jan-04
Jan-05
Jan-06
Editorial Process
•
•
•
•
•
•
•
•
•
Register title
Prepare protocol
Editorial and external review of protocol
Protocol entered on Cochrane Library
Identify trials
Complete systematic review
Peer review of systematic review
Systematic review entered on Library
Update the review regularly (every 2 years)
Editorial support
• Register title
– Check for overlap
– Find co-reviewers with similar interests
– Assigned Contact Editor
Editorial support
• Prepare protocol
–
–
–
–
–
Reviewers Handbook
RevMan software/support at editorial base
Protocol training workshop (free of charge)
Online learning materials
Help with developing search strategy (TSC)
Editorial Support
• Identify trials
• Complete systematic review
– Search the OHG Trials Register
– Analysis workshop (free of charge)
– Statistical & methodological support
from editorial base
– Help with translations
Editorial support
• Update the review regularly
– Help with identifying new trials
– Procedures for changing lead author
Challenges for the future
• Prioritisation of reviews
• Diagnostic reviews
• Reviews of no trials – including other levels
of evidence?
• Umbrella reviews
• Funding
• Dissemination
Challenges for the future
• Prioritisation of reviews
– Increasing number of reviews/overlap
– Usefulness – answering the right questions for
clinicians, consumers, funders, policy makers?
– NIDCR funding of oral cancer reviews
– OHG Symposium – identify questions/topics
– Future – ways of identifying priorities for
consumers etc
Challenges for the future
• Diagnostic reviews
– Important area within dentistry
– Cochrane Diagnostic Reviews of Test Accuracy
Initiative
– 4 phases: development of materials, piloting,
implementation & publication
– Piloting phase – 10 test review teams
– Training & implementation 2007
Challenges for the future
• Reviews of no trials – including other levels
of evidence?
– Scope of OHG includes RCTs and Quasi RCTs
– Reviews eg. Endocarditis included other levels
of evidence
– OHG currently developing guidance for
reviewers on how to deal with reviews of no
trials and when to include other levels
Challenges for the future
• Umbrella reviews
– Combine the findings of a number of related
reviews so information is in one document
– Incorporate summary of findings tables
– RevMan 5 released in 2007 will include this
feature
– Cochrane Umbrella Reviews Working Group
Challenges for the future
• Funding
– 5 years funding from DoH, UK
– Covers salary of staff, administration at
editorial base
– NIDCR grant to undertake series of oral cancer
reviews
– NICE, Italian Govn., vCIOH
– Alternative methods?
Challenges for the future
• Dissemination
–
–
–
–
The Cochrane Library
Journal articles & by products
Courses/conferences
Incorporation in guidelines (HTA, NICE,
SIGN)
– Improve visibiltiy and utilisation of reviews
Any questions?