Transcript Document

Assessing the Evidence:
Search, Evaluate, and Outline
March 26, 2015
Topics
 How to Approach the Literature Search
 Evaluating the Quality of the Identified Literature
 Framing the Story and Next Steps
2
How to Approach the Literature
Search
How to Approach the Literature Search
1. Specify your objectives
2. Formulate the key research questions
3. Build the search strategy
4. Conduct the search
4
Specify your Objectives
Why are you reviewing the literature?
Make a decision: Should we or shouldn’t we use repeat
anti-TNF-alpha therapy?
Comparative effectiveness: How does a new ACE
inhibitor compare with an existing ACE inhibitor?
Assess knowledge base: What is hereditary
angioedema and how is it treated in the US?
Prepare a summary: Drafting an introduction section in
a manuscript or an AMCP dossier
5
Determine Type of Search Based on Objectives
Systematic
•
•
•
•
Make a decision
Literature review
intended for publication
To support the
development of a core
value dossier or
specific dossier
requirements (eg, NICE
submission dossier,
AMNOG dossier)
To conduct a metaanalysis or indirect
treatment comparison
$$$
6
Narrative
•
•
•
•
•
Landscape assessment
of new therapeutic area
/ asset
Answer a specific
research question
Inform internal
planning/research
questions (broad)
To support the
development of an
AMCP dossier
Economic model
conceptualization
$$
Targeted
•
•
•
•
Answer a specific
research question
Inform internal
planning/research
question
Summary of literature
for publications
Identify model inputs
$
Formulate Your Key Research Questions
 My need is basic, so why do I have to formulate the research
questions?
7
Formulate Your Key Research Questions
 My need is basic, so why do I have to formulate the research
questions?
8
Using PICO (PICOS) to Formulate the Questions
Population
Interventions
Comparators
Outcomes
Study design
9
“The well-formed question
makes it relatively
straightforward to elicit and
combine the appropriate
terms…in the query language”.
Using PICO
Centre for Evidence-based Medicine
10
Build the Search Strategy
Purpose
11
Build the Search Strategy
Purpose
As a result…
 You can repeat what you’ve done
- science needs to be replicable
 Someone else can do the same thing and get the same results
- scalable, replicable
 You can maintain quality control
 You can modify and improve.
12
What Does a Literature Search Strategy Look Like?
Elements of a search strategy
 It depends…
Study
question
Databases
/ Sources
Search
strategy
Exclusion
criteria
Data
extraction
13
Inclusion
criteria
Language
Study
selection
Time
horizon
Pick a few
critical elements
to focus on
Conduct the Search
14
Evaluating the Quality of the
Evidence
Approaches to Grading the Evidence
 Multiple approaches exist and vary depending on the study design
 The following represents a sample of commonly used checklists
CHEERS
GRACE
CRD
NICE
Sponsor
ISPOR
NPC/Quintiles
Centre for Reviews and
Dissemination
NICE
Study Type
Evaluated
Health economic
evaluations
Observational cohort
studies
Health economic
evaluations
Health economic
evaluations
Stated Aims
To provide
recommendations, in
the form of a checklist,
to optimize reporting of
health economic
evaluations.
To assess
observational studies
of comparative
effectiveness in terms
of their quality and
usefulness for
decision-making
Evaluate the elements
of the economic
evaluation that can
have an impact on the
validity of the overall
results
To determine whether
an economic evaluation
provides evidence that
is useful to inform the
decision-making of the
GDG
Checklist
Available?
Yes
Yes
Yes
Yes
# of Items
24
11
36
22
16
Approaches to Grading the Evidence
Cont’d
 Assessment of quality of all study
types
-
Cochrane Collaboration Risk of
Bias
 Assessing reporting quality or
reporting of a RCT
-
Jadad 5-point Scale
CONSORT statement
 Assessing quality of
observational studies
-
Newcastle-Ottawa Scale 16-item
checklist
-
STROBE statement 22-tiem
checklist
 Assessing quality or reporting of
systematic literature reviews
- PRISMA Statement 27-item
checklist
- MOOSE 35-item checklist
- AMSTAR 11-item questionnaire
- NICE Methodology checklist
 Assessing the body of evidence
- GRADE
- SIGN
And many many others!
17
The Problem with Choice
18
Practical Evaluation of the Quality of the Evidence
What is the Relevance of Grading to the Research Question?
 Consider what is your purpose in grading the literature:
- Grading a body of evidence for purposes of synthesis
(quantitative)
- Grading a select number of studies for inclusion in a
dossier or other use (qualitative)
 Consider whether you are grading the quality of the evidence or the
reporting of the evidence
“The strength of evidence grade summarizes the reviewers’
confidence in the findings based on either approach to
evidence synthesis.” – AHRQ 2013
19
Practical Evaluation of the Quality of the Evidence
20
Framing the Story
and Next Steps
What to Do with the Body of Evidence?
Turn THIS
22
Into THIS
Frame the Story
Corralling the Literature
 Steps to developing the outline and framing the story
 Systematic literature reviews have their own worksteps and best practices
Create
buckets
• Identify the story or value proposition
behind the writing (theme)
• Group the available evidence for each
major topic or outcome (Level 1 outline)
Step 2
Develop
an outline
• Assess where the majority or quality of
evidence is (relative to the theme)
• Develop an outline around the evidence
(Level 2 outline)
Step 3
Drop in
the
literature
• Organize the literature around the outline
• Drop in the relevant citations and 2-3 key
points to convey from each study
• Remove the non-relevant data/studies
Gap
analysis
• Assess where there are no citations:
missed or a gap in the evidence?
• Determine if this is incorporated into the
writing or go back to your search strategy
Step 1
Step 4
23
Next Steps
 Write and publish!
24
Thank you!
25