Transcript Slide 1

The role of
qualitative evidence in
complex intervention
reviews
Professor Jane Noyes
Director, Centre for Health-Related Research
Bangor University
Lead Convenor Cochrane Qualitative
Research Methods Group
Editor Journal of Advanced Nursing
Definition of qualitative
research
Qualitative study that uses a
qualitative method of data
collection and analysis:
Eg: Ethnography,
Phenomenology
Grounded theory
Generic qualitative research:
(focus groups, interviews,
observation etc)
BUT - Many process evaluations
also include surveys/questionnaires
& ‘qual’ evidence.
Why is qualitative evidence important?
Answer: Cochrane effectiveness reviews
don’t tell the whole story
Reinterpretation of Cochrane Review:
School feeding programmes
Greenhalgh et al 2007 BMJ
Realist Synthesis principles – Children’s
feeding programmes -What worked for
whom and in what contexts?
Developed a theory as to why the
poorest children did not benefit from such
programmes
Concluded that theoretical adequacy
was equally as important as trial quality
Cochrane Intervention Review: Directly
Observed Therapy for TB
Chapter 20: Qualitative research can
contribute to Cochrane Intervention reviews
in four ways:
o Informing reviews by using evidence from
qualitative
research to help define and refine the
question, and to
ensure the review includes appropriate
studies and
addresses important outcomes;
o Enhancing reviews by synthesising
evidence from qualitative research identified
whilst looking for evidence of effectiveness;
o Extending reviews by undertaking a
search to specifically seek out evidence from
qualitative studies to address questions
directly related to the effectiveness review;
and
o Supplementing reviews by synthesising
qualitative evidence within a stand-alone, but
complementary, qualitative review to address
questions on aspects other than
effectiveness.
Some issues to
consider
1. Protocol - How and when to select a
synthesis approach?
2. Use of conceptual/theoretical
frameworks to inform synthesis
3. Organisation and management of
data for synthesis and integration
4. What challenges are people reporting
with specific approaches?
5. What methodological work still needs
to be done?
Protocol - How and
when to select a
synthesis approach?
• Upfront in protocol?
• Once pool of evidence is
known?
Decision to conduct a
qualitative evidence synthesis
Purpose of the additional
qualitative synthesis
To aggregate/summarise/ integrate
qualitative data to address specific
questions in relation to a Cochrane
intervention review
To interpret synthesised
qualitative evidence and develop
explanatory theory or models
Primarily to integrate and
interpret qualitative and
quantitative evidence within a
single approach or integrated
model
Can be used to develop
explanatory theory
· Thematic analysis without
theory generation
· Meta-aggregation
· Meta-summary
Product:
· Aggregated findings from
source papers
· Meta-ethnography
· Thematic analysis with
theory generation
· Grounded theory
·
·
·
·
·
Realist review
EPPI approach
Narrative synthesis
Bayesian synthesis
Critical interpretive synthesis
Product:
· Explanatory theory, analytical or
conceptual framework or interpretative
framework/mechanism
The Cochrane approach to
organising evidence
Cochran
Qualitati
Body
e
ve
of
Intervent
Evidenc
ion Evidence
e
Review
Synthes
is
The Cochrane approach to
integration
Cochrane
Intervention
Review
Qualitative
Evidence
Synthesis
“I think you should be more explicit here in Step Two.”
REVMAN
Narrative
Synthesis
OR
Meta Analysis
of Trials
Published
Cochrane
Intervention
Review
New
Qualitative
Evidence
Synthesis
Research Question
Cochrane
Intervention
Review
Qualitative
Evidence
Synthesis
Synthesis of Qualitative
and Quantitative Studies
Challenges of integrating the
qual/quan review products –
an example
 Topical agents or dressings for pain in
venous leg ulcers (Cochrane Intervention
Review)
Briggs M, Nelson EA
Persistent pain - No studies
Debridement pain – Pain reduction
Insufficient evidence re healing
High % ulcers not healed at end point
Topical agents or dressings
for pain in venous leg ulcers
(Cochrane)
Treatment –related pain
Qualitative synthesis
Five common themes related to the
experience of living with leg
ulceration were identified:
Physical effects of leg ulceration
(PAIN);
Describing the leg ulcer journey
Patient–professional relationships
Cost of a leg ulcer
Psychological impact.
But also new insights: The route to
healing versus chronic pain
management
Challenges with quality
appraisal and selection of
studies

Can GRADE be adapted for
qualitative evidence synthesis?

When and how is it appropriate to
sample studies?
GRADE: Quality of
evidence across studies
Outcome # 2
Outcome # 3
Quality: High
Quality: Moderate
Quality: Low
23
Outcome # 1
IB
II
V
III
IA
IV
Priority Issues and Challenges to
Consider
•
•
•
•
•
•
When should qualitative research should be included
alongside review of the effects of a complex
interventions
What types of qualitative research should be included, i.e.
whether to include only process evaluations conducted
alongside studies included in the effects review or
whether to include any qualitative study relevant to the
intervention
How to structure qualitative reviews so as to facilitate the
integration of their findings, specifically with regard to
study selection and sampling and data extraction
How to structure effects reviews so as to facilitate the
integration of their findings, e.g. with regard to data
extraction, description of the intervention
Methods for integrating and using qualitative data to
understand complexity
How to present the results of a review that has integrated
effects and qualitative data, for instance in a Summary of
Findings table