Transcript School of Health & Bioscience

Issues and challenges around
integrating qualitative and
quantitative research in
systematic reviews
Angela Harden, Professor of Community and
Family Health
InCQuiRES, 4th September, 2013
INSTITUTE FOR HEALTH AND HUMAN DEVELOPMENT
• A centre for research,
innovation and community
engagement to promote
health and wellbeing.
• Staff from a range of health
and social science
disciplines.
• Includes collaborative
programme of research with
Barts Health NHS Trust
focused on women, children
and young people
Overview of recent SR projects
Title
Funder
Main collaborating
institutions
The effects of schools and schoolenvironment interventions on health
NIHR
LSHTM, UEL, IOE
Can specific approaches to community
NIHR
engagement help to reduce inequalities in
health?
IOE, UEL, LSE
A meta-narrative review of
conceptualisations and meanings of
‘community’ within and across research
traditions
UEL
AHRC
Facilitating access to antenatal care for
EUFP7
women in India, Pakistan and Bangladesh
UEL, QMUL & other
partners
Acknowledgements
• Colleagues at the
EPPI-Centre, Social
Science Research
Unit, Institute of
Education
• Co-convenors of the
Cochrane Qualitative
Research Methods
Group
Outline
• Drivers and challenges of integration
• Review of approaches and methods
• Recent developments
• Future challenges and issues
Drivers for integration
• Greater recognition of the value of qualitative research in evidencebased policy
• ‘Empty’ reviews
• Public perspectives and experiences
• Systematic reviews of complex interventions
• Issues of process and implementation
• Extension of evidence-based health care to other areas of public
policy
• Growing tradition for integration in primary research (mixed
methods research)
• Dedicated research funding for methodological research
• Establishment of dedicated methods groups (e.g. Cochrane
Qualitative Methods Research Group)
Differences in Male Life Expectancy within a small area in London
Travelling east from Westminster, every two tube stops represent over
one year of life expectancy lost –Data revised to 2004-08
Canning Town
Male Life
Expectancy
73.6 (CI 71.9-75.2)
Male Life
Expectancy
78.5 (CI 75.5-81.6)
Westminster
London Bridge
River Thames
Canada
Bermondsey Water
Canary
Wharf
North
Greenwich
Waterloo
Southwark
London Underground
1
Jubilee Line
Electoral wards just a few miles apart geographically have life
expectancy spans varying by years. For instance, there
are eight stops between Westminster and Canning Town
on the Jubilee Line – so as one travels east, every two stops, on
average, mark over a year of shortened lifespan. 1
Source: Analysis by London Health Observatory of ONS and GLA data for 2004-08. Diagram produced by Department of Health
Mega-events and urban regeneration
• Examples of complex
interventions with multiple
anticipated outcomes
• Regeneration - a set of
activities to reverse economic,
social and physical decline
• Housing stock, service
improvements, physical
environment, creation of
socially mixed population
McCartney et al. BMJ 2010; 340:c2369
McCarthy et al. (2010) conclude…….
“There is little evidence that major multi-sport events held
between 1978 and 2008 delivered health or socioeconomic benefits for the population of the host
country…..”
“Future events such as the 2012 Olympic Games and
Paralympic Games, or the 2014 Commonwealth Games,
cannot be expected to automatically provide benefits.
Until decision makers include robust, long term
evaluations as part of their design and implementation of
events, it is unclear how the costs of major multi-sport
events can be justified in terms of benefits to the host
population.”
Unanswered questions
• What happens during
implementation?
• How do the various actors
involved experience megasporting events and the
associated regeneration?
• What are the pathways to
impact?
• Do variations in the above
relate to outcomes?
Challenges of integration
•
•
•
•
•
•
New and evolving field
Scale of task
Paradigm wars
Lack of training and expertise
Few worked examples
Infrastructure in development
Approaches and methods in 2004…
“Quantitising”
“Qualitising”
• Narrative summary
• Thematic analysis
•
•
•
•
•
•
•
•
Grounded theory
Meta-ethnography
Meta-study
Miles and Huberman's data
analysis techniques
Content analysis
Case survey
Qualitative comparative
analysis
Bayesian meta-analysis
2004
Approaches and methods 2013
Bayesian
metaanalysis
Critical
Interpretive
synthesis
Metanarrative
review
Narrative
synthesis
Integration
of qualitative
and
quantitative
research
Realist
synthesis
Mixed
methods
synthesis
Origins of approaches and methods
Method
Developed by/Exemplars
Context and purpose
Narrative synthesis
Popay et al. (2005)
Cochrane remit – to examine issues of
process, implementation and experience
Mixed methods
synthesis
Thomas, Harden et al.
(2004)
Harden and Thomas (2005)
Informing policy to promote children’s
health
Bayesian synthesis
Roberts et al. (2002)
Factors that affect the uptake of childhood
immunisation
Critical interpretive
synthesis
Dixon-Woods et al. (2006)
Access to healthcare for vulnerable groups
Meta-narrative
Greenhalgh et al. (2005)
To review research on diffusion of
innovation to inform healthcare policy
Realist synthesis
Pawson (2006)
To develop and test theories of change
underpinning complex policy interventions
In focus: Mixed methods synthesis
• Aim is to generate and test
theory from diverse body of
literature
• Exhaustive search, review
questions, inclusion criteria
and quality assessment
largely specified a priori
• Each review typically has
three syntheses:
1.
2.
3.
Statistical meta-analysis
Thematic synthesis
Cross-study synthesis
Recent examples
Teenage pregnancy and social
disadvantage (Harden et al., 2009)
Knowledge and information needs of
young people with epilepsy (Lewis
et al., 2010)
Behavioural interventions for weight
management in pregnancy
(Campbell et al., 2011)
The effects of schools and school
environment interventions (Bonell
et al., 2013)
Mixed methods systematic reviews
Working definition
Combining the findings of ‘qualitative’ and
‘quantitative’ studies within a single systematic
review, in order to address the same,
overlapping or complementary review
questions
(Harden and Thomas, 2010)
Three ways in which reviews are mixed….
1. The types of studies included and hence the
type of findings to be synthesised (i.e.
‘qualitative/ textual and
quantitative/numerical)
2. The types of synthesis method used (e.g.
statistical meta-analysis and qualitative
synthesis)
3. The mode of analysis: theory testing AND
theory building
Review questions
•What is known about the barriers to, and
facilitators of, healthy eating amongst children?
•Do interventions promote healthy eating
amongst children?
•What are children’s perspectives on healthy
eating?
•What are the implications of the above for
intervention development?
REVIEW PROCESS
Searching, screening and mapping
Focus narrowed to
‘fruit &veg’
Synthesis 1: Trials (n=33)
1. Quality assessment
2. Data extraction
3. Statistical meta-analysis
Synthesis 2: Qualitative studies
(n=8)
1. Quality assessment
2. Data extraction
3. Thematic synthesis
Synthesis 3: Trials and
qualitative studies
Synthesis 3: Across studies
Children’s views
Recommendation for
interventions
Trials
Good
quality
Other
Do not promote fruit and
vegetables in the same way
0
0
Brand fruit and vegetables as
an ‘exciting’ or child-relevant
product, as well as a ‘tasty’ one
5
5
Reduce health emphasis in
messages to promote fruit and
vegetables particularly those
which concern future health
5
6
Synthesis 3: Across studies
Increase (standardised portions per
day) in vegetable intake across trials
0.6
0.5
0.4
0.3
0.2
0.1
Study
Pe
rry
)
(b
no
ws
ki
Ba
ra
Au
ld
Au
ld
ey
no
ld
s
R
on
An
de
rs
H
en
r
y
i
Li
qu
or
dl
e
0
W
ar
Portions
Little or no
emphasis
on health
messages
Recent development
• Candy et al. (2013) Using qualitative evidence
on patients views to help understand variation
in effectiveness of complex interventions
• Used qualitative comparative analysis to
identify pathways to effectiveness
• Worked example - improving adherence to
drug therapy
Recent developments
In focus: Bayesian synthesis
• Aim is to test theory
• Findings from qualitative and
quantitative research are ‘fused’
• Only three worked examples to
date
• Variation in weight given to the
qualitative evidence
• Synthesis product is a set of
weighted factors associated
with/predicting the phenomenon
under review
Recent examples
Factors affecting the uptake of
childhood immunisation
(Roberts et al., 2002)
Factors that influence
adherence to HIV
medication regimes
(Vollis et al., 2009)
(Crandell et al., 2011)
Factors that influence adherence to HIV
medication (Crandell et al., 2011)
In focus: Critical Interpretive Synthesis
• Aim is to generate theory
from large and diverse body
of literature
• Literature itself is an object
of scrutiny (critical)
• Comprehensive search to
identify sampling frame
• Purposive and theoretical
sampling
• Analysis leads to generation
of synthetic constructs and
a synthesising argument
Examples
Access to health care for
vulnerable groups (DixonWoods et al. 2006)
Use of morphine to treat cancer
related pain (Flemming, 2009)
Nurses response to suicide and
suicidal patients (Talseth and
Gilje, 2011)
Access to health care for vulnerable groups (DixonWoods et al. 2006)
• Based on 119 papers
• Interpretive qualitative analysis of diverse types of
studies
• Conceptual and methodological problems with
measures of health service utilisation
• Synthesising argument organised around a set of
central concepts (e.g. navigation, adjudications) with
the synthetic construct of ‘candidacy’ at the core
Access to health care for vulnerable groups
(Dixon-Woods et al. 2006)
Core construct – candidacy
Central concepts
“Candidacy describes the ways in
which people's eligibility for
medical attention and
intervention is jointly
negotiated between
individuals and health
services…….candidacy is a
dynamic and contingent
process, constantly being
defined and redefined through
interactions between
individuals and professionals,
including how "cases" are
constructed.”
• Identification of candidacy
• Navigation
• Permeability of services
• Appearances at health services
• Adjudications
• Offers and resistance
• Operating conditions and local
production of candidacy
In focus: Meta-narrative review
• Aim is to make sense of and
understand diverse bodies
of literature and their
findings
• Literature itself is an object
of scrutiny (critical)
• Searching is iterative,
‘snowballing’ a key
technique
• Analysis leads to production
of a set of meta-narratives
(‘storylines of research’)
Recent examples
Spread and sustainability of
innovations in health service
delivery and organisation
(Greenhalgh et al., 2005)
Understanding the use of electronic
patient records in health care
organisations (Greenhalgh et al.,
2009)
See also the:
Realist and meta-narrative evidence
synthesis evolving standards
project (RAMASES) (Greenhalgh et
al., 2011; Wong et al., 2013)
A meta-narrative approach
• The influence of Kuhn’s ‘paradigms’ (1962) and the
makings of the first meta-narrative
• The essential technique is interpretive synthesis
exploring distinct research traditions, each with its
own meta-narrative
• Methods of ‘unpacking’ the meta-narrative:
exploratory methods; expert consultations;
snowballing; database searching
Stages of a meta-narrative review (from
Greenhalgh et al., 2009)
Meta-narratives
identified in the
electronic patient
records review
(from Greenhalgh
et al., 2009)
Comparing approaches and methods
Method
Mixed methods
synthesis
Idealist –
realist
continuum
Realist
Bayesian
synthesis
Realist
Critical
interpretive
synthesis
Idealist
Meta-narrative
Idealist
Deconstruct
body of
literature?
No
No
Yes
Yes
Mixed methods lens
Complementary
strengths stance and
Dialectical stance
Alternative or single
paradigm stance
Alternative or single
paradigm stance
Dialectical stance
Other
characteristics
Little iteration in
methods
Synthetic product
aims to directly
address policy
Iterative approach
key
Synthetic product
requires
interpretation
Future challenges
• More worked examples are key
• Focus on methods and tools for the actual integration
– Enhancing transparency
– Establishing rigour
• Further conceptual work to illuminate points of difference,
strengths and weaknesses, fit for purpose (e.g. aggregation
and configuration – Sandelowski et al., 2011; Gough et al.,
2012)
• Learning from, and contributing to, the mixed methods
literature for primary research.
• Critical appraisal and reporting standards
Thank you!
[email protected]