Transcript Document

Evidence from systematic reviews in
public health and the work of the CRD
Kath Wright
Amanda Sowden
CRD
November 2004
Background
•
•
•
1998 - Our healthier nation: a contract for health
1999 - Saving lives: our healthier nation
•
2000 - Evidence from systematic reviews of research relevant to
implementing the “wider public health” agenda. Centre for Reviews and
Dissemination http:www.york.ac.uk/inst/crd/wph.htm
•
•
cancer, CHD and stroke, accidents, mental illness
cancer, CHD and stroke, accidents, mental health, education, social
care and social welfare, crime drugs and alcohol
2002 - Review of evidence from systematic reviews relevant to
implementing the wider public health agenda. CRD.
• cancer, CHD and stroke, accidents, mental health, sexual health,
alcohol & drug misuse, smoking, immunisation , environment,
crime & violence, elderly, children, miscellaneous
CRD’s 2002 update
•
•
•
•
•
•
•
A two part document
Part one is a qualitative study to elicit the views of policy makers
and practitioners on the need for evidence and its presentation
Part two is a synthesis of existing systematic reviews relevant to
the public health agenda
Draft report produced, peer reviewed, comments incorporated
Ongoing discussions with the DoH Policy Research Programme
about dissemination of the findings
Health Development Agency now included in these discussions
under their Wider Determinants of Health programme
Aim is to pilot a new series of publications based on the “Wider
Determinants of Health” project
CRD’s 2002 update: views of policy
makers and practitioners (1)
•
•
•
•
Envisaged that the results of this would help inform how the
information collected would be presented and disseminated
Qualitative study using interviews (23), focus groups (35
individuals in 4 groups), responses (11) to e-mail discussion list
Directors of public health and public health observatories, PCT
executives, senior health promotion/development managers,
public health leads in PCTs
Data collected was analysed using N6 software
CRD’s 2002 update: views of policy
makers and practitioners (2)
•
•
•
•
•
Findings:
Range of views on evidence based policy and practice
• majority unsure why systematic reviews important
• less than 50% had heard of systematic reviews
• lack of critical appraisal skills
Amount of evidence
• information overload a common experience
• short “snappy” documents preferred
Access to evidence
• not everyone had access to the Internet
Differences between NHS and non-NHS personnel
CRD’s 2002 update: synthesis of
systematic reviews (1)
•
•
•
•
Identification, selection, appraisal and synthesis of existing
systematic reviews on the effects of interventions and
programmes relevant to the wider public health agenda
Inclusion criteria –
• have systematic methods been used?
• Is the topic of public health interest?
Originally planned to use data extraction, validity assessment,
synthesis of study results as inclusion criteria
Later decided that these would be considered in the quality
assessment of each included review
CRD’s 2002 update: synthesis of
systematic reviews (2)
•
•
•
Arranged by topic area e.g. accidents
Each chapter gives details of the size of the problem and the
current policy targets e.g. to reduce death rates by one fifth
Following this, the included SRs are briefly listed by:
•
•
•
•
•
interventions giving evidence of effect e.g. provision of free fire
alarms
interventions giving no evidence of effect - none in this section
interventions giving insufficient evidence of effect e.g. parental
education programmes aimed at increasing use of child safety seats
in motor vehicles
interventions giving evidence of harm or potential harm e.g school
based driver education programmes
Implications for policy, practice and research outlined e.g.
•
despite poisoning being the second most frequent cause of accident
and injury in children, no identified reviews addressed this issue
CRD’s 2002 update: synthesis of
systematic reviews (3)
•
•
•
•
•
•
•
•
Attewell RG, Glase K, McFadden M. Bicycle helmet efficacy: a
meta-analysis. Accident Analysis and Prevention 2001:33:345-352.
Objective - to assess the efficacy of helmet use in preventing
serious injury to cyclist
Inclusion/exclusion criteria - cyclists who crash and suffer head,
brain or facial injury and/or death
Participant description - 4 studies children, 2 studies adults, 10
studies mixed
Intervention description - wearing of bicycle helmets in crashes
Results - gives ORs for efficacy, injury reduction figures as % e.g.
head injury 45%, brain injury 33%, facial injury 27%, death 29%
Author Conclusion - helmets prevent serious injury and death
CRD comments - research question, inclusion criteria, synthesis good; search adequacy, study details - fair ; validity assessment N/A
Literature searching - problems
•
•
•
•
Range of primary sources
Volume of literature
Large number of potentially relevant databases
Variability in databases:
•
•
•
•
•
abstracts,
interface,
indexing
imprecise terminology
Internet searching
Literature searching - sources
•
Sources used
• Cochrane Database of Systematic Reviews
• Database of Abstracts of Reviews of Effects
• additional databases e.g. EMBASE, HMIC, Sociological
Abstracts, ASSIA, Health Technology Assessment database,
NRR, guidelines databases (e.g. HSTAT, NGC, USPSTF)
• websites
• contact with subject experts
Cochrane Database of Systematic
Reviews
•
•
•
•
•
Didn’t use a search strategy
Instead identified all new or substantially updated reviews on the
database for issue numbers 2000 no 1 through to 2002 number
1
Update searching done later to cover issue numbers 2002
numbers 2 and 3
Reviewers scanned lists for potentially relevant reviews and
protocols
Identified 80 potential reviews
Database of Abstracts of Reviews of
Effects
•
Identifies reviews by:
• handsearching
• contact with HTA organisations
• regular website searching
• database searching
• Current Contents Clinical Medicine
• MEDLINE
• CINAHL
• ERIC
• Biosis
• AMED
• PsycINFO
Benefits of searching DARE
•
•
•
•
•
•
A number of different databases and other sources have already
been searched
No need to use a “systematic review” search filter
The included reviews have already met agreed quality criteria.
See http://www.york.ac.uk/inst/crd/faq4.htm
So, saves time and effort and avoids many of the recognised
problems of searching in this area
Nevertheless, still undertook 18 separate searches to cover all
the topic areas
Identified 3955 potential reviews initially
• 273 on public DARE
• 3682 not on public DARE
EMBASE
•
•
•
When this project began EMBASE wasn’t searched for DARE
Wanted to include it to ensure relevant SRs were not missed
The searches undertaken demonstrate some of the problems
referred to earlier about searching in this field:
• volume of literature – especially as we were restricting our
search to 2000, 2001 and 2002.
• lack of tested systematic reviews search filter for EMBASE
EMBASE
•
•
•
Approach 1. Tried to identify records unique to EMBASE by
developing comparable EMBASE and MEDLINE strategies that
could be run across both databases, retrieving common records
and removing them. This approach failed, demonstrating the
difficulties of simply “translating” search strategies from one
database to another.
Approach 2. Relied upon using a SR strategy with no subject
terms specified. This approach failed as it retrieved too many
records (17,633) for the project (or Endnote) to cope with
Approach 3. Amended the SR search strategy to make it less
sensitive and combined it with subject terms. The 18 strategies
identified 4137 unique records
Other database searching
•
•
•
•
Had to adopt a pragmatic approach re searching for systematic
reviews e.g.
HMIC review* or guideline*
Sociological Abstracts review* or overview* or metaanal* or
meta-anal* or metanal* or guideline*
Assia – review$ or overview$ or metaanal$ or meta-anal$ or
metanal$ or guideline$
Guidelines databases and listings
•
•
•
•
HSTAT, National Guidelines Clearinghouse, Canadian Task
Force on Preventive Care, US Preventive Services Task Force
All web-based
Interfaces don’t offer very sophisticated search options e.g.
searching by date was often the most helpful approach (rather
than searching by topic or study design)
Had to rely on downloading title listings from the websites and
sifting
Literature searching issues….again
• Range of primary sources
•
mainly focused on journal literature, aware that non-journal literature
important in this area
• Volume of literature
•
in the time available had to undertake quite a narrow search (in terms of
databases used) and modify sensitivity of search strategies
• Wide number of potentially relevant databases
•
•
•
didn’t search Caredata, Social Science Citation Index, AgeLine or other
possibly relevant databases
didn’t have access to some possibly relevant databases e.g.Criminal
Justice Abstracts
had to adapt each search strategy according to availability of indexing etc
• Variable quality of search interface
•
usual access route to Assia is a web interface so used Datastar option.
• Variable indexing options
•
limited number of study types available as subject headings
• Internet searching
•
heavily reliant on scanning, sifting and downloading
Research in literature searching in public
health and the social sciences databases
•
•
•
•
•
•
CRD’s experience is similar to that of others, see:
Grayson, L, Gomersall, A. (2003). A difficult business: finding the evidence for social
science reviews. London, ESRC UK Centre for Evidence Based Policy and Practice.
• http://www.evidencenetwork.org/home.asp
Popay, J and Katrina Roen. (2003). Using evidence from diverse research designs.
London, SCIE.
• http://www.scie.org.uk/publications/reports/report03.pdf
HDA/CRD/Cardiff University (HEBW) Collaborative Group. (2004). Indexing in
databases of relevance to public health
• http://www.hda.nhs.uk/evidence/indexing_home.html
Bradshaw, J. R., S. Baldwin, et al. (2000). Methodologies for socially useful systematic
reviews in social policy. York, University of York
• http://www.regard.ac.uk/research_findings/L141250001/report.pdf
Jackson, N, Waters E and the Guidelines for Systematic Reviews of Health Promotion
and Public Health Interventions Taskforce. (2004). The challenges of systematically
reviewing public health interventions. Journal of Public Health 26 (3); 303-307.