Transcript Critically appraising research in pain management

Critically appraising research
in pain management
Making sense of the evidence
M.Sc. The nature of pain and its’ management,
2006
• Why do we need to appraise the
literature on pain ?
• What is the current state of the art ?
• How do we go about it ?
• What are good sources of information ?
Critical Appraisal
The process of “weighing
up” the evidence to assess
how useful it is in making
decisions about clinical
care
Some “drivers” promoting critical review of
research literature
• Evidence based practice
Clinical effectiveness > guidelines, standards
and pathways
• Continuing Professional Development (CPD)
• Modernisation Agenda eg. patient choice
• Research – asking the right questions
Types of Evidence
• Primary
Generated from first hand experience
eg. diaries, letters, research reports
• Secondary (synthesis of others work)
eg. narrative and systematic literature
reviews, scholarly papers
Information explosion
c. 20.000 journals, 2 million papers p.a.
Pain Management – The Evidence
•
•
•
•
•
•
•
Meta-analyses, RCT’s
Systematic reviews
Cohort studies
Case studies
Narrative reviews
Updates
Reports, editorials, working papers, product
updates
• Conference proceedings, abstracts
• Dissertations etc.
Traditional Hierarchy of Evidence
Quantitative Research
Which research has most “weight” ?
•
•
•
•
•
•
•
Meta-analysis, systematic review
High quality RCT
Controlled study without randomisation
Observational study eg. case control study
Descriptive study
Expert consensus
Clinicians and clients/patients views
Systematic review
- The “sharp end” of critical appraisal
- Used for development of clinical guidelines
Many stages including:
•
•
•
•
•
•
Development of a protocol
Systematic selection of studies
Data Extraction
Quality assessment
Statistical or other synthesis of findings
Rating of the overall body of evidence
NICE
• Cox II selective inhibitors are not recommended
for routine (regular) use in patients with
rheumatoid arthritis or osteoarthritis (2001,
Guidance 27)
• Current evidence on the safety and efficacy of
percutaneous disc decompression using
coblation for lower back pain does not appear
adequate to support the use of this procedure
without special arrangements for consent and
for audit or research (Consultation Document,
25 May 2004)
• Adalimumab, etanercept and infliximab for the
treatment of ankylosing spondylitis (in progress,
due Feb. 2007)
Assessing the quality
“critical appraisal”
a) Quality of individual publications
•
•
•
•
•
Checklists
Guidelines/reading guides
Questionnaires
Frameworks,
Criterion based scoring tools
b) Quality of bodies of evidence
• Grading schemes
• hierarchies
Quality Evaluation Tools
• Many “off-the-shelf” tools available for
different types of studies
eg. Jadad, Delphi, Maastricht, SIGN
• Debate on scales v scores v weighting
• Few have been properly validated and
the criteria for validation are unknown
• Move towards customised checklists
• Critical Appraisal Skills Programme
http://www.phru.org.uk/-casp/
• Scottish Intercollegiate Guidelines Network
http://www.sign.ac.uk/guidelines
• Jadad et. al. (1996) Control Clin Trials 17:1–12
• Newcastle Ottawa Observational Study checklist
http://www.ohri.ca/programs/clinical
_epidemiology/oxford.htm
Quality in quantitative research
eg. RCT’s
Emphasis on issues such as:
Random allocation of subjects, “blinding”
Allocation concealment
Similarity at baseline
Standard, reliable and valid outcome assessment
Drop-outs
Intention to treat analysis (ITT)
Validity
Reliability
Quality in qualitative research
• Dependability (reliability) eg.
- justification of methods
- audit trail
- providing information about the researcher
- member checking, inter-researcher comparisons, triangulation
• Credibility eg.
- providing raw quotes
- reference to accepted procedures for analysis
- discussion of how researcher may have effected outcomes
-
clear distinction between data and interpretation
• Transferability eg.
- detail about participants and context
- identifying differences between individual participants and
participants and researcher
CASP – Qualitative Studies
Rigour: has a thorough and appropriate
approach been applied to key research
methods in the study?
Credibility: are the findings well
presented and meaningful?
Relevance: how useful are the findings to
you and your organisation?
http://www.phru.nhs.uk/casp/casp_qualitative_tool.pdf
Structure of a research paper
• Title, author
• Abstract: summary of what the paper is about
• Introduction: Background including previous
research, aims, research question/hypothesis,
ethics
• Methods: patients, methods, equipment, data
analysis
• Results
• Discussion (may include results in qualitative
research)
• Conclusion, recommendations for clinical
practice and further research
General questions for any study
•
•
•
•
•
•
•
•
•
•
•
Are the aims clearly stated ?
Was it ethical ?
Does the design match the aims ?
Was the sample size justified ?
Sufficient methodological detail eg. drop outs ?
Missing data ?
Are the measurements/outcomes valid and
reliable ? (NB. study population)
Are statistical methods/data analysis described ?
Are the main findings explicit ?
Are important effects considered ?
Are results/findings compared with previous
work?
Are implications for practice discussed ?
Papers for review:
Effects of pool-based and land-based
aerobic exercise on women with fibromyalgia/
chronic widespread muscle pain
Jentoft et al
Arthritis Care and Research, 2001, 45:42-47
In the system: the lived experience of chronic
back pain from the perspectives of those
seeking help from pain clinics
Walker et al
Pain, 1999, 621-628