Transcript Evidence-Based Practice in Clinical Psychology: What It Is

Evidence-Based Practice
in Clinical Psychology:
What It Is, Why It Matters,
What You Need to Know
Bonnie Spring, Ph.D., ABPP
Northwestern University
Why it matters: EBBP Rationale



improve quality and accountability for health care
practice (IOM, 2001, Crossing the Quality Chasm)
shared vocabulary and concepts for transdisciplinary,
biopsychosocial research, practice, health care policy
stimulate development of evidence base for
behavioral treatments
Why it matters:
Potentionally Useful Infrastructure

Clinical Practice Guidelines:
• Increasingly based on ongoing systematic review of research (esp.
RCTs) (e.g., USPTF, Cochrane, CDC/AHRQ)
• Research reporting guidelines (CONSORT, TREND, QUOROM)
• Evidence grading & knowledge synthesis systems (e.g., GRADE,
AHRQ)
• Policy, often coverage/reimbursement implications (VA/DOD,
CMS, NICE) (P4P?)

Evidence-Based Practice: (life-long learning)
• Question formulation, search strategies, critical appraisal
• SUMSEARCH
• Clinical Evidence, First Consult, BMJ updates, Best Evidence
Topics, CATCRAWLER, CATBANK – clinical scenario & bottom line
Overview





History of evidence-based practice
(EBP)
Core elements of EBP
EBP pedagogy in psychology
EBP pedagogy in other health disciplines
Useful infrastructure and potential
opportunities for synergy
Origins of Evidence-Based
Practice
Emergence of EvidenceBased Medicine
1910 - Flexner report :155(31!)
96 (1915)
76(1930)
1972 - Archie Cochrane – epidemiology, health services research -
Effectiveness and Efficiency: Random Reflections on Health Services
1973 – John Wennberg – widespread practice variation
1982 - clinical epidemiology determinants and consequences of health
care decisions (McMaster U – David Sackett, Gordon Guyatt)
1985 – IOM: 15% medical practices evidence-based [2001 Crossing the
Quality Chasm]
1990 - Evidence-based medicine, Brian Haynes & Ann McKibbon – search
strategies
1992-3 -Cochrane Collaboration
2000 - Sackett - How to Practice and Teach EBM
What do we mean by
“evidence-based practice?”
Nomothetic

Alternative Definitions of
Evidence-Based Practice
Guidelines: (public health, medicine) – focus on
problem/disorder & level of evidence for practices
(based on systematic review) (e.g., NICE, VA, apa)
• ESTs: (psychology) focus on intervention (&
disorder)

EBP: (psychology, medicine, nursing, social work)
focus on decision-making about individual patients
Idiographic……….. Lifelong Learning
APA Policy Statement adopted
August 2005

“Evidence-based practice in psychology is the
integration of the best available research
with clinical expertise in the context of
patient characteristics, culture, and
preferences.”
-adapted from
IOM, 2001 & Sackett, 2000
Best available
research evidence
Clinical Decision-Making
Patient’s values,
characteristics, and
circumstances
Clinical
Expertise
Syllabus Project

Prompt: Does anyone on the list teach a
course on evidence-based practice
(EBP)? Specifically, I am searching for syllabi
that cover one or more "legs" of the threelegged EBP stool: a) research evidence, b)
clinical expertise, c) patient values,
preferences, characteristics.
November, 2006
Listservs Sampled







ABCT
APA Division 12
SSCPNET (Section III, Div 12)
CUDCP
APA Division 38
ABMR
SBM EBBM, MRBC, Obesity, CA SIGs
Outcome




39 syllabi
17 additional recommended articles and
books
273 page document
Discipline:
30 psychology
3 public health
3 medicine
1 nursing
1 PE/health/sport studies

140 requests
November, 2006
Evidence-Based Practice
Modal


Course Title: CBT, EST,
EVT, Psychological
Interventions, Psychotherapy
Research
Texts: Barlow, Handbook
Psychologic Disorders,
Bergen & Garfield Handbook
of Psychotherapy and
Behavior Change

Content: ESTs
Additional

Additional Texts:
-Persons, Case
Conceptualization
-Dawes, House of Cards

Additional Content:
-Assessment
-Case formulation, functional
analysis
-Clinical judgment
-Diversity
-Iatrogenic effects
-Research methods
courtesy of Barbara Walker, Indiana University, 2006
120
Content Distribution of Books on EBP
100
80
What is EBP?
How to do EBP
How to teach EBP
60
40
20
0
EBM (Sackett
et.al)
EBN (Craig,
Smyth, et.al)
SW (Gibbs)
LIB (McKibbon)
PSYCH
(Norcross,
Beutler, Levant)
Synthesizer
Locate
Critically appraise
Meta-analysis
Consumer
Locate
Appraise quality &
relevance
Integrate
Researcher
Design
Conduct
Analysis
Reporting
Best available
research evidence
Clinical Decision-Making
Patient’s values,
characteristics, and
circumstances
Patient
Understanding
Preferences
Access
Clinical
Expertise
Clinician
Communicate
Assess patient
Deliver EBP
Researcher Training in
Psychology versus Medicine

Psychology
• Design


Correlational
(convenience classes)
Experimental (from
animal studies)
• Conduct


Brief, tight control
Little missing data;
replace cases
• Analysis - completer
• Reporting

Clinical Medicine
• Design


Observational (population)
Clinical Trial –test of
policy applied to
population
• Conduct


Long, intercurrent events
Missing data;
• Analysis – ITT
• Reporting – CONSORT
Researcher, Synthesizer, Consumer
Training in Analysis

Psychology

• ANOVA/regression
Clinical Medicine
• Odds Ratios
Epidemiology Terminology
Absolute risk (p[disease] in a particular population)
Relative risk (p[disease/exposed]/p[disease/unexposed)
Attributable risk (p[disease/exposed] -p[disease/unexposed)
Number needed to harm (1/attributable risk)
Odds ratio (odds[disease/exposed]/odds[disease/unexposed])
Clinical Significance

NNH = 5. If 5 patients treated with

NNT = 13. 13 patients would need to be
TX1, 1 would be more likely to have AE
than if all had received TX0
treated with TX1 to see one success not
seen with TX0
Reporting: Consort Flow Diagram
Consolidated
Standards of
Reporting Trials
(CONSORT)
www.consortstatement.org
Excerpt from CONSORT checklist
METHODS
3
Eligibility criteria for participants and the settings and locations where the
data were collected.
Interventions
4
Precise details of the interventions intended for each group and how and
when they were actually administered.
Objectives
5
Specific objectives and hypotheses.
Outcomes
6
Clearly defined primary and secondary outcome measures and, when
applicable, any methods used to enhance the quality of measurements
(e.g., multiple observations, training of assessors).
Sample size
7
How sample size was determined and, when applicable, explanation of any
interim analyses and stopping rules.
Randomization -Sequence generation
8
Method used to generate the random allocation sequence, including details
of any restrictions (e.g., blocking, stratification)
Randomization -Allocation
concealment
9
Method used to implement the random allocation sequence (e.g., numbered
containers or central telephone), clarifying whether the sequence was
concealed until interventions were assigned.
Randomization -Implementation
10
Who generated the allocation sequence, who enrolled participants, and who
assigned participants to their groups.
Blinding (masking)
11
Whether or not participants, those administering the interventions, and
those assessing the outcomes were blinded to group assignment. When
relevant, how the success of blinding was evaluated.
Participants
Evidence Synthesizer
and Consumer Skills
Synthesizer
Locate
Critically appraise
Meta-analysis
Evidence User
Locate
Appraise quality &
relevance
Integrate
Researcher
Design
Conduct
Analysis
Reporting
Best available
research evidence
Clinical Decision-Making
Patient’s values,
characteristics, and
circumstances
Patient
Understanding
Preferences
Access
Clinical
Expertise
Clinician
Communicate
Assess patient
Deliver EBP
Synthesizer:
Systematic Reviewer-
explicit,
systematic, transparent to avoid bias
 Specific research question (PICO)
 Search protocol to select papers – key words


systematic search of the literature (EMBASE, CINAHL,
Cochrane Controlled Trial register, DARE)
explicit inclusion and exclusion criteria
 Explicit, transparent rating of methodological
quality
 Data extraction
 Analysis: qualitative or quantitative
 Conclusion
 Discussion of strengths and limitations
The 5 Step EBM Model for
Evidence Users (Consumers)





Ask: formulate the question
Acquire: evidence - search for
answers
Appraise: the evidence for
quality and relevance
Apply the results
Assess the outcome
Asking: Well-Built Clinical
Questions
Background: What are effective
treatments for bulimia nervosa?
 Foreground: In patients with
Patient: binge eating disorder
Intervention: does interpersonal therapy
Comparison: compared to CBT reduce
Outcome: frequency of binge episodes

Critically appraising the
evidence
Use of standardized a priori appraisal
methods to answer:
 Is the evidence valid?
• Internal validity

Is the evidence applicable/relevant?
• External validity

Is the evidence clinically significant?
Clinical Decision-Making

Clinical epidemiology discipline
• study of determinants and consequences of
clinical decisions
• apply EBP/5A’s/critical appraisal at clinical
encounter to overcome automatic,
unconscious decision-making biases (aka bad
clinical intuition)
barriers between research and practice
30 kg of guidelines per family doctor per year
25000 biomedical journals in print
8000 articles published per day
95% of studies cannot reliably guide clinical decisions
2001 Bazian Ltd
Clinical Decision-Making

Health Informatics discipline
infrastructure, resources, devices,
structures (e.g., algorithms, guidelines)
needed to store, retrieve, manage and
use health information and the time and
place that a decision needs to be made.
-Decision support.
Secondary Synthesized Evidence
(AKA “evidence-based capitulation”)


Research proliferates rapidly. Clinical performance
demands increase. Practicing clinicians too busy to use
all EBM steps will all patients.
Increased focus on pithy clinical practice guidelines,
synopses, and structured abstracts
•
•
•
•
•
MD Consult
ACP Journal Club
Cochrane Database of Systematic Reviews
“Up-to-date”
InfoPOEMS (Patient Oriented Evidence that Matters)
Synthesizer
Locate
Critically appraise
Meta-analysis
Consumer
Locate
Appraise quality &
relevance
Integrate
Researcher
Design
Conduct
Analysis
Reporting
Best available
research evidence
Clinical Decision-Making
Patient’s values,
characteristics, and
circumstances
Clinical
Expertise
Clinician
Patient
Understanding
Preferences
Access
Communicate
Assess patient
Deliver EBP
Clinically Supervised Training in
Evidence-Based Treatment

Needs work: 2005-2006 papers by
Woody and by Weissman
Synthesizer
Locate
Critically appraise
Meta-analysis
Consumer
Locate
Appraise quality &
relevance
Integrate
Researcher
Design
Conduct
Analysis
Reporting
Best available
research evidence
Clinical Decision-Making
Patient’s values,
characteristics, and
circumstances
Patient
Understanding
Preferences
Access
Clinical
Expertise
Clinician
Communicate
Assess patient
Deliver EBP
Patient Preferences

Shared decision-making requires information only available to patient

Utility assessment: All possible outcomes assigned a value between 0
(death) and 1 (perfect health).


(e.g., valuation of harms/hassles, alternative outcomes & treatments)
Time trade-off approach
• The proportion of life in a particular health state (e.g., severe
depression) that you would give up to attain perfect health (e.g.,
30%). Utility of that health state is 1-(30%) = .70
Standard gamble approach
• The point where you are indifferent to the choice between spending
the rest of your life in the health state in question and a gamble
between perfect health and instant death where the probability of
perfect health represents the utility of the health state.
Teaching evidence-based
practice = teaching a process





Didactics
Small groups, problem-based learning
Preceptorships/clinical supervision
Standardized patients and evidence
stations
Embedded throughout curriculum
Medical Decision Making in the
NU-FSM curriculum

MDM-I (first week of medical school)
• Sensitivity, specificity, pre- and post-test probabilities,
innumeracy, uncertainty in medicine

MDM-II (last two weeks of M1 year)
• Epidemiology
• Statistics

MDM-III (beginning of M2 Spring Quarter)
•
•
•
•

Decision analysis
Meta-analysis
Cost-effectiveness analysis
Clinical guidelines
M3 MDM (once a month in M3 year)
• Review papers pertaining to clinical cases
• Use of CAT
Evidence-Based Behavioral Practice (EBBP)
NIH Office of Behavioral and Social Sciences
Research contract N01-LM-6-3512:
Resources for Training in EvidenceBased Behavioral Practice,
2006 - 2011
OBSSR 5-Year Plan

Year 1: develop training website, Council,

Year 2: develop, implement a web-based,

Year 3: launch interactive web-based training
Scientific Advisory Board, white paper on
training, skills, competencies reflecting education
in evidence-based behavioral practice (EBBP)
research-focused training module(s) on EBBP;
field test in graduate curricula
courses; establish practice network, develop first
EBBP clinical practice training module
OBSSR 5-Year Plan


Year 4: With practice network, develop modules
on application of evidence-based clinical decisionmaking to intervention with specific cases. Field
test in internship/residency/post-doctoral training
programs and practice network.
Year 5: Link website to systematic reviews of
behavioral interventions, treatment manuals,
outcome assessments. Develop and field test
clinical decision-making modules that integrate
patient preference and clinical competency
assessments.
Suggestions

To enhance the evidence base for psychological
treatments and support lifelong learning, clinical
psychology training might benefit from enhanced
coverage of:
• Researcher skills in methods: clinical trial design,
analysis, reporting, synthesis
• Clinician training in 5-step (5A’s) EBP model – cover
2 A’s
Suggestions
• Psychology informatics could use infrastructure
development (PSYCinfo & Cochrane; library
access; coverage in secondary synthesized sources
like Up-to-Date; practice-based research
networks)
• Psychology could use appropriate patient
preference measures that support shared
decision-making
• A discipline of clinical psychology decision-making
needs to develop to systematize integration of
research evidence, clinical expertise, and patient
clinical data and preferences
Concluding Questions


What training modules and materials
would be helpful?
Will you partner with us to help
develop and try these out?
The Evidence Pyramid for
Treatment Effectiveness Questions
***USE THE BEST EVIDENCE AVAILABLE***
Alternatives to evidence-based
medicine




Eminence based medicine
Eloquence based medicine
Vehemence based medicine
Nervousness based medicine
(Isaacs and Fitzgerald, 1999, BMJ)
Levels of Clinical Evidence in the Primary
Literature (psycINFO, MEDLINE)
Type of
Question
Methodology
Therapy
Double-Blind
Randomized
Controlled Trial
Randomized Controlled
Trial, Double Blind, Clinical
Trials
Prognosis
Cohort Studies,
Case Control,
Case Series
Cohort Studies, Prognosis,
Survival Analysis
Etiology
Cohort Studies
Cohort Studies, Risk
Quality
Randomized
Improvement Controlled Trial
Search Filters
RCT, Practice Guideline
Consensus Development
Conference
EBM Resources





Pocket guides with web-linked
updates (Sackett; Guyatt & Rennie)
Cochrane Library
BMJ: www.clinicalevidence.com
Centre for EBM:
http://minerva.minervation.com
Centre for Evidence-based mental
health: www.cebmh.com