Transcript Service-related research: Therapy outcomes audit

Service-related research:
Therapy outcomes audit
Sarah Howley
Trainee Clinical Psychologist
UCL
Background (1)
• Big changes in NHS – current emphasis
on NICE guidance & evidence-based
practice
• NICE currently does not consider practicebased evidence as equivalent to RCT
evidence
• CBT is NICE-recommended therapy for
depression and anxiety problems based
on RCT evidence
Background (2)
• BUT – does evidence of efficacy from
RCTs prove the effectiveness of therapies
in real-life clinical settings? Many say no!
• AND – does absence of efficacy evidence
mean therapies are not effective?
• There has been very little research
generally comparing CBT (“gold standard”)
to other therapies
Rationale for study in this Service
• No recent audit of therapy outcomes in
PTS
• Little explicit information about how clients
are allocated to CBT or Exploratory
waiting lists
• Little info on comparison of therapies
(PCP, Existential and Cognitive
Behavioural therapies)
Aims of current study
• To investigate:
– Potential factors influencing allocation to
different therapy modalities: demographic
variables (age, gender, ethnicity), problem
characteristics (type, severity, duration)
– Are therapies equivalent at (a) reducing
CORE-OM scores from pre- to post-therapy
and (b) producing reliable and clinically
significant change across therapy?
Methods
• CORE-OM forms used routinely in service across all
therapeutic modalities
• Data taken from CORE database from 2002 to 2010
• CBT sample = 386, PCP = 15, ExT = 45, “Exploratory
therapy” = 5
• Therefore: PCP, ExT + Exploratory amalgamated
(“Exploratory Therapy - EPT” N = 65) and random
sample taken from CBT dataset (N = 65; representative
of full CBT dataset as determined by statistical analyses)
Methods
• Variables:
– Demographics: Age, Sex, Ethnicity
– Referral source (GP, PCMHT etc)
– Problem characteristics: type, severity &
duration
– CORE-OM mean scores at Assessment
(IAS), Pre-therapy (1st treatment session) and
Post-therapy (last treatment session)
Sample characteristics
GROUP
N (M, F)
Age (SD)
CBT
65 (21, 44)
EPT
65 (18, 47)
Age
Range
Primary
referral
source
Primary
ethnic
group (%)
41.66 (10.87) 18 - 64
GP (49%)
White
British
(96%)
41.70 (10.90) 18 - 64
GP (52%)
White
British
(94%)
Outcomes analyses
• CORE-OM scores within therapy groups
(Effectiveness):
– Assessment vs Pre-therapy (change on W/L)
– Pre-therapy vs Post-therapy (therapy-related change)
– Reliable Change Index – calculation to determine
whether clients achieved reliable and clinically
significant change (Jacobson and Truax, 1991)
Outcomes analyses
• CORE-OM scores between groups
(comparison of therapies)
– Assessment scores – do clients vary in severity at
assessment?
– Pre-therapy scores (baseline)
– Post-therapy scores – are therapies equivalent in
terms of outcome i.e. reduction in CORE scores?
– Reliable Change Index – are therapies equivalent in
terms of achieving reliable and clinically significant
change?
Results
• Allocation to therapeutic
modality:
– Only difference between
therapy groups was in
primary problem type
– Anxiety disorders
significantly more likely to be
found in CBT group as primary
presenting problem
• Chi Square test: χ2 = 6.65, p
< 0.01
GROUP
Primary
problem
Secondary
problem
CBT
Depression
(68.8%)
Anxiety
(46%)
Anxiety
(25%)
Depression
(17%)
EPT
Depression
(75.8%)
Anxiety
(6%)
Anxiety
(33%)
Inter/p
problems
(14%)
Results: Within groups
• Within groups:
– CBT & EPT: significant
reductions in CORE
domain scores from IAS to
pre-therapy
– Effectiveness?
• CBT - sig reduction in all
CORE domain scores prepost
• EPT – sig reduction in all
CORE domain scores prepost except Risk (p =
.059)
Sig differences in
CORE-OM domain
scores
Group
Ax – PreTherapy
CBT
Functioning ALL
Risk
EPT
Subjective
Wellbeing
PrePost
ALL
except
Risk
Results: Between groups
• Comparing CBT and EPT on CORE-OM:
– IAS: No statistically significant differences between
groups
– Pre-therapy: No statistically significant differences
between groups
• HOWEVER: In CBT group Problems/Symptoms domain
score was below clinical cut-off
– Post-therapy: CBT group had significantly lower
Problems/Symptoms and All Items scores compared
to EPT
Comparing CORE-OM scores
STAGE
Assessment
Pre-therapy
Post-therapy
Subjective
Wellbeing
-0.39 p < .05
-0.32
n.s.
-0.34
n.s.
Problems/
Symptoms
-0.12 n.s.
-0.24
n.s.
-0.39
p < .05
Life
-0.17 n.s.
Functioning
-0.20
n.s.
-0.22
n.s.
Risk
-0.14 n.s.
-0.14
n.s.
-0.17
n.s.
ALL Items
-0.20 n.s.
-0.17
n.s.
-0.27
p < .05
Reliable Change
• Jacobson & Truax (1991) Method used to identify whether individual
clients achieve reliable and clinically significant change
(improvement OR deterioration)
• This indicator of clinical significance is distinct from statistical
significance
• There was no statistically significant difference between the groups
in terms of no. of clients achieving reliable improvement
GROUP
Improvement No change
Deterioration
CBT (n = 53)
45.3%
54.7%
0%
EPT (n = 59)
50.8%
44.1%
5.1%
Recap of results
• No differences observed in age, sex, ethnicity, referral source,
problem severity or chronicity between therapy groups
• Anxiety disorders more likely to be main problem in CBT group
• Both therapies showed statistically significant improvement in CORE
scores across domains (apart from EPT group in Risk but score
below clinical cut off)
• CBT group showed significantly more improvement on
Problems/Symptoms domain (but were below clinical cut off at pretherapy)
• No statistically significant difference in number of clients achieving
reliable clinical improvement between groups
Interpretation of results
• Clients presenting with anxiety as main difficulty tend to be referred
more often to CBT – Service is in line with NICE guidance on
treating anxiety
• Other factors in allocation to W/L – client preference, length of
waiting lists, assessor’s preferred model?
• No significant deterioration in Risk scores in CBT or EPT groups –
effects of waiting list times on client risk?
• CBT targets specific “symptoms” – therefore unsurprising that CBT
clients show more reduction in this domain? (assuming result is
valid – small effect size…)
• Both CBT and EPT are effective in reducing CORE-OM scores from
pre- to post-therapy in this Service and clients in both achieve
comparable levels of reliable improvement
Limitations
• Relatively small sample sizes (n = 65 and smaller for
most analyses due to missing data)
• Use of data from up to 10 years ago – does this reflect
current service?
• Amalgamation of PCP, ExT & “exploratory” – difficult to
draw conclusions (but unavoidable!)
• Use of non-parametric stats (increased risk of not finding
significant differences i.e. type 1 error)
Recommendations
• Encourage all clinicians to record CORE-OM scores at
each stage of therapy – vital to practice-based evidence
for therapies offered here
• Carry out similar audits at regular intervals in order to
establish a bedrock of practice-based evidence…