Transcript Assessment of intervention outcome

Qu i c k T i m e ™ a n d a
Ph o t o - J P EG d e c o m p re s s o r
a re n e e d e d t o s e e t h i s p i c t u re .
Assessment of Intervention
Outcomes
Workshop: Thursday 7th February
IYW Conference, Llandrindod Wells
Dr Dave Daley & Dr Tracey Bywater
Overview of workshop
 Rationale for conducting your own
evaluations
 How to manage the numbers
 Suggested evaluation material
2
Why evaluate clinical work?
 NHS Trusts need require you to conduct
evaluation
 It is imperative to be able to demonstrate
improvement associated with your groups
 Helps to maintain funding and/or win new
funding
3
Is it difficult to evaluate
outcome?
 The most important decisions to make when
considering evaluation are:
1) Design
2) Evaluation measures
4
1. Design
 It is very important that you have baseline
measures (before) and outcome measures
(after)
 It is also important that you have the same
measures on everyone
5
What can we do with our
evaluation scores I?
 Even simple spreadsheet programmes like
Excel will allow you to conduct simple
statistics
 For evaluation purposes you are most
interested in change from start to end
6
Pros & Cons of change scores
 Change scores are most useful
 Easy and simple way of evaluating change
 However recent suggestion that change
scores on SDQ are less sensitive
 However formula is complicated, has not
yet been fully validated and is not
recommended for evaluations of small
groups
7
What can we do with our
evaluation scores II?
 Change scores should demonstrate
improvements in child behaviour outcome
 You can also begin to examine whether your
groups are more effective for particular
types of parents
8
What can we do with our
evaluation scores III?
 Rather than do lots of fancy statistics, it is also
interesting to examine scores at outcome
 Most evaluation measures will have scores which
indicate clinical concern
 It is very easy but also meaningful to examine the
percentage of children and/or parents who score
above cut off scores before intervention and after
intervention
9
2. Evaluation Measures
 Which ones?
 Pathfinders project uses:
 Demographic Questionnaire
 Beck Depression Inventory
 Eyberg Child Behaviour Inventory
 Strengths & Difficulties Questionnaire x 2
 Parenting Scale
 Overt Hostility Scale
 Teacher Questionnaire/Report
10
Demographics
 What do you need to know?
 Do you want to compare outcomes of:



older versus younger parents?
Boys versus girls?
Different areas?
Any other ideas?
11
The Eyberg Child Behaviour
Inventory
 36 items, ages 2-16 years, 10 minutes to complete
 Problem score has a minimum of 0, maximum of 36
(yes/no answers, problem or not)
 Intensity scale - minimum score of 36, maximum 252
(scale of 1 – 7, where 1 = never and 7 = always)
 Clinical cut-off scores, ≥127 intensity, or ≥11 problem
 Purchase from http://www3.parinc.com/
12
Strengths & Difficulties
Questionnaire
 FREE, website: http://www.sdqinfo.com/b1.html
 10 minutes to complete, 25 items:






Emotional symptoms
Conduct problems
Hyperactivity
Peer relationship problems
Prosocial behaviour
An additional impact supplement
 Parent & teacher
 Normal, borderline & atypical ranges/cut-off
13
Beck Depression Inventory
 21-item elf-report inventory measuring the
severity of characteristic attitudes & symptoms
associated with depression
 Each item contains four possible responses which
range in severity from 0 ( I do not feel sad) to 3 ( I
am so sad or unhappy that I can’t stand it)
 Score of 10-18 = mild to moderate depression
 Score of 19-29 = moderate to severe depression
14
Statistics with little maths
 Excel
 Graphs
 Mean & SD
 Change scores
 Effect sizes
15
Welsh Sure Start Data
before
BDI-I
BDI-C
ECBI-PI
ECBI-PC
ECBI-II
ECBI-IC
SD
17.74
15.04
16.56
14.75
145.92
140.11
after
SD
10.84
10.33
9.68
13.66
6.92
9.55
7.69
13.68
25.19
116.19
25.64
142.98
change scores
10.15
7.41
10.31
1.38
7.94
7.01
8.27
1.07
32.09
29.73
32.41
2.87
16
Effect sizes
Interven
fup1
BDI
Control
fup1
Difference
pooled
SD
effect
sizes
10.33
13.66
3.33
10.23
0.33
ECBI-P
9.55
13.68
4.13
8.11
0.51
ECBI-I
116.19
142.98
26.79
32.25
0.83
Cohen’s 1988 guidelines: difference between means
divided by pooled SD. 0.3 = clinically useful change,
0.5 medium effect, 0.8 = large effect
17
Any questions?????
18