Transcript Slide 1

Developing evidence based practice: what does
it mean and can it be done?
Miranda Wolpert
Director CAMHS Evidence Based Practice Unit
Chair CAMHS Outcome Research Consortium
Evidence based practice
Questions of the individual clinician
•
•
•
•
What is the best treatment for this particular child
What are the pros and cons of different treatments
What does the research show and what other
factors do I need to take into account
Are there particular reasons for adopting a
different approach in a particular case?
Evidence based practice
Questions of the service developer
• What does the evidence show should be provided
by services and in what proportions
• What skill mix is needed to provide child mental
health services
• What should be the ratio of investment in different
options eg prevention/promotion programmes as
opposed to direct interventions
Evidence based practice
questions of the academic
• What does this research really show
• Are there other interpretations
• How can research be devised to answer the
remaining questions
Evidence based practice
questions of the child and family
• What does this research really show
• Are there other interpretations
• How can research be devised to answer the
remaining questions
Answering these questions
- Few straight or clear answers
- Lots of complexity
- Lots of gaps
- Need a realistic way forward…
A realistic evidence based practice
Evidence
Audit and
Evaluation
Values
When does information become evidence?
Hierarchy of Evidence
•
•
•
•
•
•
Ia Evidence from meta-analysis of randomised controlled trials
Ib Evidence from at least one randomised controlled trial
IIa Evidence from at least one controlled study without
randomisation
IIb Evidence from at least one other type of quasi-experimental
study
III Evidence from descriptive studies such as comparative studies,
correlation studies and case-control studies
IV Evidence from expert committee reports or opinions, or from
clinical experience of a respected authority, or both.
Alternative Hierarchies?
A non-evidence based approach?
• Reliance on assumptions
• More influenced by anecdote than statistics
• Not testing theories
• Unwillingness to change in light of new evidence
• Most persuasive promoter wins out
Limitations of the evidence
•
•
•
•
•
•
•
•
•
Paucity of research
Skew in researched areas
Skew in researched populations
Generalisability to range of groups and settings
questionable
Design flaws in studies
Lack of consensus on appropriate outcomes and
perspectives
Lack of model for economic costings
Lack of focus on possible harm
Publication bias
Publication bias (from David Cottrell)
Drug
Published
Conclusions
Fluoxetine
2 Trials
Favourable risk
benefit profile
Paroxetine
1 Trial
Weak positive
risk benefit
profile
Sertraline
Citalopram
Venlafaxine
2 Trials
Weak positive
risk benefit
profile
Whittington, CJ, Kendall, T,
Fonagy, P, Cottrell, D,
Cotgrove, A & Boddington E.
(2004) Selective serotonin
reuptake inhibitors in childhood
depression: systematic review
of published versus unpublished
data. Lancet. 363, 1341-1345.
Publication bias (from David Cottrell)
Drug
Published
Conclusions
Not
Published
Conclusions
Fluoxetine
2 Trials
Favourable risk
benefit profile
Safety Data
No change
Paroxetine
1 Trial
Weak positive
risk benefit
profile
2 Trials
Risks now
outweigh benefits
Sertraline
2 Trials
Weak positive
risk benefit
profile
Additional
Data
Risks now
outweigh benefits
Citalopram
2 Trials
Unfavourable risk
benefit profile
Venlafaxine
1 Trial
Unfavourable risk
benefit profile
Drawing on the Evidence
Wolpert, Fuggle, Cottrell, Fonagy, Phillips, Target and
Stein 2002
Based on systematic review: Peter Fonagy, David
Cottrell, Mary Target, Zarrina Kurtz, Jeanette
Phillips
– DoH Mother & Child R&D Fund
Revised edition 2006
-Updated in light NICE guidance and major
randomised control trials
Possible summary of what we know works currently
Evidence based interventions
•
•
•
•
•
•
•
•
•
•
Cognitive behavioural therapies (CBT)
Behaviour therapy
Parent Training
Medications
Family Therapy
Interpersonal therapy (IPT)
Social skills training
Multi-systemic therapy (MST)
Treatment Foster Care
Individual psychodynamic therapies
BUT….complicating/mediating factors
Demographic factors ?
“attachment disturbance”
“therapeutic alliance”
Non-specific therapeutic factors
Fidelity to model
Ability to flexibly adapt model
Proposed checklist for evidence based interventions
(adapted from Kazdin 2004)
1. What are the costs, risks and benefits of this intervention
relative to no intervention?
2. What are the costs, risks and benefits of this intervention
relative to other interventions?
3. What are the key components that appear to contribute to
positive outcomes?
4. What parameters can be varied to improve outcomes (e.g.
including addition of other interventions, non specific clinical
skills etc)?
5. To what extent are effects of interventions generalizable across
a) problem areas, b) settings, c) populations of children and d)
other relevant domains
Which of these can we answer now?
How do we get answers?
Evidence base for service structures
Lots of values much less clear evidence
- Fort Bragg Studies
- Pooled budgets impact
-Suggestive work about impact of service user
involvement
Promising work on economic evaluation of early
intervention in psychosis projects
Worcestershire EIS
(2006 report Jo Smith)
National
EIS (3y) 2003-6 n=78
12-18m
5-6m
% admitted in FEP
80%
41%
% FEP using MHA
50%
27%
Readmission
50%
27.6%
% engaged @ 12m
50%
100% (79% well engaged)
Family involved
satisfied
49%
56%
91%
71%
Employed
20%
55%
48%
6.6% @ 5yrs
21%
0%
Duration of untreated psychosis
Suicide attempted
completed
Evidence base for skill mix (based on evidence based
interventions)
• 3 units of people able to provide behavioural, cognitive
and interpersonal therapies
• :1.5 units of people able to provide parent management
training
• : 1 unit of people able to provide systemic/multimodal
therapy:
• : 1 unit of a person/people able to provide physical
treatments, prescription and monitoring.
BUT doesn’t taken account of
-under-researched interventions
Non- specific therapeutic and assessment skills
Possible needs of particular populations
Evidence base for children and families
Choosing What’s Best for You
What We Know (And What We Don’t) About the best
ways of Helping Children and Teenagers With:
Eg ADHD
Information for: Children, Teenagers, Families
Choosing what's best for you
booklet aims to help children young people and their families make
informed choices about treatment options
It gives information about what research up till now has shown to help.
It is not designed to give you any general information.
In this booklet we list the most evidence based treatments a the
moment
Each treatment option is rated using the following scale:.
* * * = Very likely to help
* * - = Quite likely to help
* - - = Not that it will help
Choosing what's best for you
• Points to remember
• There are many treatments that we simply don’t know if they
work or not yet because research has not been done or is
inconclusive- they are not included here
• Even when a treatment has been shown in research to work
well for most people, as we are all different it may help some
people more than others
• You will have to weigh up the positives and negatives of any
approach, including any possible side effects
•
• Our knowledge is growing all the time so check if there have
been further developments since this was published
Choosing what's best for you
Points to remember
• There are many treatments that we simply don’t know if they
work or not yet because research has not been done or is
inconclusive- they are not included here
• Even when a treatment has been shown in research to work
well for most people, as we are all different it may help some
people more than others
• You will have to weigh up the positives and negatives of any
approach, including any possible side effects
•
• Our knowledge is growing all the time so check if there have
been further developments since this was published
Choosing what's best for you
Types of Treatment/What Might Help
Will it help?
Medication
This involves taking one or more tablets a day.
There is more than one type of medication.
You need to talk to your doctor about which
one is best for you.
***
Links to more info about ADHD/medications
Other things to think about
Any medication is likely to have side effects- you need to talk this over with your
specialist.
Behaviour Therapy
Behaviour therapy is advice and help on how
to learn behaviour that will make life easier.
**
-
Other things to think about
Can be used with medication and can mean that you don’t need to take as much
medication
Taking Omega 3 and Omega 6 Oils
This involves taking food supplements rich in
these oils
Other things to think about
This is quite a new area of research
*
- -
Reflection and evaluation
Routine outcome monitoring“Mission Impossible” ?? (Einar Heiervang)
•
Case evaluation:
To provide information about individual children and their families.
•
Clinician evaluation:
To provide information about outcomes for the range of children and
families seen by an individual clinician
•
Service evaluation:
To provide information about the outcomes of particular projects or
services
•
Strategy evaluation
To provide information about the impact of a CAMHS strategy
Underpinning values
• All services should routinely audit and evaluate their
work
• Data collected made available to clinicians, users and
commissioners
• Results used to inform service development
• Collaboration essential
Evaluating outcomes
Whose view?
Child
Parents
Clinician
Where get info from?
Conversations
Questionnaires
Written communications
Information held in a data set
Population statistics
For Whom?
What should be be evaluated
•
•
•
•
•
Change in difficulties
General adaptation
Feelings of burden and stress
Satisfaction
Population changes e.g. attendance rates;
exclusions; youth crime; reported self harm;
reported substance misuse, rates entering care,
attainment rates
CAMHS Outcome Research Consortium (CORC)
Members agreeing on a common approach
• Creating reports for reflection on individual
children/practitioners
• Collating and centrally analysing data
• Promoting use of data to inform service providers,
commissioners, users and others
• Facilitating sharing of ideas between members
• Supporting dissemination and refinement of
National CAMHS dataset
CORC aims
•
Develop and disseminate model of routine outcome
evaluation that can be used across a range of services
•
Ensure data used to inform service providers,
commissioners and users and other relevant
stakeholders
•
Collate and centrally analyse data from all member sites
•
Collaborate in using outcome information to inform and
develop good practice
•
www.corc.uk.net
CORC approach
Child/parent perspective:
Symptoms and burden:
-Strengths and Difficulties Questionnaire (SDQ) for child (11-16)
and parents of children aged 3-16
Experience of service:
- Commission for Health Improvement (CHI) for child (9+) and
parent
Practitioner perspective
Children’s Global Assessment Scale (CGAS) – measures overall
functioning
HoNOSCA where appropriate
Consultation measure
Being piloted
CORC protocol
•
•
•
•
•
Pre therapy measures for child and parent
First meeting measures for clinician
MDS
6 month follow up (or case closure if before this)
Option for repeated follow ups for longer term
contacts
Outcome measurement - SDQ
Interpreting the “evidence”
• Credibility
Self Evaluation: right measures, no.s of returns, quality of
returns
Research studies: right measures, right people, quality of
controls
• Context
Self Evaluation: specific factors to be taken into account eg
demographics, specialist focus
Research Studies: generalisability
• Comparison
Self Evaluation: with baselines, with community with
appropriate other services
Research Studies: with other findings
Towards and evidence based practice approach
Need to both acknowledge complexity and to promote
clarity- is this possible?
• More research
• Explicit recognition of values base
• Reflective practice