Transcript Evidence Based Practice, Best Practices and Promising

The Practice Review: Approaches, skills and
interventions in working with
individuals with “complex needs”
Jim Cullen, M.S.W., Ph.D
Clinic Head/Manager, IGT Concurrent Disorders and
Rainbow Services, Centre for Addiction and Mental
Health.
Assistant Professor, Faculty of Social Work, University
of Toronto
Disclosures
•
Jim Cullen has previously received funding and support from the following
sources:
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
Ontario Ministry of Health and Long Term Care
Government of Newfoundland and Labrador
Alberta Alcohol and Drug Abuse Commission/Ministry of Health
Ontario Ministry of Community and Social Services: Child Welfare Transformation Fund
Social Science and Humanities Research Council
Canadian Institute of Health Research
University of Toronto
Ryerson University
York University
University of Victoria
Ontario HIV Treatment Network
Centre for Addiction Research of British Columbia
North Bay General Hospital
City of Toronto
Toronto Hostel Training Association
Anishabe Health Services
What are people talking about when
they use the term “complex” clients.
• Clients that frustrate workers.
• Clients that are multiple service users.
• Clients that often have substance abuse
and mental health challenges.
• Clients that do not appear motivated are
willing to change.
What we are talking about are people……
Usually living with a
Concurrent Disorder
A person with a mental health problem has
a higher risk of having a substance use
problem, just as a person with a substance
use problem has an increased chance of
having a mental health problem. People
who have combined, or concurrent,
substance use and mental health problems
are said to have concurrent disorders.
Concurrent disorders can include
combinations such as:
• an anxiety disorder and a drinking
problem
• schizophrenia and cannabis dependence
• borderline personality disorder and heroin
dependence
• depression and dependence on sleeping
pills.
When do concurrent disorders
begin?
Mental health and substance use problems
can begin at any time: from childhood to
older age. Causes can include genetic,
environmental and psychological factors. We
often speak in terms of risk, but not
prediction.
WHAT ARE THE SYMPTOMS OF
CONCURRENT DISORDERS?
Concurrent disorders is a term for any combination
of mental health and substance use problems.
There is no one symptom or group of symptoms
that is common to all combinations. The
combinations of concurrent disorders can be
divided into five main groups: Substance abuse
with;
) Mood and Anxiety; 2) Persistent and Severe; 3)
Personality; 4) Eating Disorders 5) Trauma
• Some people who have a mental health
problem may use substances to feel better.
While substance use is very risky in such
cases, it can help people forget their
problems or relieve symptoms, at least in
the short-term. People sometimes talk
about using substances for “selfmedication.”
Where do people get treatment?
Most people with concurrent disorders have
mild to moderate problems that can be
treated in the community, but the referrals
are barriers!!!
People with severe problems may need
specialized care for concurrent disorders.
What is integrated treatment?
Clients with severe concurrent mental health and
substance use problems may need integrated
treatment. Integrated treatment is a way of making
sure that treatment is smooth, co-ordinated and
comprehensive for the client. It ensures that the
client receives help not only with the concurrent
disorders, but also in other life areas, such as
housing and employment.
e.g. Mental Health and Addiction Workers
Without proper screening how can we
provide interventions that work?
Screening and Assessment Tools
1st Stage; GAIN SS, any comprehensive
addiction measure.
2nd Stage: PDSQ, ASI
So we know something is
going on!
Now I can just show I care, be client
centred, express empathy and they will
connect with me and everything will get
better
Flash forward six months….
•
•
•
•
•
•
•
•
I am frustrated
The client is lazy
They dropped out of service
We have barred them from service
They are in jail
They are in their 6th time in treatment
Why am I such a bad worker
The client is bad…..must be borderline!
We didn’t our approach work?
Well what is the evidence about
what works and are we doing it?
So lets back up
Let’s start with definitions
EBP (evidence Based practice)
refers to preferential use of
mental/addiction and behavioral
health interventions for which
systematic empirical research has
provided evidence of statistically
significant effectiveness as
treatments for specific problems.
• Best Practices refers specific
mental/addiction and behavioral health
interventions recommended by (usually
government, colleges or other significant
institutions) in which EBP literature has
been reviewed and specific models
selected based on context, funding
constraints, culture etc.,
• Promising Practice refers mental/addiction and
behavioral health interventions for which
systematic empirical research has not been
provided, but due to historical use, early data
(client satisfaction questionnaires etc.), the
interventions are used and monitored.
• The inclusion of promising practice in our
discourse was as a reaction to the push for EBP.
A Little History
In recent years, EBP has been stressed by
professional organizations and colleges such as
the Ontario College of Physicians and Surgeons,
Ontario College of Social Work and Social
Service Workers, The College of Psychologists
of Ontario, College of Nurses of Ontario, Ontario
College of Pharmacists (and others)
These institutions have also strongly
encouraged their members to carry out
investigations to provide evidence supporting or
rejecting the use of specific interventions.
• Prior to this movement however government
funders began asking questions about what we
were actually doing in our programs.
• Pressure toward EBP has also come from public
and private health insurance providers, which
have sometimes refused coverage of practices,
and even closed programs lacking in systematic
evidence of usefulness.
• Outcome evaluation took on a new life and
programs were held accountable to produce
results, not just “numbers served”
• Funding contracts begun to include certain
language that insisted on EBP being used.
• Huge grass-roots reactions developed and
funders revised the rigid criteria to include
promising practice.
• Many areas of professional practice, such
as medicine, psychology, psychiatry,
nursing, social work and so forth, have
had periods in their pasts where practice
was based on loose bodies of knowledge.
Some of the knowledge was simply lore
that drew upon the experiences of
generations of practitioners, and much of it
had no truly scientific evidence on which to
justify various practices.
• In the past this has often left the door open to dubious
practice perpetrated by individuals who had no training
at all in the domain, but who wished to convey the
impression that they did for profit or other motives.
• As the scientific method became increasingly recognized
as the means to provide sound validation for such
methods, it became clear that there needed to be a way
of excluding these practitioners not only as a way of
preserving the integrity of the field but also of protecting
the public from harm.
• Evidence based practice is an approach
which tries to specify the way in which
professionals or other decision-makers
should make decisions by identifying such
evidence that there may be for a practice,
and rating it according to how scientifically
sound it may be. Its goal is to eliminate
unsound or excessively risky practices in
favour of those that have better outcomes.
• Evidence-based practice (EBP) involves
complex and conscientious decisionmaking which is based not only on the
available evidence but also on client
characteristics, situations, and
preferences. It recognizes that care is
individualized and ever changing and
involves uncertainties and probabilities.
• EBP develops individualized guidelines of which
best practices are devised to inform the
improvement of whatever professional task is at
hand. Evidence-based practice is a
philosophical approach that is in opposition to
some “ways of practice”. Examples of a reliance
on "the way it was always done" can be found in
almost every profession, even when those
practices are contradicted by new and better
information.
• Recently some authors have redefined
EBP in ways that add other factors to, the
original emphasis on empirical research
foundations. For example, EBP may be
defined as treatment choices based not
only on outcome research but also on
practice wisdom (the experience of the
clinician) and on family values (the
preferences and assumptions of a client
and his or her family or subculture).
• Because conclusions about research
results are made in a probabilistic manner,
it is impossible to work with two simple
categories of outcome research reports.
Research evidence does not fall simply
into "evidence-based" and "non-evidencebased" classes, but can be anywhere on a
continuum from one to the other,
depending on factors such as the way the
study was designed and carried out.
• The existence of this continuum makes it necessary to
think in terms of "levels of evidence", or categories of
stronger or weaker evidence that a treatment is effective.
To classify a research report as strong or weak evidence
for a treatment, it is necessary to evaluate the quality of
the research as well as the reported outcome.
• Evaluation of research quality can be a difficult task
requiring meticulous reading of research reports and
background information. It may not be appropriate simply
to accept the conclusion reported by the researchers.
• Systematic reviews help and guide us
through the literature that would otherwise
for many be overwhelming to assess.
So….what is a Systematic Review
• A systematic review is a literature review
focused on a single question which tries to
identify, appraise, select and synthesize all high
quality research evidence relevant to that
question. Systematic reviews are generally
regarded as the highest level of evidence by
professionals. An understanding of systematic
reviews and how to implement them in practice
is becoming mandatory for all professionals
involved in the delivery of health and social
service care.
• It uses explicit methods to perform a
thorough literature search and critical
appraisal of individual studies to identify
the valid and applicable evidence. It is
often applied in the healthcare context, but
is now being applied in many fields of
research.
• While many systematic reviews are based
on an explicit quantitative meta-analysis of
available data, there has been strong
critique of this rigid approach and now
there are also qualitative reviews which
nonetheless adhere to the standards for
gathering, analyzing and reporting
evidence.
• The EPPI-Centre have been influential in developing
methods for combining both qualitative and quantitative
research in systematic reviews.
• The Evidence for Policy and Practice Information and
Co-ordinating Centre (EPPI-Centre) is part of the
Social Science Research Unit at the Institute of
Education, University of London. Its work is concerned
with systematic reviews which use transparent and
explicit methodologies for reviewing research evidence
in order to be clear about what we know from research
and how we know it.
Cochrane Collaboration
• Many journals now publish systematic reviews,
but the best-known source is the Cochrane
Collaboration, a group of over 6,000 specialists
in health care who systematically review
randomized trials of the effects of treatments
and, when appropriate, the results of other
research. Cochrane reviews are published in the
Cochrane Database of Systematic Reviews
section of the Cochrane Library, which to date
contains 2,893 complete reviews and 1,646
protocols.
http://www.cochrane.org/
Campbell Collaboration
• The Cochrane Collaboration, which
systematically reviews the effects of
interventions in health care, is the Campbell
Collaboration's sibling organization.
• The Campbell Collaboration (C2) is a non-profit
organization that applies a rigorous, systematic
process to review the effects of interventions in
the social, behavioral and educational arenas, in
order to provide evidence-based information in
the shape of systematic reviews.
• The organization was founded in 1999,
and held its first Colloquium in
Philadelphia in February 2000. The idea
was to develop an international network of
social scientists in order to "produce,
maintain and disseminate systematic
reviews of research evidence on the
effectiveness of social interventions". Its
first Colloquium was held in Philadelphia in
February 2000.
• C2 has coordinating groups in education,
crime and justice, and social welfare, and
also has a methods group.
• Campbell systematic reviews are
published electronically in the C2 Register
of Interventions and Policy Evaluations
(C2-RIPE)
http://www.campbellcollaboration.org/
So what’s the evidence for
Concurrent
Across the board no matter what concurrent
Symptoms:
Do further assessment
Use of self (success and pitfalls)
Motivational Interviewing
Contingency Management
Community Reinforcement
Cognitive Behavioural Approaches
• Mood and Anxiety – CBT Mind over Mood,
Interpersonal Group Therapy.
• Personality – Dialectic Behaviour Therapy,
CBT, Multi-systemic therapy
• Trauma- 1st stage Seeking Safety, 2nd
stage – Exposure Therapy/CBT
• Psychotic Illness- life skills, art and
recreation therapy, interpersonal group
therapy, CBT.
But different agencies
have different mandates
To help you figure this out…..a
practice review may be helpful.
So what’s a practice
review?
A practice review is a systematic structure and method in
which an organization or service reviews;
1. How do we provide service/treatment/support and does
it match with our mandate. (e.g. our approach)
(Inventory)
2. What does the current literature says about “what
works”? (Literature review)
3. What are gaps between what we do and what the
literature says works? (Comparative analysis)
4. 5. How do we address these gaps (Setting practice
improvements)
Strengths
• Practice reviews can occur at micro, mezzo and macro
levels on any community that has agencies providing
service.
• They help to provide consistency, and helpful support
based on knowledge in a comprehensive way.
• Promotes knowledge exchange and sharing
• With the creation of systematic reviews and other
technology, access to evidence based
treatments/approaches has greatly expanded.
Limitations
• If done improperly can feel like evaluation.
• Agencies can focus on just one “type” of
evidence.
• Mismatch between what evidence and available
resources e.g. contingency management,
supportive housing vs. shelters etc.
• Different funders may have competing interests
in what approach to use.
Review of Templates
•
•
•
•
•
•
Summary of Steps
Next steps flow chart
Inventory
Literature Review
Comparative Analysis
Setting Practice Improvements
What we have learned in working with
complex clients from a practice review
1. Inconsistencies exist
2. The debate between labelling and “client
centred” service. (symptoms vs. diagnosis)
3. Tools are invaluable (e.g. GAIN SS), we need
to get a picture
4. Worker bias, knowledge transfer breakdown
5. Undervaluing of therapeutic alliance
6. Mental health vs. addiction stigma and bias
(Ontario disability recent ruling)
7. There are approaches that work that we have
not been using (contingency management)
Next steps for us
• Providing funders (health authorities, networks
etc) with evidence – knowledge transfer –
advocacy etc.
• Implementation of assessment tools across the
spectrum (e.g. GAIN SS), Basis 32.
• The need to establish outcome
committee/measures etc.
• Engagement of clients/patients/members – client
feedback/focus groups.
Concurrent Resources
• CD Best Practices (Health Canada): http://www.hcsc.gc.ca/ahc-asc/pubs/drugsdrogues/bp_disordermp_concomitants/index_e.html
• Treating Concurrent Disorders: A
Guide for Counsellors (CAMH, 2005)
• Structured Relapse Prevention, 2nd
edition (Herie & Watkin-Merek, CAMH, 2006)
• Canadian perspectives on women’s substance use
(Centre for Addiction and Mental Health and British
Columbia Centre of Excellence for Women’s Health).
•
Practice Review Resources
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Buysse, V., & Wesley, P.W. (2006). Evidence-based practice: How did it emerge and what does it really mean for the early
childhood field? Zero to Three, 27(2), 50-55.
a b Duffy P, Fisher C, Munroe D (February 2008). "Nursing knowledge, skill, and attitudes related to evidenced based practice:
Before or After Organizational Supports". MEDSURG Nursing 17 (1): 55-60.
Norcross, JC, Garofalo.A, Koocher.G. (2006) Discredited Psychological Treatments and Tests; A Delphi Poll. Professional
Psychology; Research and Practice. vol37. No 5. 515-522 doi:10.1037/0735-7028.37.5.515
Mercer, J., & Pignotti, M. (2007). Shortcuts cause errors in systematic research syntheses: Rethinking evaluation of mental health
interventions. Scientific Review of Mental Health Practice, 5(2), 59-77.
Rubin, A., & Parrish, D. (2007). Problematic phrases in the conclusions of published outcome studies. Research on Social Work
Practice, 17 (3), 334-347.
Chambless, D., & Hollon, S. (1998). Defining empirically supportable therapies. Journal of Consulting and Clinical Psychology, 66, 718.
Kaufman Best Practices Project. (2004). Kaufman Best Practices Project Final Report: Closing the Quality Chasm in Child Abuse
Treatment; Identifying and Disseminating Best Practices. Retrieved July 20, 2007, from
http://academicdepartments.musc.edu/ncvc/resources_prof/reports_prof.thm.
Mercer, J., & Pignotti, M. (2007). Shortcuts cause errors in systematic research syntheses. Scientific Review of Mental Health
Practice, 5(2), 59-77.
Saunders, B., Berliner, L., & Hanson, R. (2004). Child physical and sexual abuse: Guidelines for treatments. Retrieved September
15, 2006, from http://www.musc.edu/cvc.guidel.htm
Khan, K.S., et al. (2001). CRD Report 4. Stage II. Conducting the review. phase 5. Study quality assessment. York, UK: Centre for
Reviews and Dissemination, University of York. Retrieved July 20, 2007 from http://www.york.ac.uk/inst/crd/pdf/crd_4ph5.pdf
National Registry of Evidence-Based Practices and Programs (2007). NREPP Review Criteria. Retrieved March 10, 2008 from
http://www.nrepp.samsha.gov/review-criteria.htm
Mercer, J., & pignotti, M. (2007). Shortcuts cause errors in systematic research syntheses. Scientific Review of Mental Health
Practice, 5(2), 59-77
Cooper, H. (2003). Editorial. Psychological Bulletin, 129, 3-9.
Pignotti, M., & Mercer, J. (2007). Holding Therapy and Dyadic Developmental Psychotherapy are not supported and acceptable
social work interventions. Research on Social Work practice, 17(4), 513-519.
Thank you