Transcript Outcome Rating Scale (ORS)
Session by Session Monitoring of Outcome and Alliance
David C. Low Family & Systemic Psychotherapist Feedback Informed Treatment (ORS & SRS) Trainer [email protected]
Tel. 01473-280416
Aim of the Presentation
• Provide introduction to the research and clinical rational for Session by Session monitoring of outcome and alliance.
• Consider how session by session measuring fits with systemic thinking and practice.
• Introduce the Outcome Rating Scale (ORS) and Session Rating Scale (SRS) and how they can be used with families.
• Introduce how the measures can be used to bring the clients’ voice into clinical supervision.
“So What?” 5 Minute Exercise
• Get in small groups of 3.
• Introduce yourselves.
• Choose someone to feedback themes to the rest of us.
• Very briefly discuss the relevance of session by session measuring has to each of you, and why any part of it would be important to you.
The Turn to “Outcome”
• “ Outcome,” “Evidence,” “Effectiveness” &“Accountability” are
the
watchwords of the day.
• Part of a world wide discourse and trend and not specific to mental health.
What Works in Therapy: the Tale of Two Paradigms
•
The Medical Model Evidence Based Practice Paradigm
: –
Diagnosis + RCT Tx = Good Outcome?
–
NICE and PBR
“What Works for Whom.” •
To The Contextual Model Practice Based Evidence Paradigm:
– Is this relationship between this client and this therapist working for this young person/family at this time and place?
–
i.e. Session by Session Measuring
Session by Session Monitoring
the Evidence
• Currently, 13 RCT’s involving 12,374 clinically, culturally, and economically diverse clients provides evidence that routine monitoring and feedback – As much as doubles treatment “effect size”; – Decreases drop-out rates by as much as half; – Decreases deterioration by 33%; – Reduces hospitalizations and shortened length of stay by 66%; – Significantly reduced cost of care (non feedback groups increased).
Miller, S.D. (2010). Psychometrics of the ORS and SRS. Results from RCT’s and meta-analyses of routine outcome monitoring and feedback: The available evidence. http://www.scottdmiller.com/?q=blog/1&page=2.
How Does Session by Session Measures Fit
•
with Systemic Thinking?
Cybernetics
from Greek
kubernētēs
from
kubernan
to steer, control steersman, • Cybernetics concerned with self-regulating systems and how their stability and functioning were maintained through mechanisms of information, control and feedback.
• A key notion of cybernetic is the principle of feedback– how information about the results or consequences of a previous action return or loop back into the system in order regulate subsequent sessions.
• Session by session, ultra brief, reliable and valid measures provide feedback into the therapeutic system to inform when we are on track and when we are off track and need to adjust or do something different in order to have a positive outcome.
Why do I need Formal Measures as I Already ask for Feedback?
• Routine tracking of client progress using standardized measures is especially important given clinicians’ tendencies to: – be overly optimistic about the meaning of clients’ lack of progress – their over optimism of eventual improvement in the face of slow progress, – and their failure to judge when clients are headed toward a negative outcome.
Lambert, M.J. (2010)
Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice.
Washington, DC: American Psychological Association.
Self-Assessment Bias:
Clinicians Have an Overly Positive Assessment of Personal Performance • 25% of mental health professionals viewed their skill to be at the 90th percentile when compared to their peers • Most thought they were above average.
• None viewed themselves as below average. • Clinicians tended to overestimate their rates of client improvement, and underestimate their rates of client deterioration. • Using session by session measures keeps us grounded, and improving our performance and outcomes.
Walfish, S, McAlister, B., O'Donnell, P., & Lambert, M.J. (2012)An Investigation of Self-assessment Bias in Mental Health Providers. Psychological Reports: Volume 110, Issue, pp639-644.
Benefits of Session by Session Measuring
• Puts the client’s voice at center stage • Allows services to be client and family driven • Brings consumers into the inner circle of decisions • Partners in monitoring the benefit and fit of services • Collaboration, the Next Generation: Going boldly where we have not gone before.
Ok, But Every Session?
• The evidence demonstrates that frequent measurement, leads to improved outcomes, decreases deterioration rates, and reduces no shows.
• Would you advise a parent that it is ok to be consistent some of the time with parenting strategy and still expect their preferred outcome.
• Can you imagine a GP who is treating you for high blood pressure saying, “Do I need to measure the blood pressure EVERY TIME?” Lambert, M.J. (2010)
Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice.
Washington, DC: American Psychological Association.
The purpose of the ORS & SRS:
• To give therapists ongoing, formalized feedback about the progress and process of therapy, allowing them to maximize the effect of services.
• The ultra brief design and formats for different age groups including small children, allows all members of the family an important feedback voice into the therapeutic system, and enhances everyone’s engagement and participation in family work. • The ORS and SRS have good validity and reliability, and high feasibility.
Reluctant Using the ORS & SRS as It Might Infringe on Therapy Time
• Using the ORS and SRS to get feedback about the benefit & fit of treatment need not be cumbersome or intrusive on therapy time.
• Clients appreciate your dedication to getting it right, • It is painless and can fit your natural way of working.
Using the ORS and SRS to Help Level Power Imbalances
• Invites clients into inner circle of decision making • Available in multiple languages • Gives a voice to diversity and the disenfranchised • Local culture and context over privileged knowledge
ORS & CORS Integrating Outcome Into Care
• The ORS, a brief measure of global subjective distress, is administered at the beginning of each session to get real time feedback from young people and carers regarding their perspective of treatment progress. • There is a version for those 13 to adult, and a younger version with smiley faces called CORS for 6-12 year olds.
Postmodernism and ORS/CORS
• Other measures focus on symptoms and diagnostic constructs and tends to measure dysfunction (or rather peoples perception on constructed notions of dysfunction), and increase possibilities of inviting a person to adopt a 'mental illness' identity. • In contrast, the ORS is a simple generic measure that takes at face value what people construct as meaningful for themselves.
• It is easy and feasible to integrate into a variety of practice modalities including post modern models that help clients re story and develop new narratives.
Introducing the ORS/CORS to Young Person and Carer
“The ORS is an outcome measure that allows us to track where you’re at, how you’re doing, how things are changing or if they are not. It allows us to determine whether the therapy is being helpful so we can do something different if it’s not helping. It also is way to make sure that your perspective stays central here, that we are addressing what you think is most important. It only takes a minute to fill out and most clients find it to be very helpful. Would you like to give it a try? Are you ok with that?” “Ok, so let me go over the instructions with you”.
Introducing the ORS/SRS to other Members of the Family
• At most first interviews you won’t know where the distress is, so you can ask
all
family members present to complete the ORS on themselves. This allows you to "see" who is distressed. •
To carer and other family members present:
“
I would also be grateful if the rest of you would also fill the form out on yourselves to help me understand how things are going for you, and to make sure what we are doing here is benefitting you too. Even if things are going ok for you, I would be grateful if you could do this anyway, so we can monitor and ensure that whatever we do together doesn't impact you negatively.”
Outcome Rating Scale (ORS)
(Ages 13 to Adult)
Individually
(Personal well-being) I----------------------------------------------------------------------I
Interpersonally
(Family, close relationships) I----------------------------------------------------------------------I
Socially
(Work, school, friendships) I----------------------------------------------------------------------I
Overall
(General sense of well-being) I----------------------------------------------------------------------I
Child Outcome Rating Scale (CORS)
Ages 6-12
Me
(How am I doing?) I------------------------------------------------------------------------------------I
Family
(How are things in my family?) I------------------------------------------------------------------------------------I
School
(How am I doing at school?) I------------------------------------------------------------------------------------I
Everything
(How is everything going?) I------------------------------------------------------------------------------------I
The ORS: The Bare Bones
• The ORS has no specific content other than the 4 domains —a skeleton to which clients add the flesh & blood of their experiences.
• At the moment clients connect the marks with what they find distressing, the ORS is a meaningful measure of progress & potent clinical tool.
Get a good rating
First Session: Connect ORS to the Client’s Described Experience Principles: •
Where is the distress? • Constructing meaning of the scores i.e.
Connect
the client’s described experience of their lives to the marks on the scales • Systemic questioning
Graphing: A Helpful Visual
• Adds a visual component — another way to focus tx • At a glance tells the story of the client’s progress.
• Indispensable with couples & families, helps discuss multiple viewpoints
40 35 30 25 20 15 10 5 Date 0
ORS/CORS Graph
CORS Cutoff (age 6-12)
child = 32 carer = 28
ORS Cutoff
Age 13-17= 28 Age 18 + = 25
Making the Measures Work with Families
• Invite all members of the family to Complete the ORS or CORS.
• Invite all members of the family to interpret.
• Summarize progress as entry point into the meeting.
15 Year Old ORS
7.6
• What stands out from this teens scores in addition to being above the clinical cut off which is 28?
5.7
8.0
7.5
28.8
• Suppose the mother’s collateral scoring of the teen is much lower than the teens self report scoring, how could we use this information to begin or focus the session?
Linking Outcome to Treatment
• See clients
more
frequently when the slope of change is steep.
• Begin to space the visits as the rate of change lessens.
• See clients as long as there is meaningful change & they desire to continue.
ORS/CORS Reliable Change Index • Reliable Change Index = 5
Reliable Deterioration 5 point or more worsening from pre-treatment score Reliable Change 5 point or more improvement from pre treatment score Clinically Significant Change!
5 point or more improvement from pre treatment score
and
crossed the cutoff score for age group
Alliance comprised of four empirically established components which the SRS Measures:
1. agreement on goals, meaning or purpose of treatment; 2. agreement on the means and methods used; 3.
agreement on the therapist’s role (including being perceived as warm, empathic and genuine; and 4.
accommodating the client’s preferences.
Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.),
The heart and soul of change 2nd ed.: Delivering what works in therapy (pp.113-141).
Washington, D.C.: American Psychological Association
Why is Monitoring the Alliance So Crucial?
• Researchers have repeatedly found that the therapeutic alliance as rated by the client, is one of the best predictors of outcome across different types of therapy including psychopharmocology.
• The feedback from the SRS helps clinicians make the necessary adjustments to align simultaneously with all family members in the pursuit of common agreement on goals and tasks in therapy.
Martin, D.J., Garske, J.P., & Davis, M.K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68 , 438-450.
Wampold, B.E. (2001).
The great psychotherapy debate: Models, methods, and findings.
Mahwah, NJ: Erlbaum.; Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.),
The heart and soul of change 2nd ed.: Delivering what works in therapy (pp.113-141).
Washington, D.C.: American Psychological Association.
But I Already Ask My Clients for Feedback, Why do I Need use an Alliance Measure?
• Research has shown that clinicians are poor at gauging their client’s experience of the alliance and they need to request more formal real time alliance feedback. • The key here is that people THINK they ask more often than they do. • Plus, and more importantly, since the data says it’s the client's point of view that is most predictive of outcome, you don't want to leave it up to the therapist to decide WHEN to ask.
Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.),
The heart and soul of change 2nd ed.: Delivering what works in therapy (pp.113-141).
Washington, D.C.: American Psychological Association. Anker, M.G., Duncan, B.L., Owen, J., & Sparks, J.A. (2010). The alliance in couple therapy: Partner influence, early change, ad alliance patterns in a naturalistic sample.
Journal Consulting and Clinical Psychology,
78(5), 635-645.
Lambert, M.J. (2010)
Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice.
Washington, DC: American Psychological Association.
Quickest Way to Prevent Drop Out
• Clients drop out for 2 reasons: therapy is not helping (monitor outcome) & alliance problems —not engaged or turned on. • Direct way to improve effectiveness is to keep people engaged in therapy.
• Gotta measure the alliance to improve your successes!
SRS & CSRS
Using the Client’s Experience of the Alliance to Guide Practice
• The SRS is administered at the end of each session to get feedback on the alliance and therapy process from young people and carers perspective. • Like the ORS there is a version for those 13 on up, and a younger version called CORS for 6-12 year olds.
• There is also a Group Version called GSRS.
Introducing the SRS/CSRS
“Ok, we need to end in a couple minutes, but before we do I would be grateful if you would take a minute to fill out this other form which asks your opinion about our work together today? I am not hoping for a perfect score: life is not perfect and I know I am not either. I rely on this feedback to keep me on track, and let me know when I am off track and need to make some changes to ensure that what we are doing is benefitting you. The ultimate purpose of using these forms is to make every possible effort to make our work together beneficial. There might be some small things that you think you might be able to live with or are not important, but your feedback on even the small things are critical to me as it can be the small things that make the difference. If anything feels missing, or I said something that did not sit right or left you feeling blamed, you would be doing me the best favor if you let me know. Can you help me out?
Ok. Here is one for each of you. Remember please feel free to give your honest opinion when filling this out. Ok?”
Session Rating Scale (SRS)
(Ages 13 to Adult)
Relationship
I did not feel heard, understood, and respected.
I-----------------------------------------------------------------------I I did feel heard understood, and respected.
We did
not
work on or talk about what I wanted to work on and talk about.
Goals and Topics
I------------------------------------------------------------------------I We did work on or talk about what I wanted to work on and talk about.
The therapist’s approach is not a good fit for me.
Approach or Method
I-----------------------------------------------------------------------I The therapist’s approach is a good fit for me.
There was something missing in the session today.
Overall
I------------------------------------------------------------------------I Overall, today’s session was right for me.
did not always listen to me.
What we did and talked about was not really that important to me
.
I did not like what we did today.
I wish we could do something different.
Child Session Rating Scale (CSRS)
Ages 6-12
Listening
listened to me.
I-----------------------------------------------I
How Important
I-----------------------------------------------I
What We Did
What we did and talked about were important to me.
I liked what we did today.
I-----------------------------------------------I
Overall
I-----------------------------------------------I I hope we do the same kind of things next time.
40 35 30 25 20 15 10 5 0 Date
SRS/CSRS Graph
SRS Cutoff =36
Discuss
SRS Scores Above the Cutoff Score: Keep Convey Negative Feedback is a Gift
• • • • • Keep encouraging client to let you know… No bad news on the SRS! Not a measure of competence or anything negative about you or the client. Negative Feedback is a gift from the client that helps you to be better.
Unless you really want it, you are unlikely to get it.
You won’t get it from everyone.
What About Below 36 or 9cm
• Scores < 36 or 9cms should be discussed.
• Don’t expect specifics or revelations —any feedback is a godsend • “Is there anything else I could have done, something I should have done more of or less of, some question or topic I should have asked, was there anything I said that did not sit right with you?”
Low SRS Scores Can be OK
Caution:
• Don’t assume a low SRS score means that there are alliance issues.
• The most effective therapists tend to have lower SRS scores and seem to be more skilled at generating honest feedback and using negative feedback to improve performance and outcomes • Key is to consider if the ORS/CORS scores are improving. If not, there may be a rupture or problem with the alliance.
Does Not Seem to be Improving
“Okay, so things haven’t changed since the last time we talked. How do you make sense of that?
Should we be doing something different here, or should we continue on course steady as we go?
If we are going to stay on the same track, how long should we go before getting worried? When will we know when to say ‘when’ and do something different?”
Using Session by Session Measures in Supervision
• The measures and the graphs bring the feedback and voice of the young person and carer directly into the supervisory session.
• Supervisors and supervisees can utilize the measures and graphs from a variety of cases to reflect on potential areas for professional growth and development.
How to Get the Measures Free
• The ORS/CORS and SRS/CSRS measures are licensed for member groups of CORC and CYP IAPT by Scott D. Miller. • Individual clinicians may also obtain a licence and the measures at no cost in English, and languages other than English, at www.scottdmiller.com
. • Note: When down loading the measures for the first time, you may need to adjust the analogue scales to make sure they are exactly 10cm. Beyond that adaptation the license agreement involves
no
alterations to the ORS/CORS and SRS/CSRS.
Further Help and Training in using the ORS/CORS and SRS/CSRS
•
Information on Training i
n how to integrate real time outcome and alliance feedback using the ORS & SRS to improve clinical effectiveness with young people and families contact: – David C. Low, Family and Systemic Psychotherapist, [email protected]
, 01473-280416 – Scott D. Miller, Ph.D., Director of the International Center for Clinical Excellence - [email protected]