The Joy of Outcome Measurements
Download
Report
Transcript The Joy of Outcome Measurements
The Joy of Outcome
Measures
Simon Hall
Manager
of Assistive Technology/Seating
Mobility
Central Remedial Clinic
Ireland
History of Outcome Measures
in Ireland
In 1783 Arthur Guinness started to brew the
perfect pint
His customers developed their own outcome
measure which is used worldwide
Known as the perfect pint
The Pain of Outcome
Measures
Perception that outcome measures are
– Developed by Academics and Researchers
– Enforced by Management
– Suffered by Clinicians
True or False?
If this is method used to implement outcome
measurement
THE PROCESS WILL FAIL!
People focus on the pain rather
than the gain in using outcome
measures
EBP
Evidence Based Practice
EBP
Evidence based pratice is how
clinicans/researchers objectively provide
evidence through scientific means on the
outcome (positive and /or negative) of their
intervention/s with their client group/s
Newton's Third Law
For every action there is an Equal
and Opposite Reaction
We can measure forces easily
Everything we do has an outcome,
positive or negative
To manage a reliable service we need to
measure the Service provision
How we measured Outcome
Standards in the past
Complaints to the Service
Level of Referrals to the Service
Funding to the Service
Staff turnover in the Service
Growth / expansion of the Service
CRC Centres
Donegal
Monaghan
Sligo
Cavan
Louth
Galway
Meath CRC Clontarf
CRC Scoil Mochua
Midlands
CRC Limerick
CRC Waterford
Kerry
Cork
Galway
Difficulties Introducing
Outcome Measures
CRC Clinical Case Load
Mixed Adult and Children with Acquired
and Congenital Disabilities
Physically and Intellectually Impaired
clients
Sensory Impaired clients
Verbal and Non Verbal clients
Why we introduced Outcome
Measures
Service Expanding
Staff increasing
Transdiciplinary staffing
Multi grades of staff
AT / Specialised seating merging
Central Remedial Clinic
ATSS Department
To introduce outcome measures
we had to have in place:
– Structures
– Systems
– Processes
– Staffing
– Training
Introduction Process
July 2008 -2 days dedicated to Outcome Measures
1st day-meeting with managers and experts working in Assistive
Technology and Seating
Dublin, Belfast, London, USA, Wales
List of tools being used currently clinically/research
2nd day-learn about them and select measures to trial.
3 month trial August –October 2008-feedback from staff
Trial August-October 2008
One per month-One per day @ 2.00pm Ax slot
CRC South Dublin, Limerick and Clontarf
All tools used filed and clients tracked until review
3 Tools used-WhOM, FEW, Quest, (IMPT, EATs)
Feedback from (14) clinicians-therapists & engineers
Meeting for review and feedback October
Measures
Studied
WhOM- Wheelchair Outcome Measure
FEW- Functioning Everyday with a Wheelchair
WST-Wheelchair Skills Test
PIADS-Psychosocial Impact of Assistive Devices Scale
Quest-Quebec User Evaluation of Satisfaction with Assistive
Tenchology
MPT/IMPT Matching Person with Technology/Irish MPT
SPCM-Seated Postural Control Measure
Functional Independence Scale for Haemophiliacs
Chailey Levels of Sitting Ability
COPM-Canadian Occupational Performance Measure
GAS-Goal Attainment Scale
EATs-Efficiency of Assistive Technology and Services
CAS-Compass Assessment Software
WhOM
Two part questionnaire which can be administered in under 30 mins-based
on the ICF
Part I participation goals in the home and in the community identified. The
client rates perceived ‘importance’ of this goal and ‘satisfaction’ with their
current performance of this activity. (0-10) IxS=score
Part II the client answers three questions on comfort, positioning and skin
breakdown and results are recorded on the scoring sheet
Intervention planned-eg changes to wheelchair, or seating system, the
environment, or client education
Following intervention the WhOM is re-administered
Pre and post scores compared to calculate an outcome score.
WhOM
Not validated for use with paediatric clients
Too broad; difficulty coming up with goals/activities
Clients found it dificult picking a number (1-10) for
rating level of importance/satisfaction
Clients went for extremes of points on the scale
Wheelchair outcomes as opposed to AT specifically
Client centred, quick and easy to use
Can be used for all client groups
Method of rating level of importance of activities
useful for priority setting of goals
Functioning Everyday in a
Wheelchair
Self report questionnaire (ICF coded)
10 consumer generated self report items-a mix of body
function, activity and participation and environmental
elements
Scored using a 6 point scale of 6=completely agree to
1=completely disagree and a 0 score of does not apply
For slightly disagree, mostly disagree and completely
disagree ‘comments section’ for clients to describe the
features of the wheelchair system contributing to their
disagreement.
The 10 FEW items
Stability,
dependability and
durability
Comfort
Health needs
Operate
wheelchair/scooter
Reach and carry out
tasks at different
surface heights
Transfers
Personal care tasks
Indoor mobility
Outdoor mobility
Personal/public
transportation
FEW
Not suitable for complex clients, eg severe
CP
Not validated for Paediatrics
So much covered in each question it reduces
the sensitivity of the scale
Very wordy complex questions for some
clients
Repititive
Specific to wheelchair use as opposed to
technology in general
FEW
Focused on all aspects central to wheelchair
mobility
Quick and easy to administer and score
Excellent for more independent wheelchair usersmanual and powered
Supportive literature easily available
Get a good idea of the environment, ADLs and
how well mobility device is working
Can get a lot of useful information from the
questions/comments
Easy to show improvements in specific areas
Quebec User Evaluation of
Satisfaction with Assistive
Technology
Evaluates how satisfied the client is with their assistive
device and the related services they have experienced
12 items, 2 Sections rating satisfaction using a Likert scale
of 1-5
Add the ratings of the valid responses and divide the sum
by the number of valid items in the scale
Second section for clients to decide on 3/12 items of what
is most important to them eg safety, comfort, follow-up
services
Quest
Well structured with simple language and scale
Addresses all technology not just wheelchairs
It covers most of the essential aspects of an AT device
Highlights satisfaction with service provision
Review questionnaire post intervention
Didn’t cover enough aspects of clients’ needs eg
posture/pressure
Doesn’t take account of seating system as opposed to
wheelbase
Service provision-issues beyond our control repairs,
sanctioning times etc
Clients found it hard to pick 3/12 areas to prioritise goals
IMPT (3) Comprehensive but takes some
time. Research or specific client groups
EATs (2) Was not used correctly. Meant to
be used after a detailed assessment, pre and
post supply or trial of equipment. Not
before assessment has taken place.
Collaborative Approach
Therapists and Engineers involved from the start
Support from Admin staff and Research Coordinator
Management
Overall project co-ordinator
Collaboration with experts working in the field of
Assistive Technology and Seating
Assistive Technology
Outcomes
Clinical Results
Functional Results
Quality of Life
Consumer Satisfaction
Cost Factors
DeRuyter 1988
Outcome Measures should
prove your service is:
Effective
Efficient
High Quality
Client Centred
Evidence Based
Staff Effect
Problems finding the right Tools
Most Tools developed are for Adults with
language for Adults
More paperwork
Big Brother effect
What's the Gain?
Confusing at times - what was been measured
The Main Reasons (3 F’s)
Feedback - improve communication
between staff, clients and carers
Funding - we can prove the service is
working
Future - training needs, staffing needs,
trends etc
General feedback from staff
We need to have a strong activities-based questionnaire
as I found many of our clients weren’t able to independently
identify areas that their performance was restricted in, as they
were so accustomed to those restrictions.
Need more comprehensive examples of functional
performance measures (without being so exhaustive a list
as to take ages to administer)
Generally need to consider non-verbal or A.A.C
users- ? symbol based rating scales or symbol charts to
support comprehension?
There is a need for a tool that is validated for use with
children and complex clients
Where are we Now/Where are
we Going
Continue using outcomes as part of our
assessment process
Continue collaboration and research in the
development of new paediatric tools
Staff are more committed to Outcomes,
they see the benefits and the connection
between measurement and funding /
development of the service
The End