Transcript The Joy of Outcome Measurements

The Joy of Outcome
Measures
Simon Hall
Manager
of Assistive Technology/Seating
Mobility
Central Remedial Clinic
Ireland
History of Outcome Measures
in Ireland

In 1783 Arthur Guinness started to brew the
 perfect pint
 His customers developed their own outcome
measure which is used worldwide
 Known as the perfect pint
The Pain of Outcome
Measures

Perception that outcome measures are
– Developed by Academics and Researchers
– Enforced by Management
– Suffered by Clinicians
True or False?
If this is method used to implement outcome
measurement
THE PROCESS WILL FAIL!
People focus on the pain rather
than the gain in using outcome
measures
EBP
Evidence Based Practice
EBP
Evidence based pratice is how
clinicans/researchers objectively provide
evidence through scientific means on the
outcome (positive and /or negative) of their
intervention/s with their client group/s
Newton's Third Law
For every action there is an Equal
and Opposite Reaction

We can measure forces easily
 Everything we do has an outcome,
positive or negative
 To manage a reliable service we need to
measure the Service provision
How we measured Outcome
Standards in the past

Complaints to the Service
 Level of Referrals to the Service
 Funding to the Service
 Staff turnover in the Service
 Growth / expansion of the Service
CRC Centres
Donegal
Monaghan
Sligo
Cavan
Louth
Galway
Meath CRC Clontarf
CRC Scoil Mochua
Midlands
CRC Limerick
CRC Waterford
Kerry
Cork
Galway
Difficulties Introducing
Outcome Measures

CRC Clinical Case Load
 Mixed Adult and Children with Acquired
and Congenital Disabilities
 Physically and Intellectually Impaired
clients
 Sensory Impaired clients
 Verbal and Non Verbal clients
Why we introduced Outcome
Measures

Service Expanding
 Staff increasing
 Transdiciplinary staffing
 Multi grades of staff
 AT / Specialised seating merging
Central Remedial Clinic
ATSS Department

To introduce outcome measures
we had to have in place:
– Structures
– Systems
– Processes
– Staffing
– Training
Introduction Process
July 2008 -2 days dedicated to Outcome Measures
1st day-meeting with managers and experts working in Assistive
Technology and Seating
Dublin, Belfast, London, USA, Wales
List of tools being used currently clinically/research
2nd day-learn about them and select measures to trial.
3 month trial August –October 2008-feedback from staff
Trial August-October 2008

One per month-One per day @ 2.00pm Ax slot

CRC South Dublin, Limerick and Clontarf

All tools used filed and clients tracked until review

3 Tools used-WhOM, FEW, Quest, (IMPT, EATs)

Feedback from (14) clinicians-therapists & engineers

Meeting for review and feedback October













Measures
Studied
WhOM- Wheelchair Outcome Measure
FEW- Functioning Everyday with a Wheelchair
WST-Wheelchair Skills Test
PIADS-Psychosocial Impact of Assistive Devices Scale
Quest-Quebec User Evaluation of Satisfaction with Assistive
Tenchology
MPT/IMPT Matching Person with Technology/Irish MPT
SPCM-Seated Postural Control Measure
Functional Independence Scale for Haemophiliacs
Chailey Levels of Sitting Ability
COPM-Canadian Occupational Performance Measure
GAS-Goal Attainment Scale
EATs-Efficiency of Assistive Technology and Services
CAS-Compass Assessment Software
WhOM

Two part questionnaire which can be administered in under 30 mins-based
on the ICF

Part I participation goals in the home and in the community identified. The
client rates perceived ‘importance’ of this goal and ‘satisfaction’ with their
current performance of this activity. (0-10) IxS=score

Part II the client answers three questions on comfort, positioning and skin
breakdown and results are recorded on the scoring sheet

Intervention planned-eg changes to wheelchair, or seating system, the
environment, or client education

Following intervention the WhOM is re-administered

Pre and post scores compared to calculate an outcome score.
WhOM






Not validated for use with paediatric clients
Too broad; difficulty coming up with goals/activities
Clients found it dificult picking a number (1-10) for
rating level of importance/satisfaction
Clients went for extremes of points on the scale
Wheelchair outcomes as opposed to AT specifically
Client centred, quick and easy to use
 Can be used for all client groups
 Method of rating level of importance of activities
useful for priority setting of goals
Functioning Everyday in a
Wheelchair

Self report questionnaire (ICF coded)

10 consumer generated self report items-a mix of body
function, activity and participation and environmental
elements

Scored using a 6 point scale of 6=completely agree to
1=completely disagree and a 0 score of does not apply

For slightly disagree, mostly disagree and completely
disagree ‘comments section’ for clients to describe the
features of the wheelchair system contributing to their
disagreement.
The 10 FEW items





Stability,
dependability and
durability
Comfort
Health needs
Operate
wheelchair/scooter
Reach and carry out
tasks at different
surface heights





Transfers
Personal care tasks
Indoor mobility
Outdoor mobility
Personal/public
transportation
FEW

Not suitable for complex clients, eg severe
CP
 Not validated for Paediatrics
 So much covered in each question it reduces
the sensitivity of the scale
 Very wordy complex questions for some
clients
 Repititive
 Specific to wheelchair use as opposed to
technology in general
FEW







Focused on all aspects central to wheelchair
mobility
Quick and easy to administer and score
Excellent for more independent wheelchair usersmanual and powered
Supportive literature easily available
Get a good idea of the environment, ADLs and
how well mobility device is working
Can get a lot of useful information from the
questions/comments
Easy to show improvements in specific areas
Quebec User Evaluation of
Satisfaction with Assistive
Technology

Evaluates how satisfied the client is with their assistive
device and the related services they have experienced

12 items, 2 Sections rating satisfaction using a Likert scale
of 1-5

Add the ratings of the valid responses and divide the sum
by the number of valid items in the scale

Second section for clients to decide on 3/12 items of what
is most important to them eg safety, comfort, follow-up
services
Quest









Well structured with simple language and scale
Addresses all technology not just wheelchairs
It covers most of the essential aspects of an AT device
Highlights satisfaction with service provision
Review questionnaire post intervention
Didn’t cover enough aspects of clients’ needs eg
posture/pressure
Doesn’t take account of seating system as opposed to
wheelbase
Service provision-issues beyond our control repairs,
sanctioning times etc
Clients found it hard to pick 3/12 areas to prioritise goals

IMPT (3) Comprehensive but takes some
time. Research or specific client groups

EATs (2) Was not used correctly. Meant to
be used after a detailed assessment, pre and
post supply or trial of equipment. Not
before assessment has taken place.
Collaborative Approach

Therapists and Engineers involved from the start

Support from Admin staff and Research Coordinator

Management

Overall project co-ordinator

Collaboration with experts working in the field of
Assistive Technology and Seating
Assistive Technology
Outcomes

Clinical Results
 Functional Results
 Quality of Life
 Consumer Satisfaction
 Cost Factors
DeRuyter 1988
Outcome Measures should
prove your service is:

Effective

Efficient

High Quality

Client Centred

Evidence Based
Staff Effect

Problems finding the right Tools

Most Tools developed are for Adults with
language for Adults

More paperwork

Big Brother effect

What's the Gain?

Confusing at times - what was been measured
The Main Reasons (3 F’s)

Feedback - improve communication
between staff, clients and carers
 Funding - we can prove the service is
working
 Future - training needs, staffing needs,
trends etc
General feedback from staff

We need to have a strong activities-based questionnaire
as I found many of our clients weren’t able to independently
identify areas that their performance was restricted in, as they
were so accustomed to those restrictions.

Need more comprehensive examples of functional
performance measures (without being so exhaustive a list
as to take ages to administer)

Generally need to consider non-verbal or A.A.C
users- ? symbol based rating scales or symbol charts to
support comprehension?

There is a need for a tool that is validated for use with
children and complex clients
Where are we Now/Where are
we Going

Continue using outcomes as part of our
assessment process
 Continue collaboration and research in the
development of new paediatric tools
 Staff are more committed to Outcomes,
they see the benefits and the connection
between measurement and funding /
development of the service
The End