Transcript Critical Appraisal of Methods of Measuring Outcomes in AAC

Sally Boa and Joan Murphy
Funded by NHS Education Scotland
Conducted by Talking Mats Limited
Communication Matters outcome
measurement project
Aims:
“To provide information to service
providers and users in order to equip them with
knowledge about tools that are appropriate to
identify change associated with services and an
individual’s communication skills”
Points arising
• Confusion between assessment and outcome
measurement
• Lots of Outcome Measures and assessments
being used
• Why do we collect data?
– To improve our individual practice
– To provide feedback to others
Literature review
• Methods
• Review questions and data extraction
• Findings
Methods
• Rapid Review (Khangura et al 2012)
– “streamlined approach to synthesising evidence”
– Systematic search but a quick overview of existing
research on a topic
Review questions
• Which Outcome Measures (OM) are
used in Health, Education and Social
Services?
• What are the purposes of the
measures?
• How do these relate to AAC?
• Are there any gaps in the literature?
Papers screened
• Google scholar, ASSIA,
Cochrane reviews
– Total: 5121
Full papers read
• Google scholar: 5
• ASSIA: 109
• Cochrane reviews: 31
Data extracted
• Author; Year; Country; Population; OM
• Outcome measures mentioned in papers
– Total number: 72
• Categorisation:
– Global/disease specific
– Measuring what?
– Valid and reliable?
– Application to AAC?
Global outcome measures
•
•
•
•
•
•
•
•
•
•
AUSTOMS
Barthel
COPM
EQ5D
Expanded Disability status scale
FIM/FAM
Frenchay Activity Index
GAS
Nottingham Health Profile
TOMS
Other measures
• Disease specific
– Functional Assessment in Multiple Sclerosis (FAMS)
– Aphasia Communication Outcome Measure
• Age specific:
– Gross Motor Function Measure (Paediatric)
– Mental Status Questionnaire (older people)
• Aimed at patients
– COSSS Children's Overall Satisfaction with Schooling Scale
– measure yourself medical outcome profile (MYMOP)
• Aimed at carers
– Carers of Older People in Europe (COPE) Index
– Family Impact of Childhood Disability measure
Things to consider
• Validity and reliability of measures
• What are they measuring?
• Are they relevant to AAC?
Survey
•
•
•
•
•
•
Survey Monkey
Email
Twitter
Scotland, UK, International
Friday 1st – 8th March
Total responses 153
Sector
Geographical location
Profession
Extent of knowledge
Purpose
Outcome measure used
Numbers
None used
58
Care Aims
9
TOMs
9
GAS
3
Talking Mats
3
CODES Framework
2
EFFC
2
Functional Communication
2
AFROM
Advocacy projects Outcomes
1
Outcome measure used
Numbers
Carer feedback
1
East Kent Outcome Scale
1
Emotion Talks
1
Goal setting
1
In house
1
Sarah Blackstone’s Circles
1
Sign List
1
Talking Points
1
VASES
1
WAB
1
Frequency of use
Value
Why do we need Outcome Measures?
• I am not sure what an outcome measure is...
• Demonstrates the value of interventions
• ....
Strengths
• It allows therapists to be outcome led and aim
for functional, achievable goals that are
personal to the patients
• It provides consistent approach across whole
caseload
• Allows us to check as a multi agency team that
we are all assessing clients similarly within
reasonable limits
Limitations
• No way of demonstrating outcomes with different
types of AAC
• Difficult to share with service users
• Not formally evidence based or standardised. Difficult
to prove effectiveness if measured against others
• Too formal
• Not consistently used in social settings
• More difficult as intervention for higher tech users
usually cuts across a multi disciplinary/agency setting
where it is harder to agree and measure outcomes
Expert panel days
• Presentation of literature review and survey
• What is an Outcome Measure?
• Directions for future work
Main points arising from expert panel
days
• Agreed on what we mean by ‘Outcome
Measure’
• Agreed on what is the best measure to use in
AAC
• Identified the need for a measure to be
workable across agencies
Definition of an outcome measure
Outcome Measures should:
Measure change
Are repeated over time
Can be used across clients, settings and disciplines
Can be used to give feedback to services, professionals, carers and
clients (about what works as well as what doesn’t)
Are robust/standardised/well respected
Inform discharge
Are short and straightforward to use
Allow for comparison with different types of AAC/No AAC
Main contender for outcome measure
in AAC
• TOMs AAC
• (still in development – needs to be tested for
reliability and validity)
Final report available on our website
www.talkingmats.com