Transcript Adaptive Rehabilitation using Mixed-reality at Home: The ARM at Home study RIC Margaret Duff, Meghan Buell, and W.

Adaptive Rehabilitation using Mixed-reality
at Home: The ARM at Home study
RIC
Margaret Duff, Meghan Buell, and W. Zev Rymer
Emory Steven Wolf and Aimee Reiss
ASU
Pavan Turaga, Nicole Lehrer, Michael Baran, Vinay
Venkataraman, Loren Olson and Todd Ingalls
CMU Thanassis Rikakis
Adaptive mixed reality rehabilitation
- Computational assessment of movement
- Abstract audio and visual feedback, evaluation and
adaptation
- Tangible sensing objects provide functional goals
- Increase engagement and enhance motor learning
through self-assessment of movement
- Recover pre-morbid movement patterns and reduce
compensation while increasing function
Evaluating outcomes of mixed reality
compared to traditional therapy
Completed at Banner Baywood Medical Center
AMRR improves function and
kinematics
-
Both groups improved in the Wolf Motor Function Test
- Every AMRR participant saw at least a 30% improvement in
composite kinematic impairment measure (KIM), with a
much more consistent distribution of improvement
Control group
% change
% change
AMRR group
*
Issues to address
- Neither group reported a significant change in
impaired arm use / quality in ADLs
- Long-term plan to both encourage functional
and movement quality improvements
- Continue therapy at home and with a greater
variety of tasks
Scaling AMRR theories for home therapy
Home AMRR system
- An engaging therapy environment at home
- Task repetition, variability and intensity
- Easy to use and understand in a largely
unsupervised environment
- Useful information (feedback) about task
completion and movement quality
Feedback examples
Pilot study of unsupervised training
- Test feasibility and effectiveness
- Examine how people with stroke use and
accept the system
- Determine what further work is needed to
accommodate the needs of the greatest
percentage of people
Study protocol
- 1 week (3 sessions) of supervised training
- 4 weeks (12 sessions) of unsupervised
training
- Pre, post, and 4 week follow-up evaluations
Participant demographics
N=6
(6M, 0 F)
Age (years)
Months post stroke
Fugl-Meyer (/66)
Median
Range
59.5
49 - 69
27
14 - 44
48.5
37 - 55
Wolf Motor Function Test
WMFT Total time (affected)
5.0
4.0
3.0
Pre
2.0
Post
1.0
Followup
0.0
1
2
3
Subject
4
5
Total time (seconds)
Mean FAS Score (/5)
WMFT Mean FAS
60
50
40
30
20
10
0
Pre
Post
Followup
1
2
3
4
5
Subject
Both FAS and time improve after therapy and
are mostly retained at follow-up
Fugl-Meyer and Motor Activity Log
70
60
50
40
30
20
10
0
MAL Quality of Movement Mean
Score
Pre
Post
Follow up
1
2
3
Subject
4
5
Mean score (/5)
Total Score (/66)
Fugl-Meyer Total Score
5
4
3
2
1
0
Pre
Post
Followup
1
2
3
4
5
Subject
FM scores improve after therapy and are
retained at follow-up
MAL scores are inconsistent after therapy and
at follow-up
Kinematic results of trained task
Velocity peak trained to about .6 m/s
Inconsistent changes in horizontal trajectory
Participant acceptance of system
Preliminary outcomes
- System was stable throughout
- Successful unsupervised training
- FM and WMFT improved after training
- Kinematics and MAL were inconsistent
Current and future work
- Improved hand function sensing
- Track ADLs objectively and transfer therapy
gains to everyday
- Increased adaptability of therapy protocols
- Better classification of movement impairments
Hand function sensing
More adaptive therapy protocols
Current Protocol
- Two set therapy tracks
Future Considerations
- Progression based on ability
- Objects that vary more in complexity and
weight
- Variability within a set of reaches
- Dissociate objects from table
Assessing the classifiers
Problem - building high level metrics of efficiency for
complex movements with reduced sensing
Assessing new metrics for classifying movement
- Correlation to kinematic assessment of simple tasks
- Therapist ratings of simple to complex tasks, each rated
in terms of overall performance and component
performance
- Components that are being trained on do not have oneto-one mapping with therapist ratings, which implies no
supervised training data to build classifiers
- But there is weak supervision !
Kinematic classifiers that drive
feedback
Curvedness – Measure of spatial error
Too Fast / Too Slow – Measure of
deviation in velocity profile
Smoothness – Measure of variability
in velocity profile
Therapist rated tasks
Video recordings of 3 tasks (5 trials each), performed
once a week independent of therapy
Treating therapist rated each reach, presented randomly
Cone grasp
(simple, trained)
Elevated touch
(simple, semi-trained)
Transport cylinder
(complex, trained)
Therapist rated tasks
Rating system was developed to assign a score for
each of the following:
1.
2.
3.
4.
5.
6.
7.
Initial impression of overall trial (Modified FAS)
Trajectory
Compensation
Hand manipulation (grasp, touch)
Transport phase (if transport task)
Release phase (if transport task)
Final impression of overall trial (Modified FAS)
If needed, explanation recorded if final impression
is different than initial impression
How does therapist rating help in
tuning these classifiers?
Assume a linear model of kinematic classifiers
W1
W2
W3
W4
Cumulative
Classifier Score
Movement quality assessment
Therapist Rating (R)
W1F1(T1) + W2F2(T2) + W3F3(T3) + W4F4(T4) + noise = {Initial impression
of overall score}
Cost function =
Use Nelder-Mead’s Simplex (fminsearch algorithm
in MATLAB) to perform optimization
Initial Results
– 3 participants (mild impairment) recorded at
Emory
– 4 video recorded sessions each
– Total of 55 reaches to grasp the cone
Initial Results
Before optimization: Observe the overlap in score distributions, which
implies classifiers are not tuned properly
Means are
close
4
5
Initial Results
Optimizing only the combination weights: The score distribution overlap
does not get affected, suggesting that the problem really lies with the
classifiers
4
5
Initial Results
After optimization of weights and thresholds: score distribution overlap
reduces
Reduced
overlap
4
5
Conclusions
- Changes in therapy protocols and tasks
needed to benefit a larger subset of the
population
-
Movement classifiers need to be
generalized and improved, while staying
accurate
-
Monitor and encourage transfer of therapy
strategies to everyday life
Thank You!!!