Upload
verity-park
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
Towards a real-time, configurable, and affordable
system for inducing sensory conflicts in a virtual
environment for post-stroke mobility rehabilitation:
vision-based categorization of motion impairments
Babak Taati, Jennifer Campos, Jeremy Griffiths, Mona Gridseth, Alex Mihailidis
Sept 11, 2012ICDVRAT
Outline
Motivation
Background
Problem Definition
Tracking Technologies
Method
Experimental Results
Conclusions & Future Work
2
3
Post-Stroke Rehabilitation
15 million people suffer stroke each year
65% of stroke survivors have difficulty using their upper limbs
The economic cost of stroke is ~$6.3 billion a year
Training exercises that provide patients with visual feedback on the movement of the affected limb facilitate rehabilitation
Motivation
Mirror Box Illusion
View of the affected arm is replaced by the mirror reflection of the healthy arm
Simultaneous motion, or attempt of motion of both arms results in artificial visual feedback which reinforces neurorehabilitation
Mirror box therapy is effective in restoring mobility
4
Background
http://en.wikipedia.org/wiki/Mirror_box
(Ramachandran et al, Nature,1995)
Rubber Hand Illusion
5
http://www.thefrogwhocroakedblue.com/illusions.asp
(Botvinick and Cohen, Nature, 1998)
Simultaneous brushing of the rubber hand and the real hand generates a sensory conflict in the brain between the location of the visual vs. proprioceptive signal
Continued brushing typically results in a sensory takeover of proprioception and the person would feel ownership of visually perceived plastic arm
Background
Objective
Reproduce and combine the Mirror Box and the Rubber Handillusions in a virtual environment
• Computer vision: markerless skeleton tracking
• Computer graphics: visualization + pose augmentation
• Machine learning: classification + latent space projection
6
I felt my arm was here …
… but I see it here!
Problem Def.
Milestones
Duplicate the Rubber Hand Effect
• Visual skeleton tracking & visualization
• Synchronized tactile stimulation
Combine the Rubber Hand with the Mirror Box Effect
• Categorize motion impairments
• Apply a movement gain (normalize / exaggerate)
7
Problem Def.
8
Human Pose Tracking
8
[Kanaujia, 2011]
Tracking Tech.
9
Human Pose Tracking (cont’d)
Marker-based• Elaborate setup
Markerless (vision-based)• Accuracy vs. computational efficiency
9
[CMU Motion Capture Database]
[Flickr: beratus]
[Lee & Nevatia, 2009]
Tracking Tech.
10
Consumer Depth Cameras
10
KinectTM (Microsoft) WAVI Xtion (ASUS / PrimeSense)
Real-time color & depth sensing
Real-time skeleton tracking software libraries
• Microsoft SDK or NITE
Tracking Tech.
11
Challenges
Noise / inaccuracies / systematic bias Missing information Ground truth annotation Evaluation of categorical time series prediction
11
Tracking Tech.
True Labels
Predicted Labels
12
Dataset
7 healthy adults
Simulated 2 of the most common upper limb flexion synergies present in the post-stroke population
• Elbow flexion, shoulder flexion, shoulder abduction
Stereotypical post-stroke movement synergy: Rotating the Trunk
• Elbow flexion
Stereotypical post-stroke movement synergy: Elevating the Shoulder
10x normally, 10x simulating an impaired motion
Method
13
Algorithms
Features• Displacement Vectors• Principal Components Analysis (PCA)
Multi-class categorization• Logistic Regression• Support Vector Machines (SVM)• Random Forest (RF)
Cross validation• 7-fold leave-one-subject-out
Method
Dominant Modes of Motion 1
14
Results
Reaching Across(Elbow flexion, shoulder flexion, shoulder abduction)
With shoulder elevation“Normal”
Dominant Modes of Motion 2
15
Results
Reaching Up(Elbow flexion)
With trunk rotation“Normal”
Classification Accuracy
ClassifierAction N Logistic Reg. SVM RF
Reach Across1 54.3 97.1 97.93 61.4 85.7 97.1
Elbow Flexion1 64.3 95.7 90.03 59.3 85.0 92.9
16
Results
# of PCA components
17
Summary
Inherent biases were observed in the tracking of an elevated shoulder
Despite accuracy and tracking bias issues, it was possible to separate impaired motions
A single most dominant mode of motion, as identified by PCA, was sufficiently discriminative
SVM classifier obtained consistent detection accuracies >95%
Conclusions
18
Future Work
Exaggerate / normalize motions
• Latent space projection
Combine the Rubber Hand and Mirror Box illusions
Real data (real post-stroke patients)
• Transfer learning
Conclusions
19
Acknowledgements
Toronto Rehab (iDAPT research facilities)
20
Home Environment Laboratory (HEL) Challenging Environment
Assessment Laboratory (CEAL)