Virtual arm for the Phantom Limb Pain Therapy

UMR 5205

-

Virtual arm for the Phantom Limb Pain Therapy

Eynard L. and Meyer A. and Bouakaz S.

2

The Phantom Pain

Pain or discomfort felt by an amputee in the area of the missing limb

Why? Physical Amputation ≠ Psychic amputation Mismatching between brain and reality → Pain

CharacteristicsStrong pain, sometimes debilitatingHard to treat

Existing therapiesMedics, massage, relaxation, psychotherapiesNew hopeful therapies …

3

News methods for therapy

[Ramachadran96] Cognitive Science Institute ISC Lyon [Sirigu03]

4

Our idea

Computer Science Posture Tracking (vision) Augmented reality: virtual limb

Our system Camera Posture analysis Augmented picture of missing limb Mirror screen

5

Presentation summary

Previous Works

Our contributionReal-Time Posture AnalysisThe mirror effect : Virtual arm

Results

Conclusion

6

Movements Tracking (vision)

Multi-camerasVoxelic reconstruction [Mikic03,…] 3D-model : recognition of the skeleton a 3d-grid

Monocular (1 camera)Optical flow Bayesian methods [Agarwal04,…]Real-Time [Stenger03,…]

Limited movementSmall resolution

7

Our system constraints

Ours constraints Real-Time (Interactivity) Monocular (portative system)

8

Presentation summary

Previous Works

Our contributionReal-Time Posture Analysis

Background substraction Precomputation: anthropometric measures Posture Tracking

Mirror effect : Virtual arm

Results

Conclusion

9

Background substraction

Learning-based algorithm N frames → RGB values extrema for each pixel Connexity search → noise reduction

Simple and fast

10

Precomputation: anthropometric measures

Anthropometric values computation Needed for the tracking

2 postures

Silhouette extraction

11

Head and Torso measurements

Pose 1: head measure Highest  pixel in silhouette

12

Head and Torso measurements

Pose 1: head measure Highest pixel in silhouette Going down into the image while Nbi(whitepixel)> Nbi-1(whitepixel)

13

Head and Torso measurements

Pose 1: head measure« highest » pixel in frameGoing down on the image while Nbi(whitepixel)>Nbi-1(whitepixel)

→ Head widthGoing down until Nb(whitepixel) < threshold* width

→ Shoulder line

14

Head and torso measurements

Torso measure Barycenter line → Torso width Going down until separation → bottom of the torso

→ Torso height

widthheight

15

Arm measurements

Pose 1Level and side of the amputation

Each arms sizes

Horizontal extreme points

16

Arm measurements (2)

Pose 2

lbrachium

H

h /2elbow

17

Presentation summary

Previous Works

Our contributionReal-Time Posture Analysis

Background substraction Precomputation: anthropometric measures Posture Tracking

Mirror effect : Virtual arm

Results

Conclusion

18

Head and Torso detection

Head and TorsoConnexity research algorithm helped by the

anthropometrics measures

19

Seek articulations

Seeking shoulder, elbow and wrist

Anthropometrics measurements → estimation of the positions

S

Sv

v~

E~

E

Forearmaxis

P1P2

C1

20

Missing limb creation

If before-elbow amputationCoordinates systems of valid and amputee side are equivalentReprojection of pixels valid side → amputee side

Sv S

a

Xs

Ys

Xa

Ya

A s

A a

copy

Ov

Oa

Ev

W

21

Virtual arm creation

If below-elbow amputationCoordinates system of valid and amputee side are equivalentReprojection of pixels valid side → amputee side

Sv S

a

XsYs

a

Ya

A s

Aa

copy

Ov

Oa

Ev

W

Ea

X

22

Results

Webcam (640*480)

About 8 to 10 frames / s

23

Remarks

Up Follow-up the side-displacements of the silhouette Whole image reflecting Better illusion for subject Interactive time

ConstrainsSubject must be facing the cameraFor a better illusion the elbows angles have to be

symmetricNon Snake moves

24

Conclusion and future works

Experimental systemSimple posture trackingConstrains but robustVirtual arm creation

Next stepExperiment on amputeeCollaboration with ISC of Lyon

Perspectives 3D-model to be more realistic Amputees reactions to real experiments

25

Thank you …

26

Ajout du membre → réalité augmentée (RA)

Problème générique en RA

Cohérence entre réel et virtuel Texture, aspect

Élément de synthèse construit à partir d’image réelle (cf notre système)

Illumination, Ombre

27

Creation du membre manquant

amputation huméralecentre de gravité du moignon obtention d'un axe grâce a al même méthode que pour le

bras valideamputation cubitaleMême méthode que pour le bras valideObtention de l'axe du moignon grâce au centre de gravité

de la partie après le coude

28

hhead

htorsolLarm

whead

w

lRarm

yneck

ypelvis

xmin

xmax

xRshoulder

xLshoulder

torso

29

30