UMR Virtual arm for the Phantom Limb Pain Therapy Eynard L. and Meyer A. and Bouakaz S.

2 The Phantom Pain Pain or discomfort felt by an amputee in the area of the missing limb Why? Physical Amputation ≠ Psychic amputation Mismatching between brain and reality → Pain Characteristics Strong pain, sometimes debilitating Hard to treat Existing therapies Medics, massage, relaxation, psychotherapies New hopeful therapies …

3 News methods for therapy [Ramachadran96] Cognitive Science Institute ISC Lyon [Sirigu03]

4 Our idea Computer Science  Posture Tracking (vision)  Augmented reality: virtual limb Our system  Camera  Posture analysis  Augmented picture of missing limb  Mirror  screen

5 Presentation summary Previous Works Our contribution  Real-Time Posture Analysis  The mirror effect : Virtual arm Results Conclusion

6 Movements Tracking (vision) Multi-cameras  Voxelic reconstruction [Mikic03,…]  3D-model : recognition of the skeleton a 3d-grid Monocular (1 camera)  Optical flow  Bayesian methods [Agarwal04,…]  Real-Time [Stenger03,…]  Limited movement  Small resolution

7 Our system constraints Ours constraints  Real-Time (Interactivity)  Monocular (portative system)

8 Presentation summary Previous Works Our contribution  Real-Time Posture Analysis  Background substraction  Precomputation: anthropometric measures  Posture Tracking  Mirror effect : Virtual arm Results Conclusion

9 Background substraction Learning-based algorithm  N frames → RGB values extrema for each pixel  Connexity search → noise reduction  Simple and fast

10 Precomputation: anthropometric measures Anthropometric values computation  Needed for the tracking 2 postures Silhouette extraction

11 Head and Torso measurements Pose 1: head measure  Highest pixel in silhouette

12 Head and Torso measurements Pose 1: head measure  Highest pixel in silhouette  Going down into the image while Nb i (whitepixel)> Nb i-1 (whitepixel)

13 Head and Torso measurements Pose 1: head measure  « highest » pixel in frame  Going down on the image while Nb i (whitepixel)>Nb i-1 (whitepixel) → Head width  Going down until Nb(whitepixel) < threshold* width → Shoulder line

14 Head and torso measurements Torso measure  Barycenter line → Torso width  Going down until separation → bottom of the torso → Torso height width height

15 Arm measurements Pose 1  Level and side of the amputation  Each arms sizes Horizontal extreme points

16 Arm measurements (2) Pose 2

17 Presentation summary Previous Works Our contribution  Real-Time Posture Analysis  Background substraction  Precomputation: anthropometric measures  Posture Tracking  Mirror effect : Virtual arm Results Conclusion

18 Head and Torso detection Head and Torso  Connexity research algorithm helped by the anthropometrics measures

19 Seek articulations Seeking shoulder, elbow and wrist Anthropometrics measurements → estimation of the positions

20 Missing limb creation If before-elbow amputation  Coordinates systems of valid and amputee side are equivalent  Reprojection of pixels valid side → amputee side

21 Virtual arm creation If below-elbow amputation  Coordinates system of valid and amputee side are equivalent  Reprojection of pixels valid side → amputee side

22 Results Webcam (640*480) About 8 to 10 frames / s

23 Remarks Up  Follow-up the side-displacements of the silhouette  Whole image reflecting  Better illusion for subject  Interactive time Constrains  Subject must be facing the camera  For a better illusion the elbows angles have to be symmetric  Non Snake moves

24 Conclusion and future works Experimental system  Simple posture tracking  Constrains but robust  Virtual arm creation Next step  Experiment on amputee  Collaboration with ISC of Lyon Perspectives  3D-model to be more realistic  Amputees reactions to real experiments

25 Thank you …

26 Ajout du membre → réalité augmentée (RA) Problème générique en RA Cohérence entre réel et virtuel  Texture, aspect  Élément de synthèse construit à partir d’image réelle (cf notre système)  Illumination, Ombre

27 Creation du membre manquant amputation humérale  centre de gravité du moignon  obtention d'un axe grâce a al même méthode que pour le bras valide  amputation cubitale  Même méthode que pour le bras valide  Obtention de l'axe du moignon grâce au centre de gravité de la partie après le coude

28

29

30