1. UMR 5205 - Virtual arm for the Phantom Limb Pain Therapy Eynard L. and Meyer A. and Bouakaz S.

2. 2 The Phantom Pain Pain or discomfort felt by an amputee in the area of the missing limb Why? Physical Amputation ≠ Psychic amputation Mismatching between brain and reality → Pain Characteristics Strong pain, sometimes debilitating Hard to treat Existing therapies Medics, massage, relaxation, psychotherapies New hopeful therapies …

3. 3 News methods for therapy [Ramachadran96] Cognitive Science Institute ISC Lyon [Sirigu03]

4. 4 Our idea Computer Science  Posture Tracking (vision)  Augmented reality: virtual limb Our system  Camera  Posture analysis  Augmented picture of missing limb  Mirror  screen

5. 5 Presentation summary Previous Works Our contribution  Real-Time Posture Analysis  The mirror effect : Virtual arm Results Conclusion

6. 6 Movements Tracking (vision) Multi-cameras  Voxelic reconstruction [Mikic03,…]  3D-model : recognition of the skeleton a 3d-grid Monocular (1 camera)  Optical flow  Bayesian methods [Agarwal04,…]  Real-Time [Stenger03,…]  Limited movement  Small resolution

7. 7 Our system constraints Ours constraints  Real-Time (Interactivity)  Monocular (portative system)

8. 8 Presentation summary Previous Works Our contribution  Real-Time Posture Analysis  Background substraction  Precomputation: anthropometric measures  Posture Tracking  Mirror effect : Virtual arm Results Conclusion

9. 9 Background substraction Learning-based algorithm  N frames → RGB values extrema for each pixel  Connexity search → noise reduction  Simple and fast

10. 10 Precomputation: anthropometric measures Anthropometric values computation  Needed for the tracking 2 postures Silhouette extraction

11. 11 Head and Torso measurements Pose 1: head measure  Highest pixel in silhouette

12. 12 Head and Torso measurements Pose 1: head measure  Highest pixel in silhouette  Going down into the image while Nb i (whitepixel)> Nb i-1 (whitepixel)

13. 13 Head and Torso measurements Pose 1: head measure  « highest » pixel in frame  Going down on the image while Nb i (whitepixel)>Nb i-1 (whitepixel) → Head width  Going down until Nb(whitepixel) < threshold* width → Shoulder line

14. 14 Head and torso measurements Torso measure  Barycenter line → Torso width  Going down until separation → bottom of the torso → Torso height width height

15. 15 Arm measurements Pose 1  Level and side of the amputation  Each arms sizes Horizontal extreme points

16. 16 Arm measurements (2) Pose 2

17. 17 Presentation summary Previous Works Our contribution  Real-Time Posture Analysis  Background substraction  Precomputation: anthropometric measures  Posture Tracking  Mirror effect : Virtual arm Results Conclusion

18. 18 Head and Torso detection Head and Torso  Connexity research algorithm helped by the anthropometrics measures

19. 19 Seek articulations Seeking shoulder, elbow and wrist Anthropometrics measurements → estimation of the positions

20. 20 Missing limb creation If before-elbow amputation  Coordinates systems of valid and amputee side are equivalent  Reprojection of pixels valid side → amputee side

21. 21 Virtual arm creation If below-elbow amputation  Coordinates system of valid and amputee side are equivalent  Reprojection of pixels valid side → amputee side

22. 22 Results Webcam (640*480) About 8 to 10 frames / s

23. 23 Remarks Up  Follow-up the side-displacements of the silhouette  Whole image reflecting  Better illusion for subject  Interactive time Constrains  Subject must be facing the camera  For a better illusion the elbows angles have to be symmetric  Non Snake moves

24. 24 Conclusion and future works Experimental system  Simple posture tracking  Constrains but robust  Virtual arm creation Next step  Experiment on amputee  Collaboration with ISC of Lyon Perspectives  3D-model to be more realistic  Amputees reactions to real experiments

25. 25 Thank you …

26. 26 Ajout du membre → réalité augmentée (RA) Problème générique en RA Cohérence entre réel et virtuel  Texture, aspect  Élément de synthèse construit à partir d’image réelle (cf notre système)  Illumination, Ombre

27. 27 Creation du membre manquant amputation humérale  centre de gravité du moignon  obtention d'un axe grâce a al même méthode que pour le bras valide  amputation cubitale  Même méthode que pour le bras valide  Obtention de l'axe du moignon grâce au centre de gravité de la partie après le coude

28. 28

29. 29

30. 30