1. International Conference on Automatic Face and Gesture Recognition, 2006 A Layered Deformable Model for Gait Analysis Haiping Lu, K.N. Plataniotis and A.N. Venetsanopoulos The Edward S. Rogers Sr. Department of Electrical and Computer Engineering University of Toronto

2. Haiping Lu FG2006, Southampton, UK 2 Outline Motivation Overview The layered deformable model (LDM) LDM body pose recovery Experimental results Conclusions

3. Haiping Lu FG2006, Southampton, UK 3 Motivation Automated Human identification at a distance Visual surveillance and monitoring applications Banks, parking lots, airports, etc. USF HumanID Gait Challenge problem Articulated human body model for gait recognition Manually labeled silhouettes Layered, deformable

4. Haiping Lu FG2006, Southampton, UK 4 Overview Manual labeling LDM recovery Automatic extraction LDM recovery

5. Haiping Lu FG2006, Southampton, UK 5 The Layered Deformable Model (LDM) Trade-off: Complexity Vs. descriptiveness Match manual labeling: Close to human’s subjective perception Assumptions: Fronto-parallel, from right to left.

6. Haiping Lu FG2006, Southampton, UK 6 LDM – 22 Parameters Ten segments Static: Lengths (6) Widths (3) Dynamic Positions (4) Angles (9)

7. Haiping Lu FG2006, Southampton, UK 7 LDM –Layers and deformation Four layers Deformation:

8. Haiping Lu FG2006, Southampton, UK 8 LDM – Summary Summary: Realistic with moderate complexity Compact: 13 dynamic parameters Layered: model self-occlusion Deformable: realistic limbs Resemblance to manual labeling

9. Haiping Lu FG2006, Southampton, UK 9 Manual silhouettes pose estimation (ground truth & statistics) Limb joint angles: Reliable edge orientation Spatial–Orientation mean-shift (mode- seeking): dominant modes  limb orientation Others: Joint positions, limb widths and lengths Simple geometry Torso: bounding box: Head: “head top” and “front face”

10. Haiping Lu FG2006, Southampton, UK 10 Post-processing Human body constraints: Parameter variation limits Limb angles inter-dependency Temporal smoothing Moving average filtering

11. Haiping Lu FG2006, Southampton, UK 11 Automatic pose estimation Silhouette extraction (ICME06, Lu, et al.) Static parameters Coarse estimations: statistics from Gallery set Silhouette information extraction based on ideal human proportion: Height, head and waist center, joint spatial- orientation domain modes of limbs

12. Haiping Lu FG2006, Southampton, UK 12 Ideal proportion of the human eight- head-high figure in drawing

13. Haiping Lu FG2006, Southampton, UK 13 Automatic pose estimation Dynamic parameters: Geometry on static parameters and silhouette information, constraints. Limb switching detection Thighs & lower legs: variations of angles. Arms: opposite of thighs Frames between successive switch Post-processing: smoothing

14. Haiping Lu FG2006, Southampton, UK 14 Experimental results 285 sequences from five data sets, one gait cycle each sequence. Imperfection due to silhouette extraction noise and estimation algorithm Feedback LDM recovery to silhouette extraction process may help.

15. Haiping Lu FG2006, Southampton, UK 15 LDM recovery results Raw LDM manual LDM auto

16. Haiping Lu FG2006, Southampton, UK 16 LDM recovery example (revisit) Manual labeling LDM recovery Silhouette extraction LDM recovery

17. Haiping Lu FG2006, Southampton, UK 17 Angle estimation – left & right thighs From manual silhouettesFrom automatically extracted silhouettes

18. Haiping Lu FG2006, Southampton, UK 18 Error rate (in percentage) for lower limb angles

19. Haiping Lu FG2006, Southampton, UK 19 Conclusions A layered deformable model for gait analysis 13 Dynamic and 9 static parameters Body pose recovery from manual (ground truth) and automatically extracted silhouettes. Average error rate for lower limb angles: 7% Overall: close match to manual labeling, accurate & efficient model for gait analysis Future work: model-based gait recognition

20. Haiping Lu FG2006, Southampton, UK 20 Acknowledgement Thanks Prof. Sarkar from the University of South Florida (USF) for providing the manual silhouettes and Gait Challenge data sets.