1. 3D Human Body Pose Estimation from Monocular Video Moin Nabi Computer Vision Group Institute for Research in Fundamental Sciences (IPM)

2. Introduction to Human Pose Estimation Articulated pose estimation from single-view monocular image(s)

3. Application of Human Pose Estimation ■ Entertainment: Animation, Games ■ Security: Surveillance ■ Understanding: Gesture/Activity recognition

4. Difficulties of Human Pose estimation ■ Appearance/size/shape of people can vary dramatically ■ The bones and joints are observable indirectly (obstructed by clothing) ■ Occlusions ■ High dimensionality of the state space ■ Lose of depth information in 2D image projections

5. Difficulties of Human Pose estimation ■ Challenging Human Motion

6. Problem Backgrounds ■ Break up a very hard problem into smaller manageable pieces Goal: Reliable 3D Human Pose Estimation from single-camera input

7. Problem Backgrounds ■ Break up a very hard problem into smaller manageable pieces Goal: Reliable 3D Human Pose Estimation from single-camera input

8. Problem Backgrounds ■ Break up a very hard problem into smaller manageable pieces Goal: Reliable 3D Human Pose Estimation from single-camera input

9. Graphical model (definition) Nodes : Xi Random Variables Edges : P(Xj/Xi) Conditional Probability

10. Graphical model (Examples)

11. Graphical model (Inference) discrete continuous Belief propagation

12. (a) monocular input image with bottom up limb proposals overlaid (b); (c) distribution over 2D limb poses computed using nonparametric belief propagation; (d) sample of a 3D body pose generated from the 2D pose; (e) illustration of tracking. Hierarchical Inference Framework

17. Inferring 2D pose 2D Loose-Limbed Body Model

18. Graphical Modeling the Person X = {X1,X2,...,XP} in terms of 2D position, rotation, scale and foreshortening of parts, Xi € R5

19. Modeling the constraints

20. ■ Kinematic Constraints ■ Occlusion Constraints … Joint probability

21. Limb proposal 5 × 5 × 20 × 20 × 8 = 80, 000 valuated discrete states valuating the likelihood function chose the 100 most likely states for each part discretizing the state space into: 5 scales 5 foreshortenings 20 vertical positions 20 horizontal positions 8 rotations

22. Image likelihood In defining we use edge, silhouette and color features and combine them. approximate the global likelihood with a product of local terms

23. None Parametric Belief Propagation Use an Iterative method of message passing to find better poses

24. 2D Loose-Limbed Body Model (summary)

25. Result

31. Inferring 3D pose from 2D 2D Loose-Limbed Body Model Mixture of Experts (MoE)

32. Inferring 3D pose from 2D Problem: p(Y|X)is non-linear mapping, and not one-to-one

33. Inferring 3D pose from 2D Solution: p(Y|X)may be approximated by a locally linear mappings (experts)

34. MoE Formally Training of MoE is done using EM procedure (similar to learning Mixture of Gaussians)

35. Illustration of 3D pose inference

37. Inferring 3D pose from 2D 2D Loose-Limbed Body Model Mixture of Experts (MoE) Hidden Markov Model (HMM)

38. Result

40. Thank You