Shape2Pose: Human Centric Shape Analysis CMPT888 Vladimir G. Kim Siddhartha Chaudhuri Leonidas Guibas Thomas Funkhouser Stanford University Princeton University Paper Study Presentation by Shunan

Content A fundamental question: “how do people interact with objects?” 1.Problem Introduction. 2.Major Problems. 3.Proposed Method. 4.Conclusion & Discussion.

1 Problem introduction

The World need 3D models. Methods for saliency estimation, shape segmentation, feature detection, symmetry analysis, and surface correspondence; These method mainly using techniques that compute global shape properties. Object affordance – a quality of an object that allows someone to perform an action. Main technical contribution is a polynomial-time optimization algorithm that finds an approximate minimum in the combinatorial space of body-to contact assignments. Main challenge in affordance for shape analysis is to automatically predict the pose that a human will take when using a given object.

Main contributions 1.Shape2Pose, a novel affordance-inspired shape analysis tool. 2.Polynomial-time algorithm for exploring the combinatorial space. 3.A dataset of 147 models from diverse object classes with annotated ground truth poses. 4.Evaluate our method and demonstrate favorable performance as results. Shape2Pose Polynomial Datasets Results

2 Major Problems

Major Problems 01 Extend pervious toolkit with affordance analysis that predicts contact points and pose parameters, essentially detecting invariant structural relations. Shape Analysis. 02 Do not assume that there is a small set of discrete poses; instead, Search a continuous pose parameter space. This finer representation enables higher accuracy and applications beyond shape classification. Affordance. 03 Previous approaches focus on robot and virtual hand interaction, while we focus on shape analysis. Also note that a grasping interaction has a simple functional purpose. Grasp Prediction.

Shape2Pose Shape2Pose: Simultaneously predicts kinematic parameters for a static human pose and points of contact between a human body and the shape’s. Main difference: Extra critical information, the shape and deformation modes of a human body. “What significant patterns you can find in this shape?” VS “How can this human body be posed to interact with this shape?”

2 Find Pattern in Shapes. Interact with shapes.

3 Proposed Method

System pipeline Shap2epose Training Affordance Model Contact Distance Feature Compatibility. Pose Prior Symmetry & Surface Intersection Predicting Pose Model Match Inverse Kinematics

Affordance model is defined as the minimizer of an energy function above Training Affordance Model The goal in this stage is to build an energy function that can be used to evaluate the interaction between a shape S and a human pose represented by a rigid transformation T. w dist = 1000, w feat = 10, w pose = 0.3, w sym = 1, w isect = 0.05 For all.

E dist(T, Ɵ,m,S): Contact Distance High weight : parts do not touch the target points. If a body part is assigned to a surface point, we want to ensure they actually make contact. If not, we penalize. Ɵ : Joint angles m: Key body parts with constraints S: Object Shape T: Human pose transformation

E feat(m,S): Feature Compatibility High weight : if local surface is ill-suited for placement. Which means hands to graspable, butt on seat. Give a set of training shape: {S 1, S 2 …S m } with ground truth, Regression Ɵ: Joint angles m: Key body parts with constraints S: Object Shape T: Human pose transformation

E pose( Ɵ ): Pose Prior. Distinguishes unreal pose. learn the pose prior from training examples to get the distribution. Ɵ: Joint angles m: Key body parts with constraints S: Object Shape T: Human pose transformation

E sym(T,m,S): Symmetry. Observe parts of shapes with which humans interact typically have local bilateral symmetry. Ɵ: Joint angles m: Key body parts with constraints S: Object Shape T: Human pose transformation E isect(T, Ɵ,S) : Surface Intersection Helps avoid intersections between the shape and the human. Thickness of body mainly.

For training Input ： A collection of shapes with manually prescribed contact points. Output: An affordance model represented by an objective function that measures the quality of a pose for any shape. Predicting a Human Pose Input ： A novel shape. Output: A set of joint angles and contact points for a likely human interaction pose. For Predicting 2

Inverse Kinematics Predicting 1.if contact points m are assigned, one can solve for (T, Ɵ ) using inverse kinematics. 2.given T and Ɵ, one can infer contact points m via nearest neighbors. The key: possible to sample high-probability contact assignments m and high-probability poses Ɵ independently. They contribute to different major energy terms E feat (m) and E pose ( Ɵ ) respectively. M: key body part and its constraints Ɵ : Joints angles.

4 Conclusion & Discussion.

1.Predicting shape affordances is useful for computer analysis of 3D models. 2.Implementing a novel algorithm for generating the static pose. 3.Our algorithm uses a combination of local anthropometric classifiers as well as global biomechanics constraints. 4.Algorithm produces results within a 20cm tolerance in the vast majority of cases. Shape2Pose for analysis of 3D models Conclusion

Several limitations 1.Only static analysis of an object’s shape, not sufficient to understand all functional interactions with a human. 2.Discussed only a few low-level shape analysis applications in this paper. 3.Highly rely on training data.

CMPT 888 Thank You By Shunan