1 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Michael M. Bronstein Department of Computer Science Technion – Israel Institute of Technology cs.technion.ac.il/~mbron Technion 1 January 2008 Extrinsic and intrinsic similarity of shapes nonrigid

2 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Collaborators Alexander Bronstein Ron Kimmel

3 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Welcome to nonrigid world!

4 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes ? SIMILARITYCORRESPONDENCE Applications

5 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Rock Paper Scissors Rock, scissors, paper

6 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Rock Paper Scissors Hands Rock, scissors, paper

7 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Extrinsic vs. intrinsic Are the shapes congruent? Invariance to rigid motion Are the shapes isometric? Invariance to inelastic deformations EXTRINSIC SIMILARITY INTRINSIC SIMILARITY

8 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Metric model Euclidean metric Isometry = rigid motion Geodesic metric Isometry = inelastic deformation EXTRINSIC SIMILARITYINTRINSIC SIMILARITY Similarity = isometry Shape = metric space

9 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Extrinsic similarity – Iterative closest point (ICP) Chen & Medioni, 1991; Besl & McKay, PAMI 1992 Find the best rigid alignment of two shapes Hausdorff distance In Euclidean space

10 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Extrinsic similarity – limitations Suitable for nearly rigid shapes Unsuitable for nonrigid shapes EXTRINSICALLY SIMILAREXTRINSICALLY DISSIMILAR

11 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Canonical forms A. Elad, R. Kimmel, CVPR 2001 Multidimensional scaling (MDS)Isometric embedding

12 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Intrinsic similarity – canonical forms A. Elad, R. Kimmel, CVPR 2001 Compute canonical forms EXTRINSIC SIMILARITY OF CANONICAL FORMS INTRINSIC SIMILARITY = INTRINSIC SIMILARITY OF SHAPES

13 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Intrinsic similarity – limitations Intrinsically similar Intrinsically dissimilar Suitable for near-isometric shape deformations Unsuitable for deformations modifying shape topology

14 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Extrinsically dissimilar Intrinsically similar Extrinsically similar Intrinsically dissimilar Extrinsically dissimilar Intrinsically dissimilar A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007 THIS IS THE SAME SHAPE! Desired result:

15 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Joint extrinsic/intrinsic similarity DEFORM X TO MATCH Y EXTRINSICALLY CONSTRAIN THE DEFORMATION TO BE AS ISOMETRIC AS POSSIBLE A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

16 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007 Glove fitting example Stretching = Intrinsic dissimilarity Misfit = Extrinsic dissimilarity

17 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes If it doesn’t fit, you must acquit! Image: Associated Press

18 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Intrinsic dissimilarity Extrinsic dissimilarity A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007

19 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Computation of the joint similarity A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007 Optimization variable: the deformed shape vertex coordinates Assuming has the connectivity of Split into computation of and Gradients w.r.t. are required for optimization

20 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Computation of the extrinsic term A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007 Find and fix correspondence between current and Can be e.g. the closest points Compute an L 2 variant of a one-sided Hausdorff distance and its gradient Similar in spirit to ICP

21 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Computation of the intrinsic term A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007 Fix trivial correspondence between and Compute L 2 distortion of geodesic distances and gradient is a fixed matrix of all pair-wise geodesic distances on Can be precomputed using Dijkstra’s algorithm or fast marching

22 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Computation of the intrinsic term A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007 is function of the optimization variables and needs to be recomputed First option: modify the Dijkstra’s algorithm or fast marching to compute the gradient in addition to the distance itself Second option: compute and fix the path of the geodesic is a matrix of Euclidean distances between adjacent vertices is a linear operator integrating the path length along fixed path

23 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Computation of the joint similarity A. Bronstein, M. Bronstein, R. Kimmel, ICCV 2007 Alternating minimization algorithm Compute corresponding points Compute shortest paths and assemble Update to sufficiently decrease If change is small, stop; otherwise, go to Step

24 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Numerical example – dataset = topology change Data: tosca.cs.technion.ac.il

25 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Numerical example – intrinsic similarity no topological changes

26 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Numerical example – intrinsic similarity = topology change= topology-preserving Insensitive to strong deformations Sensitive to topological changes

27 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Numerical example – extrinsic similarity = topology change= topology-preserving Insensitive to topological changes Sensitive to strong deformations

28 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Numerical example – joint similarity = topology change= topology-preserving Insensitive to topological changes... …and to strong deformations

29 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Numerical example – ROC curves False acceptance rate (FAR), % False rejection rate (FRR), % Intrinsic Extrinsic Joint Intrinsic, no topological changes EER=7.7% EER=10.3% EER=1.6% EER=1.1%

30 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Intrinsic dissimilarity Extrinsic dissimilarity Set-valued joint similarity Dissimilar Similar

31 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Shape morphing Stronger intrinsic similarity (larger λ) Stronger extrinsic similarity (smaller λ)

32 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Conclusion Extrinsic similarity is insensitive to topology changes, but sensitive to nonrigid deformations Intrinsic similarity is insensitive to nearly-isometric nonrigid deformations, but sensitive to topology changes Joint similarity is insensitive to both nonrigid deformations and topology changes Can be thought of as nonrigid ICP Can be used to produce as isometric as possible morphs

33 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Open issues Efficient minimization (good initialization, multiresolution) Only topology of one shape can change: topology of Z = topology of X Mesh validity not enforced: self intersections may occur (may be important in computer graphics applications)

34 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Published by Springer To appear in early 2008 ~350 pages Over 50 illustrations Color figures tosca.cs.technion.ac.il Shameless advertisement Additional information

35 Bronstein 2 and Kimmel Extrinsic and intrinsic similarity of nonrigid shapes Workshop on Nonrigid Shape Analysis and Deformable Image Alignment (NORDIA) June 2008, Anchorage, Alaska in conjunction with CVPR’08