1 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant shape similarity © Alexander & Michael Bronstein, © Michael Bronstein, 2010 tosca.cs.technion.ac.il/book Advanced topics in vision Processing and Analysis of Geometric Shapes EE Technion, Spring 2010

2 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Invariant similarity SIMILARITY TRANSFORMATION

3 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Equivalence Equal CongruentIsometric

4 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Equivalence Equivalence is a binary relation on the space of shapes which for all satisfies Reflexivity: Symmetry: Transitivity: Can be expressed as a binary function if and only if Quotient space is the space of equivalence classes

5 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Equivalence

6 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Equivalence All deformations of the human shape are “the same”

7 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Similarity Shapes are rarely truly equivalent (e.g., due to acquisition noise or since most shapes are rigid) We want to account for “almost equivalence” or similarity -similar = -isometric (w.r.t. some metric) Define a distance on the shape space quantifying the degree of dissimilarity of shapes

8 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Similarity A monkey shape is more similar to a deformation of a monkey shape… …than to a human shape

9 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Isometry-invariant distance Non-negative function satisfying for all Corollary: is a metric on the quotient space Similarity: and are -isometric; and are -isometric (In particular, if and only if ) Symmetry: Triangle inequality: Given discretized shapes and sampled with radius Consistency to sampling:

10 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Canonical forms distance Compute Hausdorff distance over all isometries in Minimum-distortion embedding Minimum-distortion embedding No fixed embedding space will give distortion-less canonical forms

11 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Gromov-Hausdorff distance Isometric embedding Isometric embedding Mikhail Gromov Gromov-Hausdorff distance: include into minimization

12 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Properties of Gromov-Hausdorff distance Metric on the quotient space of isometries of shapes Similarity: and are -isometric; and are -isometric Consistent to sampling: given discretized shapes and sampled with radius Generalization of Hausdorff distance: Hausdorff distance between subsets of a metric space Gromov-Hausdorff distance between metric spaces Gromov, 1981

13 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Alternative definition I (metric coupling) is the disjoint union of and the (semi-) metric satisfies and where Mémoli, 2008

14 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Optimization over translates into finding the values of Mémoli, 2008 A lot of constraints! Alternative definition I (metric coupling)

15 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Correspondence A subset is called a correspondence between and if for every there exists at least one such that and similarly for every there exists such that Particular case: given and

16 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Correspondence distortion The distortion of correspondence is defined as

17 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Alternative definition II (correspondence distortion) 1. Show that for any there exists with Proof sketch Since, by definition of, and are subspaces of some such that Let By triangle inequality, for

18 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Alternative definition II (correspondence distortion) 2. Show that for any Let It is sufficient to show that there is a (semi-)metric on the disjoint union such that,, and Construct the metric as follows (in particular, for ).

19 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Alternative definition II (correspondence distortion) First, For each Since for, Second, we need to show that is a (semi-)metric on On and, it is straightforward We only need to show metric properties hold on

20 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Alternative definition III measures how much is distorted by when embedded into

21 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity measures how much is distorted by when embedded into Alternative definition III

22 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity measures how far is from being the inverse of Alternative definition III

23 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Generalized MDS A. Bronstein, M. Bronstein & R. Kimmel, 2006

24 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Difficulties How to represent points on ? Global parametrization is not always available. Some local representation is required in general case. No more closed-form expression for. Metric needs to be approximated. Minimization algorithm.

25 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Local representation is sampled at and represented as a triangular mesh. Any point falls into one of the triangles. Within the triangle, it can be represented as convex combination of triangle vertices, Barycentric coordinates. We will need to handle discrete indices in minimization algorithm.

26 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Geodesic distances Distance terms can be precomputed, since are fixed. How to compute distance terms ? No more closed-form expression. Cannot be precomputed, since are minimization variables. can fall anywhere on the mesh. Precompute for all. Approximate for any.

27 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Geodesic distance approximation Approximation from. First order accurate: Consistent with data: Symmetric: Smoothness: is and a closed-form expression for its derivatives is available to minimization algorithm. Might be only at some points or along some lines. Efficiently computed: constant complexity independent of.

28 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Geodesic distance approximation Compute for. falls into triangle and is represented as Particular case: Hence, we can precompute distances How to compute from ?

29 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Geodesic distance approximation We have already encountered this problem in fast marching. Wavefront arrives at triangle vertex at time. When does it arrive to ? Adopt planar wavefront model. Distance map is linear in the triangle (hence, linear in ) Solve for coefficients and obtain a linear interpolant

30 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Geodesic distance approximation General case: falls into triangle and is represented as Apply previous steps in triangle to obtain Apply once again in triangle to obtain

31 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity A four-step dance

32 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Minimization algorithm How to minimize the generalized stress? Particular case: L 2 stress Fix all and all except for some. Stress as a function of only becomes quadratic

33 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Quadratic stress is positive semi-definite. is convex in (but not necessarily in together).

34 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Quadratic stress Closed-form solution for minimizer of Problem: solution might be outside the triangle. Solution: find constrained minimizer Closed-form solution still exists.

35 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Minimization algorithm Initialize For each Fix and compute gradient Select corresponding to maximum. Compute minimizer If constraints are active translate to adjacent triangle. Iterate until convergence…

36 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity How to move to adjacent triangles? Three cases All : inside triangle. : on edge opposite to. : on vertex. inside on edgeon vertex

37 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Point on edge on edge opposite to. If edge is not shared by any other triangle we are on the boundary – no translation. Otherwise, express the point as in triangle. contains same values as. May be permuted due to different vertex ordering in. Complication: is not on the edge. Evaluate gradient in. If points inside triangle, update to.

38 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Point on vertex on vertex. For each triangle sharing vertex Express point as in. Evaluate gradient in. Reject triangles with pointing outside. Select triangle with maximum. Update to.

39 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity MDS vs GMDS Stress Generalized MDSMDS Generalized stress Analytic expression for Nonconvex problem Variables: Euclidean coordinates of the points must be interpolated Nonconvex problem Variables: points on in barycentric coordinates

40 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Multiresolution Stress is non convex – many small local minima. Straightforward minimization gives poor results. How to initialize GMDS? Multiresolution: Create a hierarchy of grids in, Each grid comprises Sampling: Geodesic distance matrix:

41 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Multiresolution Initialize at the coarsest resolution in. For Starting at initialization, solve the GMDS problem Interpolate solution to next resolution level Return.

42 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity GMDS Interpolation GMDS

43 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Multiresolution encore So far, we created a hierarchy of embedded spaces. One step further: create a hierarchy of embedding spaces.

44 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Labeling problem Torresani, Kolmogorov, Rother 2008 Wang, B 2010 Build a graph with vertices and edges Label each vertex Minimum distortion correspondence = graph labeling problem Efficient solvers with good global convergence properties Complexity: Hierarchical solution complexity can be lowered to

45 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity MATLAB ® intermezzo GMDS

46 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Discrete Gromov-Hausdorff distance Two coupled GMDS problems Can be cast as a constrained problem Bronstein, Bronstein & Kimmel, 2006

47 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Numerical example Canonical forms (MDS based on 500 points) Gromov-Hausdorff distance (GMDS based on 50 points) Bronstein, Bronstein & Kimmel, 2006

48 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Extrinsic similarity using Gromov-Hausdorff distance Mémoli (2008) Connection between Euclidean GH and ICP distances: CongruenceEuclidean isometry EXTRINSIC SIMILARITY ICP distance: GH distance with Euclidean metric: Mémoli, 2008

49 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Connection to canonical form distance

50 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Correspondence again A different representation for correspondence using indicator functions defines a valid correspondence if

51 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity L p Gromov-Hausdorff distance We can give an alternative formulation of the Gromov-Hausdorff distance Can we define an L p version of the Gromov-Hausdorff distance by relaxing the above definition?

52 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Measure coupling Let be probability measures defined on and The measure can be considered as a relaxed version of the indicator function or as fuzzy correspondence A measure on is a coupling of and if for all measurable sets Mémoli, 2007 (a metric space with measure is called a metric measure or mm space)

53 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Gromov-Wasserstein distance The relaxed version of the Gromov-Hausdorff distance is given by and is referred to as Gromov-Wasserstein distance [Memoli 2007] Mémoli, 2007

54 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity Earth Mover’s distance Let be a metric space, and measures supported on EMD as optimal mass transport: mass transported from to distance traveled Mémoli, 2007 The Wasserstein or Earth Mover’s distance (EMD) is given by Define the coupling of

55 Numerical Geometry of Non-Rigid Shapes Invariant shape similarity The analogy Hausdorff Mémoli, 2007 Distance between subsets of a metric space. Gromov-Hausdorff Distance between metric spaces Wasserstein Distance between subsets of a metric measure space. Gromov-Wasserstein Distance between metric measure spaces