Numerical geometry of non-rigid shapes Multidimensional scaling Alexander Bronstein, Michael Bronstein, Ron Kimmel © 2007 All rights reserved

How to measure shape similarity? Given two shapes and represented as discrete samples and , compute their similarity Extrinsic similarity Intrinsic similarity

Rigid (extrinsic) similarity Rigid shape similarity: congruence Degrees of freedom: Euclidean transformations (rotation+translation) Classical solution: iterative closest point (ICP) algorithm ROTATION MATRIX TRANSLATION VECTOR Hausdorff distance as a measure of congruence between point clouds (other distances are usually preferred) Y. Chen, G. Medioni, 1991 P. J. Besl, N. D. McKay, 1992

Cinderella measuring the glass slipper ICP in fairy tales Cinderella measuring the glass slipper Image: Disney

Non-rigid (intrinsic) similarity Congruence is not a good criterion for similarity of non-rigid shapes Geodesic distances are invariant to isometric deformations and can be easily computed using FMM Naïve approach: directly compare matrices of geodesic distances Problem: arbitrary ordering of points (permutation of rows and columns) degrees of freedom

Isometric embedding Shape Canonical form Represent the intrinsic geometry in a Euclidean space by isometrically embedding it into Treat the resulting images (canonical forms) as rigid shapes, using ICP or other rigid similarity algorithms Isometric embedding “undoes” the degrees of freedom A. Elad, R. Kimmel, CVPR, 2001

Mapmaking problem A B B A Earth (sphere) Planar map

Embedding error Theorema Egregium: a sphere has positive Gaussian curvature, the plane has zero Gaussian curvature, therefore, they are not isometric. Every cartographer knows: impossible to create a distance-preserving planar map of the Earth! Does isometric embedding into higher-dimensional spaces exist?

Conclusion: generally, isometric embedding does not exist! Linial’s example A A 1 1 C D 2 A C D 1 B C A 1 1 C D B B 2 C D D 1 1 B Conclusion: generally, isometric embedding does not exist!

Minimum distortion embedding Find an embedding that distorts the distances the least Stress function is a measure of distortion where Multidimensional scaling (MDS) problem

Near-isometric deformations of a shape Examples of canonical forms Near-isometric deformations of a shape Canonical forms A. Elad, R. Kimmel, CVPR, 2001

Matrix expression of the L2-stress - an matrix of coordinates in the embedding space - an constant matrix with values - an matrix-valued function

MDS problem variables Non-convex non-linear optimization problem Using convex optimization techniques is liable to local convergence Optimum defined up to Euclidean transformation

Iterative majorization Instead of , minimize a convex majorizing function satisfying Start with some and iteratively update

SMACOF algorithm Majorize the stress by a convex quadratic function Analytic expression for the minimum of : SMACOF (Scaling by Minimizing a COnvex Function)

SMACOF algorithm (cont) Equivalent to constant-step gradient descent Guarantees monotonically decreasing sequence of stress values No guarantee of global convergence Iteration cost:

Application: face recognition Facial expressions are approximate isometries of the facial surface Identity = intrinsic geometry Expression = extrinsic geometry A. M. Bronstein et al., IJCV, 2005

How to canonize a person? 3D surface acquisition Cropping Smoothing Canonization A. M. Bronstein et al., IJCV, 2005

Application: face recognition Facial expressions Canonical forms A. M. Bronstein et al., IJCV, 2005

Application: face recognition Rigid similarity Non-rigid similarity (canonical forms) A. M. Bronstein et al., IJCV, 2005 Alex Michael

Coarse MDS problem (N=200) Multiresolution MDS: motivation Fine MDS problem (N~1000) Coarse MDS problem (N=200)

Multiresolution MDS Bottom-up: solve coarse grid MDS problem to initialize fine grid problem Can be performed on multiple resolution levels Reduce complexity (less fine-grid iterations) Reduce the risk of local convergence (good initialization) Solve fine grid problem Fine grid solution Fine grid initialization Interpolate Solve coarse grid problem Coarse grid initialization Coarse grid solution

Multigrid MDS Top-down: start with a fine grid initialization Improve the fine grid initialization by solving a coarse grid problem and propagating the error Fine grid initialization Improved solution Fine grid residual Decimate Interpolate residual Solve coarse grid problem Coarse grid initialization Coarse grid solution Coarse grid residual

Correction Problem: the minima of fine and coarse grid problems do not coincide! Add correction term to coarse grid problem to compensate for inconsistency CORRECTION Choosing guarantees that is a coarse grid solution M. M. Bronstein et al., NLAA, 2006

Modified stress Another problem: the new coarse grid problem is unbounded! ( can be made arbitrarily large by adding a constant to without changing the stress) Modified stress: add a quadratic penalty to the stress thus resolving translation ambiguity (forcing to be centered at the origin) M. M. Bronstein et al., NLAA, 2006

Plugging everything together Hierarchy of data Interpolation and decimation operators to transfer variables and residuals from one resolution level to another Hierarchy of optimization problems Relaxation: optimization algorithm used to improve solution M. M. Bronstein et al., NLAA, 2006

Solve coarsest grid problem V-cycle Relax Decimate Interpolate and correct Relax Relax Decimate Interpolate and correct Solve coarsest grid problem M. M. Bronstein et al., NLAA, 2006

Convergence of SMACOF and MG MDS (N=2145) Convergence example Stress Time (sec) Convergence of SMACOF and MG MDS (N=2145) Order of magnitude speedup, especially pronounced for large M. M. Bronstein et al., NLAA, 2006

How to choose the embedding space? A generic, non-Euclidean embedding space Must result in small embedding error (good representation) Convenient representation of points in (local or preferably global parametrization) Simple (preferably analytic) expression for distances The isometry group is simple (few degrees of freedom)

Problem: embedding error can be reduced, but not made zero! Possible choices Schwartz et al. 1989: Elad & Kimmel 2001: Elad & Kimmel 2002: BBK 2005: Walter & Ritter 2002: Euclidean Spherical Hyperbolic Problem: embedding error can be reduced, but not made zero!

Generalized MDS Embedding space = triangular mesh Generalized stress where is the image on triangular mesh Generalized MDS (GMDS) problem A. M. Bronstein et al., PNAS, 2006

Main differences Difference 1: the distances have no analytic expression Consequence 1: geodesic distance interpolation Difference 2: points represented in local barycentric coordinates Consequence 2: optimization with a modified line search (unfolding) A. M. Bronstein et al., SIAM, 2006

Distance interpolation How to approximate the distances between points ? Precompute the pair-wise distances between all mesh vertices using FMM Find triangles and enclosing Interpolate from known distances Interpolate from A. M. Bronstein et al., SIAM, 2006

Modified line search: unfolding Optimization on triangular mesh requires displacing a point along a ray (line search) Line search in barycentric coordinates requires unfolding Result: polylinear path A. M. Bronstein et al., SIAM, 2006

Conclusions so far Geodesic distances are intrinsic descriptors of non-rigid shapes invariant to isometric deformations MDS is an efficient method for representing and comparing intrinsic invariants Multiresolution and multigrid methods can yield a significant convergence speedup and reduce the risk on local convergence Generalized MDS allows avoiding the embedding error by embedding one surface into another