In the Rigid Kingdom Alexander Bronstein, Michael Bronstein ...He had Cinderella sit down, and, putting the slipper to her foot, he found that it went on very easily, fitting her as if it had been made of wax. C. Perrault, Cinderella In the Rigid Kingdom Alexander Bronstein, Michael Bronstein © 2008 All rights reserved. Web: tosca.cs.technion.ac.il

Imagine a glamorous ball…

A fairy tale shape similarity problem

Extrinsic shape similarity Given two shapes and , find the degree of their incongruence. Compare and as subsets of the Euclidean space . Invariance to rigid motion: rotation, translation, (reflection): is a rotation matrix, is a translation vector

How to get rid of Euclidean isometries? How to remove translation and rotation ambiguity? Find some “canonical” placement of the shape in . Extrinsic centroid (a.k.a. center of mass, or center of gravity): Set to resolve translation ambiguity. Three degrees of freedom remaining…

How to get rid of the rotation ambiguity? Find the direction in which the surface has maximum extent. Maximize variance of projection of onto is the covariance matrix Second-order geometric moments of : is the first principal direction

How to get rid of the rotation ambiguity? Project on the plane orthogonal to . Repeat the process to find second and third principal directions .

Canonical basis span a canonical orthogonal basis for in .

How to get rid of the rotation ambiguity? Direction maximizing = largest eigenvector of . and correspond to the second and third eigenvectors of . admits unitary diagonalization . Setting aligns with the standard basis axes . Principal component analysis (PCA), a.k.a. Karhunen-Loéve transform (KLT), or Hotelling transform. Bottom line: the transformation brings the shape into a canonical configuration in .

Second-order geometric moments Eigenvalues of are second-order moments of . In the canonical basis, mixed moments vanish. Ratio describe eccentricity of . Magnitudes of express shape scale.

Higher-order geometric moments Second-order moments allow some discrimination. Use higher-order moments gives more discrimination. -th order moment Computed in the canonical basis. Invariant to rigid motion. Signature of moments A fingerprint of the extrinsic geometry of .

A signal decomposition intuition Moments are decomposition coefficients in the monomial basis is a Dirac delta function for and elsewhere. span .

A signal decomposition intuition uniquely identify a shape (up to a rigid motion). can be reconstructed exactly from is the bi-orthonormal basis, i.e. The monomial basis is not orthogonal. The bi-orthonormal basis is ugly, but we do not need to reconstruct .

Truncated signatures of moments Compute the truncated moment signature Construct a moments distance function, e.g. A distance function on the shape of spaces. Quantifies the extrinsic dissimilarity of and .

Moments distance is small for nearly congruent and . is large for strongly non-congruent and . If and are truly congruent, . However, does not imply that and are congruent (unless ). Which shapes are indistinguishable by ? Ideally, congruent at a coarse resolution (“low frequency”) and differing in fine details (“high frequency”). Degree of coarseness is controlled by the moments order . Geometric moments do not satisfy this requirement.

Other moments Instead of the monomial basis, other bases can be chosen Fourier basis Spherical harmonics, Zernike polynomials, wavelets, etc, etc.

Moments of joy, moments of sorrow Shape similarity is translated to similarity of moment signatures. Comparison of moments signatures is fast (e.g. Euclidean distance). Sorrow: Do not allow for partial similarity!

Iterative closest point (ICP) algorithms Given two shapes and , find the best rigid motion bringing as close as possible to : is some shape-to-shape distance. Minimum = extrinsic dissimilarity of and . Minimizer = best rigid alignment between and . ICP is a family of algorithms differing in The choice of the shape-to-shape distance. The choice of the numerical minimization algorithm.

Shape-to-shape distance The Hausdorff distance is the distance between a point and the shape . is the distance between a point and

Shape-to-shape distance A non-symmetric version is preferred to allow for partial similarity The (max-min) formulation is sensitive to outliers. Use the variant is a point-to-shape distance. Different possibilities to define .

Point-to-point distance Treat as a cloud of points. Find the closest point to on . Define the distance as

Point-to-plane distance Treat as a plane, and define the point-to-plane distance is the normal to the surface at point . Can be approximated as

Second-order point-to-shape distance Point-to-plane distance is a first-order approximation of the true point-to-shape distance. Construct a second-order approximation are the principal curvature radii at . are the principal directions. is the signed distance to the closest point.

Second-order point-to-shape distance The second-order distance approximant may become negative for some values of . Use a non-negative quadratic approximant

Second-order point-to-shape distance “Near-field” case – point-to-plane distance “Far-field” case – point-to-point distance

Second-order point-to-shape distance Second-order distance generalizes the point-to-point and the point-to- plane distances. Gives more accurate alignment between shapes. Requires principal curvatures and directions (second-order quantities).

Iterative closest point algorithm Initialize Find the closest point correspondence Minimize the misalignment between corresponding points Update Iterate until convergence…

Closest points How to find closest points efficiently? Straightforward complexity: number of points on , number of points on . divides the space into Voronoi cells Given a query point , determine to which cell it belongs.

Closest points

Approximate nearest neighbors To reduce search complexity, approximate Voronoi cells. Use binary space partition trees (e.g. kd-trees or octrees). Approximate nearest neighbor search complexity: .

Best alignment Given two sets and of corresponding points. Find best alignment A numerical minimization algorithm can be used. For some point-to-shape distances, a closed-form solution exists.

MATLAB® intermezzo Iterative closest point algorithm

Until convergence… ICP should find the solution of Instead, it solves Correspondence fixed to instead of . Not guaranteed to produce a monotonically decreasing sequence of values of . Not guaranteed to converge!

Enter numerical optimization Treat as a numerical minimization problem. Express the distance terms as a quadratic function is a 3×3 symmetric positive definite matrix, is 3×1 vector, and is a scalar.

Local quadratic approximant Point-to-point distance: Point-to-plane distance:

Local quadratic approximant Minimize over . Dependence of and on might be complicated. For small motion , hence

Minimization variables is required to be unitary (orthonormal). Enforcing orthonormality is cumbersome. Minimization w.r.t. to the rotation angles involves nonlinear functions. Under small motion assumption, Linearize rotation matrix

Let Newton be! Linearized rotation yields a quadratic objective w.r.t . Use a Newton step to find the steepest descent direction. Approximation is valid only for small steps. Use Armijo rule to find a fractional step ensuring sufficient decrease of objective function. What is a fractional step?

Fractional step Let be a small transformation, which applied times gives . is a rotation by . Hence

Iterative closest point algorithm revisited Initialize Find closest point correspondence Construct local quadratic approximant of Find Newton direction Use Armijo rule to find such that Update Iterate until convergence…

Iterative closest point algorithm revisited Coefficients of the quadratic approximant can be computed on demand using efficient nearest neighbor search. Alternative: approximate the values of in the space using a space partition tree.

Summary and suggested reading Rigid shape similarity R. C. Veltkamp and M. Hagedoorn, Shape similarity measures, properties, and constructions, (2001). Moment-based shape descriptors R. J. Prokop and A. P. Reeves, A survey of moment-based techniques for unoccluded object representation and recognition, Graphical Models and Image Processing 54 (1992), no. 5, 438–460. D. Zhang and G. Lu, Review of shape representation and description techniques, Pattern Recognition 37 (2004), 1–19. Signal decomposition and reconstruction insights M. Elad, P. Milanfar, and G. H. Golub, Shape from moments-an estimation theory perspective, IEEE Trans. Signal Processing 52 (2004), no. 7, 1814–1829. Partial shape similarity More on partial similarity in upcoming lectures…

Summary and suggested reading Iterative closest point algorithms S. Rusinkiewicz and M. Levoy, Efficient variants of the ICP algorithm, Proc. 3D Digital Imaging and Modeling (2001), 145–152. Point-to-shape distances: point-to-point, point-to-plane, second-order. Fast and approximate nearest neighbor search S. Arya and D. M. Mount, Approximate nearest neighbor searching, Proc. 4th ACM-SIAM Symposium on Discrete Algorithms (1993), 271–280. Convergence of ICP E. Ezra, M. Sharir, and A. Efrat, On the ICP algorithm, Proc. Symp. Computational Geometry (2006), 95–104. D. Arthur and S. Vassilvitskii, Worst-case and smoothed analysis of the ICP algorithm, with an application to the k-means method, Proc. Symp. Foundations of Computer Science (2006), 153–164.

Summary and suggested reading ICP from the point of view of numerical optimization N. J. Mitra, N. Gelfand, H. Pottmann, and L. Guibas, Registration of point cloud data from a geometric optimization perspective, Proc. Symp. Geometry Processing (SGP), 2004, pp. 23–32. ICP initialization and global optimality N. Gelfand, N. J. Mitra, L. Guibas, and H. Pottmann, Robust global registration, Proc. Symp. Geometry Processing (SGP) (2005). H. Li and R. Hartley, The 3D-3D registration problem revisited, Proc. ICCV, 2007.