Invariant correspondence and calculus of shapes © Alexander & Michael Bronstein, 2006-2010 tosca.cs.technion.ac.il/book VIPS Advanced School on Numerical Geometry of Non-Rigid Shapes University of Verona, April 2010 1 1

“Natural” correspondence?

‘ ‘ ‘ Geometric Semantic Aesthetic accurate makes sense beautiful Correspondence Geometric Semantic Aesthetic accurate ‘ makes sense ‘ beautiful ‘

Correspondence Correspondence is not a well-defined problem!  Chances to solve it with geometric tools are slim.  If objects are sufficiently similar, we have better chances.  Correspondence between deformations of the same object.

Invariant correspondence Ingredients: Class of shapes Class of deformations Correspondence procedure which given two shapes returns a map Correspondence procedure is -invariant if it commutes with i.e., for every and every ,

Invariant similarity (reminder) Ingredients: Class of shapes Class of deformations Distance Distance is -invariant if for every and every

Rigid similarity Class of deformations: congruences Congruence-invariant (rigid) similarity: Closest point correspondence between , parametrized by Its distortion Minimize distortion over all possible congruences

Rigid correspondence Class of deformations: congruences Congruence-invariant similarity: Congruence-invariant correspondence: INVARIANT SIMILARITY  INVARIANT CORRESPONDENCE RIGID SIMILARITY  RIGID CORRESPONDENCE

Invariant representation (canonical forms) Ingredients: Class of shapes Class of deformations Embedding space and its isometry group Representation procedure which given a shape returns an embedding Representation procedure is -invariant if it translates into an isometry in , i.e., for every and , there exists such that

INVARIANT SIMILARITY = INVARIANT REPRESENTATION + RIGID SIMILARITY

Invariant parametrization Ingredients: Class of shapes Class of deformations Parametrization space and its isometry group Parametrization procedure which given a shape returns a chart Parametrization procedure is -invariant if it commutes with up to an isometry in , i.e., for every and , there exists such that

INVARIANT CORRESPONDENCE = INVARIANT PARAMETRIZATION + RIGID CORRESPONDENCE

Representation errors Invariant similarity / correspondence is reduced to finding isometry in embedding / parametrization space. Such isometry does not exist and invariance holds approximately Given parametrization domains and , instead of isometry find a least distorting mapping . Correspondence is

Dirichlet energy Minimize Dirchlet energy functional Equivalent to solving the Laplace equation Boundary conditions Solution (minimizer of Dirichlet energy) is a harmonic function. N. Litke, M. Droske, M. Rumpf, P. Schroeder, SGP, 2005

Dirichlet energy Caveat: Dirichlet functional is not invariant Not parametrization-independent Solution: use intrinsic quantities Frobenius norm becomes Hilbert-Schmidt norm Intrinsic area element Intrinsic Dirichlet energy functional N. Litke, M. Droske, M. Rumpf, P. Schroeder, SGP, 2005

The harmony of harmonic maps Intrinsic Dirichlet energy functional is the Cauchy-Green deformation tensor Describes square of local change in distances Minimizer is a harmonic map. N. Litke, M. Droske, M. Rumpf, P. Schroeder, SGP, 2005

Physical interpretation RUBBER SURFACE METAL MOULD = ELASTIC ENERGY CONTAINED IN THE RUBBER

Minimum-distortion correspondence Ingredients: Class of shapes Class of deformations Distortion function which given a correspondence between two shapes assigns to it a non-negative number Minimum-distortion correspondence procedure

Minimum-distortion correspondence Correspondence procedure is -invariant if distortion is -invariant, i.e., for every , and ,

Minimum-distortion correspondence Euclidean norm Dirichlet energy Quadratic stress CONGRUENCES CONFORMAL ISOMETRIES

Minimum distortion correspondence

Uniqueness & symmetry The converse in not true, i.e. there might exist two distinct minimum-distortion correspondences such that for every Intrinsic symmetries create distinct isometry-invariant minimum- distortion correspondences, i.e., for every

Partial correspondence

Measure coupling Let be probability measures defined on and (a metric space with measure is called a metric measure or mm space) A measure on is a coupling of and if for all measurable sets The measure can be considered as a fuzzy correspondence Mémoli, 2007

Intrinsic similarity Hausdorff Wasserstein Gromov-Hausdorff Distance between subsets of a metric space . Distance between subsets of a metric measure space . Gromov-Hausdorff Gromov-Wasserstein Distance between metric spaces Distance between metric measure spaces Mémoli, 2007

Minimum-distortion correspondence Gromov-Hausdorff Minimum-distortion correspondence between metric spaces Gromov-Wasserstein Minimum-distortion fuzzy correspondence between metric measure spaces Mémoli, 2007

Texture transfer TIME Reference Transferred texture

Virtual body painting

Texture substitution I’m Alice. I’m Bob. I’m Alice’s texture on Bob’s geometry

How to add two dogs? + = 1 2 1 2 C A L C U L U S O F S H A P E S

Affine calculus in a linear space Subtraction creates direction Addition creates displacement Affine combination spans subspace Convex combination ( ) spans polytopes

Affine calculus of functions Affine space of functions Subtraction Addition Affine combination Possible because functions share a common domain

? Affine calculus of shapes A. Bronstein, M. Bronstein, R. Kimmel, IEEE TVCG, 2006

Temporal super-resolution TIME

Motion-compensated interpolation

Metamorphing 100% Alice 75% Alice 25% Bob 50% Alice 50% Bob 75% Alice

Face caricaturization EXAGGERATED EXPRESSION 1 1.5

Affine calculus of shapes

What happened? SHAPE SPACE IS NON-EUCLIDEAN!

Shape space Shape space is an abstract manifold Deformation fields of a shape are vectors in tangent space Our affine calculus is valid only locally Global affine calculus can be constructed by defining trajectories confined to the manifold Addition Combination

Choice of trajectory Equip tangent space with an inner product Riemannian metric on Select to be a minimal geodesic Addition: initial value problem Combination: boundary value problem

Choice of metric Deformation field of is called Killing field if for every Infinitesimal displacement by Killing field is metric preserving and are isometric Congruence is always a Killing field Non-trivial Killing field may not exist

Choice of metric Inner product on Induces norm measures deviation of from Killing field – defined modulo congruence Add stiffening term

Minimum-distortion trajectory Geodesic trajectory Shapes along are as isometric as possible to Guaranteeing no self-intersections is an open problem