Shape Matching and Anisotropy Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz Princeton University Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz Princeton University

Motivation 3D data is becoming more commonly available Someday 3D models will be as common as images are today Cheap ScannersWorld Wide Web 3D Cafe Cyberware Fast Graphics Cards ATI Images courtesy of Cyberware, ATI, & 3Dcafe

Motivation When 3D models are ubiquitous, there will be a shift in research focus Future research will ask: “How do we find 3D models?” Future research will ask: “How do we find 3D models?” Utah VW BugUtah TeapotStanford Bunny Images courtesy of Stanford & Utah Previous research has asked: “How do we construct 3D models?” Previous research has asked: “How do we construct 3D models?”

Challenge Given: A database of 3D models and a query model Find: The k database models most similar to the query Images courtesy of Google & Princeton

Approach To retrieve the nearest k models: Compute the distance between the query and every database model. Sort the database models by proximity. Return the first k matches. 3D Query Database Models Best Match(es) Sorted Models Sort by proximity Query comparison

Comparing 3D Models Direct Approach: Establish pair-wise correspondences between points on the surfaces of the two models. Define the distance between the models as the distance between corresponding points. pnpn pnpn qnqn qnqn p n-1 q n-1 q3q3 q3q3 q2q2 q1q1 q1q1 p3p3 p2p2 p1p1 Similarity defined as distance between models  Establishing correspondences is difficult and slow

Comparing 3D Models Practical Approach: Represent each 3D model by a shape descriptor. Define the distance between two models as the distance between their shape descriptors. Shape Descriptors :  Extended Gaussian Images, Horn  Complex Extended Gaussian Images, Kang et al.  Spherical Attribute Images, Delingette et al.  Crease Histograms, Besl  Shape Histograms, Ankerst et al.  Shape Distributions, Osada et al.  Spherical Extent Functions, Vranic et al.  Gaussian EDTs, Funkhouser et al.  Symmetry Descriptors, Kazhdan et al. Approximates distance between models Correspondences are implicit Comparison is easy and fast

Observation It is not enough to consider the distance between two 3D surfaces… We also need to consider how the surfaces transform into each other. M1M1 Q Q M2M2 M2M2 Q M1M1 ?

Match models in two steps: 1. Factor out low-frequency alignment of the models 2. Match the aligned models Define similarity by combining low-frequency alignment info with high-frequency difference Our Approach Isotropic Models Anisotropy

Outline Introduction Aligning Anisotropic Scales Related Work Anisotropy Normalization Convergence Properties Shape Matching Conclusion and Future Work

Point Set Alignment Given point sets P={p 1,…,p n } and Q={q 1,…q n }, what is the optimal alignment A minimizing: PQ Original [Horn, 1987]

Point Sets (Translation) Translate so that the center is at the origin: A model can be aligned for translation independent of what it will be compared against PQ Original [Horn, 1987] Translated P’Q’

Point Sets (Isotropic Scale) Scale so that mean variance from center is equal to 1: A model can be aligned for isotropic scale independent of what it will be compared against PQ OriginalTranslated P’Q’P”Q” Scaled [Horn, 1987]

Point Sets (Anisotropic Scale) Scale so that the variance in every direction equal to 1: A model can be aligned for anisotropic scale independent of what it will be compared against Anisotropic ModelsIsotropic Models

Unit Variance Unit Variance in Every Direction Covariance Matrix is Identity (Covariance Ellipse is a Sphere) Anisotropic Model Isotropic Model Initial Point Set Rescaled Point Set Covariance Ellipse For point sets, transform by inverse square root of the covariance matrix

From Points to Surfaces Points samples from a surface become isotropic but the sampled surface does not. Point Set ModelSurface Model

From Points to Surfaces Uniform samples do not stay uniform… Initial Point Set Isotropic Point Set

From Points to Surfaces Uniform samples do not stay uniform… But the model gets more isotropic. Iteratively rescale to get models that are progressively more isotropic

Convergence of Iteration Provably convergent  Show that in the worst case smallest eigenvalue doesn’t get smaller and largest one doesn’t get larger.  Use the triangle inequality to show that at least one of the eigenvalues has to change. In practice, converges very quickly Tested on 1890 Viewpoint models

Outline Introduction Aligning Anisotropic Scales Shape Matching Extending Shape Descriptors Experimental Results Conclusion and Future Work

Product Descriptor For any shape descriptor, we define a new shape descriptor that is the product of: The descriptor of the isotropic model, and The anisotropic scales Initial Model Isotropic Model Rescaling Ellipse Descriptor Eigenvalues New Descriptor

Factored Matching Isotropic ModelsAnisotropy  Parameterized family of shape metrics, as a function of anisotropy importance .

Experimental Database Princeton Shape Benchmark ~900 models, 90 classes 14 biplanes50 human bipeds7 dogs17 fish 16 swords6 skulls15 desk chairs13 electric guitars

Example Query Results Without Anisotropy Factorization Results With Anisotropy Factorization (  =3) Query Gaussian EDT, Funkhouser et al. 2003

Retrieval Results (  =3) DescriptorDimImprovement SHELLS1D63% D21D36% EGI * 1D64% CEGI * 1D28% Sectors * 1D31% EXT * 1D39% REXT * 2D16% Voxel * 2D23% Sectors + Shells * 2D35% Gaussian EDT * 2D4% Rotation Invariant Descriptors DescriptorDimImprovement EGI2D29% CEGI2D20% Sectors2D5% EXT2D7% REXT3D4% Voxel3D1% Sectors + Shells3D7% Gaussian EDT3D4% Rotation Varying Descriptors * Spherical Power Spectrum Representation

Outline Introduction Aligning Anisotropic Scales Shape Matching Conclusion and Future Work

Conclusion Presented and iterative approach for transforming anisotropic models into isotropic ones: Provides a method for factoring shape matching Improves matching for all descriptors Facilitates registration of models Initial Model Isotropic Model Rescaling Ellipse Descriptor Eigenvalues New Descriptor

Conclusion Presented and iterative approach for transforming anisotropic models into isotropic ones: Provides a method for factoring shape matching Gives rise to improved matching retrieval results Facilitates registration of models Anisotropic Models

Conclusion Presented and iterative approach for transforming anisotropic models into isotropic ones: Provides a method for factoring shape matching Gives rise to improved matching retrieval results Facilitates registration of models Isotropic Models

Future Work Address cross-class anisotropy variance Factor out higher order transformations Without Anisotropy Factorization With Anisotropy Factorization Without Anisotropy Factorization With Anisotropy Factorization

Future Work Address cross-class anisotropy variance Factor out higher order transformations Translation Anisotropic Scale ? Isotropic Scale Rotation

Thank You Funding National Science Foundation Source Code Dan Rockmore and Peter Kostelec Databases Viewpoint Data Labs, Cacheforce, De Espona Infografica Princeton Shape Matching Group Patrick Min and Phil Shilane

Thank You