1. Shape Matching and Anisotropy Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz Princeton University Michael Kazhdan, Thomas Funkhouser, and Szymon Rusinkiewicz Princeton University

2. Motivation 3D data is becoming more commonly available Someday 3D models will be as common as images are today Cheap ScannersWorld Wide Web 3D Cafe Cyberware Fast Graphics Cards ATI Images courtesy of Cyberware, ATI, & 3Dcafe

3. Motivation When 3D models are ubiquitous, there will be a shift in research focus Future research will ask: “How do we find 3D models?” Future research will ask: “How do we find 3D models?” Utah VW BugUtah TeapotStanford Bunny Images courtesy of Stanford & Utah Previous research has asked: “How do we construct 3D models?” Previous research has asked: “How do we construct 3D models?”

4. Challenge Given: A database of 3D models and a query model Find: The k database models most similar to the query Images courtesy of Google & Princeton

5. Approach To retrieve the nearest k models: Compute the distance between the query and every database model. Sort the database models by proximity. Return the first k matches. 3D Query Database Models Best Match(es) Sorted Models Sort by proximity Query comparison

6. Comparing 3D Models Direct Approach: Establish pair-wise correspondences between points on the surfaces of the two models. Define the distance between the models as the distance between corresponding points. pnpn pnpn qnqn qnqn p n-1 q n-1 q3q3 q3q3 q2q2 q1q1 q1q1 p3p3 p2p2 p1p1 Similarity defined as distance between models  Establishing correspondences is difficult and slow

7. Comparing 3D Models Practical Approach: Represent each 3D model by a shape descriptor. Define the distance between two models as the distance between their shape descriptors. Shape Descriptors :  Extended Gaussian Images, Horn  Complex Extended Gaussian Images, Kang et al.  Spherical Attribute Images, Delingette et al.  Crease Histograms, Besl  Shape Histograms, Ankerst et al.  Shape Distributions, Osada et al.  Spherical Extent Functions, Vranic et al.  Gaussian EDTs, Funkhouser et al.  Symmetry Descriptors, Kazhdan et al. Approximates distance between models Correspondences are implicit Comparison is easy and fast

8. Observation It is not enough to consider the distance between two 3D surfaces… We also need to consider how the surfaces transform into each other. M1M1 Q Q M2M2 M2M2 Q M1M1 ?

9. Match models in two steps: 1. Factor out low-frequency alignment of the models 2. Match the aligned models Define similarity by combining low-frequency alignment info with high-frequency difference Our Approach Isotropic Models Anisotropy

10. Outline Introduction Aligning Anisotropic Scales Related Work Anisotropy Normalization Convergence Properties Shape Matching Conclusion and Future Work

11. Point Set Alignment Given point sets P={p 1,…,p n } and Q={q 1,…q n }, what is the optimal alignment A minimizing: PQ Original [Horn, 1987]

12. Point Sets (Translation) Translate so that the center is at the origin: A model can be aligned for translation independent of what it will be compared against PQ Original [Horn, 1987] Translated P’Q’

13. Point Sets (Isotropic Scale) Scale so that mean variance from center is equal to 1: A model can be aligned for isotropic scale independent of what it will be compared against PQ OriginalTranslated P’Q’P”Q” Scaled [Horn, 1987]

14. Point Sets (Anisotropic Scale) Scale so that the variance in every direction equal to 1: A model can be aligned for anisotropic scale independent of what it will be compared against Anisotropic ModelsIsotropic Models

15. Unit Variance Unit Variance in Every Direction Covariance Matrix is Identity (Covariance Ellipse is a Sphere) Anisotropic Model Isotropic Model Initial Point Set Rescaled Point Set Covariance Ellipse For point sets, transform by inverse square root of the covariance matrix

16. From Points to Surfaces Points samples from a surface become isotropic but the sampled surface does not. Point Set ModelSurface Model

17. From Points to Surfaces Uniform samples do not stay uniform… Initial Point Set Isotropic Point Set

18. From Points to Surfaces Uniform samples do not stay uniform… But the model gets more isotropic. Iteratively rescale to get models that are progressively more isotropic

19. Convergence of Iteration Provably convergent  Show that in the worst case smallest eigenvalue doesn’t get smaller and largest one doesn’t get larger.  Use the triangle inequality to show that at least one of the eigenvalues has to change. In practice, converges very quickly Tested on 1890 Viewpoint models

20. Outline Introduction Aligning Anisotropic Scales Shape Matching Extending Shape Descriptors Experimental Results Conclusion and Future Work

21. Product Descriptor For any shape descriptor, we define a new shape descriptor that is the product of: The descriptor of the isotropic model, and The anisotropic scales Initial Model Isotropic Model Rescaling Ellipse Descriptor Eigenvalues New Descriptor

22. Factored Matching Isotropic ModelsAnisotropy  Parameterized family of shape metrics, as a function of anisotropy importance .

23. Experimental Database Princeton Shape Benchmark ~900 models, 90 classes 14 biplanes50 human bipeds7 dogs17 fish 16 swords6 skulls15 desk chairs13 electric guitars http://shape.cs.princeton.edu/benchmark/

24. Example Query Results Without Anisotropy Factorization Results With Anisotropy Factorization (  =3) Query 1 234 87 65 1 234 87 65 Gaussian EDT, Funkhouser et al. 2003

25. Retrieval Results (  =3) DescriptorDimImprovement SHELLS1D63% D21D36% EGI * 1D64% CEGI * 1D28% Sectors * 1D31% EXT * 1D39% REXT * 2D16% Voxel * 2D23% Sectors + Shells * 2D35% Gaussian EDT * 2D4% Rotation Invariant Descriptors DescriptorDimImprovement EGI2D29% CEGI2D20% Sectors2D5% EXT2D7% REXT3D4% Voxel3D1% Sectors + Shells3D7% Gaussian EDT3D4% Rotation Varying Descriptors * Spherical Power Spectrum Representation

26. Outline Introduction Aligning Anisotropic Scales Shape Matching Conclusion and Future Work

27. Conclusion Presented and iterative approach for transforming anisotropic models into isotropic ones: Provides a method for factoring shape matching Improves matching for all descriptors Facilitates registration of models Initial Model Isotropic Model Rescaling Ellipse Descriptor Eigenvalues New Descriptor

28. Conclusion Presented and iterative approach for transforming anisotropic models into isotropic ones: Provides a method for factoring shape matching Gives rise to improved matching retrieval results Facilitates registration of models Anisotropic Models

29. Conclusion Presented and iterative approach for transforming anisotropic models into isotropic ones: Provides a method for factoring shape matching Gives rise to improved matching retrieval results Facilitates registration of models Isotropic Models

30. Future Work Address cross-class anisotropy variance Factor out higher order transformations 1 234 5678 1 234 5678 Without Anisotropy Factorization With Anisotropy Factorization 1 234 5678 Without Anisotropy Factorization 1 234 5678 With Anisotropy Factorization

31. Future Work Address cross-class anisotropy variance Factor out higher order transformations Translation Anisotropic Scale ? Isotropic Scale Rotation

32. Thank You Funding National Science Foundation Source Code Dan Rockmore and Peter Kostelec http://www.cs.dartmouth.edu/~geelong/sphere http://www.cs.dartmouth.edu/~geelong/soft Databases Viewpoint Data Labs, Cacheforce, De Espona Infografica http://www.viewpoint.com http://www.cacheforce.com http://www.deespona.com Princeton Shape Matching Group Patrick Min and Phil Shilane http://shape.cs.princeton.edu

33. Thank You