1. Harmonic 3D Shape Matching Michael Kazhdan Thomas Funkhouser Princeton University Michael Kazhdan Thomas Funkhouser Princeton University

2. Motivation Large databases of 3D models Mechanical CAD (National Design Repository) Molecular Biology (Audrey Sanderson) Computer Graphics (Princeton 3D Search Engine)

3. Goal Find 3D models with similar shapes 3D ModelShape Descriptor Model Database Nearest Neighbor

4. Research Challenge Need shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match Need shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match 3D ModelShape Descriptor Model Database Nearest Neighbor

5. Research Challenge Finding a 3D shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match Finding a 3D shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match 3D ModelShape Descriptor Nearest Neighbor Model Database

6. Research Challenge Finding a 3D shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match Finding a 3D shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match Model Database Nearest Neighbor Shape Descriptor 3D Model

7. Research Challenge Finding a 3D shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match Finding a 3D shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match 3D Model Nearest Neighbor Shape Descriptor Model Database

8. Research Challenge Finding a 3D shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match Finding a 3D shape descriptor that is: DiscriminatingDiscriminating Concise to storeConcise to store Quick to computeQuick to compute Efficient to matchEfficient to match 3D Model Nearest Neighbor Shape Descriptor Model Database Many possible alignments

9. 3D Model Matching Approaches Search over all possible alignments Too slow for large databaseToo slow for large database Search over all possible alignments Too slow for large databaseToo slow for large database min - - - - - -

10. 3D Model Matching Approaches Search over all possible alignments Too slow for large databaseToo slow for large database Normalize alignment (e.g., with moments) OK for translation and scale, not for rotationOK for translation and scale, not for rotation Search over all possible alignments Too slow for large databaseToo slow for large database Normalize alignment (e.g., with moments) OK for translation and scale, not for rotationOK for translation and scale, not for rotation PCA Aligned Models

11. 3D Model Matching Approaches Search over all possible alignments Too slow for large databaseToo slow for large database Normalize alignment (e.g., with moments) OK for translation and scale, not for rotationOK for translation and scale, not for rotation Build alignment invariance into descriptor Previous methods not very discriminatingPrevious methods not very discriminating Search over all possible alignments Too slow for large databaseToo slow for large database Normalize alignment (e.g., with moments) OK for translation and scale, not for rotationOK for translation and scale, not for rotation Build alignment invariance into descriptor Previous methods not very discriminatingPrevious methods not very discriminating Shape Histograms [Ankerst et al., 1999] Shape Histograms [Ankerst et al., 1999]

12. Outline IntroductionApproachImplementation Experimental Results Conclusion and Future Work IntroductionApproachImplementation Experimental Results Conclusion and Future Work

13. Our Approach Harmonic 3D shape descriptor Decompose 3D shapes into irreducible set of rotation independent componentsDecompose 3D shapes into irreducible set of rotation independent components Store “how much” of the model resides in each componentStore “how much” of the model resides in each component Harmonic 3D shape descriptor Decompose 3D shapes into irreducible set of rotation independent componentsDecompose 3D shapes into irreducible set of rotation independent components Store “how much” of the model resides in each componentStore “how much” of the model resides in each component 3D Model Shape Descriptor Rotation Independent Components

14. Our Approach Harmonic 3D shape descriptor Decompose 3D shapes into irreducible set of rotation independent componentsDecompose 3D shapes into irreducible set of rotation independent components Store “how much” of the model resides in each componentStore “how much” of the model resides in each component Harmonic 3D shape descriptor Decompose 3D shapes into irreducible set of rotation independent componentsDecompose 3D shapes into irreducible set of rotation independent components Store “how much” of the model resides in each componentStore “how much” of the model resides in each component Concentric Spheres 3D Model Shape Descriptor Rotation Independent Components

15. Our Approach Harmonic 3D shape descriptor Decompose 3D shapes into irreducible set of rotation independent componentsDecompose 3D shapes into irreducible set of rotation independent components Store “how much” of the model resides in each componentStore “how much” of the model resides in each component Harmonic 3D shape descriptor Decompose 3D shapes into irreducible set of rotation independent componentsDecompose 3D shapes into irreducible set of rotation independent components Store “how much” of the model resides in each componentStore “how much” of the model resides in each component Frequency Decomposition 3D Model Shape Descriptor Rotation Independent Components

16. Our Approach Harmonic 3D shape descriptor Decompose 3D shapes into irreducible set of rotation independent componentsDecompose 3D shapes into irreducible set of rotation independent components Store “how much” of the model resides in each componentStore “how much” of the model resides in each component Harmonic 3D shape descriptor Decompose 3D shapes into irreducible set of rotation independent componentsDecompose 3D shapes into irreducible set of rotation independent components Store “how much” of the model resides in each componentStore “how much” of the model resides in each component 3D Model Shape Descriptor Rotation Independent Components Amplitudes

17. Outline IntroductionApproachImplementation Experimental Results Conclusion and Future Work IntroductionApproachImplementation Experimental Results Conclusion and Future Work

18. Voxelization Convert polygonal model to 3D voxel grid Rasterize surfaces (no solid reconstruction)Rasterize surfaces (no solid reconstruction) Normalize for translation and scaleNormalize for translation and scale Convert polygonal model to 3D voxel grid Rasterize surfaces (no solid reconstruction)Rasterize surfaces (no solid reconstruction) Normalize for translation and scaleNormalize for translation and scale 3D Model3D Voxel Grid

19. Spherical Decomposition Intersect with concentric spheres

20. Frequency Decomposition Represent each spherical function as a sum of different frequencies Spherical Functions Frequency Components

21. Fourier Analysis Circular Function

22. Fourier Analysis +++=+ … Cosine/Sine Decomposition Circular Function

23. Fourier Analysis = +++ Constant =+ … Frequency Decomposition Circular Function

24. Fourier Analysis = +++ + Constant1 st Order =+ … + Frequency Decomposition Circular Function

25. Fourier Analysis = +++ ++ Constant1 st Order2 nd Order =+ … + Frequency Decomposition Circular Function

26. Fourier Analysis = +++ +++ Constant1 st Order2 nd Order3 rd Order =+ … + … + Frequency Decomposition Circular Function

27. ++++ … + Fourier Analysis = +++ Constant1 st Order2 nd Order3 rd Order + … Frequency Decomposition = Amplitudes invariant to rotation Circular Function

28. Harmonic Analysis Spherical Function

29. Harmonic Analysis +++=+ … Spherical Function Harmonic Decomposition

30. Harmonic Analysis = +++ +++ Constant1 st Order2 nd Order3 rd Order =+ … + … Spherical Function

31. Building Shape Descriptor Store “how much” (L 2 -norm) of the shape resides in each frequency of each sphere Frequency Decomposition Amplitudes Harmonic Shape Descriptor

32. Matching Model similarity defined as L 2 -distance between their descriptors Bounds proximity of voxel grids over all rotationsBounds proximity of voxel grids over all rotations Model similarity defined as L 2 -distance between their descriptors Bounds proximity of voxel grids over all rotationsBounds proximity of voxel grids over all rotations, = - - - - - Sim

33. Outline IntroductionApproachImplementation Experimental Results Conclusion and Future Work IntroductionApproachImplementation Experimental Results Conclusion and Future Work

34. Princeton 3D Search Engine Query

35. Retrieval Experiment Viewpoint “household” database 1,890 models, 85 classes 153 dining chairs25 livingroom chairs16 beds12 dining tables 8 chests28 bottles39 vases36 end tables

36. Retrieval Results Precision-recall curve (mean for all queries) 00.20.40.60.81 0 0.2 0.4 0.6 0.8 1 Recall Precision Random 3D Harmonics Query

37. Retrieval Results Precision versus recall (mean for all queries) 00.20.40.60.8 0 0.2 0.4 0.6 0.8 1 Recall Precision 3D Harmonics (Our Method) D2 Shape Distributions [Osada et al., 2001] Shape Histograms [Ankerst, 1999] EGI [Horn, 1984] Moments [Elad et al., 2001] Random 1

38. Retrieval Results Precision versus recall (mean for all queries) 00.20.40.60.8 0 0.2 0.4 0.6 0.8 1 Recall Precision 3D Harmonics (Our Method) D2 Shape Distributions [Osada et al., 2001] Shape Histograms [Ankerst, 1999] EGI [Horn, 1984] Moments [Elad et al., 2001] Random 1

39. Retrieval Results Precision versus recall (mean for all queries) 00.20.40.60.8 0 0.2 0.4 0.6 0.8 1 Recall Precision 3D Harmonics (Our Method) D2 Shape Distributions [Osada et al., 2001] Shape Histograms [Ankerst, 1999] EGI [Horn, 1984] Moments [Elad et al., 2001] Random 1

40. Retrieval Results Precision versus recall (mean for all queries) 00.20.40.60.8 0 0.2 0.4 0.6 0.8 1 Recall Precision 3D Harmonics (Our Method) D2 Shape Distributions [Osada et al., 2001] Shape Histograms [Ankerst, 1999] EGI [Horn, 1984] Moments [Elad et al., 2001] Random 1

41. Summary and Conclusion Harmonic shape descriptor is a rotation invariant representation that is: Discriminating (46%-245% better than others tested)Discriminating (46%-245% better than others tested) Concise to store (2048 bytes)Concise to store (2048 bytes) Quick to compute (1-5 seconds)Quick to compute (1-5 seconds) Efficient to match (0.45 seconds: 20,000 model DB)Efficient to match (0.45 seconds: 20,000 model DB) Harmonic shape descriptor is a rotation invariant representation that is: Discriminating (46%-245% better than others tested)Discriminating (46%-245% better than others tested) Concise to store (2048 bytes)Concise to store (2048 bytes) Quick to compute (1-5 seconds)Quick to compute (1-5 seconds) Efficient to match (0.45 seconds: 20,000 model DB)Efficient to match (0.45 seconds: 20,000 model DB)

42. Future Work Extensions Partial object matching?Partial object matching? Other 3D shape functions?Other 3D shape functions? Other applications Molecular biologyMolecular biology MedicineMedicine PaleontologyPaleontology ForensicsForensicsExtensions Partial object matching?Partial object matching? Other 3D shape functions?Other 3D shape functions? Other applications Molecular biologyMolecular biology MedicineMedicine PaleontologyPaleontology ForensicsForensics Query http://shape.cs.princeton.edu/

43. Thank You Funding National Science FoundationNational Science Foundation Sloan FoundationSloan FoundationPeople Bernard Chazelle, David Dobkin, David Jacobs, David Kazhdan, Allison Klein, Patrick Min, Szymon Rusinkiewicz, Peter Sarnak, Julianna TymoczkoBernard Chazelle, David Dobkin, David Jacobs, David Kazhdan, Allison Klein, Patrick Min, Szymon Rusinkiewicz, Peter Sarnak, Julianna TymoczkoFunding National Science FoundationNational Science Foundation Sloan FoundationSloan FoundationPeople Bernard Chazelle, David Dobkin, David Jacobs, David Kazhdan, Allison Klein, Patrick Min, Szymon Rusinkiewicz, Peter Sarnak, Julianna TymoczkoBernard Chazelle, David Dobkin, David Jacobs, David Kazhdan, Allison Klein, Patrick Min, Szymon Rusinkiewicz, Peter Sarnak, Julianna Tymoczko http://shape.cs.princeton.edu/

45. Analysis The Harmonic Descriptor is not invertible Different spheres rotate independentlyDifferent spheres rotate independently Different orders rotate independentlyDifferent orders rotate independently Rotations are not transitive within an orderRotations are not transitive within an order The Harmonic Descriptor is not invertible Different spheres rotate independentlyDifferent spheres rotate independently Different orders rotate independentlyDifferent orders rotate independently Rotations are not transitive within an orderRotations are not transitive within an order

46. Analysis The Harmonic Descriptor is not invertible Different spheres rotate independentlyDifferent spheres rotate independently Different orders rotate independentlyDifferent orders rotate independently Rotations are not transitive within an orderRotations are not transitive within an order The Harmonic Descriptor is not invertible Different spheres rotate independentlyDifferent spheres rotate independently Different orders rotate independentlyDifferent orders rotate independently Rotations are not transitive within an orderRotations are not transitive within an order l=0 l=1 l=2 l=3

47. Inter-Radial Coherence Force same orders on different spheres to rotate together by setting up the rotation invariant matrix M k with: where f k,j is the k-th order component of the restriction of f to the j-th radius. (The diagonal is precisely the collection of L 2 -norms.) Force same orders on different spheres to rotate together by setting up the rotation invariant matrix M k with: where f k,j is the k-th order component of the restriction of f to the j-th radius. (The diagonal is precisely the collection of L 2 -norms.)

48. Polygon Rasterization Rasterize using the Euclidean Distance Transform to measure how much models miss by: Polygonal Model Voxel Model