1. Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors Michael Kazhdan Thomas Funkhouser Szymon Rusinkiewicz Princeton University Michael Kazhdan Thomas Funkhouser Szymon Rusinkiewicz Princeton University

2. Motivation Large databases of 3D models Mechanical CAD (National Design Repository) Molecular Biology (Audrey Sanderson) Computer Graphics (Princeton 3D Search Engine)

3. Retrieval Approach 3D ModelShape Descriptor Model Database Nearest Neighbor

4. Shape Unchanged by Rotation =

5. Problem Many shape descriptors are functions that rotate with the shape Extended Gaussian Image [Horn ’84] Spherical Attribute Image [Ikeuchi ’95] Shape Histogram [Ankerst ’99] Spherical Extent Function [Vranic ’00] Reflective Symmetry Descriptor [Kazhdan ’02] Gaussian EDT [Funkhouser ’03]

6. Goal Compute similarity of shape descriptors independent of rotation -= ?

7. Brute Force Approach - - - - min (rotation) -= Impractical for databases

8. Normalization Use PCA to place models into a canonical coordinate frame Covariance Matrix Computation Principal Axis Alignment

9. Normalization Doesn’t always work Only second order informationOnly second order information Doesn’t always work Only second order informationOnly second order information

10. Our Approach Eliminate rotation dependence in spherical and 3D descriptors Shape Descriptor EGI [Horn ’84] SAI [Ikeuchi ’95] EXT [Vranic ’00] RSD [Kazhdan ’02] EDT [Funkhouser ’03] etc. Shape Descriptor

11. Our Approach Eliminate rotation dependence in spherical and 3D descriptors Rotation Invariant Representation Shape Descriptor

12. Outline IntroductionBackground Harmonic Representation Properties Experimental Results Conclusion and Future Work IntroductionBackground Harmonic Representation Properties Experimental Results Conclusion and Future Work

13. Key Idea Obtain rotation invariant representation by storing amplitude and eliminating phase + + ++ … = [Lo 1989] [Burel 1995]

14. Fourier Descriptors Circular Function

15. Fourier Descriptors +++=+ … Cosine/Sine Decomposition Circular Function

16. Fourier Descriptors = +++ Constant =+ … Frequency Decomposition Circular Function

17. Fourier Descriptors = +++ + Constant1 st Order =+ … + Frequency Decomposition Circular Function

18. Fourier Descriptors = +++ ++ Constant1 st Order2 nd Order =+ … + Frequency Decomposition Circular Function

19. Fourier Descriptors = +++ +++ Constant1 st Order2 nd Order3 rd Order =+ … + … + Frequency Decomposition Circular Function

20. ++++ … + Fourier Descriptors = +++ Constant1 st Order2 nd Order3 rd Order + … Frequency Decomposition = Amplitudes invariant to rotation Circular Function

21. Harmonic Representation Spherical Function

22. Harmonic Representation = Spherical Function ++++ … Harmonic Decomposition

23. Harmonic Representation = Spherical Function ++++ … + + ++ … = Constant1 st Order2 nd Order3 rd Order

24. Harmonic Representation + + ++ … = Norms Invariant to Rotation Store “how much” (L 2 -norm) of the shape resides in each frequency

25. 3D Function (Voxel Grid) Restrict to concentric spheres

26. 3D Function (Voxel Grid) = = = + + ++ + + + + + + + + Compute harmonic representation of each sphere independently

27. 3D Function (Voxel Grid) Combine harmonic representations Radius Frequency

28. Matching L 2 -difference of harmonic representations… Harmonic Representation - 2

29. Matching min (rotations) - - 22 … bounds proximity of descriptors over all rotations

30. Outline IntroductionBackground Harmonic Representation Properties Experimental Results Conclusion and Future Work IntroductionBackground Harmonic Representation Properties Experimental Results Conclusion and Future Work

31. Advantages The harmonic representations is: Rotation invariantRotation invariant Multi-resolutionMulti-resolution CompactCompact DiscriminatingDiscriminating The harmonic representations is: Rotation invariantRotation invariant Multi-resolutionMulti-resolution CompactCompact DiscriminatingDiscriminating

32. Compact … … …

33. …… … …

34. …… … …

35. …… … …

36. …… … …

37. Information Loss Intra-frequency information lossIntra-frequency information loss Cross-frequency information lossCross-frequency information loss Cross-radial information lossCross-radial information loss Intra-frequency information lossIntra-frequency information loss Cross-frequency information lossCross-frequency information loss Cross-radial information lossCross-radial information loss

38. Information Loss (Spherical Descriptor) Intra-frequency information lossIntra-frequency information loss Cross-frequency information lossCross-frequency information loss Intra-frequency information lossIntra-frequency information loss Cross-frequency information lossCross-frequency information loss

39. Information Loss (Spherical Descriptor) Intra-frequency information lossIntra-frequency information loss Cross-frequency information lossCross-frequency information loss Intra-frequency information lossIntra-frequency information loss Cross-frequency information lossCross-frequency information loss + + 22.5 o 90 o = =

40. Information Loss (3D Descriptor) Cross-radial information lossCross-radial information loss

41. Outline IntroductionBackground Harmonic Representation Properties Experimental Results Conclusion and Future Work IntroductionBackground Harmonic Representation Properties Experimental Results Conclusion and Future Work

42. Shape Descriptors Extended Gaussian Image Horn 1984 Spherical Extent Function Vranic 2000 Shape Histogram Ankerst 1999 Gaussian EDT Funkhouser 2003

43. Experimental Database Viewpoint “household” database 1,890 models, 85 classes 153 dining chairs25 livingroom chairs16 beds12 dining tables 8 chests28 bottles39 vases36 end tables

44. Gaussian EDT Results PCA-Normalized Results Harmonic Representation Results Query123456 78910 123456 78910

45. Gaussian EDT Results Precision vs. Recall 50%100% 0% 50% Recall Precision 0% Harmonics PCA

46. Retrieval Results Harmonics PCA Harmonics PCA Harmonics PCA Harmonics PCA EGI EDTEXT SECT 50%100% 0% 50% Recall Precision 0% 50%100% 0% 50% Recall Precision 0% 50%100% 0% 50% Recall Precision 0% 50%100% 0% 50% Recall Precision 0% EGI: Extended Gaussian Image SECT : Shape Histogram (Sectors) EXT : Spherical Extent Function EDT : Gaussian Euclidean Distance Transform

47. Retrieval Results Harmonics PCA EGI Harmonics PCA SECT 50%100% 0% 50% Recall Precision 0% Harmonics PCA EDT 50%100% 0% 50% Recall Precision 0% Harmonics PCA EXT 50%100% 0% 50% Recall Precision 0% 50%100% 0% 50% Recall Precision 0% EGI: Extended Gaussian Image SECT : Shape Histogram (Sectors) EXT : Spherical Extent Function EDT : Gaussian Euclidean Distance Transform

48. Exhaustive Gaussian EDT Results Harmonic PCA min L 2 100% 50% 0% 50%100% Recall Precision Gaussian EDT - - - - min (rotation)

49. Summary and Conclusion Provide a rotation invariant representation of shape descriptors that: Eliminates PCA dependenceEliminates PCA dependence Gives better matching performanceGives better matching performance Is more compactIs more compact Is a multi-resolution representationIs a multi-resolution representation Provide a rotation invariant representation of shape descriptors that: Eliminates PCA dependenceEliminates PCA dependence Gives better matching performanceGives better matching performance Is more compactIs more compact Is a multi-resolution representationIs a multi-resolution representation

50. Future Work Managing Information Loss Obtain cross radial information for 3D descriptorsObtain cross radial information for 3D descriptors Obtain cross frequency informationObtain cross frequency information Get finer resolution of rotation invariance within frequenciesGet finer resolution of rotation invariance within frequencies More Generally Consider new shape descriptorsConsider new shape descriptors Managing Information Loss Obtain cross radial information for 3D descriptorsObtain cross radial information for 3D descriptors Obtain cross frequency informationObtain cross frequency information Get finer resolution of rotation invariance within frequenciesGet finer resolution of rotation invariance within frequencies More Generally Consider new shape descriptorsConsider new shape descriptors

51. Thank You Funding National Science Foundation Sloan Foundation Spherical Harmonics Dan Rockmore and Peter Kostelec SpharmonicKit:http://www.cs.dartmouth.edu/~geelong/sphere 3D Shape Matching Patrick Min, Alex Halderman, Phil Shilane, David Jacobs, Joyce Chen Princeton 3D Model Search Engine: http://shape.cs.princeton.edu http://shape.cs.princeton.eduFunding National Science Foundation Sloan Foundation Spherical Harmonics Dan Rockmore and Peter Kostelec SpharmonicKit:http://www.cs.dartmouth.edu/~geelong/sphere 3D Shape Matching Patrick Min, Alex Halderman, Phil Shilane, David Jacobs, Joyce Chen Princeton 3D Model Search Engine: http://shape.cs.princeton.edu http://shape.cs.princeton.edu