1. TEMPLATE BASED SHAPE DESCRIPTOR Raif Rustamov Department of Mathematics and Computer Science Drew University, Madison, NJ, USA

2. Components of descriptors in general  Selection of surface feature  Mapping  Signal Processing  Need this discussion to set up the context for our approach

3. Selection of surface feature  A function on the surface that captures a property relevant to shape description:  constant function (restriction of the surface's characteristic function to the surface itself)  distance to the center of mass  curvature  components of the normal vector  We refer to the selected function as the feature function.

4. Mapping  The feature function is used to construct a new function defined on some predetermined domain  The new domain called the “mapping domain”  the new function the “mapped feature function”  Common mapping domains:  Spheres  Planes  the 3D space (surface's bounding volume)  surface itself  Mapping procedures:  projection  Identity, if mapping domain = surface itself

5. Signal Processing  Extract concise noise-robust numerical descriptor from the mapped feature function.  Depends on the mapping domain:  Sphere – Spherical Harmonic Transform  Plane or box volume – 2D or 3D Fourier transform  Ball volume – 3D Zernike Transform.  Mapped feature function is expanded in a series in terms of the relevant basis  Expansion coefficients are used as the shape descriptor

6. Example I: Saupe, Vranic 2001  Shoot rays from the origin (center of mass), determine the distance to the farthest intersection point with the bounding mesh  Parameterize the rays by the unit sphere to obtain a function on the sphere  Use spherical harmonic transform on this function to extract the numerical shape descriptor.

7. Example I: Saupe, Vranic 2001  Surface feature  Distance to the origin  Mapping  Mapping domain: the unit sphere  Mapping procedure: project onto the sphere, resolve collisions by selecting the larger function value  Signal processing  Spherical harmonic transform

8. Example II: Depth Buffer  Heczko, Keim, Saupe, Vranic 2002  Place a normalized mesh into a unit cube  Generate six gray-scale images on each face of the cube by parallel projection  The grayness value is the distance from the cube face to the model  Apply 2D Fourier transform to each of the six gray- scale images

9. Example II: Depth Buffer  For each cube face:  Surface feature  distance from mesh point to the face  Mapping  Mapping domain: cube face  Mapping procedure: project onto the face, resolve collisions by selecting the smaller function value  Signal processing  2D Fourier transform

10. Generality  More examples easily generated  Compare to classification in Bustos et al. survey  Mapping ≈ object abstraction  Signal processing ≈ numerical transformation

11. Observations  Mapping:  Mapping domain:  ≠ original surface  a primitive geometry: sphere, plane etc  Mapping procedure:  Projection  Signal processing  well established: Fourier, Zernike, Spherical Harmonics  limits possible mapping domains

12. Contributions  Mapping:  Mapping domain:  any fixed surface – template  Mapping procedure:  interpolation: mean-value coordinates, Shepard  Signal processing  via manifold harmonics – eigenfunctions of Laplace- Beltrami operator

13. Why templates?  Mademlis, Daras, Tzovaras, Strintzis 2008:  Since ellipses approximate elongated shapes better than spheres: Mapping domain: ellipsoid Signal processing: ellipsoidal harmonics  Showed experimentally better retrieval results than sphere + spherical harmonics  We take this idea further:  Mapping domain: any fixed surface

14. Why template ≠surface itself?  Expand the feature function in terms of the manifold harmonics of the original surface?  Problem: notoriously difficult to match the harmonics coming from different surfaces  Sign flipping  Eigenfunction switching  Linear combinations  Fixed template: extracted expansion coefficients are in direct correspondence

15. Why interpolation?  Projection  Mapped feature function can be discontinuous at overlaps  Gibbs effect may render low-frequency expansion coefficients used as the shape descriptor inadequate for representing the function Feature function is distance to the origin Jump discontinuity

16. Why interpolation?  Projection  Redundancy the value sets of the mapped feature functions on various templates will be almost the same limits the gains of concatenating descriptors obtained from different templates

17. Why interpolation?  Interpolation  No Gibbs effect mapped feature function is smooth  Less redundancy the value sets of the mapped feature functions on various templates depend on relative positions mean-value coordinates can inject more shape information into the mapped feature function a mesh can be reconstructed given the mean-value coordinates

18. Construction of the descriptor  Selection of surface feature  Mapping  Signal Processing  Now discuss details

19. Selection of surface feature  All models are normalized using shift, continuous PCA, isotropic scaling  Many possibilities, but not:  the characteristic function  nor linear function of coordinates  To focus discussion  f = distance from a mesh point to the origin  Similar to Saupe, Vranic 2001

20. Mapping  Model surface S, Template surface T  Given  Construct  Shepard interpolation  Mean-value interpolation

21. Mapping: Shepard  Model surface S, Template surface T  Given  Construct  0th order precision: constant functions reproduced

22. Mapping: Barycentric  Model surface S, Template surface T  Given  Construct  are barycentric coordinates of point p with respect to vertex  1st order precision: linear functions reproduced

23. Mapping: Barycentric  A few different kinds of barycentric coordinates  Mean-value, positive mean-value  Harmonic  Maximum Entropy  Green coordinates, Complex in 2D  We use mean-value coordinates  Closed formula  Fastest to evaluate

24. Signal processing  We have a function  Need a compact representation  Expand the function into series  Use low-frequency coefficients  Need a function basis on template surface T  Manifold harmonics = Laplace-Beltrami eigenfunctions

25. Signal Processing  Manifold harmonics generalize Fourier basis to Riemannian manifolds  Spherical harmonics = manifold harmonics on the sphere  Have similar properties  Orthogonal  Concept of frequency low-frequency coefficients are noise-robust convey essential information about function

26. Signal Processing  Egenvalues, eigenfunctions solve  Evaluation procedures well known  Solve symmetric eigenvalue problem for a matrix  We use cotangent Laplacian with voronoi point-areas  Pre-compute for the given templates and store  Our templates have about 500 vertices, the process takes less than 3 seconds  The storage for each template = 10,500 floats= =20*500+500 = #eigs * #vertices + #vertices to store eigenvectors to store point areas

27. Resulting shape descriptor  Feature function  Mapped feature function  The template’s feature function  Quotient function  Expand into series

28. Resulting shape descriptor  Feature vector, N=20  For template surface T,  Normalization: scale to get a = 1  Use L2 distance

29. Experiments: Benchmark  Models: watertight benchmark  400 closed surface models  20 equal object classes

30. Experiments: Implementation  Implemented in MATLAB  Use C++ for mean-value coordinate computation  Timing  About 1 minute per model when mean-value coordinates used  Could make faster if simplified the models

31. Experiments: Templates  Templates: randomly chosen models  Simplified using Qslim  Makes mean-value computation faster

32. Compare Mapping Methods  Template = sphere  Projection vs. Shepard (a=1,2,3) vs. Mean-value  At long distance behavior of mean-value interpolant is similar to that of Shepard with a=2

33. Compare templates  Mapping via mean-value interpolation  M

34. Compare templates  Beneficial to combine – relative independence  All templates are normalized as objects in the benchmark – span similar spatial regions  The descriptors could have been made even more independent if the templates were differently posed.

35. Future work  Investigate dependency between the nature of the template and the produced retrieval results  No “ideal" template for all kinds of shapes  Flexibility of our approach – choose optimal templates based on the shape database at hand  How to choose?  Can we design a rotationally invariant descriptor?