1. Spectral bases for 3D shapes representation

2. Spectral bases – reminder Last week we spoke of Fourier analysis  1D sine/cosine bases for signals:

3. Spectral bases – reminder Last week we spoke of Fourier analysis  2D bases for 2D signals (images)

4. How about 3D shapes? Problem: general 3D shapes cannot be described as height functions Height function, regularly sampled above a 2D domain General 3D shapes

5. Irregular meshes In graphics, shapes are mostly represented by triangle meshes

6. Irregular meshes In graphics, shapes are mostly represented by triangle meshes Geometry: Vertex coordinates (x 1, y 1, z 1 ) (x 2, y 2, z 2 )... (x n, y n, z n ) Connectivity: List of triangles (i 1, j 1, k 1 ) (i 2, j 2, k 2 )... (i m, j m, k m )

7. Irregular meshes In graphics, shapes are mostly represented by triangle meshes  The connectivity (the graph) can be very efficiently encoded About 2 bits per vertex only  The geometry (x,y,z) is heavy! When stored naively, at least 12 bits per coordinate are needed, i.e. 36 bits per vertex

8. How to define efficient bases? Extension of the 2D DCT basis to a general (irregular) mesh DCT???

9. Basis shapes We need a collection of basis functions  First basis functions will be very smooth, slowly-varying  Last basis functions will be high-frequency, oscillating We will represent our shape (mesh geometry) as a linear combination of the basis functions We will induce the basis functions from the mesh connectivity!  We assume that the connectivity is known to the decoder side (it is transferred first)

10. The Mesh Laplacian operator Measures the local smoothness at each mesh vertex

11. Laplacian operator in matrix form L matrix

12. Laplacian matrix L is constructed based on the connectivity alone So, if the decoder got the connectivity first, it can construct L

13. Spectral bases L is a symmetric n  n matrix Perform spectral analysis of L ! L = T 1 2 n Basis vectorsFrequencies, sorted in ascending order

14. How to represent our mesh geometry in the spectral basis? X, Y, Z  R n. Decompose in the spectral basis:

15. How to represent our mesh geometry in the spectral basis? Decompose the mesh geometry in the spectral basis: The first components are low-frequency The last components are high-frequency

16. The spectral basis First functions are smooth and slow, last oscillate a lot horse connectivity spectral basis of L = the DCT basis chain connectivity 2 nd basis function 10 th basis function 100 th basis function

17. Why this works? Low-frequency basis vectors are smooth and slowly-varying:  The first basis vector:  It is the smoothest possible function (Laplacian is constant zero on it)

18. Why this works? Low-frequency basis vectors are smooth and slowly-varying:  The first basis vector:  Lb 1 = 0 because all the rows of L sum to zero

19. Why this works? Low-frequency basis vectors are smooth and slowly-varying:  The following basis vectors: The frequency is small  the Laplacian operator gives small values  the shape of the function is smooth

20. Why this works? High-frequency basis vectors are non- smooth, oscillating: The frequency is large  the Laplacian operator gives large values  the shape of the function is less smooth!

21. Geometry compression As in JPEG compression, drop the high-frequency components!

22. Geometry compression Increasing number of participating basis vectors. The less coefficients are dropped, the less shape information is lost

23. Technical algorithm The encoding side:  Compute the spectral bases from mesh connectivity  Represent the shape geometry in the spectral basis and decide how many coeffs. to leave (K)  Store the connectivity and the K non-zero coefficients The decoding side:  Compute the first K spectral bases from the connectivity  Combine them using the K received coefficients and get the shape

24. Challenges and trade-offs Performing spectral decomposition of a large matrix ( n>1000 ) is prohibitively expensive ( O(n 3 ) )  Today’s meshes come with 50,000 and more vertices  We don’t want the decompressor to work forever! Solution: like in JPEG, work on small blocks:

25. Challenges and trade-offs As in JPEG, blocking artifacts appear for low bit rates, because the compression is lossy (too much information is lost)

26. Further info “Spectral Mesh Compression”, Zachi Karni and Craig Gotsman, ACM SIGGRAPH 2000 http://www.mpi-inf.mpg.de/~karni/publications/spectral.pdf “3D Mesh compression using fixed spectral bases”, Z. Karni and C. Gotsman, Graphics Interface, 2001 http://www.mpi-inf.mpg.de/~karni/publications/fixed.pdf

27. See you next time