1. Signal Processing and Representation Theory Lecture 3

2. Outline: Review Spherical Harmonics Rotation Invariance Correlation and Wigner-D Functions

3. Representation Theory Review Given a representation  of a group G onto an inner product space V, decomposing V into the direct sum of irreducible sub-representations: V=V 1  …  V n makes it easier to: –Compute the correlation between two vectors: fewer multiplications are needed –Obtain G-invariant information: more transformation invariant norms can be obtained

4. Representation Theory Review In the case that the group G is commutative, the irreducible sub-representations V i are all one- complex-dimensional, (Schur’s Lemma). Example: If V is the space of functions on a circle, represented by n- dimensional arrays, and G is the group of 2D rotations: –Correlation can be done in O(n log n) time (using the FFT) –We can obtain n/2-dimensional, rotation invariant descriptors

5. Representation Theory What happens when the group G is not commutative? Example: If V is the space of functions on a sphere and G is the group of 3D rotations: –How quickly can we correlate? –How much rotation invariant information can we get?

6. Outline: Review Spherical Harmonics Rotation Invariance Correlation and Wigner-D Functions

7. Representation Theory Spherical Harmonic Decomposition Goal: Find the irreducible sub-representations of the group of 3D rotation acting on the space of spherical functions.

8. Representation Theory Spherical Harmonic Decomposition Preliminaries: If f is a function defined in 3D, we can get a function on the unit sphere by looking at the restriction of f to points with norm 1.

9. Representation Theory Spherical Harmonic Decomposition Preliminaries: A polynomial p(x,y,z) is homogenous of degree d if it is the linear sum of monomials of degree d:

10. Representation Theory Spherical Harmonic Decomposition Preliminaries: We can think of the space of homogenous polynomials of degree d in x, y, and z as: where P d (x,y) is the space of homogenous polynomials of degreed d in x and y.

11. Representation Theory Spherical Harmonic Decomposition Preliminaries: If we let P d (x,y,z) be the set of homogenous polynomials of degree d, then P d (x,y,z) is a vector- space of dimension:

12. Representation Theory Spherical Harmonic Decomposition Observation: If M is any 3x3 matrix, and p(x,y,z) is a homogenous polynomial of degree d: then p(M(x,y,z)) is also a homogenous polynomial of degree d:

13. Representation Theory Spherical Harmonic Decomposition If V is the space of functions on the sphere, we can consider the sub-space of functions on the sphere that are restrictions of homogenous polynomials of degree d. Since a rotation will map a homogenous polynomial of degree d back to a homogenous polynomial of degree d, these sub-spaces are sub-representations.

14. Representation Theory Spherical Harmonic Decomposition In general, the space of homogenous polynomials of degree d has dimension (d+1)+(d)+(d-1)+…+1:

15. Representation Theory Spherical Harmonic Decomposition If (x,y,z) is a point on the sphere, we know that this point satisfies: Thus, if q(x,y,z)  P d (x,y,z), then even though in general, the polynomial: is a homogenous polynomial of degree d+2, its restriction to the sphere is actually a homogenous polynomial of degree d.

16. Representation Theory Spherical Harmonic Decomposition So, while the sub-spaces P d (x,y,z) are sub- representations, they are not irreducible as P d-2 (x,y,z)  P d (x,y,z). To get the irreducible sub-representations, we look at the spaces:

17. Representation Theory Spherical Harmonic Decomposition And the dimension of these sub-representations is:

18. Representation Theory Spherical Harmonic Decomposition The spherical harmonics of frequency d are an orthonormal basis for the space of functions V d. If we represent a point on a sphere in terms of its angle of elevation and azimuth: with 0  π and 0  <2π …

19. Representation Theory Spherical Harmonic Decomposition The spherical harmonics are functions Y l m, with l  0 and -l  m  l spanning the sub-representations V l :

20. Representation Theory Spherical Harmonic Decomposition Fact: If we have a function defined on the sphere, sampled on a regular nxn grid of angles of elevation and azimuth, the forward and inverse spherical harmonic transforms can be computed in O(n 2 log 2 n). Like the FFT, the fast spherical harmonic transform can be thought of as a change of basis, and a brute force method would take O(n 4 ) time.

21. Representation Theory What are the spherical harmonics Y l m ( ,  )?

22. Representation Theory What are the spherical harmonics Y l m ? Conceptually: The Y l m are the different homogenous polynomials of degree l:

23. Representation Theory What are the spherical harmonics Y l m ? Technically: Where the P l m are the associated Legendre polynomials: Where the P l are the Legendre polynomials:

24. Representation Theory What are the spherical harmonics Y l m ? Functionally: The Y l m are the eigen-values of the Laplacian operator:

25. Representation Theory What are the spherical harmonics Y l m ? Visually: The Y l m are spherical functions whose number of lobes get larger as the frequency, l, gets bigger: l=1l=1 l=2l=2 l=3l=3 l=0l=0

26. Representation Theory What are the spherical harmonics Y l m ? What is important about the spherical harmonics is that they are an orthonormal basis for the (2d+1)- dimensional sub-representations, V d, of the group of 3D rotations acting on the space of spherical functions.

27. Representation Theory Sub-Representations

28. Representation Theory Sub-Representations

29. Representation Theory Sub-Representations

30. Representation Theory Sub-Representations

31. Outline: Review Spherical Harmonics Rotation Invariance Correlation and Wigner-D Functions

32. Representation Theory Invariance Given a spherical function f, we can obtain a rotation invariant representation by expressing f in terms of its spherical harmonic decomposition: where each f l  V l :

33. Representation Theory Invariance We can then obtain a rotation invariant representation by storing the size of each f l independently: where:

34. Representation Theory Invariance Spherical Harmonic Decomposition + +=+

35. Representation Theory Invariance + +=+ +++ Constant1 st Order2 nd Order 3 rd Order

36. Representation Theory Invariance +++ Constant1 st Order2 nd Order 3 rd Order Ψ

37. Representation Theory Invariance Limitations: By storing only the energy in the different frequencies, we discard information that does not depend on the pose of the model: –Inter-frequency information –Intra-frequency information

38. + Representation Theory Invariance Inter-Frequency information: 22.5 o 90 o = = +

39. Representation Theory Invariance Intra-Frequency information:

40. Representation Theory Invariance … … … O(n2)O(n2) O(n)O(n)

41. Representation Theory Invariance … … … O(n2)O(n2) O(n)O(n)

42. Representation Theory Invariance … … … O(n2)O(n2) O(n)O(n)

43. Representation Theory Invariance … … … O(n2)O(n2) O(n)O(n)

44. Representation Theory Invariance … … … O(n2)O(n2) O(n)O(n)

45. Outline: Review Spherical Harmonics Rotation Invariance Correlation and Wigner-D Functions

46. Representation Theory Wigner-D Functions The Wigner-D functions are an orthogonal basis of complex-valued functions defined on the space of rotations: with l  0 and -l  m,m’  l.

47. Representation Theory Wigner-D Functions Fact: If we are given a function defined on the group of 3D rotations, sampled on a regular nxnxn grid of Euler angles, the forward and inverse spherical harmonic transforms can be computed in O(n 4 ) time. Like the FFT and the FST, the fast Wigner-D transform can be thought of as a change of basis, and a brute force method would take O(n 6 ) time.

48. Representation Theory Motivation Given two spherical functions f and g we would like to compute the distance between f and g at every rotation. To do this, we need to be able to compute the correlation: Corr(f,g,R)=  f,R(g)  at every rotation R.

49. Representation Theory Correlation If we express f and g in terms of their spherical harmonic decompositions:

50. Representation Theory Correlation Then the correlation of f with g at a rotation R is given by:

51. Representation Theory Correlation So that we get an expression for the correlation of f with g as some linear combination of the Wigner-D functions:

52. Representation Theory Correlation The complexity of correlating two spherical functions sampled on a regular nxn grid is: –Forward spherical harmonic transform: O(n 2 log 2 n)

53. Representation Theory Correlation The complexity of correlating two spherical functions sampled on a regular nxn grid is: –Forward spherical harmonic transform: O(n 2 log 2 n) –Multiplying frequency terms:O(n 3 )

54. Representation Theory Correlation The complexity of correlating two spherical functions sampled on a regular nxn grid is: –Forward spherical harmonic transform: O(n 2 log 2 n) –Multiplying frequency terms:O(n 3 ) –Inverse Wigner-D transform:O(n 4 )

55. Representation Theory Correlation The complexity of correlating two spherical functions sampled on a regular nxn grid is: –Forward spherical harmonic transform: O(n 2 log 2 n) –Multiplying frequency terms:O(n 3 ) –Inverse Wigner-D transform:O(n 4 ) Total complexity of correlation is: O(n 4 ) (Note that a brute force approach would take O(n 5 ): For each of O(n 3 ) rotations we would have to perform an O(n 2 ) dot-product computation.)