1. Non-Euclidean Example: The Unit Sphere

2. Differential Geometry
Is the mathematical theory which…
…works with small ‘patches’ looking Euclidean…
…and do calculus over these patches.

3. Manifolds Open Sets Coordinate neighbourhood Compatible neighbourhoods
Talk about analytic manifold

4. Tangent Space Tangent Vectors Tangent Space, Inner Product along D
Norm:
depends on
varies smoothly

5. Geodesics and Metrics The shortest path between two points X, Y
is the geodesic in the
manifold.
The length of the geodesic is the distance between the two points

6. Exponential and Logarithm Maps
: Maps tangents to the manifold
: Maps points on the manifold to
Both maps are locally defined only

7. Gradient
In euclidean space: direction of fastest increase
On the manifold: tangent of fastest increase
is a real valued function on the manifold
The gradient at X is is unique satisfying
directional derivative along D

8. Vector Space vs. Manifold Space
X the “origin”, Y an other point
X. Pennec, P. Fillard and N. Ayache , “A Riemannian Framework for Tensor Computing,” International. Journal of Computer Vision., 66(1), 41–66, 2006.

9. Sets of matrices, general linear group GL(n,R)
Lie Groups
Sets of matrices, general linear group GL(n,R)
form a group under matrix multiplication
are the simplest Riemannian manifolds
Examples
Rigid body transformations SE(n) (n+1)x(n+1)
Rotations SO(n) nxn
Affine motions A(n) (n+1)x(n+1)
R t
0T 1
W. Rossman, “Lie Groups: An Introduction through Linear Groups,” Oxford University Press, 2003.

10. Therefore the representation is not unique!
Grassmann Manifolds,
Each point on the Grassmann manifold, , represents a dimensional subspace of
Numerically, represented by an orthonormal basis (does not matter which one)
matrix such that
Therefore the representation is not unique!
computation should account for this
A. Edelman, T. A. Arias and S. T. Smith, “The Geometry of Algorithms with Orthogonality Constraints,” SIAM Journal on Matrix Analysis and Applications, 20(2), 303–353, 1998.

11. Equivalent to SO(3)xSO(3)
Essential Manifold
calibrated cameras
Set of matrices with
two equal and one zero singular value
let the two equal singular values be 1
Equivalent to SO(3)xSO(3)
two-time covering of the essential manifold
Correction by Dubbelman(2012).
Subbarao&Meer(2009) had two solutions from eight possible ones, and one still had to be eliminated
S. Soatto, R. Frezza and P. Perona , “Recursive Estimation on the Essential Manifold,” 3rd Europan Conference on Computer Vision, Stockholm, Sweden, May 1994, vol.II, p

12. Twisted pairs gives the same essential matrix
Four different camera geometries,
but the same essential matrix.

13. Symmetric Positive Define (SPD) Manifold
contains symmetric positive definite matrices. e.g. diffusion tensor MRI
has two different metrics
Affine Invariant metric (precise)
Log-Euclidean Metric (this is used in comp.vision)
practically similar to the affine invariant metric
computationally easier to work with
V. Arsigny , P. Fillard, X. Pennec and N. Ayache , “Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices,” SIAM Journal of Matrix Analysis and Applications, 29(1), 328–347, 2006.

14. Mean Shift for Euclidean Spaces
The kernel density estimate
Mean shift as normalized gradient of
where
The iteration

15. Gradient ascent on the kernel density on the closest mode.
Original Mean Shift
Gradient ascent on the kernel density on the closest mode.
Equivalent to expectation-maximization for Gaussian kernels
Mean shift is a nonparametric clustering
D. Comaniciu and P. Meer , “Mean Shift: A Robust Approach Towards Feature Space Analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.24, 603–619, 2002.
D. Comaniciu, V. Ramesh and P. Meer , “Kernel-based Object Tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.25, 564–577, 2003.

16. Mean Shift for Manifolds
The kernel density estimate on the manifold
Mean shift as normalized gradient of , tangent space
The iteration

17. Mean Shift for Riemannian Manifolds
Map locally the points to tangent space
Compute the weighted average of the tangent vectors and obtain the mean shift vector
Map the mean shift vector back to the manifold

18. Properties for Nonlinear Mean Shift
Still the gradient ascent on the kernel density
Nonlinear mean shift is convergent but depend on an upper limit on allowed bandwidth in the tangent space!
nonlinear mean shift equivalent to EM for Gaussians kernels. Valid for the groups mentioned.
All significant modes may come out in the manifold.

19. Clustering by nonlinear mean shift
Motion Segmentation
Hypothesis Generation matrix Lie groups Randomly pick N elemental subset of corresponding points. May have outlier elemental subsets too.
Generate N parameter hypotheses R, t
Clustering by nonlinear mean shift
the clustering needs h(-s)
Returns
Number of dominant modes
Positions of dominant modes

20. 2D Affine Motion A(2) matrix 3x3
Total of 80 matches.
Affine has 6 parameters.
Three point pairs per elemental subset.
500 clustering hypotheses.

21. Camera Pose Estimation SE(3) 4x4 matrix
Calibrated camera. Three point clouds, triangulated in 3D have different coordinate systems. With all three scenes present, segmentation in 2D can be used to establish
2D to 3D correspondences for the entire scene.
3D poses are estimated with SE(3) and h=0.1.

22. Results with all three scenes together
M1 books on right
M2 books on left
M3 box in middle

23. Factorization G10,3 uncalibrated affine camera
Grassmann. 5 frames n=2x5=10.
planar move each object k=3. e.subset 3 points in all 5 frames.
257 corners hypotheses.
h = 0.07

24. Essential Matrix SO(3)xSO(3)
Two motions. 100 point matches.
1000 hypotheses. h = 0.001
Second solution comes from rotation +-p around the y-axis, which maps into the same tangent vector.

25. Discontinuity Preserving Filtering
An image is a mapping from a lattice in to data lying on a manifold
Filtering: Run mean shift in the space
2D hs
Mean shift iterations update the spatial and parameter values.
The iteration from converges to
set in the filtered image

26. Chromatic Noise Filtering G3,1
Chromaticity affects direction of the color vector but not the intensity. s = 0.2
original mean shift hs=11 hp= chromatic hs=11 hp=0.7

27. Chromatic Noise Filtering
original mean shift hs=11 hp= chromatic hs=11 hp=0.3

28. Diffusion tensor Image R3xSym+3
Images encoded with 3x3 Symmetric Positive Definite (SPD)
matrices at each pixel.
Smoothing the DTI
improves the result.

29. Group theory starts to be used as part of a deep learning method.
Example:
Lohit,Turaga: Learning Invariant Riemannian Geometric Representations Using Deep Nets
Grassmannian manifold
multi-class
classification
on a unit
hypersphere
Exmp: Face to illumination subspace
28x28 images G784,d Orthonornal matrix U:784xd
The grassmannian manifold is invariant to orthogonal
transformations, like illuminations, which is in a low dim. subspace. The 90% of the variances are in the first d=5 principal components of a database.
250(200+50) 3D random faces. 33 illuminations from
Yale database B, which had a total of 64 illuminations.
Neural network has 3 convolution layers and 2 full connected. || Frechet (geometric) mean in the tangent space for the ground-truth subspace of the training set. || All the results are projected in the manifold.

30. gives RQ*, a new point of the manifold.
d= d=4 T
DG the mean subspace distance of principal angles obtain from Uavg Utest
Returns the subspace of illumination with a point of the manifold R which spans the same subspace as the columns of U. The rotation Q* = min ||Uavg – RQ||F
gives RQ*, a new point of the manifold.
orthogonal Procrustes problem