1. Multivariate Analysis: Theory and Geometric Interpretation
David Chelidze

2. Proper Orthogonal Decomposition (POD)
We consider a scalar field , where and POD decomposes it using orthonormal basis functions and the corresponding time coordinates
, where
This is equivalent to the following maximization problem:
, subject to
This results into the following integral eigenvalue problem

3. Smooth Orthogonal Decomposition (SOD)
SOD is looking for a projective function , such that the has maximal variance subject to its minimal roughness, expressed by the following maximization problem:
subject to
This translates into the following generalized integral eigenvalue problem:
Using the solution to the above eigenvalue problem, the scalar field can be reconstructed as: , where
A set of smooth orthogonal modes form a bi-orthogonal set with smooth projective modes

4. Practical Calculations for POD and SOD
When the field is sampled such that , where , then POD can be solved by singular value decomposition , where are time coordinates and contains the corresponding mode shapes.
The corresponding SOD problem can be solved by generalized singular value decomposition:
where: are unitary matrices, are diagonal matrices, and
columns of are smooth orthogonal modes;
columns of are smooth orthogonal coordinates;
columns of are smooth projective modes;
are smooth orthogonal values, and

5. Proper Orthogonal Decomposition: Geometric Interpretation

6. Geometric Interpretation of SOD
SOD identifies the subspaces where the scalar field projection is maximally smooth
Smooth orthogonal modes (SOMs) are not orthogonal to each other but are linearly independent
SOMs span smooth modal subspaces
Smooth projective modes (SPMs) form a bi- orthonormal set with SOMs and are used to obtain smooth orthogonal coordinates (SOCs)
SOCs are orthogonal to each other (i.e., their covariance matrix is diagonal)
SOCs are invariant under invertible linear coordinate transform

7. SOD Geometric Interpretation
Given a cloud of trajectory points, we identify its center of mass.
Then a point and its velocity can be visualized by two vectors in the figure
Now we identify the first SPM by maximizing the ratio of the projections of the velocity and the position onto this mode
This is followed by the similar maximization for the second SPM and so on
Smooth orthogonal modes (SOMs) form a bi- orthonormal set with SPMs and are used to obtain describe the points in the cloud