1. Outline
Singular Value Decomposition
Example of PCA: Eigenfaces

2. Singular Value Decomposition
A sample set of M N-dimensional points can be written as a matrix each row of which represents a sample point.
PCA of the sample is then equivalent to solving the SVD
problem for this matrix; that is, finding the decomposition
W is diagonal matrix, V is orthogonal square matrix, columns of U are orthogonal, columns of V are orthogonal :
Also

3. SVD-remarks
Meaning: columns of V represent the KL transform axes, ordered by respective values in W (singular values), which are amount of variation, in descending order.
The new axes (columns of V) are also eigenvectors of XXT if X is square matrix
From the orthogonal basis vectors given as columns of V we omit those to which correspond small values in W.
SVD provides unique decomposition for the given data.
Taking the first m<N eigenvectors (rows in VT) we get the optimal approximation in the sense of L2 norm.

4. SVD-remarks
When M<N the singular values wj for j=M+1,…,N are zero.