1. Principal Component Analysis IML 2004-5

2. Outline Max the variance of the output coordinates Optimal reconstruction Generating data Limitations of PCA

3. Eigenfaces Variance in Face Pictures Figure/ground Orientation Lighting Hairline

4. Eigenfaces 100 images 30x30 pixels A 900 subtract mean 100 AA T 900

5. Maximizing Output Variance The first eigenvector (highest eigenvalue) characterizes the maximal variance in the image: figure - background

6. Maximizing Output Variance The second eigenvector characterizes right orientation

7. Maximizing Output Variance

8. Variance Dimensionality

9. Optimal Reconstruction q=1q=2q=4q=8 q=16q=32q=64q=100… Original Image

10. e.g. n=80x80 pixels >> m=100 images Problem: finding the eigenvectors of a 6400x6400 matrix = O(6400 3 ) Solution: extract the eigenvectors Q of A T A If n>>m

11. If n>m q=1q=2q=4q=8 q=16q=32q=64q=100 Original Image

12. Generating Data

15. Kernel PCA

16. Limits of PCA Should the goal be finding independent rather than pair-wise uncorrelated dimensions Independent Component Analysis (ICA) ICA PCA

17. Limits of PCA Relevant Component Analysis (RCA) Fisher Discriminant analysis (FDA) Are the maximal variance dimensions the relevant dimensions for preservation?