1. Principal Component Analysis

2. Objective: Project the data onto a lower dimensional space while minimizing the information loss

3. Principal Component Analysis load mnist m = mean(data); for i=1:size(data_m,2) data_m(:,i) = data(:,i) - m(i); end [pc,evals] = pca_OF(data_m); pc_data = data_m*pc(1:200,:)';

4. Principal Component Analysis function [pc,evals] = pca_OF(x) [pc,evals] = eig(cov(x)); evals = diag(evals); [evals, si] = sort(-evals); %Sort eigenvalues evals = -evals; pc = pc(:,si)'; %Sort eigenvectors by magnitude of eigenvalues

5. Sorted Eigenvalues

6. Normalized Cumulative Variance (information preserved)

7. Projecting the digits onto the first two PCs

8. Projecting the digits onto PCs 1 and 3

9. Projecting the digits onto PCs 2 and 3

10. Recognition Accuracies and Running Times with MNIST dataset PCs% VarianceRunning timeAccuracy 323.1%29.1 s0.4574 533.3%55.0 s0.7179 1048.9%98.4 s0.9251 1558.0 %172.5 s0.9577 2569.3%282.6 s0.9742 5082.6 %587.1 s0.9761 10091.5 %1183.6 s0.9738 20096.7 %2267.4 s0.9720