1. Irena Váňová

2. B A1A1. A2A2. A3A3.

3. repeat until no sample is misclassified … labels of classes Perceptron algorithm for i=1...N if then end * * * * * * * * * o o o o oo

4. repeat for i=1...N if then end until no sample is misclassified Find the coefficients is equivalent to find  In the dual representation, the data points only appear inside dot products  Many algorithms have dual form Rewitten algorithm – dual form Gram matrix

5.  Perceptron works for linear separable problems  There is a computational problem (very large vectors)  Kernel trick:

6.  Polynomial kernels  Gaussian kernels ◦ Infinit dimensions ◦ Separated by a hyperplane  Good kernel?  Bad kernel! ◦ Almost diagonal

7.  We precompute  We are in implicitly in higher dimensions (too high?)  Generalization problem - easy to overfit in high dimensional spaces repeat for i=1...N if then end until no sample is misclassified

8.  Kernel function  Use: replacing dot products with kernels  Implicit mapping to feature space ◦ Solve the computational problem ◦ Can make it possible to use infinite dimensions  Conditons: continuous, symmetric, positive definite  Information ‘bottleneck’: contains all necessary information for the learning algorithm  Fuses information about the data AND the kernel

9.  Orthogonal linear transformation  The greatest variance = first coordinate, …  Rotation around mean value  Dimensionality reduction ◦ many dimensions = high correlation

10.  W,T – unitary matrix ( )  Columns of W,V? ◦ Basis vector, eigenvectors of X T X, resp. XX T n n n n m m m

11.  Data with zero mean, SVD n n m m m m covariance matrix (1/n) eigenvectors

12.  Projections of data onto few larger eigenvectors equation for PCA kernel function equation for high-dim. PCA We don’t know eigenvector explicitly - only vector of numbers which identify the vector projection onto k-th eigenvector

14. PCA is blind

15.  fundamental assumption: normal distribution  First: same covariance matrix, full rank

16.  fundamental assumption: normal distribution  First: only full rank  Kernel variant

17.  Face recognition – eigenfaces