1. Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction Presented by Xianwang Wang Masashi Sugiyama

2. Dimensionality Reduction Goal  Embed high-dimensional data to low-dimensional space  Preserve intrinsic information Example High dimension 3-dimension

3. Categories Nonlinear  ISOMAP  Locally Linear Embedding (LLE)  Laplacian Eigenmap (LE) Linear  Principal Components Analysis (PCA)  Locality-Preserving Projection (LPP)  Fisher Discriminant Analysis (FDA) Unsupervised  S-ISOMAP, S-LLE, PCA Supervised  LPP, FDA

4. Formulation Number of samples: d-dimensional samples: Class labels : Number of samples in the class : Data matrix : Embedded samples:

5. Goal for linear dimensionality Reduction Find a transformation matrix Use Iris data for demos (http://archive.ics.uci.edu/ml/machine-learning- databases/iris/iris.data)http://archive.ics.uci.edu/ml/machine-learning- databases/iris/iris.data  Attribute Information: sepal length in cm sepal width in cm petal length in cm petal width in cm  class: Iris Setosa; Iris Versicolour; Iris Virginica

6. FDA(1) Mean of samples in the class Mean of all samples Within-class scatter matrix Between-class scatter matrix

7. FDA(2) Maximize the following objective Maximize the following constrained optimization problem equivalently Use the lagrangian, Apply KKT conditions Demo

8. LPP Minimize Equivalently We can get Demo

9. Local Fisher Discriminant Analysis(LFDA) FDA can perform poorly if samples in some class form several separate clusters LPP can make samples of different classes overlapped if they are close in the original high dimensional space LFDA combines the idea of FDA and LPP

10. LFDA(1) Reformulating FDA

11. LFDA(2) Definition of LFDA

12. LFDA(3) Maximize the following objective Equivalently, Similarly, we can get Demo

13. Conclusion LFDA provided more separate embedding than FDA and LPP FDA (globally), while LFDA(locally) More discussion about efficiently computing of LFDA transformation matrix and Kernel LFDA in the paper

14. Questions?