吳育德 陽明大學放射醫學科學研究所 台北榮總整合性腦功能研究室 Introduction To Linear Discriminant Analysis

Linear Discriminant Analysis For a given training sample set, determine a set of optimal projection axes such that the set of projective feature vectors of the training samples has the maximum between-class scatter and minimum within-class scatter simultaneously.

Linear Discriminant Analysis Linear Discriminant Analysis seeks a projection that best separate the data. Sb : between-class scatter matrix Sw : within-class scatter matrix

Sol: LDA Fisher discriminant analysis

where, = k 1 +k 2 and let LDA Fisher discriminant analysis

LDA Fisher discriminant analysis

Let M be a real symmetric matrix with largest eigenvalue then and the maximum occurs when, i.e. the unit eigenvector associated with. Proof : LDA Generalized eigenvalue problem.....Theorem 2

LDA Generalized eigenvalue problem.....proof of Theorem 2

If M is a real symmetric matrix with largest eigenvalue. And the maximum is achieved whenever,where is the unit eigenvector associated with. Cor: LDA Generalized eigenvalue problem.....proof of Theorem 2

LDA Generalized eigenvalue problem…….. Theorem 1 Let Sw and Sb be n*n real symmetric matrices. If Sw is positive definite, then there exists an n*n matrix V which achieves The real numbers λ 1 ….λ n satisfy the generalized eiegenvalue equation : : generalized eigenvector : generalized eigenvalue

Generalized eigenvalue problem.....proof of Theorem 1 Let and be the unit eigenvectors and eigenvalues of S w, i.e Now define then where Since r i ﹥ 0 (S w is positive definite), exist LDA

Generalized eigenvalue problem.....proof of Theorem 1

LDA We need to claim : (applying a unitary matrix to a whitening process doesn’t affect it!) (V T ) -1 exists since det(V T S w V) = det (I ) → det(V T ) det( S w ) det(V) = det(I) Because det(V T )= det(V) → [det(V T )] 2 det(S w ) = 1 > 0 → det(V T ) 0 Generalized eigenvalue problem.....proof of Theorem 1

Procedure for diagonalizing S w (real symmetric and positive definite) and S b (real symmetric) simultaneously is as follows : 1. Find λ i by solving And then find normalized, i=1,2…..,n 2. normalized LDA Generalized eigenvalue problem.....proof of Theorem 1