Linear Discriminant Analysis

Linear Discriminant Analysis Why To identify variables into one of two or more mutually exclusive and exhaustive categories. To examine whether significant differences exist among the groups in terms of the predictor variables. What The analysis helps determine what predictor variables contribute most to intergroup differences. It then classifies cases to one of the groups based on the values of the predictor variables. How Using a combination of MANOVA, PCA and MLP.

LDA Assumptions Absence of outliers Equal samples size Many data Homogeneity of variance-covariance Linear relationship No multicolinearity

LDA Toy example IVs DVs = X

LDA First step: Significance testing of the overall classifier in order to know if a set of discriminant functions can significantly predict group membership or not Second step: Significance testing for each discriminant function. Third step: Computation of the (standardized, unstandardized) discriminant functions

LDA - Overall Testing Sum of Square and Cross Product SSCP=

LDA - Overall Testing Canonical Correlation Matrix Error and hypothesis matrices

LDA - Overall Testing Computing W (WLR) where s = min(df, q), lk is kth eigenvalue extracted from HiE-1 and |E| (as well as |E+Hi|) is the determinant. The overall test is significant

LDA - Individual Testing Eigenvalues and eigenvectors decomposition of the matrix: E-1H E-1H= PCA E-1H

LDA - Individual Testing Canonical Discriminant Analysis Squared canonical correlation (Can also obtained from the eigenvalues of the correlation matrix R) Canonical correlation

LDA - Individual Testing Significance test for the canonical correlations A significant output indicates that there is a variance share between IV and DV sets Procedure: We test for all the variables (m=1,…,min(p,q)) If significant, we removed the first variable (canonical correlate) and test for the remaining ones (m=2,…, min(p,q) Repeat

LDA - Individual Testing Significance test for the canonical correlations Since all canonical variables are significant, we will keep them all.

LDA – Projection of the solution Second group First group P=VY Third group Second discriminant function First discriminant function

LDA – Discriminant Functions b0 b1 b2 b3 b4 Class membership is given by: Max(D1, D2, D3) Example x=(86, 6, 35, 6.5); D1= 122.817 (MAX) D2= 103.706 D3= 105.642