1. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.1 Lecture 9: Discriminant function analysis (DFA) l Rationale and use of DFA l The underlying model (what is a discriminant function anyway?) l Finding discriminant functions: principles and procedures l Rationale and use of DFA l The underlying model (what is a discriminant function anyway?) l Finding discriminant functions: principles and procedures l Linear versus quadratic discriminant functions l Significance testing l Rotating discriminant functions l Component retention, significance, and reliability.

2. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.2 What is discriminant function analysis? l Given a set of p variables X 1, X 2,…, X p, and a set of N objects belonging to m known groups (classes) G 1, G 2,…, G m, we try and construct a set of functions Z 1, Z 2,…, Z min{m-1,p} that allow us to classify each object correctly. l The hope (sometimes faint) is that “good” classification results (i.e., low misclassification rate, high reliability) will be obtained through a relatively small set of simple functions.

3. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.3 What is a discriminant function anyway? l A discriminant function is a function: which maximizes the “separation” between the groups under consideration, or (more technically) maximizes the ratio of between group/within group variation. l A discriminant function is a function: which maximizes the “separation” between the groups under consideration, or (more technically) maximizes the ratio of between group/within group variation. Group 1 Group 2 Group 1 Group 2 (not so good) (better) Frequency

4. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.4 The linear discriminant model l For a set of p variables X 1, X 2,…, X p, the general model is l where the X j s are the original variables and the a ij s are the discriminant function coefficients. l For a set of p variables X 1, X 2,…, X p, the general model is l where the X j s are the original variables and the a ij s are the discriminant function coefficients. l Note: unlike in PCA and FA, the discriminant functions are based on the raw (unstandardized) variables, since the resulting classifications are unaffected by scale. l For p variables and m groups, the maximum number of DFs is min{p, m-1}.

5. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.5 The geometry of a single linear discriminant function l 2 groups with measurements of two variables (X 1 and X 2 ) on each object. l In this case, the linear DF Z* results in no misclassifications, whereas another possible DF (Z) gives two misclassifications. l 2 groups with measurements of two variables (X 1 and X 2 ) on each object. l In this case, the linear DF Z* results in no misclassifications, whereas another possible DF (Z) gives two misclassifications. Group 1 Group 2 X2X2 X1X1 Z Z* Misclassified under Z but not under Z*

6. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.6 Finding discriminant functions: principles l The first discriminant function is that which maximizes the differences between groups compared to the differences within groups… l …which is equivalent to maximizing F in a one- way ANOVA. l The first discriminant function is that which maximizes the differences between groups compared to the differences within groups… l …which is equivalent to maximizing F in a one- way ANOVA. a = (a 1,…, a p ) F(Z) Z1Z1

7. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.7 Finding discriminant functions: principles l The second discriminant function is that which maximizes the differences between groups compared to the differences within groups unaccounted for by Z 1... l …which is equivalent to maximizing F in a one-way ANOVA given the constraint that Z 1, Z 2 are uncorrelated. l The second discriminant function is that which maximizes the differences between groups compared to the differences within groups unaccounted for by Z 1... l …which is equivalent to maximizing F in a one-way ANOVA given the constraint that Z 1, Z 2 are uncorrelated. a = (a 1,…, a p ) F(Z) Z2Z2

8. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.8 The geometry of several linear discriminant functions l 2 groups with measurements of two variables (X 1 and X 2 ) on each individual. l Using only Z 1, 4 objects are misclassified, whereas using both Z 1 and Z 2, only one object is misclassified. l 2 groups with measurements of two variables (X 1 and X 2 ) on each individual. l Using only Z 1, 4 objects are misclassified, whereas using both Z 1 and Z 2, only one object is misclassified. X2X2 X1X1 Z1Z1 Group 1 Group 2 Z2Z2 Misclassified using both Z 1 and Z 2 Misclassified using only Z 1

9. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.9 SSCP matrices: within, between, and total l The total (T) SSCP matrix (based on p variables X 1, X 2,…, X p ) in a sample of objects belonging to m groups G 1, G 2,…, G m with sizes n 1, n 2,…, n m can be partitioned into within- groups (W) and between- groups (B) SSCP matrices: Value of variable X k for ith observation in group j Mean of variable X k for group j Overall mean of variable X k Element in row r and column c of total (T, t) and within (W, w) SSCP

10. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.10 Finding discriminant functions: analytic procedures l Calculate total (T), within (W) and between (W) SSCPs. l Determine eigenvalues and eigenvectors of the product W -1 B.  is ratio of between to within SSs for the ith discriminant function Z i … l …and the elements of the corresponding eigenvectors are the discriminant function coefficients. l Calculate total (T), within (W) and between (W) SSCPs. l Determine eigenvalues and eigenvectors of the product W -1 B.  is ratio of between to within SSs for the ith discriminant function Z i … l …and the elements of the corresponding eigenvectors are the discriminant function coefficients.

11. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.11 AssumptionsAssumptions l Equality of within-group covariance matrices (C 1 = C 2 =...) implies that each element of C 1 is equal to the corresponding element in C 2, etc. Variance Covariance G1G1 G2G2

12. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.12 The quadratic discriminant model l For a set of p variables X 1, X 2,…, X p, the general quadratic model is l where the X j s are the original variables and the a ij s are the linear coefficients and the b ij s are the 2nd order coefficients. l For a set of p variables X 1, X 2,…, X p, the general quadratic model is l where the X j s are the original variables and the a ij s are the linear coefficients and the b ij s are the 2nd order coefficients. l Because the quadratic model involves many more parameters, sample sizes must be considerably larger to get reasonably stable estimates of coefficients. X2X2 X1X1 Group 1 Group 2 Quadratic Z 1 Linear Z 1

13. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.13 Fitting discriminant function models: the problems l Goal: find the “best” model, given the available data l Problem 1: what is “best”? l Problem 2: even if “best” is defined, by what method do we find it? l Possibilities: n If there are m variables, we might compute DFs using all possible subsets (2 m -1) of variables models and choose the best one n use some procedure for winnowing down the set of possible models. l Goal: find the “best” model, given the available data l Problem 1: what is “best”? l Problem 2: even if “best” is defined, by what method do we find it? l Possibilities: n If there are m variables, we might compute DFs using all possible subsets (2 m -1) of variables models and choose the best one n use some procedure for winnowing down the set of possible models.

14. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.14 Criteria for choosing the “best” discriminant model l Discriminating ability: better models are better able to distinguish among groups l Implication: better models will have lower misclassification rates. l N.B. Raw misclassification rates can be very misleading. l Discriminating ability: better models are better able to distinguish among groups l Implication: better models will have lower misclassification rates. l N.B. Raw misclassification rates can be very misleading. l Parsimony: a discriminant model which includes fewer variables is better than one with more variables. l Implication: if the elimination/addition of a variable does not significantly increase/decrease the misclassification rate, it may not be very useful.

15. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.15 Criteria for choosing the “best” discriminant model (cont’d) l Model stability: better models have coefficients that are stable as judged through cross- validation. l Procedure: Judge stability through cross- validation (jackknifing, bootstrapping). l Model stability: better models have coefficients that are stable as judged through cross- validation. l Procedure: Judge stability through cross- validation (jackknifing, bootstrapping). l NB.1. In general, linear discriminant functions will be more stable than quadratic functions, especially if the sample is small. l NB.2. If the sample is small, then ”outliers” may dramatically decrease model stability.

16. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.16 Fitting discriminant function models: the problems l Goal: find the “best” model, given the available data l Problem 1: what is “best”? l Problem 2: even if “best” is defined, by what method do we find it? l Possibilities: n If there are m variables, we might compute DFs using all possible subsets (2 m -1) of variables models and choose the best one n use some procedure for winnowing down the set of possible models. l Goal: find the “best” model, given the available data l Problem 1: what is “best”? l Problem 2: even if “best” is defined, by what method do we find it? l Possibilities: n If there are m variables, we might compute DFs using all possible subsets (2 m -1) of variables models and choose the best one n use some procedure for winnowing down the set of possible models.

17. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.17 Analytic procedures: general approach l Evaluate significance of a variable (X i ) in DF by computing the difference in group resolution between two models, one with the variable included, the other with it excluded. Evaluate change in discriminating ability (  DA) associated with inclusion of the variable in question l Unfortunately, change in discriminating ability may depend on what other variables are in model! l Evaluate significance of a variable (X i ) in DF by computing the difference in group resolution between two models, one with the variable included, the other with it excluded. Evaluate change in discriminating ability (  DA) associated with inclusion of the variable in question l Unfortunately, change in discriminating ability may depend on what other variables are in model! Model A (X i in) Model B (X i out)  DA Delete X i (  small) Retain X i (  large)

18. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.18 Strategy I: computing all possible models l compute all possible models and choose the “best” one. l Impractical unless number of variables is relatively small. l compute all possible models and choose the “best” one. l Impractical unless number of variables is relatively small. {X 1, X 2, X 3 } {X2}{X2} {X1}{X1} {X3}{X3} {X1, X2}{X1, X2} {X2, X3}{X2, X3} {X1, X3}{X1, X3} {X1, X2, X3}{X1, X2, X3}

19. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.19 Strategy II: forward selection l start with variable for which differences among group means are the largest (largest F-value) l add others one at a time based on F to enter (p to enter) until no further significant increase in discriminating ability is achieved. l problem: if X j is included, it stays in even if it contributes little to discriminating ability once other variables are included. l start with variable for which differences among group means are the largest (largest F-value) l add others one at a time based on F to enter (p to enter) until no further significant increase in discriminating ability is achieved. l problem: if X j is included, it stays in even if it contributes little to discriminating ability once other variables are included. F 2 > F 1, F 3, F 4 F 2 > F to enter (< p to enter) F 1 > F 3, F 4 F 1 > F to enter (< p to enter) (X 1, X 2, X 3, X 4 ) All variables (X2)(X2) (X 1, X 2 ) F 4 > F 3 ; F 4 < F to enter (> p to enter) Final model

20. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.20 What is F to enter/remove (p to enter/remove) anyway? l When no variables are in the model, F to enter is the F-value from a univariate one-way ANOVA comparing group means with respect to the variable in question, and p to enter is the Type I probability associated with the null that all group means are equal. l When other variables are in the model, F to enter corresponds to the F-value for an ANCOVA comparing group means with respect to the variable in question, where the covariates are the variables already entered.

21. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.21 Strategy III: backward selection l Start with all variables and drop that for which differences among group means are the smallest (smallest F-value) l Delete others one at a time based on F to remove (p to remove) until further removal results in a significant reduction in the ability to discriminate groups. l problem: if X j is excluded, it stays out even if it contributes substantially to discriminating ability once other variables are excluded. l Start with all variables and drop that for which differences among group means are the smallest (smallest F-value) l Delete others one at a time based on F to remove (p to remove) until further removal results in a significant reduction in the ability to discriminate groups. l problem: if X j is excluded, it stays out even if it contributes substantially to discriminating ability once other variables are excluded. F 2 < F 1, F 3, F 4 F 2 < F to remove (> p to remove) F 1 < F 3, F 4 F 1 < F to remove (> p to remove) (X 1, X 2, X 3, X 4 ) All variables in (X 3, X 4 ) F 4 F to remove (< p to remove) Final model (X 1, X 3, X 4 )

22. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.22 Canonical scores l Because discriminant functions are functions, we can “plug in” the values for each variable for each observation, and calculate a canonical score for each observation and each discriminant function.

23. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.23 Canonical scores plots l Plots of canonical scores for each object. l The better the model, the greater the separation between clouds of points representing individual groups, e.g. Fisher’s famous irises. l Plots of canonical scores for each object. l The better the model, the greater the separation between clouds of points representing individual groups, e.g. Fisher’s famous irises. Canonical scores of group means 1 2 1 7.6080.215 2-1.825-0.728 3-5.7830.513 95% confidence ellipse

24. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.24 PriorsPriors In standard DFA, it is assumed that in the absence of any information, the a priori (prior) probability  i of a given object belonging to one of i = 1,…,m groups is the same for all groups: l But, if each group is not equally likely, then priors should be adjusted so as to reflect this bias. l E.g. in species with biased sex-ratios, males and females should have unequal priors.

25. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.25 Caveats: unequal priors l For a given set of discriminant functions, misclassification rates will usually depend on the priors… l …so that artificially low misclassification rates can be obtained simply by strategically adjusting the priors. l For a given set of discriminant functions, misclassification rates will usually depend on the priors… l …so that artificially low misclassification rates can be obtained simply by strategically adjusting the priors. l So, only adjust priors if you are confident that the true frequency of each group in the population is (reasonably) accurately estimated by the group frequencies in the sample.

26. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.26 Significance testing l Question: which discriminant functions are statistically “significant”? For testing significance of all r DFs for m groups based on p variables, calculate Bartlett’s V and compare to  2 distribution with p(m-1) degrees of freedom l Question: which discriminant functions are statistically “significant”? For testing significance of all r DFs for m groups based on p variables, calculate Bartlett’s V and compare to  2 distribution with p(m-1) degrees of freedom Eigenvalue associated with ith discriminant function

27. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.27 Significance testing (cont’d) l Each DF is tested in a hierarchical fashion by first testing significance of all DFs combined. l If all DFs combined not significant, then no DF is significant. l If all DFs combined are significant, then remove first DF and recalculate V (= V 1 ) and test. l Continue until residual V j no longer significant at df = (p – j)(m – j - 1) l Each DF is tested in a hierarchical fashion by first testing significance of all DFs combined. l If all DFs combined not significant, then no DF is significant. l If all DFs combined are significant, then remove first DF and recalculate V (= V 1 ) and test. l Continue until residual V j no longer significant at df = (p – j)(m – j - 1)

28. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.28 Caveats/assumptions: tests of significance l Tests of significance assume that within-group covariance matrices are the same for all groups, and that within groups, observations have a multivariate normal distribution l Tests of significance can be very misleading because j th discriminant function in the population may not appear as j th discriminant function in the sample due to sampling errors… l So be careful, especially if the sample is small! l Tests of significance assume that within-group covariance matrices are the same for all groups, and that within groups, observations have a multivariate normal distribution l Tests of significance can be very misleading because j th discriminant function in the population may not appear as j th discriminant function in the sample due to sampling errors… l So be careful, especially if the sample is small!

29. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.29 Caveats/assumptions: tests of significance l If stepwise (forward or backward) procedures are used, significance tests are biased because given enough variables, significant discriminant functions can be produced by chance alone. l In such cases, it is advisable to (1) test results with more standard analyses or (2) use randomization procedures whereby objects are randomly assigned to groups. l If stepwise (forward or backward) procedures are used, significance tests are biased because given enough variables, significant discriminant functions can be produced by chance alone. l In such cases, it is advisable to (1) test results with more standard analyses or (2) use randomization procedures whereby objects are randomly assigned to groups.

30. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.30 Assessing classification accuracy I. Raw classification results l The derived discriminant functions are used to classify all objects in the sample, and a classification table is produced. l Classification accuracy is likely to be overestimated, since the data used to generate the DFs in the first place are themselves being classified. l The derived discriminant functions are used to classify all objects in the sample, and a classification table is produced. l Classification accuracy is likely to be overestimated, since the data used to generate the DFs in the first place are themselves being classified. GroupTotal Group12 143548 281422 Total511970 Misclassification (G 2 ) = 8/22 Misclassification (G 1 ) = 5/48 Overall misclassification = 13/70

31. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.31 Assessing classification accuracy II. Jackknifed classification l Discriminant functions are derived using N – 1 objects, and the Nth object is then classified. l This procedure is repeated for all N objects, each time leaving a different one out, and a classification table produced. l In general, jackknifed classification results are worse than raw classification results, but more reliable. l Discriminant functions are derived using N – 1 objects, and the Nth object is then classified. l This procedure is repeated for all N objects, each time leaving a different one out, and a classification table produced. l In general, jackknifed classification results are worse than raw classification results, but more reliable. GroupTotal Group12 141748 291322 Total511970 Misclassification (G 2 ) = 9/22 Misclassification (G 1 ) = 7/48 Overall misclassification = 16/70

32. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.32 Assessing classification accuracy III. Data splitting l Use 2/3 of sample data (randomly) selected to generate discriminant functions (learning set) l Use derived discriminant functions to classified other 1/3 (test set) and produce classification table. l In general, data-splitting classification results are worse than both raw and jackknifed classification results, but more reliable. l Use 2/3 of sample data (randomly) selected to generate discriminant functions (learning set) l Use derived discriminant functions to classified other 1/3 (test set) and produce classification table. l In general, data-splitting classification results are worse than both raw and jackknifed classification results, but more reliable. GroupTotal Group12 140848 291322 Total511970 Misclassification (G 2 ) = 9/22 Misclassification (G 1 ) = 8/48 Overall misclassification = 17/70

33. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.33 Assessing classification accuracy IV. Bootstrapped data splitting l Use 2/3 of sample data (randomly sampled) to generate discriminant functions (learning set) l Use derived discriminant functions to classify other 1/3 (test set) and produce classification results. l Repeat a large number (e.g. 1000) times, each time sampling with replacement. l Generate classification statistics over bootstrapped samples, e.g. mean classification results, standard errors, etc. l Use 2/3 of sample data (randomly sampled) to generate discriminant functions (learning set) l Use derived discriminant functions to classify other 1/3 (test set) and produce classification results. l Repeat a large number (e.g. 1000) times, each time sampling with replacement. l Generate classification statistics over bootstrapped samples, e.g. mean classification results, standard errors, etc. GroupTotal Group12 1 41.2  1.7 6.8  0.6 48 2 9.3  0.5 12.7  1.1 22 Total511970 Misclassification (G 2 ) = 42.3% Misclassification (G 1 ) = 14.2% Overall misclassification = 23.0%

34. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.34 Interpreting discriminant functions l Examine standardized coefficients (coefficients of discriminant functions based on standardized values) l For interpretation, use variables with large absolute standardized coefficients. l Examine standardized coefficients (coefficients of discriminant functions based on standardized values) l For interpretation, use variables with large absolute standardized coefficients. l Examine the discriminant-variable correlations. l For interpretation, use variables with high correlations with important discriminant functions.

35. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.35 Example: Fisher’s famous irises l Data: four variables (sepal length, sepal width, petal length, petal width), 3 species, N = 150 (50 for each species). l Problem: find the “best” set of DFs. l Data: four variables (sepal length, sepal width, petal length, petal width), 3 species, N = 150 (50 for each species). l Problem: find the “best” set of DFs.

36. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.36 Example: Fisher’s famous irises: between-groups F- matrix l Matrix entries are F – values from one-way MANOVA comparing group means, and can be considered measures of the distance between group centroids. l Do not use associated probabilities to determine “significance” unless you correct for multiple tests. l Matrix entries are F – values from one-way MANOVA comparing group means, and can be considered measures of the distance between group centroids. l Do not use associated probabilities to determine “significance” unless you correct for multiple tests. Species 123 10.0 2550.20.0 31098.3105.30.0

37. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.37 Example: Fisher’s famous irises: canonical discriminant functions l Four variables (sepal length, sepal width, petal length, petal width), 3 species, N = 150 (50 for each species). Canonical discriminant functions 1 2 Constant 2.105-6.661 SEPALLEN 0.829 0.024 SEPALWID 1.534 2.165 PETALLEN-2.201-0.932 PETALWID-2.810 2.839 Note: discriminant functions are derived using equal priors.

38. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.38 Example: Fisher’s famous irises: standardized canonical discriminant functions l Four variables (sepal length, sepal width, petal length, petal width), 3 species, N = 150 (50 for each species). Standardized canonical discriminant functions 1 2 SEPALLEN 0.427 0.012 SEPALWID 0.521 0.735 PETALLEN-0.942-0.401 PETALWID-2.810 0.581 Note: standardized canonical discriminant functions are based on standardized values.

39. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.39 l Eigenvalues give amount of differences among groups captured by a a particular discriminant function, and cumulative proportion of dispersion is the corresponding proportion. Discriminant function Parameter12 Eigenvalues32.1920.285 Canonical correlation 0.9850.471 Cumulative proportion of dispersion 0.9911.000 l Canonical correlation is the correlation between a given canonical variate (DF) and a set of two dummy variables representing each group.

40. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.40 Fisher’s irises: raw and jackknifed classification results l In this case, results are identical (a relatively rare occurrence!) Species % correct Species123 15000100 2048296 3014998 Total50495198 Species % correct Species123 15000100 2048296 3014998 Total50495198

41. Université d’Ottawa / University of Ottawa 2001 Bio 8100s Applied Multivariate Biostatistics L9.41 Dicriminant function analysis: caveats and notes l Unless the ratio of number of objects/number of variables is large (> 20), standardized coefficients and correlations are unstable. l DFA is unaffected by differences among variables in scale, so standardization is not required (unlike PCA, FA, etc.) l Unless the ratio of number of objects/number of variables is large (> 20), standardized coefficients and correlations are unstable. l DFA is unaffected by differences among variables in scale, so standardization is not required (unlike PCA, FA, etc.) l Linear DFA is quite sensitive to the assumption of equality of covariance matrices among groups. If this assumption is violated, use quadratic classification. l However, quadratic DFA is more unstable when N is small and normality does not hold.