1. Discriminant analysis
Chapter 11

2. 2-groups discriminant analysis
Discriminant analysis is a statistical procedure which allows us to classify cases in separate categories to which they belong on the basis of a set of characteristic independent variables called predictors or discriminant variables
The target variable (the one determining allocation into groups) is a qualitative (nominal or ordinal) one, while the characteristics are measured by quantitative variables.
DA looks at the discrimination between two groups
Multiple discriminant analysis (MDA) allows for classification into three or more groups.

3. Applications of DA
DA is especially useful to understand the differences and factors leading consumers to make different choices allowing them to develop marketing strategies which take into proper account the role of the predictors.
Examples
Determinants of customer loyalty
Shopper profiling and segmentation
Determinants of purchase and non-purchase

4. Example on the Trust data-set
Purchasers of chicken at the butcher’s shop (recorded in question q8d)
Respondents may belong to one of two groups
those who purchase chicken at the butcher’s shop
those who do not
Discrimination between these groups through a set of consumer characteristics
expenditure on chicken in a standard week (q5)
age of the respondent (q51)
whether respondents agree (on a seven-point ranking scale) that butchers sell safe chicken (q21d)
trust (on a seven-point ranking scale) towards supermarkets (q43b)
Does a linear combination of these four characteristics allow one to discriminate between those who buy chicken at the butcher’s and those who do not?

5. Discriminant analysis(DA)
Two groups only, thus a single discriminating value (discriminating score)
For each respondent a score is computed using the estimated linear combination of the predictors (the discriminant function)
Respondents with a score above the discriminating value are expected to belong to one group, those below to the other group.
When the discriminant score is standardized to have zero mean and unity variance it is called Z score
DA also provides information about the discriminating power of each of the original predictors

6. Multiple discriminant analysis(MDA)(1)
Discriminant analysis may involve more than two groups, in which case it is termed multiple discriminant analysis (MDA).
Example from the Trust data-set
Dependent variable: Type of chicken purchased ‘in a typical week’, choosing among four categories: value (good value for money), standard, organic and luxury
Predictors: age (q50), stated relevance of taste (q24a), value for money (q24b) and animal welfare (q24k), plus an indicator of income (q60)

7. Multiple discriminant analysis(2)
In this case there will be more than one discriminant function.
The exact number of discriminant functions is equal to either (g-1), where g is the number of categories in classification or to k, the number of independent variables, whichever is the smaller
Trust example: four groups and five explanatory variables, the number of discriminant functions is three (that is g-1 which is smaller than k=5).

8. The output of MDS
Similarities with factor (principal component) analysis
the first discriminant function is the most relevant for discriminating across groups, the second is the second most relevant, etc.
the discriminant functions are also independent, which means that the resulting scores are non-correlated.
Once the coefficients of the discriminant functions are estimated and standardized, they are interpreted in a similar fashion to the factor loadings.
The larger the standardised coefficients (in absolute terms), the more relevant the respective variables to discriminating between groups
There is no single discriminant score in MDA
group means are computed (centroids) for each of the discriminant functions to have a clearer view of the classification rule

9. Running discriminant analysis (two groups)
Discriminant function (Target variable: purchasers of chicken at the butcher’s shop)
Discriminant score
Predictors
weekly expenditure on chicken
age
safety of butcher’s chicken
trust in supermarkets
The a discriminant coefficients need to be estimated

10. Fisher’s linear discriminant analysis
The discrimant function is the starting point
Two key assumptions behind linear DA
the predictors are normally distributed;
the covariance matrices for the predictors within each of the groups are equal.
Departure from condition (a) should suggest use of alternative methods (logistic regression, see lecture 16)
Departure from condition (b) requires the use of different discriminant techniques (usually quadratic discriminant functions).
In most empirical cases, the use of linear DA is appropriate

11. Estimation
The first step is the estimation of the a coefficients, also termed as discriminant coefficients or weights
Estimation is similar to factor analysis or PCA, as the coefficients are those which maximize the variability between groups
In MDA the first discriminating function is the one with the highest between-group variability, the second discriminating function is independent from the first and maximizes the remaining between-group variability and so on

12. SPSS – two groups case 1. Choose the target variable
2. Define the range of the dependent variable
3.Select the predictors

13. Coefficient estimates
Additional statistics and diagnostics
Fisher’s and standardized estimates of the discriminant function coefficients need to be asked for

14. Classification options
Decide whether prior probabilities are equal across groups or group sizes reflect different allocation probabilities
These are diagnostic indicators to evaluate how well the discriminant function predict the groups

15. Save classification
Create new variables in the data-set, containing the predicted group membership and/or the discriminant score for each case and each function

16. Output – coefficient estimates
Most important predictor
Trust in supermarkets has a – sign (thus it reduces the discriminant score)
Standardized coefficients do not depend on the measurement unit
Unstandardized coefficients depend on the measurement unit

17. Centroids
These are the means of the discriminant score for each of the two groups
Thus, the group of those not purchasing chicken at the butcher’s shop have a negative centroid
With two groups, the discriminating score is zero
This can be computed by weighting the centroids with the initial probabilities
From these prior probabilities it follows that the discriminating score is x x 0.34 = 0

18. Output – classification success
Using the discriminant function, it is possible to correctly classify 71.2% of original cases (244 no-no + 55 yes-yes)/420

19. Diagnostics (1)
Box’s M test. This tests whether covariances are equal across groups
Wilks’ Lambda (or U statistic) tests discrimination between groups. It is related to analysis of variance.
Individual Wilks’Lambda for each of the predictors in a discriminant function; univariate ANOVA (are there significant differences in the predictor’s means between the groups?), p-value from the F distribution.
Wilks’ Lambda for the function as a whole. Are there significant differences in the group means for the discriminant function p-value from the Chi-square distribution?
The overall Wilks’ Lambda is especially helpful in multiple discriminant analysis as it allows one to discard those functions which do not contribute towards explaining differences between groups.

20. Diagnostics (2)
DA returns one eigenvalue (or more eigenvalues for MDA) of the discriminant function.
These can be interpreted as in principal component analysis
In MDA (more than one discriminant function) eigenvalues are exploited to compute how each function contributes to explain variability
The canonical correlation measures the intensity of the relationship between the groups and the single discriminant function

21. Trust example: diagnostics
Covariance matrices are not equal
Statistic
P-value
Box's M statistic
37.3
0.000
Overall Wilks' Lambda
0.85
Wilks Lambda for
Expenditure
0.98
0.002
Age
0.97
0.001
Safer for Butcher
0.91
Trust in Supermarket
Eigenvalue
0.18
Canonical correlation
0.39
% of correct predictions
71.2%
The overall discriminating power of the DF is good
All of the predictors are relevant to discriminating between the two groups
The eigenvalue is the ratio between variances between and variance within groups (the larger the better)
Square root of the ratio between variability between and total variability

22. MDA
To run MDA in SPSS the only difference is that the range has more than two categories

23. Predictors
Three predictors only appear to be relevant in discriminating among preferred types of chicken
Null rejected at 95% c.l., but not at 99% c.l.

24. Discriminant functions
Three discriminant functions (four groups minus one) can be estimated
The first two discriminant functions have a significant discriminating power.

25. Coefficients Value for money is very relevant for the second function
Income is very relevant for the first function

26. Structure matrix
The values in the structure matrix are the correlations between the individual predictors and the scores computed on the discriminant functions.
For example, the income variable has a strong correlation with the scores of the first function
The structure matrix help interpreting the functions
Value and taste
Income
Age

27. Centroids
The second allows some discrimination standard-organic, value-standard, organic-luxury (taste and value matter)
The first function discriminates well between value and organic (income matters to organic buyers)

28. Plot of two functions
Tick ‘separate-groups’ to show graphs of the first two functions for each individual group
The ‘territorial map’ shows the scores for the first two functions considering all groups

29. Plots: individual groups
Example: organic chicken
Most cases tend to be relatively high on function 1 (income)
Example: organic chicken
Most cases tend to be relatively high on function 1 (income)

30. Plots – all groups

31. Prediction results
The functions do not predict well; most units are allocated to standard chicken – on average only 56.3% of the cases are allocated correctly

32. Stepwise discriminant analysis
As for linear regression it is possible to decide whether all predictors should appear in the equation regardless of their role in discriminating (the Enter option) or a sub-set of predictors is chosen on the basis of their contribution to discriminating between groups (the Stepwise method)

33. The step-wise method
A one-way ANOVA test is run on each of the predictors, where the target grouping variable determines the treatment levels. The ANOVA test provides a criterion value and tests statistics (usually the Wilks Lambda). According to the criterion value, it is possible to identify the predictor which is most relevant in discriminating between the groups
The predictor with the lowest Wilks Lambda (or which meets an alternative optimality criterion) enters the discriminating function, provided the p-value is below the set threshold (for example 5%).
An ANCOVA test is run on the remaining predictors, where the covariates are the target grouping variables and the predictors that have already entered the model. The Wilks Lambda is computed for each of the ANCOVA options.
Again, the criteria and the p-value determine which variable (if any) enter the discriminating function (and possibly whether some of the entered variables should leave the model).
The procedure goes back to step 3 and continues until none of the excluded variables have a p-value below the threshold and none of the entered variables have a p-value above the threshold (the stopping rule is met).

34. Alternative criteria Unexplained variance Smallest F ratio
Mahalanobis distance
Rao’s V

35. In SPSS
The step-wise method allows selection of relevant predictors

36. Output of the step-wise method
Only two predictors are kept in the model