Outline of Today’s Discussion 1.Introduction to Discriminant Analysis 2.Assumptions for Discriminant Analysis 3.Discriminant Analysis in SPSS

Part 1 Introduction To Discriminant Analysis

Definition 1.Discriminant Analysis - A procedure for (1) determining whether two or more mutually exclusive groups can be distinguished from each other based on linear combinations of predictor variables, and (2) determining which variables contribute to that separation. 2.Generic Linear Discriminant Analysis Equation: Score = aX 1 + bX 2 + cX 3 + dX 4 + intercept 3.Why is the equation above said to be linear?

Definition 1.Generic Linear Discriminant Analysis Equation: Score = aX 1 + bX 2 + cX 3 + dX 4 + intercept 2.To the extent that the scores resulting from the Linear Discriminant Analysis (LDA) form two or more distinct (mutually exclusive) groups, the LDA is successful. 3.Example: Scores ranging from… 10 to 20  Group 1 60 to 70  Group 2

Definition 1.Additional LDAs could be used to sort scores on separate, ORTHOGONAL, dimensions of variability! 2.Dim 2 Score = eX 5 + fX 6 + gX 7 + hX 8 + intercept 3.Dim 3 Score = eX 9 + fX 10 + gX 11 + hX 12 + intercept 4.Dim 4 Score = iX 13 + jX 14 + kX 15 + lX 16 + intercept

Questions Addressed by LDA 1.What are the linear combinations of predictors whose average values best separate the groups? (On average, which combos work best?) 2.Are the linear combinations useful in actually classifying cases into groups? (What’s the accuracy?) 3.Is there a smaller subset of predictors that captures the differences between the groups? (What’s the parsimony?)

Sample LDA Research Questions 1.Predict which universities will fail financially, based on endowment size, debt, operating budget, and alumni support. 2.Predict which students should skip PSYC 100, based on SAT scores, grades in HS psyc classes, HS class rank, and Psyc AP score. 3.Predict which clients will relapse, based on education level, income, number of mental health hospitalizations, family stability, frequency of contact with drug / alcohol abusers. 4.Predict which children on the autistic spectrum will benefit from Applied Behavior Analysis (ABA), based on family SES, parental attachment scores, IQ, parental motivation, number of siblings.

Relation to Other Stats 1.LDA is similar to the ANOVA family of stats because… 2.It is a linear model. 3.It is based on means (it is parametric). 4.It can be univariate (anova) or multivariate (manova).

Relation to Other Stats 1.In the special case of prediction for exactly two groups, LDA is similar to multiple regression. 2.In the two-group case, the LDA coefficients and multiple regression coefficients will be proportional to each other (will differ from each other by only a simple multiplicative scaling factor) if the MR is computed with “Group” as the DV (criterion). This proportionality is only true in the 2-group case. 3.Informally, LDA is the “reverse” of MR and (M)ANOVA. In MR and (M)ANOVA, we typically use group as an IV, not a DV. In LDA, one or more DVs are used to predict group membership, a typical IV!

Relation to Other Stats 1.LDA is also similar to Factor Analysis in that both procedures use linear combinations of measured variables to “extract” (latent) ORTHOGONAL trends. 2.However, in LDA the researcher *must* explicitly identify variables as DVs or IVs. 3.By contrast, in Factor Analysis, variables are just variables; researchers do not identify DVs or IVs as such!

Part 2 Assumptions for Discriminant Analysis

Discriminant Analysis Assumptions 1.Perhaps counter-intuitively, a discriminant analysis requires a data set for which each participant’s group membership is already known. 2.So why bother “predicting” group membership if it’s already known? Answer: We use the current known- memberships to derive LDA equations for classifying future cases whose group membership is NOT known. (We’re building a model!)

Discriminant Analysis Assumptions 1.If the cases in your data set are not already assigned to groups, you can’t use LDA.  2.Note: There is an alternate multivariate procedure called Cluster Analysis, which discerns groups (“clusters”) in data sets where group membership is NOT known. 3.We won’t do a cluster analysis (nor its mighty big-sibling, hierarchical cluster analysis) this semester. 

Discriminant Analysis Assumptions 1.LDA’s assumption set is similar to that of many other linear models in the ANOVA family. 2.Assumptions: Normalcy Equal Variance Independence Note: The LDA and other ANOVA-like linear models can be robust to violations of the normalcy assumption to the extent that the groups have equal n. No such luck with the other two assumptions.

Discriminant Analysis Assumptions 1.If your LDA data set does not meet the normalcy or equal variance assumptions, there is another statistical procedures that might help... 2.Logistic Regression can predict binary group membership, like LDA, but without assuming either normalcy or equal variance. 3.Note: If the assumptions underlying LDA are met, LDA is often more sensitive (more likely to produce significance) than LR is. We won’t learn LR this semester. 

Part 3 Discriminant Analysis In SPSS

Discriminant Analysis in SPSS 1.Assume we would like to build an LDA to address the issue of criminal recidivism. 2.We will use a data set to classify 20 current prison inmates based on whether they had or had not been previously incarcerated. 3.In our SPSS Variable View, we will use a “1” for the group that had NOT been previously incarcerated, and a “2” for prior incarceration. 4.We will use the values on the following variables for the prediction: An Alcohol Abuse Test A Drug Abuse Test An Anger Test

Discriminant Analysis in SPSS 1.Click Analyze, Classify, and Discriminant. 2.Click your group variable and move it to the Grouping Variable box. 3.Click Define Range, and enter your minimum & maximum values. 4.Click Continue. 5.Click on the variables to be used for prediction; move to Independent box. 6.Click Statistics and click Means and univariate ANOVAs. 7.Click all four options in the Matrices box. 8.Click continue and Classify. 9.Click All Groups Equal (if equal n, or group size proportional to N). Else, click “Compute Group from Group Sizes”. 10.Click Combined Groups in the Plot Box and Within Groups in the Covariance Matrix box. 11.Click Summary Table in the Display box and Continue. 12.Click Save and Predicted Group Membership. 13.Click Continue and OK.

Discriminant Analysis in SPSS First of Three Helpful Output Boxes (occurs in the “Discriminant” Section) Here, you might informally ask yourself, “based on the means, which variables distinguish the groups?”

Discriminant Analysis in SPSS Second of Three Helpful Output Boxes (occurs under “Analysis 1. Summary of Cannonical Discriminant Functions”) Wilks’ Lambda also occurs in MANOVA. It reflects the proportion of variance NOT explained by the model. Lower Wilks’ Lambda scores are better. Our model’s predictions are significantly better than chance! ( p = )

Discriminant Analysis in SPSS Third of Three Helpful Output Boxes (occurs under “Classification Statistics”) Our model has 80% accuracy…beats a coin flip!

Discriminant Analysis in SPSS Hey Look! A New Variable Appears Automatically In Our Data View! (Better than Christmas!) The new “Dis_1” variable shows us what group (1 vs 2) was predicted for each case. Our Linear Discriminant Analysis misclassified cases 6,8,10, & 20.

Discriminant Analysis in SPSS 1.To summarize, we used a discriminant analysis to assess how well three variables (alcohol score, drug score, anger score) could predict whether an inmate had been previously incarcerated. 2.The Wilks’ Lambda was significant, meaning that our discriminant function produced a significant difference between the two target groups. 3.Our linear classifier (our LDA) “grouped” the inmates with an accuracy of 80%. 4.More generally, we can use LDA to predict membership in a wide range of groups…including clinically relevant DSM groups! (And there was much rejoicing!)