Factor Analysis Basics

Why Factor? Combine similar variables into more meaningful factors. Reduce the number of variables dramatically while retaining most of the explanatory power, for subsequent analyses, such as Clustering.

Factor Types, Variables R-type (commonly used) Q-type (more like clustering, rare) Variables are generally metric. Must have some correlations. Dummies can be used with ‘Boolean Factor Analysis’.

Principal Components Analysis Common Factor uses only the portion of variance of each variable that is in common with other variables, in the diagonal of the correlation matrix. Specific and Error Variances are excluded. Used when identifying latent factors is primary motive. Principal components uses the entire variance – puts ‘1’ in the diagonal of the correlation matrix. Used when prediction is primary motive.

What it does Creates a factor – linear combination of all variables – that best explains the combined variance in all variables. Defines a second factor – orthogonal to the first – best explains the residual variance. Defines a third – orthogonal to the first two, and so on.

Result First factor is correlated highly with most variables, second one less so, and so on – difficult to interpret meaningfully. Solution: Rotate the factor matrix Factor 1 Factor 2 Rotated Factor 1 Rotated Factor 2

Types of Rotation Quartimax Simplify rows – variable loads high on one factor, low on others Varimax Simplify columns – clearer separation of factors – each factor has variables that either load high or load very low Equimax Compromise between the two – rarely used

How many factors in the end? Use latent root (eigen value) criterion – Pick only factors that explain at least as much variance as one of the original variables. Use Scree plot – find parsimonious solution – stop when additional factors do not add sufficient explaining power.