1. An introduction to exploratory factor analysis in IBM SPSS Statistics
Dr Anne Laure Humbert
York University, 10 November 2017

2. In this session:
Distinguishing between exploratory and confirmatory uses of Factor Analysis (FA)
Illustrate how to carry out a FA in SPSS using real data
Determining the number of factors to extract
Explaining the concept of rotation of factors
Describing how to name a factor
State the major limitations of factor analytic techniques

3. Definition and purpose
Factor Analysis (FA) is a statistical technique which analyses the underlying covariance structure between variables in order to identify its underlying structure
Using FA allows us to:
Examine the interrelationships among a large number of variables
Attempt to explain these variables in terms of their common underlying dimensions (factors)
Summarize and reduce data without independent and dependent variables

4. We have data about a number of aspects of team climate
Example: Team climate
We have data about a number of aspects of team climate
Are all of these elements separate, or can we identify/group them into an underlying structure?

5. Example: Team climate
v_151: I am clear about what my team's objectives are
v_152: I am in agreement with the team's objectives
v_153: The team's objectives are clearly understood by other members of the team
v_154: The team's objectives can actually be achieved
v_155: The team's objectives are worthwhile to the organisation
v_156: We keep each other informed about work-related issues in the team
v_157: There are real attempts to share information throughout the team
v_158: We have a 'we are in it together' attitude
v_159: We are prepared to question the basis of what the team is doing
v_160: We critically appraise potential weaknesses in what we are doing in order to achieve the best possible outcome
v_161: We build on each other's ideas in order to achieve the best possible outcome
v_162: We are always searching for fresh, new ways of looking at problems
v_163: We take the time needed to develop new ideas
v_164: We co-operate in order to help each other and apply new ideas

6. Example: team climate

7. Example: team climate

8. Example: team climate

9. Stages in Factor Analysis
Stage 1: Objectives of FA
Stage 2: Designing an FA
Stage 3: Assumptions
Stage 4: Factors & Overall Fit
Stage 5: Interpreting the factors
Stage 6: Validation of FA
Stage 7: Additional uses of FA results

10. Objectives of FA
Exploratory Factor Analysis (EFA): Data reduction / summarisation
To discover a the factor structure of a construct and examine its reliability
Data driven
Confirmatory Factor Analysis (CFA): Structure detection
To confirm the fit of the hypothesised factor structure to the observed data
Theory driven

11. Example EFA vs CFA in a research paper
Paper produces a model of implementation leadership using EFA
Paper then uses another sample to test the framework through CFA

12. Summarise vs reduce
Summarising data: derives underlying dimensions that describe the data in a much smaller number of concepts
Reducing data: extends summarisation by deriving an empirical value (factor score) for each dimension and then substitute this value for the original values

13. Warning: FA will always provide factors… but do they make sense?
Variable selection
Consider conceptual underpinnings
Use judgment on the appropriateness of variables for factor analysis
Warning: FA will always provide factors… but do they make sense?

14. Variable selection and measurement
What type of variables can be used in FA?
A correlation value can be calculated among all variables.
Numerical or pseudo-numerical (e.g. Likert scales) variables only
Reflective variables, i.e. variables that reflect a latent construct
How many variables should be included?
Ideally five or more variables for each proposed factor
Little use in identifying factors with one variable

15. Sample size
At least 50, preferably 100 or more At least five times as many observations as the number of variables, better at least 10:1 Employ the most parsimonious set of variables Interpret findings cautiously at lower cases-to variable ratio

16. Assumptions in FA More conceptual than statistical
Some underlying structure does exist
Researcher should ensure that observed patterns are conceptually valid
Sample homogeneity
Some degree of multicollinearity is desirable

17. Assumptions in FA Check the correlation matrix and avoid:
Correlations < .30
Anti-image correlation matrix: partial correlations > .7
Check the validity of the model:
Bartlett test of sphericity (sig < .05)
Measure of sampling adequacy (MSA)
> .8 meritorious
> .7 middling
> .6 mediocre
> .5 miserable
< .5 unacceptable
Examine for each variable

18. Determining the number of factors to extract
Iterative factor extraction
Combine conceptual foundation (how many factors should be in the structure?) with some empirical evidence (how many factors can be reasonably supported?)

19. Determining the number of factors to extract
Latent root criterion
Latent roots (eigenvalues) > 1, a.k.a Kaiser criterion
Cut-off is most reliable when number of variables is between 20 and 50
A priori criterion
Number of factors to extract is known
Used when testing a theory or a hypothesis or when replicating previous work

20. Determining the number of factors to extract
Percentage of variance criterion
Achieving a specified cumulative percentage of total variance extracted by successive factors
~ 60% or more can be regarded satisfactory
Scree test criterion
Identifies the optimum number of factors that can be extracted before the amount of unique variance begins to dominate the common variance structure

21. Scree plot

22. Interpreting the factors
Strong conceptual foundation for the anticipated structure required
Factor loadings: correlation of each variable and the factor, the higher the better
Initial unrotated factor matrix
Rotated factor matrix: simplifies factor structure

23. Factor rotation Orthogonal (axes are maintained at 90 degrees):
Quartimax
Varimax
Equimax
Preferred when the goal is data reduction for subsequent use in other multivariate techniques
Oblique (axes are not maintained at 90 degrees):
Oblimin
Promax
Orthoblique
Preferred when the goal is to obtain theoretically meaningful factors or constructs

24. Orthogonal factor rotation

25. Oblique factor rotation

26. Varimax Rotation
Varimax –simplifies columns –as many values in each column as close to zero as possible. Frequently used. –default option

27. Varimax Rotation
Varimax –simplifies columns –as many values in each column as close to zero as possible. Frequently used. –default option

28. Interpreting the factor matrix
Examine the factor matrix of loadings
Identify significant loadings for each variable
Cross-loadings
Assess the communalities of the variables
<.5 suggests no sufficient explanation
Respecify the factor model if needed
No significant loading
Significant loading but low communality
Cross-loading
Label the factors

29. Validation Check and report:
Adequacy (KMO, BTS, no communalities < 0.3, total variance > 0.6 and no less than 0.5)
Convergent validity (all factors loading > 0.6 and no less than 0.5, factors loadings average to > 0.7)
Discriminant validity (no strong cross loadings, factor correlation matrix < 0.7)
Reliability (Cronbach alpha scores within each factors > 0.7 and no less than 0.6)

30. Example using Team Climate data in IBM SPSS Statistics

31. Correlations Analyze > Correlations > Bivariates CORRELATIONS
/VARIABLES=v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164
/PRINT=TWOTAIL NOSIG
/MISSING=PAIRWISE.

32. Correlations

33. Correlation Matrix
Check for evidence of strong correlations

34. Factor Analysis
Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

40. Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

41. Univariate analysis

42. Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

43. Anti-image Correlation Matrix
Check for MSAs value along the diagonal, ideally >0.5 MSA = Measure of Sampling Adequacy

44. Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

45. Validity
KMO = 0.934, well above 0.7 BTS < 0.01, evidence of valid model

46. Communalities > 0.5 Evidence that most variables in the FA model provide sufficient explanation

47. Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

48. Total variance explained
The FA model consists of 2 components (eigenvalues > 1) that explain 59% of the variance

49. Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

50. Scree Plot

51. Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA MINEIGEN(1) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

52. Check: Loadings > 0.5 No cross- loadings Take remedial action

53. Interpreting the factors
Information and cooperation
Objectives
Are other more fine-grained models possible?

54. Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA FACTOR(3) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

55. Remember last variable communality was very low
Problem with cross loading

56. Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_161 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA FACTOR(4) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

58. Analyze > Dimension Reduction > Factor… FACTOR /VARIABLES v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_162 v_163 v_164 /MISSING LISTWISE /ANALYSIS v_151 v_152 v_153 v_154 v_155 v_156 v_157 v_158 v_159 v_160 v_162 v_163 v_164 /PRINT UNIVARIATE INITIAL KMO AIC EXTRACTION ROTATION /FORMAT SORT BLANK(.5) /PLOT EIGEN /CRITERIA FACTOR(4) ITERATE(25) /EXTRACTION PC /CRITERIA ITERATE(25) /ROTATION VARIMAX /METHOD=CORRELATION.

60. Interpreting the factors
Objectives
Shared information
Critical outlook
Innovation

61. An introduction to exploratory factor analysis in IBM SPSS Statistics
Dr Anne Laure Humbert