1. © LOUIS COHEN, LAWRENCE MANION AND KEITH MORRISON
FACTOR ANALYSIS, CLUSTER ANALYSIS AND STRUCTURAL EQUATION MODELLING
© LOUIS COHEN, LAWRENCE MANION AND KEITH MORRISON
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

2. STRUCTURE OF THE CHAPTER
Factor analysis
What to look for in factor analysis output
Cluster analysis
A note on structural equation modelling
A note on multilevel modelling
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

3. WHAT IS FACTOR ANALYSIS?
A method of grouping together variables which have something in common.
The researcher can take a set of variables and reduce them to a smaller number of underlying factors (latent variables) which account for as many variables as possible.
It detects structures and commonalities in the relationships between variables. Researchers can identify where different variables in fact are addressing the same underlying concept.
It detects latent (unobservable) factors.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

4. TWO MAIN FORMS OF FACTOR ANALYSIS
Exploratory factor analysis: the use of factor analysis (principal components analysis in particular) to explore previously unknown groupings of variables, to seek underlying patterns, clusterings and groups.
Confirmatory factor analysis is more stringent, testing a found set of factors against a hypothesized model of groupings and relationships.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

5. SAFETY CHECKS FOR FACTOR ANALYSIS
Sample size.
Number of variables.
Ratio of sample size to number of variables.
Interval and ratio data.
Sampling adequacy.
Intercorrelations between variables.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

6. SAFETY CHECKS FOR FACTOR ANALYSIS
Intercorrelations between factors.
Normal distributions.
Linearity.
Outliers.
Selection bias/proper specification.
Theoretical underpinning of factors.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

7. STAGE ONE IN FACTOR ANALYSIS
Check that the data are suitable for factor analysis:
(a) Sample size (varies in the literature, from a minimum of 30 to a minimum of 300); if the sample size is small then the factor loadings should be high to be included).
(b) Number of variables.
(c) Ratio of sample size to number of variables (different ratios given in literature, from 5:1 to 30:1).
(c) Strength of intercorrelations should be no less than 0.3.
(d) Bartlett’s test of sphericity should be statistically significant ( < 0.05).
(e) Kaiser-Mayer-Olkin measure of sampling adequacy should be 0.6 or higher (maximum is 1).
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

8. STAGE TWO IN FACTOR ANALYSIS
Decide which form of extraction method to use:
(a) Principal components analysis is widely used.
(b) Set the Kaiser criterion (the Eigenvalues to be set at greater than 1); the Eigenvalue of a factor indicates the amount of the total variance explained by that factor – if it is less than 1.00 then it does not have any additional explanatory value and should be ignored (SPSS does this automatically).
(c) Unrotated factor solution to be set.
(d) Scree plot to be set.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

9. SCREE PLOT IN SPSS
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

10. STAGE THREE IN FACTOR ANALYSIS
Conduct the factor rotation:
(a) Decide which of the two main approaches to use:
Oblique (related variables): Direct Oblimin
Orthogonal (unrelated variables): Varimax.
(b) People often use the varimax solution when it should not be used, as it is sometimes easier to use than other kinds.
(c) Check that the rotated solution is set.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

11. ROTATION
Rotation keeps together those items that are closely related and separates them clearly from other items, i.e. it includes and excludes (keeps together a group of homogeneous items and keeps them apart from other groups).
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

12. EXAMPLE OF FACTOR ANALYSIS USING SPSS
Factor analysis for an oblique rotation.
Direct Oblimin rotation.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

13. Analyze  Dimension Reduction  Factor  Move the variables to be included to the ‘Variables’ box 
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

14. Click on ‘Descriptives’  Click on ‘KMO and Bartlett’s test of sphericity’  Click on Coefficients’  Click ‘Continue’ 
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

15. Click on ‘Extraction’  Click on ‘Principal components’  Click on ‘Correlation matrix’  Click on ‘Unrotated factor solution’  Click on ‘Scree plot’  Click on ‘Based on Eigenvalue’  Click ‘Continue’ 
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

16. Click on ‘Rotation’  Click on ‘Direct Oblimin’ or ‘Varimax’ (depending on whether the rotation is oblique or orthogonal)  Click ‘Continue’  return to main screen and click ‘OK’
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

17. ANALYSIS OF THE EXAMPLE FROM SPSS
SPSS produces many tables for factor analysis. Be selective but fair to the data.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

18. Check correlation coefficients (most should be over 0
Check correlation coefficients (most should be over 0.3) (selection only reproduced here, not the full table)
How much do you feel that working with colleagues all day is really a strain for you?
How much do you feel emotionally drained by your work?
How much do you worry that your job is hardening you emotionally?
How much frustration do you feel in your job?
Correlation
1.000
0.554
0.507
0.461
0.580
0.518
0.646
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

19. SUITABILITY FOR FACTOR ANALYSIS
KMO and Bartlett's Test
Kaiser-Meyer-Olkin Measure of Sampling Adequacy.
0.845
Bartlett's Test of Sphericity
Approx. Chi-Square df 36
Sig.
0.000
KMO > 0.6
Bartlett’s test Sig.:  < 0.05
The data are suitable for factor analysis
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

20. How much of the variance is explained by each item (lower than 0
How much of the variance is explained by each item (lower than 0.3 and the item is a poor fit)
Communalities
Initial
Extraction
How hard do you feel you are working in your job?
1.000
.779
How much do you feel exhausted by the end of the workday?
.818
How much do you feel that you cannot cope with your job any longer?
.578
How much do you feel that you treat colleagues as impersonal objects?
How much do you feel that working with colleagues all day is really a strain for you?
.602
How much do you feel emotionally drained by your work?
.629
How tired do you feel in the morning, having to face another school day?
.595
How much do you worry that your job is hardening you emotionally?
.661
How much frustration do you feel in your job?
Extraction Method: Principal Component Analysis.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

21. Total Variance Explained
Two factors found: factor one explains per cent of total variance; factor two explains per cent of total variance.
Total Variance Explained
Component
Initial Eigenvalues
Extraction Sums of Squared Loadings
Rotation Sums of Squared Loadingsa
Total
% of Variance
Cumulative % 1
4.139
45.985
4.028 2
1.697
18.851
64.836
1.991 3
.661
7.342
72.178 4
.542
6.023
78.202 5
.531
5.900
84.102 6
.451
5.006
89.107 7
.395
4.390
93.497 8
.323
3.593
97.090 9
.262
2.910
Extraction Method: Principal Component Analysis.
a. When components are correlated, sums of squared loadings cannot be added to obtain a total variance.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

22. Pattern Matrixa
Component 1 2
How hard do you feel you are working in your job?
.005
.882
How much do you feel exhausted by the end of the workday?
.252
.834
How much do you feel that you cannot cope with your job any longer?
.691
.234
How much do you feel that you treat colleagues as impersonal objects?
.674
-.459
How much do you feel that working with colleagues all day is really a strain for you?
.782
-.158
How much do you feel emotionally drained by your work?
.774
.096
How tired do you feel in the morning, having to face another school day?
.697
.247
How much do you worry that your job is hardening you emotionally?
.814
-.008
How much frustration do you feel in your job?
.752
.097
Extraction Method: Principal Component Analysis.
Rotation Method: Oblimin with Kaiser Normalization.
a. Rotation converged in 6 iterations.
Decide the cut-off points and which variables to include.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

23. WHICH VARIABLES TO INCLUDE IN A FACTOR
For each variable:
Include the highest scoring variables.
Omit the low scoring variables.
Look for where there is a clear scoring distance between those included and those excluded.
Review your selection to check that no lower scoring variables have been excluded which are conceptually close to those included.
Review your selection to check whether some higher scoring variables should be excluded if they are not sufficiently conceptually close to the others that have been included.
Review your final selection to see that they are conceptually similar.
NB. Inclusion and exclusion are an art, not a science; there is no simple formula, so you have to use your judgement.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

24. WHAT TO REPORT
Method of factor analysis used (Principal components; Direct Oblimin); KMO and Bartlett test of sphericity; Eigenvalues greater than 1; scree test; rotated solution).
How many factors were extracted with Eigenvalues greater than 1.
How many factors were included as a result of the scree test.
Give a name/title to each of the factors.
Indicate how much of the total variance was explained by each factor.
Report the cut-off point for the variables included in each factor.
Indicate the factor loadings of each variable in the factor.
What the results tell us.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

25. FIVE STAGES IN FACTOR ANALYSIS
Safety checks
Stage 2
Data processing and initial analysis
Stage 3
Constructing the factors from the variables
Stage 4
Naming the factors
Stage 5
Reporting the factor analysis
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

26. CLUSTER ANALYSIS
Factor analysis enables the researcher to group together factors and variables, but cluster analysis enables the researcher to group together similar and homogeneous sub-samples of people.
SPSS creates a dendrogram of clusters of people into groups.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

27. TWO MAIN FORMS OF CLUSTER ANALYSIS
1. Hierarchical cluster analysis.
2. Non-hierarchical analysis.
Researchers often use K-Means Cluster (non‑hierarchical cluster analysis), Hierarchical Cluster and Two-step Cluster, of which Hierarchical Cluster analysis is the most widely used.
Cluster analysis can work with interval, ratio, ordinal and nominal data, using different statistics for each kind of data.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

28. DENDROGRAM IN CLUSTER ANALYSIS
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

29. INTERPRETING THE DENDROGRAM
There are two main clusters
Cluster One: Persons 19, 20, 2, 13, 15, 9, 11, 18, 14, 16, 1, 10, 12, 5, 17
Cluster Two: Persons 7, 8, 4, 3, 6
If one wishes to have smaller clusters then three clusters can be found
Cluster One: Persons 19, 20, 2, 13, 15, 9, 11, 18
Cluster Two: Persons 14, 16, 1, 10, 12, 5, 17
Cluster Three: Persons 7, 8, 4, 3, 6
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

30. ANALYSING AND REPORTING CLUSTERS
What is the similarity criterion that combines individual cases into a single cluster?
How many cases (and who) are in each cluster?
How similar are the cases within each cluster?
What differentiates that cluster from another?
What is the criterion or characteristic that combines clusters?
How similar/dissimilar are the clusters?
At what level in the hierarchy is it most advisable to cease combining clusters?
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

31. STRUCTURAL EQUATION MODELLING
The name given to a group of techniques that enable researchers to construct models of putative causal relations, and to test those models against data.
It is designed to enable researchers to confirm, modify and test their models of causal relations between variables.
It is based on multiple regression and factor analysis.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

32. STRUCTURAL EQUATION MODELLING
It works with observed and unobserved variables, not latent factors (as in factor analysis).
It is a particular kind of multiple regression analysis that enables the researcher to see the relative weightings of observed independent variables on each other and on a dependent variable, to establish pathways of causation, and to determine the direct and indirect effects of independent variables on a dependent variable.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

33. A CAUSAL MODEL (USING AMOS WITH SPSS)
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

34. THE CAUSAL MODEL WITH CALCULATIONS ADDED
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

35. INTERPRETING THE CAUSAL MODEL
Socio-economic’ status exerts a direct powerful influence on class of degree (.18), which is higher than the direct influence of either ‘part-time work’ (–.01) or ‘level of motivation for academic study’ (.04).
‘Socio-economic status’ exerts a powerful direct influence on ‘level of motivation for academic study’ (.52), which is higher than the influence of ‘socio-economic status’ on ‘class of degree’ (.18).
‘Socio-economic status’ exerts a powerful direct and negative influence on ‘part-time work’ (–.21), i.e. the higher the socio-economic status, the lower the amount of part-time work undertaken.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

36. INTERPRETING THE CAUSAL MODEL
‘Part-time work’ exerts a powerful direct influence on ‘level of motivation for academic study’ (1.37), and this is higher than the influence of ‘socio-economic status’ on ‘level of motivation for academic study’ (.52).
‘Level of motivation for academic study’ exerts a powerful negative direct influence on ‘part-time work’ (–1.45), i.e. the higher the level of motivation for academic study, the lower the amount of part-time work undertaken.
‘Level of motivation for academic study’ exerts a slightly more powerful influence on ‘class of degree’ (.04) than does ‘part-time work’ (–.01).
‘Part-time work’ exerts a negative influence on the class of degree (–.01), i.e. the more one works part-time, the lower the class of degree obtained.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

37. A STRUCTURAL EQUATION MODEL (USING AMOS IN SPSS)
Ovals = factors
Rectangles = variables for each factor
E = Error factor
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

38. A NOTE ON MULTILEVEL MODELLING
Data and variables exist at individual and group levels
between students over all groups
between groups
between students within groups
individual
group
class
school
local
regional
national
international
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

39. A NOTE ON MULTILEVEL MODELLING
Data are ‘nested’, e.g. individual-level data are nested within group, class, school, regional etc. levels.
A dependent variable is affected by independent variables at different levels, i.e. data are hierarchical.
Multilevel modelling uses regression analysis and multilevel regression.
Multilevel modelling enables the researcher to calculate the relative impact on a dependent variable of one or more independent variables at each level of the hierarchy, and thereby to identify factors at each level of the hierarchy that are associated with the impact of that level.
© 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors