1. Factor Analysis and Inference for Structured Covariance Matrices
Shyh-Kang Jeng
Department of Electrical Engineering/
Graduate Institute of Communication/
Graduate Institute of Networking and Multimedia

2. History Early 20th-century attempt to define and measure intelligence
Developed primarily by scientists interested in psychometrics
Advent of computers generated a renewed interest
Each application must be examined on its own merits

3. Essence of Factor Analysis
Describe the covariance among many variables in terms of a few underlying, but unobservable, random factors.
A group of variables highly correlated among themselves, but having relatively small correlations with variables in different groups represent a single underlying factor

4. Example 9.8 Examination Scores

5. Orthogonal Factor Model

6. Orthogonal Factor Model

7. Orthogonal Factor Model

8. Orthogonal Factor Model

9. Orthogonal Factor Model

10. Example 9.1: Verification

11. Example 9.2: No Solution

12. Ambiguities of L When m>1

13. Principal Component Solution

14. Principal Component Solution

15. Residual Matrix

16. Determination of Number of Common Factors

17. Example 9.3 Consumer Preference Data

18. Example 9.3 Determination of m

19. Example 9.3 Principal Component Solution

20. Example 9.3 Factorization

21. Example 9.4 Stock Price Data
Weekly rates of return for five stocks
X1: Allied Chemical
X2: du Pont
X3: Union Carbide
X4: Exxon
X5: Texaco

22. Example 9.4 Stock Price Data

23. Example 9.4 Principal Component Solution

24. Example 9.4 Residual Matrix for m=2

25. Maximum Likelihood Method

26. Result 9.1

27. Factorization of R

28. Example 9.5: Factorization of Stock Price Data

29. Example 9.5 ML Residual Matrix

30. Example 9.6 Olympic Decathlon Data

31. Example 9.6 Factorization

32. Example 9.6 PC Residual Matrix

33. Example 9.6 ML Residual Matrix

34. A Large Sample Test for Number of Common Factors

35. A Large Sample Test for Number of Common Factors

36. Example 9.7 Stock Price Model Testing

37. Example 9.8 Examination Scores

38. Example 9.8 Maximum Likelihood Solution

39. Example 9.8 Factor Rotation

40. Example 9.8 Rotated Factor Loading

41. Varimax Criterion

42. Example 9.9: Consumer-Preference Factor Analysis

43. Example 9.9 Factor Rotation

44. Example 9.10 Stock Price Factor Analysis

45. Example 9.11 Olympic Decathlon Factor Analysis

46. Example 9.11 Rotated ML Loadings

47. Factor Scores

48. Weighted Least Squares Method

49. Factor Scores of Principal Component Method

50. Orthogonal Factor Model

51. Regression Model

52. Factor Scores by Regression

53. Example 9.12 Stock Price Data

54. Example 9.12 Factor Scores by Regression

55. Example 9.13: Simple Summary Scores for Stock Price Data

56. A Strategy for Factor Analysis
1. Perform a principal component factor analysis
Look for suspicious observations by plotting the factor scores
Try a varimax rotation
2. Perform a maximum likelihood factor analysis, including a varimax rotation

57. A Strategy for Factor Analysis
3. Compare the solutions obtained from the two factor analyses
Do the loadings group in the same manner?
Plot factor scores obtained for PC against scores from ML analysis
4. Repeat the first 3 steps for other numbers of common factors
5. For large data sets, split them in half and perform factor analysis on each part. Compare the two results with each other and with that from the complete data set

58. Example 9.14 Chicken-Bone Data

59. Example 9.14:Principal Component Factor Analysis Results

60. Example 9.14: Maximum Likelihood Factor Analysis Results

61. Example 9.14 Residual Matrix for ML Estimates

62. Example 9.14 Factor Scores for Factors 1 & 2

63. Example 9.14 Pairs of Factor Scores: Factor 1

64. Example 9.14 Pairs of Factor Scores: Factor 2

65. Example 9.14 Pairs of Factor Scores: Factor 3

66. Example 9.14 Divided Data Set

67. Example 9.14: PC Factor Analysis for Divided Data Set

68. WOW Criterion
In practice the vast majority of attempted factor analyses do not yield clear-cut results
If, while scrutinizing the factor analysis, the investigator can shout “Wow, I understand these factors,” the application is deemed successful

69. Structural Equation Models
Sets of linear equations to specify phenomena in terms of their presumed cause-and-effect variables
In its most general form, the models allow for variables that can not be measured directly
Particularly helpful in the social and behavioral science

70. LISREL (Linear Structural Relationships) Model

71. Example h: performance of the firm x: managerial talent Y1: profit
Y2: common stock price
X1: years of chief executive experience
X2: memberships on board of directors

72. Linear System in Control Theory

73. Kinds of Variables
Exogenous variables: not influenced by other variables in the system
Endogenous variables: affected by other variables
Residual: associated with each of the dependent variables

74. Construction of a Path Diagram
Straight arrow
to each dependent (endogenous) variable from each of its source
also to each dependent variables from its residual
Curved, double-headed arrow
between each pair of independent (exogenous) variables thought to have nonzero correlation

75. Example 9.15 Path Diagram

76. Example 9.15 Structural Equation

77. Covariance Structure

78. Estimation

79. Example 9.16 Artificial Data

80. Example 9.16 Artificial Data

81. Example 9.16 Artificial Data

82. Assessing the Fit of the Model
The number of observations p for Y and q for X must be larger than the total number of unknown parameters t < (p+q)(p+q+1)/2
Parameter estimates should have appropriate signs and magnitudes
Entries in the residual matrix S – S should be uniformly small

83. Model-Fitting Strategy
Generate parameter estimates using several criteria and compare the estimates
Are signs and magnitudes consistent?
Are all variance estimates positive?
Are the residual matrices similar?
Do the analysis with both S and R
Split large data sets in half and perform the analysis on each half