1 Factor Analysis and Inference for Structured Covariance Matrices Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking and Multimedia

2 History Early 20 th -century attempt to define and measure intelligence Developed primarily by scientists interested in psychometrics Advent of computers generated a renewed interest Each application must be examined on its own merits

3 Essence of Factor Analysis Describe the covariance among many variables in terms of a few underlying, but unobservable, random factors. A group of variables highly correlated among themselves, but having relatively small correlations with variables in different groups represent a single underlying factor

4 Example 9.8 Examination Scores

5 Orthogonal Factor Model

6

7

8

9

10 Example 9.1: Verification

11 Example 9.2: No Solution

12 Ambiguities of L When m>1

13 Principal Component Solution

14 Principal Component Solution

15 Residual Matrix

16 Determination of Number of Common Factors

17 Example 9.3 Consumer Preference Data

18 Example 9.3 Determination of m

19 Example 9.3 Principal Component Solution

20 Example 9.3 Factorization

21 Example 9.4 Stock Price Data Weekly rates of return for five stocks –X 1 : Allied Chemical –X 2 : du Pont –X 3 : Union Carbide –X 4 : Exxon –X 5 : Texaco

22 Example 9.4 Stock Price Data

23 Example 9.4 Principal Component Solution

24 Example 9.4 Residual Matrix for m=2

25 Maximum Likelihood Method

26 Result 9.1

27 Factorization of R

28 Example 9.5: Factorization of Stock Price Data

29 Example 9.5 ML Residual Matrix

30 Example 9.6 Olympic Decathlon Data

31 Example 9.6 Factorization

32 Example 9.6 PC Residual Matrix

33 Example 9.6 ML Residual Matrix

34 A Large Sample Test for Number of Common Factors

35 A Large Sample Test for Number of Common Factors

36 Example 9.7 Stock Price Model Testing

37 Example 9.8 Examination Scores

38 Example 9.8 Maximum Likelihood Solution

39 Example 9.8 Factor Rotation

40 Example 9.8 Rotated Factor Loading

41 Varimax Criterion

42 Example 9.9: Consumer- Preference Factor Analysis

43 Example 9.9 Factor Rotation

44 Example 9.10 Stock Price Factor Analysis

45 Example 9.11 Olympic Decathlon Factor Analysis

46 Example 9.11 Rotated ML Loadings

47 Factor Scores

48 Weighted Least Squares Method

49 Factor Scores of Principal Component Method

50 Orthogonal Factor Model

51 Regression Model

52 Factor Scores by Regression

53 Example 9.12 Stock Price Data

54 Example 9.12 Factor Scores by Regression

55 Example 9.13: Simple Summary Scores for Stock Price Data

56 A Strategy for Factor Analysis 1. Perform a principal component factor analysis –Look for suspicious observations by plotting the factor scores –Try a varimax rotation 2. Perform a maximum likelihood factor analysis, including a varimax rotation

57 A Strategy for Factor Analysis 3. Compare the solutions obtained from the two factor analyses –Do the loadings group in the same manner? –Plot factor scores obtained for PC against scores from ML analysis 4. Repeat the first 3 steps for other numbers of common factors 5. For large data sets, split them in half and perform factor analysis on each part. Compare the two results with each other and with that from the complete data set

58 Example 9.14 Chicken-Bone Data

59 Example 9.14:Principal Component Factor Analysis Results

60 Example 9.14: Maximum Likelihood Factor Analysis Results

61 Example 9.14 Residual Matrix for ML Estimates

62 Example 9.14 Factor Scores for Factors 1 & 2

63 Example 9.14 Pairs of Factor Scores: Factor 1

64 Example 9.14 Pairs of Factor Scores: Factor 2

65 Example 9.14 Pairs of Factor Scores: Factor 3

66 Example 9.14 Divided Data Set

67 Example 9.14: PC Factor Analysis for Divided Data Set

68 WOW Criterion In practice the vast majority of attempted factor analyses do not yield clear-cut results If, while scrutinizing the factor analysis, the investigator can shout “ Wow, I understand these factors, ” the application is deemed successful