1. Techniques for studying correlation and covariance structure Principal Components Analysis (PCA) Factor Analysis

2. Principal Component Analysis

3. Then where are eigenvectors of  of length 1 and are eigenvalues of  Lethave a p-variate Normal distribution with mean vector

4. or and have covariance matrix The Principal Components are defined by are independent with Var(C j ) = j

5. Many times for large value of j, Var(C j ) = j, is small and contributes little to the total variance In this case the number of variables can be reduced to the small number of principal components. In regression analysis it is sometimes useful to transform the independent variables into their principal components

6. Scree Plot Proportion of variance Principal Components

7. Scree Plot Cumulative Proportion of variance Principal Components

8. Example In this example wildlife (moose) population density was measured over time (once a year) in three areas.

9. picture Area 3 Area 1 Area 2

10. The Sample Statistics The mean vector The covariance matrix The correlation matrix

11. Principal component Analysis The eigenvalues of S The eigenvectors of S The principal components

12. The Example

13. Scree Plots

14. More Examples

20. Computation of the eigenvalues and eigenvectors of  Recall:

21. continuing we see that: For large values of n

22. The algorithm for computing the eigenvector 1.Compute rescaling so that the elements do not become to large in value. i.e. rescale so that the largest element is 1. 2.Computeusing the fact that: 3.Compute 1 using

23. 4.Repeat using the matrix 5.Continue with i = 2, …, p – 1 using the matrix Example – Using Excel - EigenEigen

24. Factor Analysis An Alternative technique for studying correlation and covariance structure

25. Let The Factor Analysis Model: have a p-variate Normal distribution with mean vector Let F 1, F 2, …, F k denote independent standard normal observations (the Factors) Suppose that there exists constants ij (the loadings) such that: Let  1,  2, …,  p denote independent normal random variables with mean 0 and var(  i ) =  p x 1 = 11 F 1 + 12 F 2 + … +  k F k +   x 2 = 21 F 1 + 22 F 2 + … +  k F k +   … x p = p1 F 1 + p2 F 2 + … + pk F k +  p

26. Using matrix notation where and with

27. Note: hence i.e. the component of variance of x i that is due to the common factors F 1, F 2, …, F k. and i.e. the component of variance of x i that is specific only to that observation.

28. Determine cov(x i,F j ) Recall

29. Also where Thus Now, if also then ij is the correlation between x i and F j.

30. Rotating Factors Recall the factor Analysis model This gives rise to the vectorhaving covariance matrix: Let P be any orthogonal matrix, then and

31. Hence if is a Factor Analysis model then so also is with where P is any orthogonal matrix. with

32. The process of exploring other models through orthogonal transformations of the factors is called rotating the factors There are many techniques for rotating the factors VARIMAX Quartimax Equimax VARIMAX rotation attempts to have each individual variables load high on a subset of the factors

33. Example: Olympic decathlon Scores Data was collected for n = 160 starts (139 athletes) for the ten decathlon events (100-m run, Long Jump, Shot Put, High Jump, 400-m run, 110-m hurdles, Discus, Pole Vault, Javelin, 1500-m run). The sample correlation matrix is given on the next slide

34. Correlation Matrix

36. Identification of the factors