1. 1 Principal Components Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking and Multimedia

2. 2 Concept of Principal Components x1x1 x2x2

3. 3 Principal Component Analysis Explain the variance-covariance structure of a set of variables through a few linear combinations of these variables Objectives –Data reduction –Interpretation Does not need normality assumption in general

4. 4 Principal Components

5. 5 Result 8.1

6. 6 Proof of Result 8.1

7. 7 Result 8.2

8. 8 Proof of Result 8.2

9. 9 Proportion of Total Variance due to the kth Principal Component

10. 10 Result 8.3

11. 11 Proof of Result 8.3

12. 12 Example 8.1

13. 13 Example 8.1

14. 14 Example 8.1

15. 15 Geometrical Interpretation

16. 16 Geometric Interpretation

17. 17 Standardized Variables

18. 18 Result 8.4

19. 19 Proportion of Total Variance due to the kth Principal Component

20. 20 Example 8.2

21. 21 Example 8.2

22. 22 Principal Components for Diagonal Covariance Matrix

23. 23 Principal Components for a Special Covariance Matrix

24. 24 Principal Components for a Special Covariance Matrix

25. 25 Sample Principal Components

26. 26 Sample Principal Components

27. 27 Example 8.3

28. 28 Example 8.3

29. 29 Scree Plot to Determine Number of Principal Components

30. 30 Example 8.4: Pained Turtles

31. 31 Example 8.4

32. 32 Example 8.4: Scree Plot

33. 33 Example 8.4: Principal Component One dominant principal component –Explains 96% of the total variance Interpretation

34. 34 Geometric Interpretation

35. 35 Standardized Variables

36. 36 Principal Components

37. 37 Proportion of Total Variance due to the kth Principal Component

38. 38 Example 8.5: Stocks Data Weekly rates of return for five stocks –X 1 : Allied Chemical –X 2 : du Pont –X 3 : Union Carbide –X 4 : Exxon –X 5 : Texaco

39. 39 Example 8.5

40. 40 Example 8.5

41. 41 Example 8.6 Body weight (in grams) for n =150 female mice were obtained after the birth of their first 4 litters

42. 42 Example 8.6

43. 43 Comment An unusually small value for the last eigenvalue from either the sample covariance or correlation matrix can indicate an unnoticed linear dependency of the data set One or more of the variables is redundant and should be deleted Example: x 4 = x 1 + x 2 + x 3

44. 44 Check Normality and Suspect Observations Construct scatter diagram for pairs of the first few principal components Make Q-Q plots from the sample values generated by each principal component Construct scatter diagram and Q-Q plots for the last few principal components

45. 45 Example 8.7: Turtle Data

46. 46 Example 8.7

47. 47 Large Sample Distribution for Eigenvalues and Eigenvectors

48. 48 Confidence Interval for i

49. 49 Approximate Distribution of Estimated Eigenvectors

50. 50 Example 8.8

51. 51 Testing for Equal Correlation

52. 52 Example 8.9

53. 53 Monitoring Stable Process: Part 1

54. 54 Example 8.10 Police Department Data *First two sample cmponents explain 82% of the total variance

55. 55 Example 8.10: Principal Components

56. 56 Example 8.10: 95% Control Ellipse

57. 57 Monitoring Stable Process: Part 2

58. 58 Example 8.11 T 2 Chart for Unexplained Data

59. 59 Example 8.12 Control Ellipse for Future Values *Example 8.10 data after dropping out-of-control case

60. 60 Example 8.12 99% Prediction Ellipse

61. 61 Avoiding Computation with Small Eigenvalues