1. Principal Components Shyh-Kang Jeng
Department of Electrical Engineering/
Graduate Institute of Communication/
Graduate Institute of Networking and Multimedia

2. Concept of Principal Componentsx2 x1

3. Principal Component Analysis
Explain the variance-covariance structure of a set of variables through a few linear combinations of these variables
Objectives
Data reduction
Interpretation
Does not need normality assumption in general

4. Principal Components

5. Result 8.1

6. Proof of Result 8.1

7. Result 8.2

8. Proof of Result 8.2

9. Proportion of Total Variance due to the kth Principal Component

10. Result 8.3

11. Proof of Result 8.3

12. Example 8.1

13. Example 8.1

14. Example 8.1

15. Geometrical Interpretation

16. Geometric Interpretation

17. Standardized Variables

18. Result 8.4

19. Proportion of Total Variance due to the kth Principal Component

20. Example 8.2

21. Example 8.2

22. Principal Components for Diagonal Covariance Matrix

23. Principal Components for a Special Covariance Matrix

24. Principal Components for a Special Covariance Matrix

25. Sample Principal Components

26. Sample Principal Components

27. Example 8.3

28. Example 8.3

29. Scree Plot to Determine Number of Principal Components

30. Example 8.4: Pained Turtles

31. Example 8.4

32. Example 8.4: Scree Plot

33. Example 8.4: Principal Component
One dominant principal component
Explains 96% of the total variance
Interpretation

34. Geometric Interpretation

35. Standardized Variables

36. Principal Components

37. Proportion of Total Variance due to the kth Principal Component

38. Example 8.5: Stocks Data Weekly rates of return for five stocks
X1: Allied Chemical
X2: du Pont
X3: Union Carbide
X4: Exxon
X5: Texaco

39. Example 8.5

40. Example 8.5

41. Example 8.6
Body weight (in grams) for n=150 female mice were obtained after the birth of their first 4 litters

42. Example 8.6

43. Comment
An unusually small value for the last eigenvalue from either the sample covariance or correlation matrix can indicate an unnoticed linear dependency of the data set
One or more of the variables is redundant and should be deleted
Example: x4 = x1 + x2 + x3

44. Check Normality and Suspect Observations
Construct scatter diagram for pairs of the first few principal components
Make Q-Q plots from the sample values generated by each principal component
Construct scatter diagram and Q-Q plots for the last few principal components

45. Example 8.7: Turtle Data

46. Example 8.7

47. Large Sample Distribution for Eigenvalues and Eigenvectors

48. Confidence Interval for li

49. Approximate Distribution of Estimated Eigenvectors

50. Example 8.8

51. Testing for Equal Correlation

52. Example 8.9

53. Monitoring Stable Process: Part 1

54. Example 8.10 Police Department Data
*First two sample cmponents explain 82%
of the total variance

55. Example 8.10: Principal Components

56. Example 8.10: 95% Control Ellipse

57. Monitoring Stable Process: Part 2

58. Example 8.11 T2 Chart for Unexplained Data

59. Example 8.12 Control Ellipse for Future Values
*Example 8.10 data after dropping out-of-control case

60. Example 8.12 99% Prediction Ellipse

61. Avoiding Computation with Small Eigenvalues