1 Multivariate Normal Distribution Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking and Multimedia

2 Multivariate Normal Distribution Generalized from univariate normal density Base of many multivariate analysis techniques Useful approximation to “ true ” population distribution Central limit distribution of many multivariate statistics Mathematical tractable

3 Univariate Normal Distribution

4 Table 1, Appendix

5 Square of Distance (Mahalanobis distance)

6 p-dimensional Normal Density

7 Example 4.1 Bivariate Normal

8 Example 4.1 Squared Distance

9 Example 4.1 Density Function

10 Example 4.1 Bivariate Distribution  11 =  22,  12 = 0

11 Example 4.1 Bivariate Distribution  11 =  22,  12 = 0.75

12 Contours

13 Result 4.1

14 Example 4.2 Bivariate Contour

15 Example 4.2 Positive Correlation

16 Probability Related to Squared Distance

17 Probability Related to Squared Distance

18 Result 4.2

19 Example 4.3 Marginal Distribution

20 Result 4.3

21 Proof of Result 4.3: Part 1

22 Proof of Result 4.3: Part 2

23 Example 4.4 Linear Combinations

24 Result 4.4

25 Example 4.5 Subset Distribution

26 Result 4.5

27 Example 4.6 Independence

28 Result 4.6

29 Proof of Result 4.6

30 Proof of Result 4.6

31 Example 4.7 Conditional Bivariate

32 Example 4.1 Density Function

33 Example 4.7

34 Result 4.7

35  2 Distribution

36  2 Distribution Curves

37 Table 3, Appendix

38 Proof of Result 4.7 (a)

39 Proof of Result 4.7 (b)

40 Result 4.8

41 Proof of Result 4.8

42 Example 4.8 Linear Combinations

43 Example 4.8 Linear Combinations

44 Multivariate Normal Likelihood

45 Maximum-likelihood Estimation

46 Trace of a Matrix

47 Result 4.9

48 Proof of Result 4.9 (a)

49 Proof of Result 4.9 (b)

50 Likelihood Function

51 Result 4.10

52 Proof of Result 4.10

53 Result 4.11 Maximum Likelihood Estimators of  and 

54 Proof of Result 4.11

55 Invariance Property

56 Sufficient Statistics

57 Distribution of Sample Mean

58 Sampling Distribution of S

59 Wishart Distribution

60 Univariate Central Limit Theorem

61 Result 4.12 Law of Large Numbers

62 Result 4.12 Multivariate Cases

63 Result 4.13 Central Limit Theorem

64 Limit Distribution of Statistical Distance

65 Evaluating Normality of Univariate Marginal Distributions

66 Evaluating Normality of Univariate Marginal Distributions

67 Q-Q Plot

68 Example 4.9

69 Example 4.9

Histogram of MidTerm Scores of Students of This Course in

Q-Q Plot of MidTerm Scores of Students of This Course in n = 33, r Q =

72 Example 4.10 Radiation Data of Closed-Door Microwave Oven

73 Measurement of Straightness

74 Table 4.2 Q-Q Plot Correlation Coefficient Test

75 Example 4.11

76 Evaluating Bivariate Normality

77 Example 4.12

78 Example 4.12

79 Chi-Square Plot

80 Example 4.13 Chi-Square Plot for Example 4.12

81 Example 4.13 Chi-Square Plot for Example 4.12

82 Chi-Square Plot for Computer Generated 4-variate Normal Data

83 Steps for Detecting Outliers Make a dot plot for each variable Make a scatter plot for each pair of variables Calculate the standardized values. Examine them for large or small values Calculated the squared statistical distance. Examine for unusually large values. In chi-square plot, these would be points farthest from the origin.

84 Helpful Transformation to Near Normality Original Scale Transformed Scale Counts, y Proportions, Correlations, r

85 Box and Cox’s Univariate Transformations

86 Example 4.16 ( ) vs. Example 4.16 ( ) vs.

87 Example 4.16 Q-Q Plot

88 Transforming Multivariate Observations

89 More Elaborate Approach

90 Example 4.17 Original Q-Q Plot for Open-Door Data

91 Example 4.17 Q-Q Plot of Transformed Open-Door Data

92 Example 4.17 Contour Plot of for Both Radiation Data

93 Transform for Data Including Large Negative Values