1. Chapter-1 Multivariate Normal Distributions
Dr. A. PHILIP AROKIADOSS
Assistant Professor
Department of Statistics
St. Joseph’s College (Autonomous)
Tiruchirappalli

2. 1.The Normal distribution – parameters m and s (or s2)
Comment: If m = 0 and s = 1 the distribution is called the standard normal distribution
Normal distribution with m = 50 and s =15
Normal distribution with m = 70 and s =20

3. The probability density of the normal distribution
If a random variable, X, has a normal distribution with mean m and variance s2 then we will write:

4. The multivariate Normal distribution

5. Let
= a random vector
Let
= a vector of constants (the mean vector)

6. Let
= a p × p positive definite matrix

7. Definition The matrix A is positive semi definite if
Further the matrix A is positive definite if

8. Suppose that the joint density of the random vector
The random vector, [x1, x2, … xp] is said to have a p-variate normal distribution with mean vector and covariance matrix S
We will write:

9. Example: the Bivariate Normal distribution
with
and

10. Now
and

12. Hence
where

13. Note:
is constant when
is constant.
This is true when x1, x2 lie on an ellipse centered at m1, m2 .

15. Surface Plots of the bivariate Normal distribution

16. Contour Plots of the bivariate Normal distribution

17. Scatter Plots of data from the bivariate Normal distribution

18. Trivariate Normal distribution - Contour mapx3
mean vector x2 x1

19. Trivariate Normal distributionx3 x2 x1

20. Trivariate Normal distributionx3 x2 x1

21. Trivariate Normal distributionx3 x2 x1

22. example
In the following study data was collected for a sample of n = 183 females on the variables
Age,
Height (Ht),
Weight (Wt),
Birth control pill use (Bpl - 1=no pill, 2=pill)
and the following Blood Chemistry measurements
Cholesterol (Chl),
Albumin (Abl),
Calcium (Ca) and
Uric Acid (UA). The data are tabulated next page:

23. The data :

25. Alb, Chl, Bp

26. Marginal and Conditional distributions

27. Theorem: (Marginal distributions for the Multivariate Normal distribution)
have p-variate Normal distribution
with mean vector
and Covariance matrix
Then the marginal distribution of is qi-variate Normal distribution (q1 = q, q2 = p - q)
with mean vector
and Covariance matrix

28. Theorem: (Conditional distributions for the Multivariate Normal distribution)
have p-variate Normal distribution
with mean vector
and Covariance matrix
Then the conditional distribution of given is qi-variate Normal distribution
with mean vector
and Covariance matrix

29. Proof: (of Previous two theorems)is
The joint density of ,
and
where

30. where ,
and

31. also
and ,

32. ,

34. The marginal distribution of is

35. The conditional distribution of given is:

36. is called the matrix of partial variances and covariances.
is called the partial covariance (variance if i = j) between xi and xj given x1, … , xq.
is called the partial correlation between xi and xj given x1, … , xq.

37. is called the matrix of regression coefficients for predicting xq+1, xq+2, … , xp from x1, … , xq.
Mean vector of xq+1, xq+2, … , xp given x1, … , xqis:

38. Example:
Suppose that
Is 4-variate normal with

39. The marginal distribution of
is bivariate normal with
The marginal distribution of
is trivariate normal with

40. Find the conditional distribution of
given
Now
and

42. The matrix of regression coefficients for predicting x3, x4 from x1, x2.

44. Thus the conditional distribution of
given
is bivariate Normal with mean vector
And partial covariance matrix

45. Using SPSS
Note: The use of another statistical package such as Minitab is similar to using SPSS

46. The first step is to input the data.
The data is usually contained in some type of file.
Text files
Excel files
Other types of files

47. After starting the SSPS program the following dialogue box appears:

48. If you select Opening an existing file and press OK the following dialogue box appears

49. Once you selected the file and its type

50. The following dialogue box appears:

51. If the variable names are in the file ask it to read the names
If the variable names are in the file ask it to read the names. If you do not specify the Range the program will identify the Range:
Once you “click OK”, two windows will appear

52. A window containing the output

53. The other containing the data:

54. To perform any statistical Analysis select the Analyze menu:

55. To compute correlations select Correlate then Bivariate To compute partial correlations select Correlate then Partial

56. for Bivariate correlation the following dialogue appears

57. the output for Bivariate correlation:

58. for partial correlation the following dialogue appears

59. the output for partial correlation:
P A R T I A L C O R R E L A T I O N C O E F F I C I E N T S
Controlling for.. AGE HT WT
CHL ALB CA UA
CHL
( 0) ( 178) ( 178) ( 178)
P= P= P= P= .002
ALB
( 178) ( 0) ( 178) ( 178)
P= P= P= P= .101 CA
( 178) ( 178) ( 0) ( 178)
P= P= P= P= .020 UA
( 178) ( 178) ( 178) ( 0)
P= P= P= P= .
(Coefficient / (D.F.) / 2-tailed Significance)
" . " is printed if a coefficient cannot be computed

60. Compare these with the bivariate correlation:

61. Bivariate Correlations
Partial Correlations
CHL ALB CA UA
CHL
ALB CA UA
Bivariate Correlations

62. In the last example the bivariate and partial correlations were roughly in agreement.
This is not necessarily the case in all stuations
An Example:
The following data was collected on the following three variables:
Age
Calcium Intake in diet (CAI)
Bone Mass density (BMI)

63. The data

64. Bivariate correlations

65. Partial correlations

66. Scatter plot CAI vs BMI (r = -0.447)

67. 25 35 45 55 65 75

68. 3D Plot
Age, CAI and BMI

72. Independence

73. Note: two vectors, , are independent if
Then the conditional distribution of given
is equal to the marginal distribution of
If is multivariate Normal with mean vector
and Covariance matrix
Then the two vectors, , are independent if

74. The components of the vector, , are independent if
s ij = 0 for all i and j (i ≠ j )
i. e. S is a diagonal matrix

75. Transformations

76. Transformations Theorem
Let x1, x2,…, xn denote random variables with joint probability density function
f(x1, x2,…, xn )
Let u1 = h1(x1, x2,…, xn).
u2 = h2(x1, x2,…, xn). ⁞
un = hn(x1, x2,…, xn).
define an invertible transformation from the x’s to the u’s

77. Then the joint probability density function of u1, u2,…, un is given by:
where
Jacobian of the transformation

78. Example
Suppose that u1, u2 are independent with a uniform distribution from 0 to 1
Find the distribution of
Solving for u1 and u2 we get the inverse transformation

79. also
and
Hence

80. The Jacobian of the transformation

82. The joint density of u1, u2 is
f(u1, u2) = f1 (u1) f2(u2)
Hence the joint density of z1 and z2 is:

83. Thus z1 and z2 are independent Standard normal.
The transformation
is useful for converting uniform RV’s into independent standard normal RV’s

84. Example
Suppose that x1, x2 are independent with density functions f1 (x1) and f2(x2)
Find the distribution of
u1 = x1+ x2
u2 = x1 - x2
Solving for x1 and x2 we get the inverse transformation

85. The Jacobian of the transformation

86. The joint density of x1, x2 is
f(x1, x2) = f1 (x1) f2(x2)
Hence the joint density of u1 and u2 is:

87. un = an1 x1+ an2 x2 +…+ annxn + cn
Theorem
Let x1, x2,…, xn denote random variables with joint probability density function
f(x1, x2,…, xn )
Let u1 = a11x1+ a12x2+…+ a1nxn + c1
u2 = a21x1 + a22x2+…+ a2nxn + c2 ⁞
un = an1 x1+ an2 x2 +…+ annxn + cn
define an invertible linear transformation from the x’s to the u’s

88. Then the joint probability density function of u1, u2,…, un is given by:
where

89. Theorem
Suppose that The random vector, [x1, x2, … xp] has a p-variate normal distribution with mean vector and covariance matrix S
then
has a p-variate normal distribution
with mean vector
and covariance matrix

90. Theorem
Suppose that The random vector, [x1, x2, … xp] has a p-variate normal distribution with mean vector and covariance matrix S
then
has a p-variate normal distribution
with mean vector
and covariance matrix

91. Proof
then

92. since
and
Also
and
hence
QED

93. Theorem
(Linear transformations of Normal RV’s)
Suppose that The random vector,
has a p-variate normal distribution with mean vector and covariance matrix S
then
has a p-variate normal distribution
Let A be a q × p matrix of rank q ≤ p
with mean vector
and covariance matrix

94. proof Let B be a (p - q) × p matrix so that is invertible. then
is p–variate normal with mean vector
and covariance matrix

95. Thus the marginal distribution of
is q–variate normal with mean vector
and covariance matrix

96. Summary – Distribution Theory for Multivariate Normal
Marginal distribution
Conditional distribution

97. (Linear transformations of Normal RV’s)
Suppose that The random vector,
has a p-variate normal distribution with mean vector and covariance matrix S
then
has a p-variate normal distribution
Let A be a q × p matrix of rank q ≤ p
with mean vector
and covariance matrix

98. Recall: Definition of eigenvector, eigenvalue
Let A be an n × n matrix
Let
then l is called an eigenvalue of A and
and is called an eigenvector of A and

99. Thereom If the matrix A is symmetric with distinct eigenvalues, l1, … , ln, with corresponding eigenvectors
Assume

100. Applications of these results to Statistics
Suppose that The random vector, [x1, x2, … xp] has a p-variate normal distribution with mean vector and covariance matrix S
Then and covariance matrix S is positive definite.
Suppose l1, … , lp are the eigenvalues of S corresponding eigenvectors of unit length
Note l1 > 0, … , lp > 0

101. Let

104. Suppose that the random vector, [x1, x2, … xp] has a p-variate normal distribution with mean vector and covariance matrix S
then
has a p-variate normal distribution
with mean vector
and covariance matrix

105. Thus the components of
are independent normal with mean 0 and variance 1.
and
Has a c2 distribution with p degrees of freedom