The Multivariate Normal Distribution, Part 2 BMTRY 726 5/22/2018

More Properties of MVN Last lecture we discussed: The form of the MVN distribution Contours of constant density obtained by taking a slice of the MVN distribution as some set height Some of the properties of the MVN distribution Impact of linear combinations of X Partitions of X Conditions for Independence of vectors in X We will continue this discussion with some additional useful properties

Conditional Distributions Result 4.6: Suppose Then the conditional distribution of X1 given that X2 = x2 is a normal distribution Note the covariance matrix does not depend on the value of x2

Proof of Result 4.6

Proof of Result 4.6

Proof of Result 4.6

Example Consider Find the conditional distribution of the 1st and 3rd components

Example

Example

Results 4.6 & Multiple Regression Consider The conditional distribution of Y|X=x is univariate normal with

Result 4.7: If and S is positive definite, then Proof:

Result 4.7: If and S is positive definite, then Proof cont’d:

Result 4.8: If are mutually independent with And c1, c2, …,cn are n constants. Then Additionally if we have and which are r x p matrices of constants we can also say

Sample Data Let’s say that X1, X2, …, Xn are i.i.d. random vectors If the data vectors are sampled from a MVN distribution then

Multivariate Normal Likelihood We can also look at the joint likelihood of our random sample

Some needed Results (1) Given A > 0 and are eigenvalues of A (a) (b) (c) (2) From (c) we can show that:

Some needed Results (2) Proof that:

Some needed Results (2) Proof that:

Some needed Results (2) Proof that:

Some needed Results (1) Given A > 0 and are eigenvalues of A (a) (b) (c) (2) From (c) we can show that: (3) Given Spxp > 0, Bpxp > 0 and scalar b > 0

MLE’s for .

MLE’s for .

MLE’s for .

MLE’s for .

A Few Notes About The MLE’s for Variance As in the univariate setting, the MLE for the variance matrix is biased Thus we generally use an alternative to the MLE…

Sampling Distributions So we’ve discussed that we can estimate the mean vector, m, and the covariance matrix, S, using and S But we need to understand how these are distributed..

Sample Mean Vector We can estimate a sample mean for X1, X2, …, Xn

Sample Mean Vector Now we can estimate the mean of our sample But what about the properties of ? It is an unbiased estimate of the mean It is a sufficient statistic Also, the sampling distribution is:

.

.

Sample Covariance And the sample covariance for X1, X2, …, Xn Sample variance Sample Covariance

Sample Mean Vector So we can also estimate the variance of our sample And like , S also has some nice properties It is an unbiased estimate of the variance It is also a sufficient statistic It is also independent of But what about the sampling distribution of S?

Wishart Distribution Given , the distribution of is called a Wishart distribution with n degrees of freedom. has a Wishart distribution with n -1 degrees of freedom The density function is where A and S are positive definite

Wishart cont’d The Wishart distribution is the multivariate analog of the central chi-squared distribution. If are independent then If then CAC’ is distributed The distribution of the (i, i) element of A is

Large Sample Behavior Let X1, X2, …, Xn be a random sample from a population with mean and variance (not necessarily normally distributed) Then and S are consistent estimators for m and S. This means

Large Sample Behavior If we have a random sample X1, X2, …, Xn a population with mean and variance, we can apply the multivariate central limit theorem as well The multivariate CLT says

Next Time Checking Normality How can we check MVN and what do we do if our data don’t appear MVN? SAS and R Begin our discussion of statistical inference for MV vectors