1. Maximum Likelihood Estimation
Multivariate Normal distribution

2. The Method of Maximum Likelihood
Suppose that the data x1, … , xn has joint density function
f(x1, … , xn ; q1, … , qp)
where q = (q1, … , qp) are unknown parameters assumed to lie in W (a subset of p-dimensional space).
We want to estimate the parametersq1, … , qp

3. Definition: The Likelihood function
Suppose that the data x1, … , xn has joint density function
f(x1, … , xn ; q1, … , qp)
Then given the data the Likelihood function is defined to be
= L(q1, … , qp)
= f(x1, … , xn ; q1, … , qp)
Note: the domain of L(q1, … , qp) is the set W.

4. Definition: Maximum Likelihood Estimators
Suppose that the data x1, … , xn has joint density function
f(x1, … , xn ; q1, … , qp)
Then the Likelihood function is defined to be
= L(q1, … , qp)
= f(x1, … , xn ; q1, … , qp)
and the Maximum Likelihood estimators of the parameters q1, … , qp are the values that maximize

5. i.e. the Maximum Likelihood estimators of the parameters q1, … , qp are the values
Such that
Note:
is equivalent to maximizing
the log-likelihood function

6. The Multivariate Normal Distribution
Maximum Likelihood Estiamtion

7. Let
denote a sample (independent)
from the p-variate normal distribution
with mean vector
and covariance matrix
Note:

8. The matrix
is called the data matrix.

9. The vector
is called the data vector.

10. The mean vector

11. The vector
is called the sample mean vector
note

12. also

13. In terms of the data vector
where

14. Graphical representation of sample mean vector
The sample mean vector is the centroid of the data vectors.

15. The Sample Covariance matrix

16. The sample covariance matrix:
where

17. There are different ways of representing sample covariance matrix:

21. Maximum Likelihood Estimation
Multivariate Normal distribution

22. Let
denote a sample (independent)
from the p-variate normal distribution
with mean vector
and covariance matrix
Then the joint density function of
is:

23. The Likelihood function is:
and the Log-likelihood function is:

24. To find the Maximum Likelihood estimators of
we need to find
to maximize
or equivalently maximize

25. Note:
thus
hence

26. Now

27. Now

28. Summary:
the Maximum Likelihood estimators of
are
and

29. Sampling distribution of the MLE’s

30. Note
is:
The joint density function of

31. This distribution is np-variate normal with mean vector

32. Thus the distribution of
is p-variate normal with mean vector

34. Summary
The sampling distribution of is p-variate normal with

35. The sampling distribution of the sample covariance matrix S and

36. The Wishart distribution
A multivariate generalization of the c2 distribution

37. Definition: the p-variate Wishart distribution
Let
be k independent random p-vectors
Each having a p-variate normal distribution with
Then U is said to have the p-variate Wishart distribution with k degrees of freedom

38. The density ot the p-variate Wishart distribution
Suppose
Then the joint density of U is:
where Gp(·) is the multivariate gamma function.
It can be easily checked that when p = 1 and S = 1 then the Wishart distribution becomes the c2 distribution with k degrees of freedom.

39. Theorem
Suppose
then
Corollary 1:
Corollary 2:
Proof

40. Theorem
Suppose
are independent, then
Theorem
are independent and
Suppose
then

41. Theorem
Let
be a sample from
then
Theorem
Let
be a sample from
then

42. Theorem
Proof
etc

43. Theorem
Let
be a sample from
then
is independent of
Proof
be orthogonal
Then

44. Note H* is also orthogonal

45. Properties of Kronecker-product

48. This the distribution of
is np-variate normal with mean vector

49. Thus the joint distribution of
is np-variate normal with mean vector

50. Thus the joint distribution of
is np-variate normal with mean vector

51. Summary: Sampling distribution of MLE’s for multivatiate Normal distribution
Let
be a sample from
then
and