1. The Multivariate Normal Distribution, Part 1 BMTRY 726 1/10/2014

2. Univariate Normal Recall the density function of the univariate normal We can rewrite this as

3. Multivariate Normal Distribution We denote the MVN distribution as What is the density function of X ?

4. Multivariate Normal Distribution What is the density function of X ?

5. Multivariate Normal Note, the density does not exist if –  is not positive definite – = 0 – does not exist We will assume that is positive definite for most of the MVN methods we discuss

6. Multivariate Density Function If we assume that  is positive definite is the square of the generalized distance from x to . Also called – Squared statistical distance of x to . – Squared Mahalanobis distance of x to  – Squared standardized distance of x to 

7. Why Multivariate Normal The MVN distribution makes a good choice in statistics for several reasons – Mathematical simplicity – Multivariate central limit theorem – Many naturally occurring phenomenon approximately exhibit this distribution

8. Bivariate Normal Example Consider samples from Let’s write out the joint distribution of x 1 and x 2

9. Bivariate Normal Example Joint distribution of x 1 and x 2

10. Bivariate Normal Example Joint distribution of x 1 and x 2

11. Bivariate Normal Example This yields joint distribution of x 1 and x 2 in the form

12. Bivariate Normal Example The density if a function of  1,  2,  1,  2, and  – The density is well defined if - 1 <  < 1 – If  = 0, then …

13. Contours of constant density What if we take a slice of this bivariate distribution at a constant height? – i.e.

14. Contours of constant density The density is constant for all points for which This is an equation for an ellipse centered at

15. Bivariate Normal Example Let’s look at an example of the bivariate normal when we vary some of the parameters…

16. Examples X1X1 X1X1 X1X1 X1X1 X2X2 X2X2 X2X2 X2X2

17. Contours of constant density What happens when

18. Contours of constant density How do we find the axes of the ellipse? – Axes are in the direction of the eigenvectors of   – Axes lengths are proportional to the reciprocals of the square root of the eigenvalues of   – We can get these from  (avoid calculating   ) Let’s look at this for the bivariate case... We must find the eigenvalues and eigenvectors for  – Eigenvalues: – Eigenvectors:

19. Eigenvalues of  :

21. The corresponding eigenvector, e 1, of  :

23. Similarly we can find e 2, which corresponds to 2 : The axes of the contours of constant density will have length

24. If we let then are the eigenvalues of  and e 1 and e 2 are the corresponding eigenvectors

25. The ratio of the lengths of the axes The actual lengths depend on the contour being considered. For the (1-a)x100% contour, the ½ lengths are given by Thus the solid ellipsoid of x values satisfying has probability 1- .

26. Univariate case: length of the interval containing the central 95% of the population is proportional to  Bivariate case: the area of the region containing 95% of the population is proportional to.

27. We can call this “smallest” region the central (1-  )x100% of the multivariate normal population. The “area” of this smallest ellipse in the 2-D case is: This extends to higher dimensions (think volume) – Consider – The smallest region for which there is 1-  that a randomly selected observation falls in the region is a p-dimensional ellipsoid centered at  with volume

28. Visual of the 3-dimensional case

29. Next Time Properties of the MVN…