1. 1 Probability and Statistical Inference (9th Edition) Chapter 4 Bivariate Distributions November 4, 2015

2. 2 Joint Probability Mass Function Let X and Y be two discrete random variables defined on the same outcome set. The probability that X=x and Y=y is denoted by P X,Y (x,y)= P(X=x,Y=y) and is called the joint probability mass function (joint pmf) of X and Y P X,Y (x,y) satisfies the the following 3 properties:

3. 3 Example: Roll a pair of unbiased dice. For each of the 36 possible outcomes, let X denote the smaller number and Y denote the larger number The joint pmf of X and Y is: Joint Probability Mass Function

4. 4 Note that we can always create a common outcome set for any two or more random variables. For example, let X and Y correspond to the outcomes of the first and second tosses of a coin, respectively. Then, the outcome set of X is {head up, tail up} and the outcome set of Y is also {head up, tail up}. The common outcome set of X and Y is {(head up, head up),(head up, tail up),(tail up, head up),(tail up, tail up)} Joint Probability Mass Function

5. 5 Another Example: Assume that we toss a dice once. Let random variable X correspond to whether the outcome is less than or equal to 2, and random variable Y correspond to whether the outcome is an even number. Then, the joint pmf of X and Y is shown on the next page Joint Probability Mass Function

6. 6 1 01 P XY (0,0)=1/3 P XY (0,1)=1/3 P XY (1,1)=1/6 P XY (1,0)=1/6 Outcome123456 X110000 Y010101 X Y Joint Probability Mass Function

7. 7 Marginal Probability Mass Function Let P XY (x,y) be the joint pmf of discrete random variables X and Y. Then is called the marginal pmf of X Similarly, is called the marginal pmf of Y

8. 8 Independent Random Variables Two discrete random variables X and Y are said to be independent if and only if Otherwise, X and Y are said to be dependent

9. 9 Uncorrelated Random Variables Let X and Y be two random variables. Then, E[(X- µ X )(Y-µ Y )] is called the covariance of X and Y (denoted by Cov(X,Y)) Covariance is a measure of how much two random variables change together A positive value of Cov(X,Y) indicates that Y tends to increase as X increases Two discrete random variables X and Y are said to be uncorrelated if Cov(X,Y)=0 Otherwise, X and Y are said to be correlated

10. 10 Independent Implies Uncorrelated Cov(X,Y) = E[(X-µ X )(Y-µ Y )] = E[XY- µ Y X- µ X Y+ µ X µ Y ] = E[XY]- µ Y E[X]- µ X E[Y]+E[ µ X µ Y ] = E[XY]- µ X µ Y If X and Y are independent, then Therefore, if X and Y are independent, then Cov(X,Y)=0 The converse statement is not true (example later)

11. 11 Correlation Coefficient Correlation coefficient of X and Y: Insights: If X and Y are above or below their respective means simultaneously, then ρ XY > 0. If X is above µ X whenever Y is below µ Y, and X is below µ X whenever Y is above µ Y, then ρ XY < 0

12. 12 Addition of Two Random Variables Let X and Y be two random variables. Then, E[X+Y]=E[X]+E[Y] Note that the above equation holds even if X and Y are dependent Proof of the discrete case:

13. 13 On the other hand, Addition of Two Random Variables

14. 14 Note that if X and Y are independent, then E[XY]=E[X]E[Y] Therefore, if X and Y are independent, then Var[X+Y]=Var[X]+Var[Y] Addition of Two Random Variables

15. 15 Examples of Correlated Random Variables Assume that a supermarket collected the following statistics of customers’ purchasing behavior: Purchasing Wine Not Purchasing Wine Male45255 Female70630 Purchasing Juice Not Purchasing Juice Male60240 Female210490

16. 16 Examples of Correlated Random Variables Let random variable M correspond to whether a customer is male, random variable W correspond to whether a customer purchases wine, random variable J correspond to whether a customer purchases juice

17. 17 The joint pmf of M and W is Cov(M,W)= E[MW] - E[M]E[W] = 0.045 – 0.3*0.115 = 0.0105 > 0 M and W are positively correlated (outcome M=1 makes it more likely that W=1) W M P MW (1,1) = 0.045 P MW (1,0) = 0.255 P MW (0,1) = 0.07 P MW (0,0) = 0.63 Examples of Correlated Random Variables

18. 18 The joint pmf of M and J is Cov(M,J)= E[MJ] - E[M]E[J] = 0.06 – 0.3*0.27 = -0.021 < 0 M and J are negatively correlated W M P MJ (1,1) = 0.06 P MJ (1,0) = 0.24 P MJ (0,1) = 0.21 P MJ (0,0) = 0.49 Examples of Correlated Random Variables

19. 19 Example of Uncorrelated Random Variables Assume X and Y have the following joint pmf: P XY (0,1)= P XY (1,0)= P XY (2,1)= 1/3 We can derive the following marginal pmfs:

20. 20 Example of Uncorrelated Random Variables Since P XY (0,1) = 1/3, and P X (0) x P Y (1) = 1/3 x 2/3 = 2/9, X and Y are not independent However, Cov(X,Y) = E[XY] – E[X]E[Y] = [2/9 x 1 + 2/9 x 2] – [1 x 2/3] = 0. Thus, X and Y are uncorrelated Thus, uncorrelated does not imply independence

21. 21 Conditional Distributions Let X and Y be two discrete random variables. The conditional probability mass function (pmf) of X, given that Y=y, is defined by Similarly, if X and Y are continuous random variables, then the conditional probability density function (pdf) of X, given that Y=y, is defined by

22. 22 Conditional Distributions Assume that X and Y are two discrete random variables. Then, Similarly, for two continuous random variables X and Y, we have

23. 23 Conditional Distributions The conditional mean of X, given that Y=y, is defined by The conditional variance of X, given that Y=y, is defined by

24. 24 Example 1 Let X and Y have the joint pmf It can be easily shown that Then, the conditional pmf of X, given that Y=y, is

25. 25 Example 1 (Cont.) Similarly, the conditional pmf of Y, given that X=x, is

26. 26 Example 2 3 blue balls (labeled A, B, C) and 2 red balls (labeled D, E) are in a bag Randomly taking a ball out of the bag, what is the probability of getting a blue ball? (Ans: 3/5) What is the probability of getting A? (Ans: 1/5) What is the probability of getting A, given that the ball we get is a blue ball? (Ans: 1/3) X = label of the ball we get Y = color of the ball we get P(X=A | Y=blue) = P(X=A, Y=blue) / P(Y=blue) = (1/5) / (3/5) = 1/3

27. 27 Bivariate Normal Distribution The joint pdf of bivariate normal The joint pdf of multivariate normal where in the case of bivariate and | | denotes the determinant of a matrix

28. 28 Bivariate Normal Distribution Graphic representations of bivariate (2D) normal

29. 29 Bivariate Normal Distribution

30. 30 Bivariate Normal Distribution

31. 31 Bivariate Normal Distribution

32. 32 Bivariate Normal Distribution

33. 33 Example Let us assume that in a certain population of college students, the respective grade point average (GPA)—say X and Y—in high school and the first year in college have an approximate bivariate normal distribution with parameters Then, for example, where

34. 34 Example (Cont.) The conditional pdf of Y, given that X=x, is normal, with mean and variance

35. 35 Example (Cont.) Since the conditional pdf of Y, given that X=3.2, is normal with mean and standard deviation we have

36. 36 Correlations and Independence for Normal Random Variables In general, random variables may be uncorrelated but statistically dependent (i.e., uncorrelated does not imply independence) But if a random vector has a multivariate normal distribution, then any two or more of its components that are uncorrelated are independent (i.e., uncorrelated does imply independence in this case)

37. 37 The fact that two random variables X and Y both have a normal distribution does not imply that the pair (X, Y) has a joint normal distribution. Example: Suppose X has a normal distribution with expected value 0 and variance 1. Let where c is a positive number X and Y are not jointly normally distributed, even though they are separately normally distributed Correlations and Independence for Normal Random Variables

38. 38 If X and Y are normally distributed and independent, this implies they are "jointly normally distributed", i.e., the pair (X, Y) must have multivariate normal distribution However, a pair of jointly normally distributed variables need not be independent (would only be so if uncorrelated) Correlations and Independence for Normal Random Variables