1. Jointly distributed Random variables
Multivariate distributions

2. Quite often there will be 2 or more random variables (X, Y, etc) defined for the same random experiment.
Example:
A bridge hand (13 cards) is selected from a deck of 52 cards.
X = the number of spades in the hand.
Y = the number of hearts in the hand.
In this example we will define:
p(x,y) = P[X = x, Y = y]

3. The function
p(x,y) = P[X = x, Y = y]
is called the joint probability function of X and Y.

4. The possible values of X are 0, 1, 2, …, 13
Note:
The possible values of X are 0, 1, 2, …, 13
The possible values of Y are also 0, 1, 2, …, 13 and X + Y ≤ 13.
The number of ways of choosing the y hearts for the hand
The number of ways of choosing the x spades for the hand
The number of ways of completing the hand with diamonds and clubs.
The total number of ways of choosing the 13 cards for the hand

5. Table: p(x,y)

6. Bar graph: p(x,y)
p(x,y) y x

7. Example:
A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears.
Now
p(x,y) = P[X = x, Y = y]
The possible values of X are 0, 1, 2, 3, 4, 5.
The possible values of Y are 0, 1, 2, 3, 4, 5.
and X + Y ≤ 5

8. A typical outcome of rolling a die n = 5 times will be a sequence F5FF6 where F denotes the outcome {1,2,3,4}. The probability of any such sequence will be:
where
x = the number of sixes in the sequence and
y = the number of fives in the sequence

9. Now
p(x,y) = P[X = x, Y = y]
Where K = the number of sequences of length 5 containing x sixes and y fives.

10. Thus
p(x,y) = P[X = x, Y = y]
if x + y ≤ 5 .

11. Table: p(x,y)

12. Bar graph: p(x,y)
p(x,y) y x

13. General properties of the joint probability function;
p(x,y) = P[X = x, Y = y]

14. Example:
A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears.
What is the probability that we roll more sixes than fives
i.e. what is P[X > Y]?

15. Table: p(x,y)

16. Marginal and conditional distributions

17. Definition:
Let X and Y denote two discrete random variables with joint probability function
p(x,y) = P[X = x, Y = y]
Then
pX(x) = P[X = x] is called the marginal probability function of X.
and
pY(y) = P[Y = y] is called the marginal probability function of Y.

18. Note: Let y1, y2, y3, … denote the possible values of Y.
Thus the marginal probability function of X, pX(x) is obtained from the joint probability function of X and Y by summing p(x,y) over the possible values of Y.

19. Also

20. Example:
A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears.

21. Conditional Distributions

22. Definition:
Let X and Y denote two discrete random variables with joint probability function
p(x,y) = P[X = x, Y = y]
Then
pX |Y(x|y) = P[X = x|Y = y] is called the conditional probability function of X given Y = y
and
pY |X(y|x) = P[Y = y|X = x] is called the conditional probability function of Y given X = x

23. Note
and

24. Marginal distributions describe how one variable behaves ignoring the other variable.
Conditional distributions describe how one variable behaves when the other variable is held fixed

25. Example:
A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears. y x

26. The conditional distribution of X given Y = y.
pX |Y(x|y) = P[X = x|Y = y] y x

27. The conditional distribution of Y given X = x.
pY |X(y|x) = P[Y = y|X = x] y x

28. Example
A Bernoulli trial (S - p, F – q = 1 – p) is repeated until two successes have occurred.
X = trial on which the first success occurs
and
Y = trial on which the 2nd success occurs.
Find the joint probability function of X, Y.
Find the marginal probability function of X and Y.
Find the conditional probability functions of Y given X = x and X given Y = y,

29. Solution
A typical outcome would be:
FFF…FSFFF…FS x y
x - 1
y – x - 1

30. p(x,y) - Table y 1 2 3 4 5 6 7 8 x p2
p2q
p2q2
p2q3
p2q4
p2q5
p2q6

31. The marginal distribution of X
This is the geometric distribution

32. The marginal distribution of Y
This is the negative binomial distribution with k = 2.

33. The conditional distribution of X given Y = y
This is the geometric distribution with time starting at x.

34. The conditional distribution of Y given X = x
This is the uniform distribution on the values 1, 2, …(y – 1)

35. Summary Discrete Random Variables

36. The joint probability function;
p(x,y) = P[X = x, Y = y]

37. Continuous Random Variables

38. Definition: Two random variable are said to have joint probability density function f(x,y) if

39. If
then
defines a surface over the x – y plane

40. Multiple Integration

41. A
f(x,y) A

42. If the region A = {(x,y)| a ≤ x ≤ b, c ≤ y ≤ d} is a rectangular region with sides parallel to the coordinate axes: x y d c a b
Then A
f(x,y)

43. First evaluate the inner integral
To evaluate A
First evaluate the inner integral
Then evaluate the outer integral
f(x,y)

44. = area under surface above the line where y is constantx y d c a b dy y
= area under surface above the line where y is constant
Infinitesimal volume under surface above the line where y is constant
f(x,y)

45. First evaluate the inner integral
The same quantity can be calculated by integrating first with respect to y, than x. A
First evaluate the inner integral
Then evaluate the outer integral
f(x,y)

46. = area under surface above the line where x is constanty d c a b dx x
= area under surface above the line where x is constant
Infinitesimal volume under surface above the line where x is constant
f(x,y)

47. Example: Compute
Now
f(x,y)

48. The same quantity can be computed by reversing the order of integration
f(x,y)

49. Integration over non rectangular regions

50. Suppose the region A is defined as follows
A = {(x,y)| a(y) ≤ x ≤ b(y), c ≤ y ≤ d} y d c x
Then
b(y)
a(y) A

51. If the region A is defined as follows
A = {(x,y)| a ≤ x ≤ b, c(x) ≤ y ≤ d(x) } y
d(x)
c(x) x
Then a b A

52. In general the region A can be partitioned into regions of either typex

53. Example:
Compute the volume under f(x,y) = x2y + xy3 over the region A = {(x,y)| x + y ≤ 1, 0 ≤ x, 0 ≤ y} y
(0, 1)
x + y = 1
(1, 0) x
f(x,y)

54. Integrating first with respect to x than y
x + y = 1
(1, 0)
(0, 1)
(0, y)
(1 - y, y) A
f(x,y)

55. and

56. Now integrating first with respect to y than x
(0, 1)
x + y = 1
(x, 1 – x )
(1, 0)
(x, 0) x A

57. Hence

58. Continuous Random Variables

59. Definition: Two random variable are said to have joint probability density function f(x,y) if

60. Definition: Let X and Y denote two random variables with joint probability density function f(x,y) then
the marginal density of X is
the marginal density of Y is

61. Definition: Let X and Y denote two random variables with joint probability density function f(x,y) and marginal densities fX(x), fY(y) then
the conditional density of Y given X = x
conditional density of X given Y = y

62. The bivariate Normal distribution

63. Let
where
This distribution is called the bivariate Normal distribution.
The parameters are m1, m2 , s1, s2 and r.

64. Surface Plots of the bivariate Normal distribution

65. Note:
is constant when
is constant.
This is true when x1, x2 lie on an ellipse centered at m1, m2 .

67. Marginal and Conditional distributions

68. Marginal distributions for the Bivariate Normal distribution
Recall the definition of marginal distributions for continuous random variables:
and
It can be shown that in the case of the bivariate normal distribution the marginal distribution of xi is Normal with mean mi and standard deviation si.

69. Proof:
The marginal distributions of x2 is
where

70. Now:

71. Hence
Also
and

72. Finally

73. and

74. Summarizing
where
and

75. Thus

76. Thus the marginal distribution of x2 is Normal with mean m2 and standard deviation s2.
Similarly the marginal distribution of x1 is Normal with mean m1 and standard deviation s1.

77. Conditional distributions for the Bivariate Normal distribution
Recall the definition of conditional distributions for continuous random variables:
and
It can be shown that in the case of the bivariate normal distribution the conditional distribution of xi given xj is Normal with:
and
mean
standard deviation

78. Proof

79. where
and
Hence
Thus the conditional distribution of x2 given x1 is Normal with:
and
mean
standard deviation

80. Bivariate Normal Distribution with marginal distributions

81. Bivariate Normal Distribution with conditional distribution

82. x2 ( m1, m2) x1 Major axis of ellipses Regression
Regression to the mean x1

83. Example:
Suppose that a rectangle is constructed by first choosing its length, X and then choosing its width Y.
Its length X is selected form an exponential distribution with mean m = 1/l = 5. Once the length has been chosen its width, Y, is selected from a uniform distribution form 0 to half its length.
Find the probability that its area A = XY is less than 4.

84. Solution:

85. xy = 4
y = x/2

87. This part can be
evaluated
This part may require Numerical evaluation

89. multivariate distributions
k ≥ 2

90. Definition
Let X1, X2, …, Xk denote k discrete random variables, then
p(x1, x2, …, xk )
is joint probability function of X1, X2, …, Xk if

91. Definition
Let X1, X2, …, Xk denote k continuous random variables, then
f(x1, x2, …, xk )
is joint density function of X1, X2, …, Xk if

92. Example: The Multinomial distribution
Suppose that we observe an experiment that has k possible outcomes {O1, O2, …, Ok } independently n times.
Let p1, p2, …, pk denote probabilities of O1, O2, …, Ok respectively.
Let Xi denote the number of times that outcome Oi occurs in the n repetitions of the experiment.
Then the joint probability function of the random variables X1, X2, …, Xk is

93. Note: is the probability of a sequence of length n containing
x1 outcomes O1
x2 outcomes O2 …
xk outcomes Ok

94. is the number of ways of choosing the positions for the x1 outcomes O1, x2 outcomes O2, …, xk outcomes Ok

95. is called the Multinomial distribution

96. Example:
Suppose that a treatment for back pain has three possible outcomes:
O1 - Complete cure (no pain) – (30% chance)
O2 - Reduced pain – (50% chance)
O3 - No change – (20% chance)
Hence p1 = 0.30, p2 = 0.50, p3 = 0.20.
Suppose the treatment is applied to n = 4 patients suffering back pain and let X = the number that result in a complete cure, Y = the number that result in just reduced pain, and Z = the number that result in no change.
Find the distribution of X, Y and Z. Compute P[X + Y ≥ Z]

97. Table: p(x,y,z)

98. P [X + Y ≥ Z] =

99. Example: The Multivariate Normal distribution
Recall the univariate normal distribution
the bivariate normal distribution

100. The k-variate Normal distribution
where

101. Marginal distributions

102. Definition
Let X1, X2, …, Xq, Xq+1 …, Xk denote k discrete random variables with joint probability function
p(x1, x2, …, xq, xq+1 …, xk )
then the marginal joint probability function
of X1, X2, …, Xq is

103. Definition
Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xq, xq+1 …, xk )
then the marginal joint probability function
of X1, X2, …, Xq is

104. Conditional distributions

105. Definition
Let X1, X2, …, Xq, Xq+1 …, Xk denote k discrete random variables with joint probability function
p(x1, x2, …, xq, xq+1 …, xk )
then the conditional joint probability function
of X1, X2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is

106. Definition
Definition
Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xq, xq+1 …, xk )
then the conditional joint probability function
of X1, X2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is

107. Definition – Independence of sets of vectors
Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xq, xq+1 …, xk )
then the variables X1, X2, …, Xq are independent of Xq+1, …, Xk if
A similar definition for discrete random variables.

108. Definition – Mutual Independence
Let X1, X2, …, Xk denote k continuous random variables with joint probability density function
f(x1, x2, …, xk )
then the variables X1, X2, …, Xk are called mutually independent if
A similar definition for discrete random variables.

109. Example
Let X, Y, Z denote 3 jointly distributed random variable with joint density function then
Find the value of K.
Determine the marginal distributions of X, Y and Z.
Determine the joint marginal distributions of
X, Y
X, Z
Y, Z

110. Solution
Determining the value of K.

111. The marginal distribution of X.

112. The marginal distribution of X,Y.

113. Find the conditional distribution of:
Z given X = x, Y = y,
Y given X = x, Z = z,
X given Y = y, Z = z,
Y , Z given X = x,
X , Z given Y = y
X , Y given Z = z
Y given X = x,
X given Y = y
X given Z = z
Z given X = x,
Z given Y = y
Y given Z = z

114. The marginal distribution of X,Y.
Thus the conditional distribution of Z given X = x,Y = y is

115. The marginal distribution of X.
Thus the conditional distribution of Y , Z given X = x is