1. ASV Chapters 1 - Sample Spaces and Probabilities
2 - Conditional Probability and Independence
3 - Random Variables
4 - Approximations of the Binomial Distribution
5 - Transforms and Transformations
6 - Joint Distribution of Random Variables
7 - Sums and Symmetry
8 - Expectation and Variance in the Multivariate Setting
9 - Tail Bounds and Limit Theorems
10 - Conditional Distribution
11 - Appendix A, B, C, D, E, F

2. Suppose X and Y are two discrete random variables with pmfs pX(x) and pY(y) respectively:
Suppose Z is a third discrete random variable that depends on X and Y, i.e., there exists a joint pmf for every ordered pair (x, y)… x
pX(x) x1
pX (x1) x2
pX (x2)  xr
pX (xr) 1 y
pY(y) y1
pY(y1) y2
pY(y2)  yc
pY(yc) 1 Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc) x2
p(x2, y1)
p(x2, y2)
p(x2, yc) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (x1)
pX (x2)
pX (xr)
pY (y1)
pY (y2)
pY (yc) 1

3. Suppose X and Y are two discrete random variables with pmfs pX(x) and pY(y) respectively:
Suppose Z is a third discrete random variable that depends on X and Y, i.e., there exists a joint pmf for every ordered pair (x, y)… x
pX(x) x1
pX (x1) x2
pX (x2)  xr
pX (xr) 1 y
pY(y) y1
pY(y1) y2
pY(y2)  yc
pY(yc) 1 Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (yc) 1

4. Suppose X and Y are two discrete random variables with pmfs pX(x) and pY(y) respectively:
Suppose Z is a third discrete random variable that depends on X and Y, i.e., there exists a joint pmf for every ordered pair (x, y)… x
pX(x) x1
pX (x1) x2
pX (x2)  xr
pX (xr) 1 y
pY(y) y1
pY(y1) y2
pY(y2)  yc
pY(yc) 1 Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (yc) 1

5. Suppose X and Y are two discrete random variables with pmfs pX(x) and pY(y) respectively:
Joint pmf x
pX(x) x1
pX (x1) x2
pX (x2)  xr
pX (xr) 1 y
pY(y) y1
pY(y1) y2
pY(y2)  yc
pY(yc) 1 Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (yc) 1

6. p

7. p

8. p

9. p

10. p

11. Joint Probability Mass Function
DISCRETE
Joint Probability Mass Function Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (y2)
pY (yc) 1

12. Joint Probability Mass Function
DISCRETE
Joint Probability Mass Function Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (y2)
pY (yc) 1

13. Joint Probability Mass Function
DISCRETE
Joint Probability Mass Function Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (y2)
pY (yc) 1
Def: X and Y are statistically independent if
i.e., each cell probability is equal to the product of its marginal probabilities.

14. Joint Probability Mass Function
DISCRETE
Joint Probability Mass Function Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc) x2
p(x2, y1)
p(x2, y2)
p(x2, yc) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)

15. Joint Probability Mass Function
DISCRETE
In principle, one can construct a probability histogram much as before, with the height of each rectangle centered at the point (x, y) equal to the pmf z = p(x, y).
Joint Probability Mass Function
Extend this to the continuous scenario….
What happens as the partition of the X and Y axes becomes arbitrarily small (i.e., the number of rows and columns  ∞)?
Recall…

16. Time intervals = 0.5 secs
Time intervals = 5.0 secs
Time intervals = 1.0 secs
Time intervals = 2.0 secs
DISCRETE
CONTINUOUS
“Density”
Interval widths can be made arbitrarily small, i.e, the scale at which X is measured can be made arbitrarily fine, since it is continuous.
As x  0 and # rectangles  ∞, this “Riemann sum” approaches the area under the density curve f(x), expressed as a definite integral.
Similarly….

17. Joint Probability Mass Function
DISCRETE
Joint Probability Mass Function Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (y2)
pY (yc) 1

18. ... CONTINUOUS DISCRETE DISCRETE Joint Probability Density Function
Joint Probability Mass Function Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (y2)
pY (yc) 1

19. ... CONTINUOUS DISCRETE DISCRETE Joint Probability Density Function
Joint Probability Mass Function Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (y2)
pY (yc) 1

20. ... CONTINUOUS DISCRETE DISCRETE Joint Probability Density Function
Joint Probability Mass Function Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (y2)
pY (yc) 1

22. Joint Probability Density Function
CONTINUOUS
Joint Probability Density Function
Volume under density f(x, y) over A.
“area element”
Area A

23. Joint Probability Density Function
CONTINUOUS
Example:
Uniform Distribution
Joint Probability Density Function
Recall for one r.v. X…

24. Joint Probability Density Function
CONTINUOUS
Example:
Uniform Distribution
Joint Probability Density Function

25. Joint Probability Density Function
CONTINUOUS
Example:
Uniform Distribution 
Joint Probability Density Function 

26. Joint Probability Density Function
CONTINUOUS
Example: 
Joint Probability Density Function

27. Joint Probability Density Function
CONTINUOUS
Example: 
Joint Probability Density Function 

28. Joint Probability Density Function
CONTINUOUS
Example:
Joint Probability Density Function A

29. Joint Probability Density Function
CONTINUOUS
Example:
Joint Probability Density Function

30. Example:

31. ... CONTINUOUS DISCRETE DISCRETE Joint Probability Density Function
Joint Probability Mass Function Y y1 y2 … yc X x1
p(x1, y1)
p(x1, y2)
...
p(x1, yc)
pX (x1) x2
p(x2, y1)
p(x2, y2)
p(x2, yc)
pX (x2) xr
p(xr, y1)
p(xr, y2)
p(xr, yc)
pX (xr)
pY (y1)
pY (y2)
pY (yc) 1
What about these marginal pdfs?

32. Joint Probability Density Function
CONTINUOUS
Joint Probability Density Function
This is the density curve that corresponds to our fixed value of y*.
This is a plane parallel to the XZ-plane. Every point in it has the form (x, y*, z).

33. Joint Probability Density Function
CONTINUOUS
Joint Probability Density Function
This is the density curve that corresponds to our fixed value of y*.
Integrate w.r.t. x from - to  to obtain…
This is a plane parallel to the XZ-plane. Every point in it has the form (x, y*, z).
(vis-à-vis row marginals)
(vis-à-vis column marginals)

34. Joint Probability Density Function
CONTINUOUS
Example (revisted):
Joint Probability Density Function

35. Joint Probability Density Function
CONTINUOUS
Example (revisted):
Joint Probability Density Function 
Check? 

36. Joint Probability Density Function
CONTINUOUS
Example (revisted):
Joint Probability Density Function

37. Joint Probability Density Function
CONTINUOUS
Example (revisted):
Joint Probability Density Function
F increases continuously and monotonically from 0 to 1.

38. Joint Probability Density Function
CONTINUOUS
Example (revisted):
Joint Probability Density Function

39. Joint Probability Density Function
CONTINUOUS
Example (revisted):
Joint Probability Density Function

40. Joint Probability Density Function
CONTINUOUS
Example (revisted):
Joint Probability Density Function A

41. Joint Probability Density Function
CONTINUOUS
To summarize…
Joint Probability Density Function X
f(x1, y1)
f(x1, y2)
...
f(x1, yc)
marginal pdf of Y Y
f(x2, y1)
f(x2, y2)
f(x2, yc)
f(xr, y1)
f(xr, y2)
f(xr, yc) 1

42. Joint Probability Density Function
CONTINUOUS
Example
Joint Probability Density Function
Exercise

43. Joint Probability Density Function
CONTINUOUS
Example
Joint Probability Density Function
Exercise
Exercise

44. Joint Probability Density Function
CONTINUOUS
Example
Joint Probability Density Function A
Exercise

45. Joint Probability Density Function
CONTINUOUS
Example
Joint Probability Density Function A
Exercise

46. Joint Probability Density Function
CONTINUOUS
Example
Joint Probability Density Function A
Exercise

47. More Than Two Random Variables…?
CONTINUOUS
Joint Probability Density Function
“Hypervolume” under density f over A.
Volume under density f(x, y) over A.
More Than Two Random Variables…?
Definition of statistical independence of X and Y
can be extended to any number of variables.
Area A