ASV Chapters 1 - Sample Spaces and Probabilities 2 - Conditional Probability and Independence 3 - Random Variables 4 - Approximations of the Binomial Distribution 5 - Transforms and Transformations 6 - Joint Distribution of Random Variables 7 - Sums and Symmetry 8 - Expectation and Variance in the Multivariate Setting 9 - Tail Bounds and Limit Theorems 10 - Conditional Distribution 11 - Appendix A, B, C, D, E, F

Suppose X and Y are two discrete random variables with pmfs pX(x) and pY(y) respectively: Suppose Z is a third discrete random variable that depends on X and Y, i.e., there exists a joint pmf for every ordered pair (x, y)… x pX(x) x1 pX (x1) x2 pX (x2)  xr pX (xr) 1 y pY(y) y1 pY(y1) y2 pY(y2)  yc pY(yc) 1 Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) x2 p(x2, y1) p(x2, y2) p(x2, yc) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (x1) pX (x2) pX (xr) pY (y1) pY (y2) pY (yc) 1

Suppose X and Y are two discrete random variables with pmfs pX(x) and pY(y) respectively: Suppose Z is a third discrete random variable that depends on X and Y, i.e., there exists a joint pmf for every ordered pair (x, y)… x pX(x) x1 pX (x1) x2 pX (x2)  xr pX (xr) 1 y pY(y) y1 pY(y1) y2 pY(y2)  yc pY(yc) 1 Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (yc) 1

Suppose X and Y are two discrete random variables with pmfs pX(x) and pY(y) respectively: Suppose Z is a third discrete random variable that depends on X and Y, i.e., there exists a joint pmf for every ordered pair (x, y)… x pX(x) x1 pX (x1) x2 pX (x2)  xr pX (xr) 1 y pY(y) y1 pY(y1) y2 pY(y2)  yc pY(yc) 1 Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (yc) 1

Suppose X and Y are two discrete random variables with pmfs pX(x) and pY(y) respectively: Joint pmf x pX(x) x1 pX (x1) x2 pX (x2)  xr pX (xr) 1 y pY(y) y1 pY(y1) y2 pY(y2)  yc pY(yc) 1 Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (yc) 1

p

p

p

p

p

Joint Probability Mass Function DISCRETE Joint Probability Mass Function Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (y2) pY (yc) 1

Joint Probability Mass Function DISCRETE Joint Probability Mass Function Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (y2) pY (yc) 1

Joint Probability Mass Function DISCRETE Joint Probability Mass Function Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (y2) pY (yc) 1 Def: X and Y are statistically independent if i.e., each cell probability is equal to the product of its marginal probabilities.

Joint Probability Mass Function DISCRETE Joint Probability Mass Function Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) x2 p(x2, y1) p(x2, y2) p(x2, yc) xr p(xr, y1) p(xr, y2) p(xr, yc)

Joint Probability Mass Function DISCRETE In principle, one can construct a probability histogram much as before, with the height of each rectangle centered at the point (x, y) equal to the pmf z = p(x, y). Joint Probability Mass Function Extend this to the continuous scenario…. What happens as the partition of the X and Y axes becomes arbitrarily small (i.e., the number of rows and columns  ∞)? Recall…

Time intervals = 0.5 secs Time intervals = 5.0 secs Time intervals = 1.0 secs Time intervals = 2.0 secs DISCRETE CONTINUOUS “Density” Interval widths can be made arbitrarily small, i.e, the scale at which X is measured can be made arbitrarily fine, since it is continuous. As x  0 and # rectangles  ∞, this “Riemann sum” approaches the area under the density curve f(x), expressed as a definite integral. Similarly….

Joint Probability Mass Function DISCRETE Joint Probability Mass Function Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (y2) pY (yc) 1

... CONTINUOUS DISCRETE DISCRETE Joint Probability Density Function Joint Probability Mass Function Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (y2) pY (yc) 1

... CONTINUOUS DISCRETE DISCRETE Joint Probability Density Function Joint Probability Mass Function Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (y2) pY (yc) 1

... CONTINUOUS DISCRETE DISCRETE Joint Probability Density Function Joint Probability Mass Function Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (y2) pY (yc) 1

Joint Probability Density Function CONTINUOUS Joint Probability Density Function Volume under density f(x, y) over A. “area element” Area A

Joint Probability Density Function CONTINUOUS Example: Uniform Distribution Joint Probability Density Function Recall for one r.v. X…

Joint Probability Density Function CONTINUOUS Example: Uniform Distribution Joint Probability Density Function

Joint Probability Density Function CONTINUOUS Example: Uniform Distribution  Joint Probability Density Function 

Joint Probability Density Function CONTINUOUS Example:  Joint Probability Density Function

Joint Probability Density Function CONTINUOUS Example:  Joint Probability Density Function 

Joint Probability Density Function CONTINUOUS Example: Joint Probability Density Function A

Joint Probability Density Function CONTINUOUS Example: Joint Probability Density Function

Example:

... CONTINUOUS DISCRETE DISCRETE Joint Probability Density Function Joint Probability Mass Function Y y1 y2 … yc X x1 p(x1, y1) p(x1, y2) ... p(x1, yc) pX (x1) x2 p(x2, y1) p(x2, y2) p(x2, yc) pX (x2) xr p(xr, y1) p(xr, y2) p(xr, yc) pX (xr) pY (y1) pY (y2) pY (yc) 1 What about these marginal pdfs?

Joint Probability Density Function CONTINUOUS Joint Probability Density Function This is the density curve that corresponds to our fixed value of y*. This is a plane parallel to the XZ-plane. Every point in it has the form (x, y*, z).

Joint Probability Density Function CONTINUOUS Joint Probability Density Function This is the density curve that corresponds to our fixed value of y*. Integrate w.r.t. x from - to  to obtain… This is a plane parallel to the XZ-plane. Every point in it has the form (x, y*, z). (vis-à-vis row marginals) (vis-à-vis column marginals)

Joint Probability Density Function CONTINUOUS Example (revisted): Joint Probability Density Function

Joint Probability Density Function CONTINUOUS Example (revisted): Joint Probability Density Function  Check? 

Joint Probability Density Function CONTINUOUS Example (revisted): Joint Probability Density Function

Joint Probability Density Function CONTINUOUS Example (revisted): Joint Probability Density Function F increases continuously and monotonically from 0 to 1.

Joint Probability Density Function CONTINUOUS Example (revisted): Joint Probability Density Function

Joint Probability Density Function CONTINUOUS Example (revisted): Joint Probability Density Function

Joint Probability Density Function CONTINUOUS Example (revisted): Joint Probability Density Function A

Joint Probability Density Function CONTINUOUS To summarize… Joint Probability Density Function X f(x1, y1) f(x1, y2) ... f(x1, yc) marginal pdf of Y Y f(x2, y1) f(x2, y2) f(x2, yc) f(xr, y1) f(xr, y2) f(xr, yc) 1

Joint Probability Density Function CONTINUOUS Example Joint Probability Density Function Exercise

Joint Probability Density Function CONTINUOUS Example Joint Probability Density Function Exercise Exercise

Joint Probability Density Function CONTINUOUS Example Joint Probability Density Function A Exercise

Joint Probability Density Function CONTINUOUS Example Joint Probability Density Function A Exercise

Joint Probability Density Function CONTINUOUS Example Joint Probability Density Function A Exercise

More Than Two Random Variables…? CONTINUOUS Joint Probability Density Function “Hypervolume” under density f over A. Volume under density f(x, y) over A. More Than Two Random Variables…? Definition of statistical independence of X and Y can be extended to any number of variables. Area A