1. Chapter 5a:Functions of Random Variables zlyang@smu.edu.sg http://www.mysmu.edu/faculty/zlyang/ Yang Zhenlin

2. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU Chapter 5a Contents Functions of One Random Variable --- change of variable technique Functions of Two Random Variables --- change of variable technique Sum of Independent Random variables --- The Moment Generating Function Technique 2 The main purpose of this chapter: Introducing methods for finding the distribution of a function of the random variable(s). The main purpose of this chapter: Introducing methods for finding the distribution of a function of the random variable(s).

3. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 3 Functions of One Random Variable Definition 5a.1 (Change-of-Variable Technique) Let X be a continuous type random variable with pdf f(x). Let Y = u(X) be a one-to-one transformation of X with inverse function X = v(Y). Then the pdf of Y is given by g(y) = f[v(y)] |v(y)|, where v(y) is the derivative of v(y). If the possible values of X are c 1 < x <c 2, then the possible values of Y are u(c 1 )< y <u(c 2 ). Definition 5a.1 (Change-of-Variable Technique) Let X be a continuous type random variable with pdf f(x). Let Y = u(X) be a one-to-one transformation of X with inverse function X = v(Y). Then the pdf of Y is given by g(y) = f[v(y)] |v(y)|, where v(y) is the derivative of v(y). If the possible values of X are c 1 < x <c 2, then the possible values of Y are u(c 1 )< y <u(c 2 ). Example 5a.1. Let X have a gamma distribution with pdf Let Y = e X. Find the pdf of Y. The case of a continuous random variable.

4. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 4 Functions of One Random Variable Solution: Since the inverse function is X= v(Y) = log (Y), v(y) = 1/y. Thus, by Definition 5.1, the pdf of Y is given by Since the support of X is (0,  ), the support of Y is (1,  ). The pdf of Y is thus, The way to see the change-of-variable technique is through CDF: G(y) = P{Y  y} = P{X  v(y)} = F[v(y)]. Taking derivatives leads to g(y) = f[v(y)] |v(y)|. So, the change- of-variable technique is essentially the CDF technique. The way to see the change-of-variable technique is through CDF: G(y) = P{Y  y} = P{X  v(y)} = F[v(y)]. Taking derivatives leads to g(y) = f[v(y)] |v(y)|. So, the change- of-variable technique is essentially the CDF technique.

5. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 5 Functions of One Random Variable The Change-of-variable technique can be applied to a random variable X of the discrete type, but there is major difference: pmf p(x) = P{X = x} represents probability, but pdf f(x) does not. For a one-to-one transformation, Y = u(X), with inverse X = v(Y), we can easily see that the pmf g(y) of Y is g(y) = P{Y = y} = P{X = v(y)} = p[v(y)] The possible values of Y are found directly from the possible values of X through the functional relation Y = u(X). The case of a discrete random variable. Example 5a.2. Let X have a Poisson distribution with = 4. Find the pmf of Y = X 1/2. Since X = Y 2, we have,

6. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 6 Functions of One Random Variable The case of a non-one-to-one function of a continuous r.v. The change-of-variable technique requires that the function is one-to-one, thus cannot be applied when the function is not one-to-one. However, as noted earlier, the distribution of functions of a random variable are essentially developed from the CDF. Thus, the distribution of a non-one-to-one function can still be derived from the CDF! We will demonstrate this idea by showing an important result: The square of a standard normal random variable is a gamma r.v. with parameters (1/2, 2), this special gamma r.v. is called the chi- squared random variable with degrees of freedom equal to 1.

7. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 7 Functions of One Random Variable Example 5a.3. Let Z be a standard normal r.v. and let X = Z 2. The CDF of X is Taking derivative with respect to x, we obtain the pdf of X: Recognizing that, the above is the pdf of a gamma r.v. with parameters (1/2, 2), called the chi-squared with 1 d.f.

8. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 8 Functions of Two Random Variables The above change-of-variable technique can be extended to the case of joint distributions involving two or more random variables. Many interesting problems solved. Definition 5a.2. (Change-of-Variable Technique) Let X 1 and X 2 be two continuous type random variables with joint pdf f(x 1, x 2 ). Let Y 1 = u 1 (X 1, X 2 ) and Y 2 = u 2 (X 1, X 2 ) be two continuous functions, which have single-valued inverse: X 1 = v 1 (Y 1, Y 2 ) and X 2 = v 2 (Y 1, Y 2 ). Then the joint pdf of Y 1 and Y 2 is where J, called the Jacobian, is the determinant of the matrix of partial derivatives: Definition 5a.2. (Change-of-Variable Technique) Let X 1 and X 2 be two continuous type random variables with joint pdf f(x 1, x 2 ). Let Y 1 = u 1 (X 1, X 2 ) and Y 2 = u 2 (X 1, X 2 ) be two continuous functions, which have single-valued inverse: X 1 = v 1 (Y 1, Y 2 ) and X 2 = v 2 (Y 1, Y 2 ). Then the joint pdf of Y 1 and Y 2 is where J, called the Jacobian, is the determinant of the matrix of partial derivatives:

9. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 9 Functions of Two Random Variables Example 5a.4. Let X 1 and X 2 be two independent r.v.s, each with pdf f(x) = e  x, 0 < x < . Consider Y 1 = X 1  X 2, and Y 2 = X 1 + X 2. (a)Find the joint pdf of Y 1 and Y 2 (b)Find the marginal pdfs of Y 1 and Y 2, respectively.

10. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 10 Functions of Two Random Variables where the possible values of Y 1 and Y 2 can be found as follows: Y 2 = X 1 + X 2 implies 0 < Y 2 <  ; X 1 = (Y 1 + Y 2 )/2 > 0 implies Y 1 >  Y 2 ; X 2 = (Y 2  Y 1 )/2 > 0 implies Y 1 < Y 2. y1y1 y2y2 y 2 = y 1 y 2 = -y 1 The region of (Y 1, Y 2 ) values

11. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU The latter expression can simply be written as That is called the double exponential pdf. 11 Functions of Two Random Variables (b)The marginal pdf of Y 2 : The marginal pdf of Y 1 :

12. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 12 Functions of Two Random Variables Definition 5a.3. Let X and Y be jointly distributed r.v.s with joint pmf p(x, y), or a joint pdf f(x, y). Let u(X, Y) be a continuous function of X and Y. Then, u(X, Y) is also a random variable. If X and Y are both discrete, And if X and Y are both continuous, Definition 5a.3. Let X and Y be jointly distributed r.v.s with joint pmf p(x, y), or a joint pdf f(x, y). Let u(X, Y) be a continuous function of X and Y. Then, u(X, Y) is also a random variable. If X and Y are both discrete, And if X and Y are both continuous, If X and Y are jointly distributed r.v.s, then, Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y) If further X and Y are independent, then Var(X + Y) = Var(X) + Var(Y) If X and Y are jointly distributed r.v.s, then, Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y) If further X and Y are independent, then Var(X + Y) = Var(X) + Var(Y)

13. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 13 Functions of Two Random Variables Example 5a.5. The joint probability distribution of variables X and Y is shown in the table below, Y 1 X2X23 10.200.180.12 20.150.090.06 30.070.030.10 (a) Determine the marginal probability distributions of X and Y. (b) Are X and Y independent? Explain. (c) Find the probability mass function of X+Y. (d) Find the probability of P(X  Y). X123 Y123 pX(x)pX(x)0.420.300.28pY(y)pY(y)0.500.300.20 Solution: (a) The marginal pmfs are:

14. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 14 Functions of Two Random Variables (b)No. Because p(1, 1) = 0.20, but = 0.42  0.50 = 0.21. (c)Let Z = X+Y, then the pmf of Z is Where, for example, p Z (3) = P(X+Y = 3) = P(X = 1, Y = 2) + P(X = 2, Y = 1) = 0.15 + 0.18 = 0.33. (d) P(X  Y) = 1  P(X = Y) = 1  P(X = 1, Y = 1)  P(X = 2, Y = 2)  P(X = 3, Y = 3) = 1  0.20  0.09  0.10 = 0.61 Z23456 pZ(z)pZ(z)0.200.330.280.090.10

15. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 15 Sum of Independent Random Variables Using the above property, one can easily see the following results: Sum of independent binomial r.v.s with the same probability of success  is again a binomial r.v. Sum of independent Poisson r.v.s is again a Poisson r.v. Sum of independent exponential r.v.s with the same mean is a gamma r.v. Sum of independent normal r.v.s is again a normal r.v. And more.... Recall the Uniqueness Property of MGF: The MGF of a r.v. X uniquely determines its distribution, and vise versa, e.g., if the MGF of X is the same of that of a normal r.v., then, X must be normally distributed.

16. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU To demonstrate the above result using the MGF technique, consider two independent normal random variables and. Let Y = X 1 + X 2. The MGF of Y is 16 Sum of Independent Random Variables Sum of independent normal r.v.s is again a normal r.v. Recall the MGF of X ~ N( ,  2 ): If one can show that the MGF of a random variable has the same for as above, then one can conclude that this random variable is normal with mean and variance being, respectively, the quantities in front of ‘t’ and ‘t 2 ’. Recall the MGF of X ~ N( ,  2 ): If one can show that the MGF of a random variable has the same for as above, then one can conclude that this random variable is normal with mean and variance being, respectively, the quantities in front of ‘t’ and ‘t 2 ’. Follows from the independence between X 1 and X 2

17. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU It follows that Recognizing that this MGF is in the same form of the MGF of a normal random variable, Y must be normally distributed. In particular, This result can easily be extended to the case of many normal r.v.s 17 Sum of Independent Random Variables

18. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 18 Sum of Independent Random Variables Sum of independent binomial r.v.s with the same probability of success  is again a binomial r.v. To see this, using MGF technique. White board presentation.

19. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 19 Sum of Independent Random Variables Sum of independent Poisson r.v.s is again a Poisson r.v. To see this, using MGF technique. White board presentation.

20. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 20 Sum of Independent Random Variables Sum of independent exponential r.v.s with the same mean is a gamma r.v. To see this, using MGF technique. White board presentation.

21. STAT306, Term II, 09/10 Chapter 5a STAT151, Term I 2015-16© Zhenlin Yang, SMU 21 Functions of Normal R.V.s In Example 5a.3, we have shown that if Z is a standard normal r.v., then X = Z 2 follows a chi-squared distribution with 1 d.f., which is seen to be a special gamma r.v. We have also shown using the MGF technique that the sum of two independent normal r.v.s is again normally distributed. There are many other functions of normal r.v.(s) of which the distributions are of interest. In particular in the context of statistical inference, functions of a random sample drawn from a normal population, or functions of two random samples drawn from two independent normal populations, are needed for the purposes of drawing statistical inferences about the normal populations. We put these into a general topic: “Sampling Distribution”, with details presented in Chapter 5b.