1. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-1 Supplement 3: Expectations, Conditional Expectations, Law of Iterated Expectations *The ppt is a joint effort: Ms Jingwen zHANG discussed the law of iterated expectations with Dr. Ka-fu Wong on 1 March 2007; Ka-fu explained the concept with an example; Jingwen drafted the ppt; Ka-fu revised it. Use it at your own risks. Comments, if any, should be sent to kafuwong@econ.hku.hk.

2. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-2 Joint, conditional and marginal probability, when there are two random variables. Let (X,Y) be two random variables with a joint probability of P(X,Y). From the joint probability, we can compute the marginal probability P X (X) and P Y (Y). P X (X=k) = ∑ Y P (X=k,Y); P Y (Y=k) = ∑ X P (X,Y=k) the conditional probability P x|y (X) and P y|x (Y). P X|Y=k (X) = P(X,Y=k)/ P Y (Y=k) ; P Y|X=k (Y) = P(X=k,Y)/ P X (X=k) Unconditional expectation E(Y) =∑ Y ∑ X Y*P (X,Y) Conditional expectations: E(Y|X) and E(X|Y) E(Y|X) = ∑ Y Y*P Y|X (Y) E(X|Y) = ∑ X X*P X|Y (X)

3. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-3 Conditional expectations are random variables XE(Y|X)P X (X) x1x1 E(Y|X=x 1 )P X (X=x 1 ) x2x2 E(Y|X=x 2 )P X (X=x 2 ) … xnxn E(Y|X=x n )P X (X=x n ) The conditional expectation can take different values. The probability of the conditional expectation taking a particular value.

4. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-4 Expectation of conditional expectations XE(Y|X)P X (X) x1x1 E(Y|X=x 1 )P X (X=x 1 ) x2x2 E(Y|X=x 2 )P X (X=x 2 ) … xnxn E(Y|X=x n )P X (X=x n ) E[E(Y|X)] = ∑ X {E(Y|X)*P X (X)} = ∑ X {[∑ Y Y*P y|x (Y)] *P X (X)} since E(Y|X) = ∑ y Y*P y|x (Y) = ∑ X {[∑ Y Y* P(X,Y)/ P X (X) ] *P X (X)} since P Y|X=k (Y) = P(X=k,Y)/ P X (X=k) = ∑ X ∑ Y Y* P(X,Y) = E(Y)

5. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-5 Let X and Y be random variables. X = education attainment (1=degree holder, 0 without degree) Y = income (only three groups for simplicity; 1, 2, 3 thousands) P(X,Y)Y=1Y=2Y=3 X=0P(X=0, Y=1)P(X=0, Y=2)P(X=0, Y=3) X=1P(X=1, Y=1)P(X=1, Y=2)P(X=1, Y=3) P(X=0) P(X=1) P(Y=1)P(Y=2)P(Y=3) P(X | Y)Y=1Y=2Y=3 X=0P(X=0 | Y=1)P(X=0 | Y=2)P(X=0 | Y=3) X=1P(X=1 | Y=1)P(X=1| Y=2)P(X=1 | Y=3) P(Y | X)Y=1Y=2Y=3 X=0P(Y=1 | X=0)P(Y=2 | X=0)P(Y=3 | X=0) X=1P(Y=1 | X=1)P(Y=2 | X=1)P(Y=3 | X=1) E(X |Y=1)E(X |Y=2)E(X |Y=3) E(Y | X=0) E(Y | X=1)

6. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-6 Let X and Y be random variables. X = education attainment (1=degree holder, 0 without degree) Y = income (only three groups for simplicity; 1, 2, 3) P(X | Y)Y=1Y=2Y=3 X=0P(X=0| Y=1)P(X=0| Y=2)P(X=0| Y=3) X=1P(X=1| Y=1)P(X=1| Y=2)P(X=1| Y=3) P(Y | X)Y=1Y=2Y=3 X=0P(Y=1|X=0)P(Y=2|X=0)P(Y=3|X=0) X=1P(Y=1|X=1)P(Y=2|X=1)P(Y=3|X=1) E(X|Y=1)E(X|Y=2)E(X|Y=3) E(Y|X=0) E(Y|X=1) Expected education of a person randomly drawn from the income group Y=1. Expected income of a person randomly drawn from the education group X=1.

7. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-7 Joint, conditional and marginal probability, when there are three random variables. Let (X,Y, Z) be three random variables with a joint probability of P(X,Y, Z). From the joint probability, we can compute The marginal probability P X (X), P Y (Y), P Z (Z). P X (X=k) = ∑ Y ∑ Z P (X=k,Y, Z); P Y (Y=k) = ∑ X ∑ Z P (X,Y=k, Z) P Z (Z=k) = ∑ X ∑ Y P (X,Y, Z=k) The bivariate distribution of any pair of the three random variables P XY (X,Y), P XZ (X,Z), P YZ (Y,Z) The conditional probability P X|Y,Z (X), P Y|X,Z (Y), P Z|X,Y (Z). P X|Y=k,Z=m (X) = P(X,Y=k,Z=m)/ P YZ (Y=k,Z=m) ; P Y|X=k,Z=m (Y) = P(X=k,Y,Z=m)/ P XZ (X=k,Z=m) P Z|X=k,Y=m (Z) = P(X=k,Y=m,Z)/ P XY (X=k,Y=m) The conditional bivariate probability P XY|Z (X,Y), P YZ|X (Y,Z), P XZ|Y (X,Z). P XY|Z=m (X,Y) = P(X,Y,Z=m)/ P Z (Z=m)

8. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-8 Joint, conditional and marginal probability, when there is only three random variables. Let (X,Y, Z) be three random variables with a joint probability of P(X,Y, Z). Unconditional expectation E(Y) =∑ y ∑ x ∑ Z Y*P (X,Y,Z) Conditional expectations: E(Y|X,Z) and E(X|Y,Z) E(Y|X,Z) = ∑ Y Y*P y|x,Z (Y) E(X|Y,Z) = ∑ X X*P X|Y,Z (X)

9. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-9 Conditional expectations are random variables. XZP(X,Z)E(Y|X,Z)...… …… xixi zizi P(X=x i,Z=z i )E(Y|X=x i,Z=z i ) ………… E[E(Y|X,Z)|Z] = ∑ X {E(Y|X,Z)*P X|Z (X)} = … = E(Y|Z) E[E(Y|Z)]=E(Y)

10. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-10 X, Y and Z random variables. X = education attainment (1=degree holder, 0 without degree) Y = income (only three groups for simplicity; 1, 2, 3 thousands) Z = gender (1= male, 2=female) E(Y|X=1,Z=1) The expected income of a person randomly drawn from the group of male degree holders. E(Y|X=1,Z=2) The expected income of a person randomly drawn from the group of female degree holders. E(Y|X=1,Z=1) - E(Y|X=1,Z=2) >0 and E(Y|X=0,Z=1) - E(Y|X=0,Z=2) >0 For the same education attainment, male’s expected income of a person is higher than female’s. Sometimes, it is interpreted as a piece of evidence of sex discrimination against female.

11. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-11 X, Y and Z random variables. X = education attainment (1=degree holder, 0 without degree) Y = income (only three groups for simplicity; 1, 2, 3 thousands) Z = gender (1= male, 2=female) E(Y|X=1,Z=1) The expected income of a person randomly drawn from the group of male degree holders. E(Y|X=0,Z=1) The expected income of a person randomly drawn from the group of male non-degree holders. E(Y|X=1,Z=1) - E(Y|X=0,Z=1) >0 and E(Y|X=1,Z=2) - E(Y|X=0,Z=2) >0 The return to education/schooling is positive. Education/schooling thus helps to accumulate the “human capital” embodied in us.

12. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-12 X, Y and Z random variables. X = education attainment (1=degree holder, 0 without degree) Y = income (only three groups for simplicity; 1, 2, 3 thousands) Z = gender (1= male, 2=female) E(Y | X=1,Z=1) The expected income of a person randomly drawn from the group of male degree holders. E(Y | Z=1) The expected income of a person randomly drawn from the group of male, regardless of education attainment. XZP(X|Z)E(Y|X,Z) 010.41.5 110.62.5 020.61.3 120.42.1 E(Y|Z=1)=E[E(Y|X,Z)|Z=1] = 1.5*0.4+2.5*0.6 E(Y|Z=2)=E[E(Y|X,Z)|Z=2] = 1.3*0.6+2.1*0.4 E(Y|Z)=E[E(Y|X,Z)|Z]

13. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-13 Law of iterated expectations Given E(  |X) = 0, find E(  X). E(  X) = E[E(  X|X)] =E[E(  |X)X] =E[0*X] =0 P( ,X)  X XX 0.11 0.1010 111 0.2-22-4 0.22-2 0.322-4 E(  X) =P(X=1) E(  X|X=1) + P(X=2) E(  X|X=2) =0.4*E(  |X=1)*1+0.6*E(  |X=2)*2 =0.4*0 + 0.6*0 = 0 E(  X) =0.1*(-1) + 0.1*0 + 0.1*1 + 0.2*(-4) + 0.2*(-2) + 0.3*(4) =0 + 0.

14. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-14 Law of iterated expectations Given E(Y|X,Z) = 0, E(XY) = 2, E(Z) =4, find E(XYZ). E(XYZ) = E[E(YXZ|X,Z)] = E[E(Y|X,Z) X Z] = E[ 0* X Z] = 0.

15. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-15 Definition: Estimator Estimator is a formula or a rule that takes a set of data and returns an estimate of the population quantity (also known as population parameter) we are interested in. θ(x 1,x 2,...,x n )

16. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-16 Example: An estimator for the population mean If we are interested in the population mean, a very intuitive estimator of the population mean based on a sample (x 1,x 2,...,x n ) is θ(x 1,x 2,...,x n )= (x 1 +x 2 +...+x n )/n Suppose someone suggest θ(x 1,x 2,...,x n )= (x 1 +x 2 +...+x n +1)/n

17. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-17 Desired property: unbiased. That is, on average, the estimator correctly estimates the population mean. θ(x 1,x 2,...,x n )= (x 1 +x 2 +...+x n )/n E [θ(x 1,x 2,...,x n )] = E [(x 1 +x 2 +...+x n )/n] = (1/n)*{E(x 1 ) +E(x 2 )+...+E(x n )} = (1/n)*n*E(x) = E(x) θ(x 1,x 2,...,x n )= (x 1 +x 2 +...+x n +1)/n E [θ(x 1,x 2,...,x n )] = E [(x 1 +x 2 +...+x n +1)/n] = (1/n)*{E(x 1 ) +E(x 2 )+...+E(x n ) + 1} = (1/n)*{n*E(x) + 1} = E(x) + 1/n Approaches zero as sample size increases. i.e., the estimator is asymptotically unbiased.

18. Ka-fu Wong © 2007 ECON1003: Analysis of Economic Data Supplement3-18 - END - Supplement 3: Expectations, Conditional Expectations, Law of Iterated Expectations