1 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance The covariance of two random variables X and Y, often written  XY, is defined to be the expected value of the product of their deviations from their population means.

2 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance If two variables are independent, their covariance is zero. To show this, start by rewriting the covariance as the product of the expected values of its factors. If X and Y are independent,

3 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance We are allowed to do this because (and only because) X and Y are independent (see the earlier sequence on independence. If X and Y are independent,

The expected values of both factors are zero because E(X) =  X and E(Y) =  Y. E(  X ) =  X and E(  Y ) =  Y because  X and  Y are constants. Thus the covariance is zero. 4 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance If X and Y are independent,

5 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance rules There are some rules that follow in a perfectly straightforward way from the definition of covariance, and since they are going to be used frequently in later chapters it is worthwhile establishing them immediately. First, the addition rule. 1.If Y = V + W,

2.If Y = bZ, where b is a constant, 6 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance rules Next, the multiplication rule, for cases where a variable is multiplied by a constant. 1.If Y = V + W,

Finally, a primitive rule that is often useful. 7 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION 3.If Y = b, where b is a constant, 2.If Y = bZ, where b is a constant, Covariance rules 1.If Y = V + W,

8 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance rules The proofs of the rules are straightforward. In each case the proof starts with the definition of cov(X, Y). 1.If Y = V + W, Since Y = V + W,  Y =  V +  W Proof

9 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance rules We now substitute for Y and re-arrange. Since Y = V + W,  Y =  V +  W Proof 1.If Y = V + W,

Since Y = V + W,  Y =  V +  W 10 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance rules This gives us the result. Proof 1.If Y = V + W,

Since Y = bZ,  Y = b  Z 11 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance rules 2.If Y = bZ, where b is a constant, Next, the multiplication rule, for cases where a variable is multiplied by a constant. The Y terms have been replaced by the corresponding bZ terms. Proof

Since Y = bZ,  Y = b  Z 12 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance rules b is a common factor and can be taken out of the expression, giving us the result that we want. Proof 2.If Y = bZ, where b is a constant,

The proof of the third rule is trivial. Since Y = b,  Y = b 13 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Covariance rules 3.If Y = b, where b is a constant, Proof

14 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Example use of covariance rules Here is an example of the use of the covariance rules. Suppose that Y is a linear function of Z and that we wish to use this to decompose cov(X, Y). We substitute for Y (first line) and then use covariance rule 1 (second line). Suppose Y = b 1 + b 2 Z

cov(X, b 1 ) = 0, using covariance rule 3. The multiplicative factor b 2 may be taken out of the second term, using covariance rule 2. Suppose Y = b 1 + b 2 Z 15 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Example use of covariance rules

16 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules Corresponding to the covariance rules, there are parallel rules for variances. First the addition rule. 1.If Y = V + W,

17 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules Next, the multiplication rule, for cases where a variable is multiplied by a constant. 1.If Y = V + W, 2.If Y = bZ, where b is a constant,

18 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules A third rule to cover the special case where Y is a constant. 1.If Y = V + W, 2.If Y = bZ, where b is a constant, 3.If Y = b, where b is a constant,

Finally, it is useful to state a fourth rule. It depends on the first three, but it is so often of practical value that it is worth keeping it in mind separately. 19 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules 1.If Y = V + W, 2.If Y = bZ, where b is a constant, 3.If Y = b, where b is a constant, 4.If Y = V + b, where b is a constant,

20 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules The proofs of these rules can be derived from the results for covariances, noting that the variance of Y is equivalent to the covariance of Y with itself. 1.If Y = V + W, Proof

21 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules We start by replacing one of the Y arguments by V + W. 1.If Y = V + W, Proof

22 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules We then use covariance rule 1. 1.If Y = V + W, Proof

23 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules We now substitute for the other Y argument in both terms and use covariance rule 1 a second time. 1.If Y = V + W, Proof

24 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules This gives us the result. Note that the order of the arguments does not affect a covariance expression and hence cov(W, V) is the same as cov(V, W). 1.If Y = V + W, Proof

25 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules The proof of the variance rule 2 is even more straightforward. We start by writing var(Y) as cov(Y, Y). We then substitute for both of the iYi arguments and take the b terms outside as common factors. Proof 2.If Y = bZ, where b is a constant,

3.If Y = b, where b is a constant, 26 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules The third rule is trivial. We make use of covariance rule 3. Obviously if a variable is constant, it has zero variance. Proof

4.If Y = V + b, where b is a constant, 27 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules The fourth variance rule starts by using the first. The second term on the right side is zero by variance rule 3. The third is also zero by covariance rule 3. Proof

The intuitive reason for this result is easy to understand. If you add a constant to a variable, you shift its entire distribution by that constant. The expected value of the squared deviation from the mean is unaffected. 0 0V + b V VV  V + b 28 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Variance rules Proof 4.If Y = V + b, where b is a constant,

cov(X, Y) is unsatisfactory as a measure of association between two variables X and Y because it depends on the units of measurement of X and Y. 29 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Correlation population correlation coefficient

A better measure of association is the population correlation coefficient because it is dimensionless. The numerator possesses the units of measurement of both X and Y. 30 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Correlation population correlation coefficient

The variances of X and Y in the denominator possess the squared units of measurement of those variables. 31 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Correlation population correlation coefficient

However, once the square root has been taken into account, the units of measurement are the same as those of the numerator, and the expression as a whole is unit free. 32 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Correlation population correlation coefficient

If X and Y are independent,  XY will be equal to zero because  XY will be zero. 33 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Correlation population correlation coefficient

If there is a positive association between them,  XY, and hence  XY, will be positive. If there is an exact positive linear relationship,  XY will assume its maximum value of 1. Similarly, if there is a negative relationship,  XY will be negative, with minimum value of –1. 34 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Correlation population correlation coefficient

If X and Y are independent,  XY will be equal to zero because  XY will be zero. 35 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Correlation population correlation coefficient

If there is a positive association between them,  XY, and hence  XY, will be positive. If there is an exact positive linear relationship,  XY will assume its maximum value of COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Correlation population correlation coefficient

Similarly, if there is a negative relationship,  XY will be negative, with minimum value of –1. 37 COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION Correlation population correlation coefficient

Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section R.4 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course EC2020 Elements of Econometrics