1. EC220 - Introduction to econometrics (review chapter)
Christopher Dougherty
EC220 - Introduction to econometrics (review chapter)
Slideshow: covariance, covariance and variance rules, and correlation
Original citation:
Dougherty, C. (2012) EC220 - Introduction to econometrics (review chapter). [Teaching Resource]
© 2012 The Author
This version available at:
Available in LSE Learning Resources Online: May 2012
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms.

2. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The covariance of two random variables X and Y, often written sXY, is defined to be the expected value of the product of their deviations from their population means. 1

3. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
If two variables are independent, their covariance is zero. To show this, start by rewriting the covariance as the product of the expected values of its factors. 2

4. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
We are allowed to do this because (and only because) X and Y are independent (see the earlier sequence on independence. 3

5. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The expected values of both factors are zero because E(X) = mX and E(Y) = mY. E(mX) = mX and E(mY) = mY because mX and mY are constants. Thus the covariance is zero. 4

6. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
1. If Y = V + W,
cov(X, Y) = cov(X, V) + cov(X,W)
There are some rules that follow in a perfectly straightforward way from the definition of covariance, and since they are going to be used frequently in later chapters it is worthwhile establishing them immediately. First, the addition rule. 5

7. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
1. If Y = V + W,
cov(X, Y) = cov(X, V) + cov(X,W)
2. If Y = bZ, where b is a constant,
cov(X, Y) = bcov(X, Z)
Next, the multiplication rule, for cases where a variable is multiplied by a constant. 6

8. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
1. If Y = V + W,
cov(X, Y) = cov(X, V) + cov(X,W).
2. If Y = bZ, where b is a constant,
cov(X, Y) = bcov(X, Z)
3. If Y = b, where b is a constant,
cov(X, Y) = 0
Finally, a primitive rule that is often useful. 7

9. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
1. If Y = V + W,
cov(X, Y) = cov(X, V) + cov(X,W)
Proof:
Since Y = V + W, mY = mV + mW
The proofs of the rules are straightforward. In each case the proof starts with the definition of cov(X, Y). 8

10. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
1. If Y = V + W,
cov(X, Y) = cov(X, V) + cov(X,W).
Proof:
Since Y = V + W, mY = mV + mW
We now substitute for Y and re-arrange. 9

11. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
1. If Y = V + W,
cov(X, Y) = cov(X, V) + cov(X,W).
Proof:
Since Y = V + W, mY = mV + mW
This gives us the result. 10

12. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
2. If Y = bZ,
cov(X, Y) = bcov(X, Z)
Proof:
Since Y = bZ, mY = bmZ
Next, the multiplication rule, for cases where a variable is multiplied by a constant. The Y terms have been replaced by the corresponding bZ terms. 11

13. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
2. If Y = bZ,
cov(X, Y) = bcov(X, Z).
Proof:
Since Y = bZ, mY = bmZ
b is a common factor and can be taken out of the expression, giving us the result that we want. 12

14. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Covariance rules
3. If Y = b,
cov(X, Y) = 0.
Proof:
Since Y = b, mY = b
The proof of the third rule is trivial. 13

15. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Example use of covariance rules
Suppose Y = b1 + b2Z
cov(X, Y) = cov(X, [b1 + b2Z])
= cov(X, b1) + cov(X, b2Z)
= 0 + cov(X, b2Z)
= b2cov(X, Z)
Here is an example of the use of the covariance rules. Suppose that Y is a linear function of Z and that we wish to use this to decompose cov(X, Y). We substitute for Y (first line) and then use covariance rule 1 (second line). 14

16. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Example use of covariance rules
Suppose Y = b1 + b2Z
cov(X, Y) = cov(X, [b1 + b2Z])
= cov(X, b1) + cov(X, b2Z)
= 0 + cov(X, b2Z)
= b2cov(X, Z)
Next we use covariance rule 3 (third line), and finally covariance rule 2 (fourth line). 15

17. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z)
3. If Y = b, where b is a constant,
var(Y) = 0
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
Corresponding to the covariance rules, there are parallel rules for variances. First the addition rule. 16

18. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z)
3. If Y = b, where b is a constant,
var(Y) = 0
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
Next, the multiplication rule, for cases where a variable is multiplied by a constant. 17

19. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z)
3. If Y = b, where b is a constant,
var(Y) = 0
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
A third rule to cover the special case where Y is a constant. 18

20. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z)
3. If Y = b, where b is a constant,
var(Y) = 0
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
Finally, it is useful to state a fourth rule. It depends on the first three, but it is so often of practical value that it is worth keeping it in mind separately. 19

21. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
The proofs of these rules can be derived from the results for covariances, noting that the variance of Y is equivalent to the covariance of Y with itself. 20

22. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W)
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
We start by replacing one of the Y arguments by V + W. 21

23. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W).
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
We then use covariance rule 1. 22

24. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W).
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
We now substitute for the other Y argument in both terms and use covariance rule 1 a second time. 23

25. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
1. If Y = V + W,
var(Y) = var(V) + var(W) + 2cov(V, W).
Proof:
var(Y) = cov(Y, Y) = cov([V + W], Y)
= cov(V, Y) + cov(W, Y)
= cov(V, [V + W]) + cov(W, [V + W])
= cov(V, V) + cov(V,W) + cov(W, V) + cov(W, W)
= var(V) + 2cov(V, W) + var(W)
This gives us the result. Note that the order of the arguments does not affect a covariance expression and hence cov(W, V) is the same as cov(V, W). 24

26. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
2. If Y = bZ, where b is a constant,
var(Y) = b2var(Z).
Proof:
var(Y) = cov(Y, Y) = cov(bZ, bZ)
= b2cov(Z, Z)
= b2var(Z).
The proof of the variance rule 2 is even more straightforward. We start by writing var(Y) as cov(Y, Y). We then substitute for both of the iYi arguments and take the b terms outside as common factors. 25

27. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
3. If Y = b, where b is a constant,
var(Y) = 0
Proof:
var(Y) = cov(b, b) = 0
The third rule is trivial. We make use of covariance rule 3. Obviously if a variable is constant, it has zero variance. 26

28. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
Proof:
var(Y) = var(V) + 2cov(V, b) + var(b)
= var(V)
The fourth variance rule starts by using the first. The second term on the right side is zero by covariance rule 3. The third is also zero by variance rule 3. 27

29. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
4. If Y = V + b, where b is a constant,
var(Y) = var(V)
Proof:
var(Y) = var(V) + 2cov(V, b) + var(b)
= var(V) V mV
V + b
mV + b
The intuitive reason for this result is easy to understand. If you add a constant to a variable, you shift its entire distribution by that constant. The expected value of the squared deviation from the mean is unaffected. 28

30. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
cov(X, Y) is unsatisfactory as a measure of association between two variables X and Y because it depends on the units of measurement of X and Y. 29

31. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
A better measure of association is the population correlation coefficient because it is dimensionless. The numerator possesses the units of measurement of both X and Y. 30

32. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
The variances of X and Y in the denominator possess the squared units of measurement of those variables. 31

33. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
However, once the square root has been taken into account, the units of measurement are the same as those of the numerator, and the expression as a whole is unit free. 32

34. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
If X and Y are independent, rXY will be equal to zero because sXY will be zero. 33

35. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
If there is a positive association between them, sXY, and hence rXY, will be positive. If there is an exact positive linear relationship, rXY will assume its maximum value of 1. Similarly, if there is a negative relationship, rXY will be negative, with minimum value of –1. 34

36. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
If X and Y are independent, rXY will be equal to zero because sXY will be zero. 35

37. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
If there is a positive association between them, sXY, and hence rXY, will be positive. If there is an exact positive linear relationship, rXY will assume its maximum value of 1. 36

38. COVARIANCE, COVARIANCE AND VARIANCE RULES, AND CORRELATION
Similarly, if there is a negative relationship, rXY will be negative, with minimum value of –1. 37

39. Copyright Christopher Dougherty 2011.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author.
The content of this slideshow comes from Section R.4 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press.
Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre
Individuals studying econometrics on their own and who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course
EC212 Introduction to Econometrics
or the University of London International Programmes distance learning course
20 Elements of Econometrics