1. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-1 Basic Mathematical tools Today, we will review some basic mathematical tools. Then we will use these tools to derive general properties of estimators. We can also use these tools to derive the “best” estimators.

2. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-2 Summations The  symbol is a shorthand notation for discussing sums of numbers. It works just like the + sign you learned about in elementary school.

3. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-3 Algebra of Summations

4. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-4 Summations: A Useful Trick

5. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-5 Double Summations The “Secret” to Double Summations: keep a close eye on the subscripts.

6. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-6 Descriptive Statistics How can we summarize a collection of numbers? – Mean: the arithmetic average. The mean is highly sensitive to a few large values (outliers). – Median: the midpoint of the data. The median is the number above which lie half the observed numbers and below which lie the other half. The median is not sensitive to outliers.

7. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-7 Descriptive Statistics (cont.) – Mode: the most frequently occurring value. – Variance: the mean squared deviation of a number from its own mean. The variance is a measure of the “spread” of the data. – Standard deviation: the square root of the variance. The standard deviation provides a measure of a typical deviation from the mean.

8. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-8 Descriptive Statistics (cont.) – Covariance: the covariance of two sets of numbers, X and Y, measures how much the two sets tend to “move together.” If Cov(X,Y)  0, then if X is above its mean, we would expect that Y would also be above its mean.

9. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-9 Descriptive Statistics (cont.) – Correlation Coefficient: the correlation coefficient between X and Y “norms” the covariance by the standard deviations of X and Y. You can think of this adjustment as a unit correction. The correlation coefficient will always fall between -1 and 1.

10. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-10 A Quick Example

11. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-11 A Quick Example (cont.)

12. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-12 A Quick Example (cont.)

13. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-13 Populations and Samples Two uses for statistics: – Describe a set of numbers – Draw inferences from a set of numbers we observe to a larger population The population is the underlying structure which we wish to study. Surveyors might want to relate 6000 randomly selected voters to all the voters in the United States. Macroeconomists might want to relate data about unemployment and inflation from 1958–2004 to the underlying process linking unemployment and inflation, to predict future realizations.

14. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-14 Populations and Samples (cont.) We cannot observe the entire population. Instead, we observe a sample drawn from the population of interest. In the Monte Carlo demonstration from last time, an individual dataset was the sample and the Data Generating Process described the population.

15. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-15 Populations and Samples (cont.) The descriptive statistics we use to describe data can also describe populations. What is the mean income in the United States? What is the variance of mortality rates across countries? What is the covariance between gender and income?

16. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-16 Populations and Samples (cont.) In a sample, we know exactly the mean, variance, covariance, etc. We can calculate the sample statistics directly. We must infer the statistics for the underlying population. Means in populations are also called expectations.

17. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-17 Populations and Samples (cont.) If the true mean income in the United States is , then we expect a simple random sample to have sample mean . In practice, any given sample will also include some “sampling noise.” We will observe not , but  + . If we have drawn our sample correctly, then on average the sampling error over many samples will be 0. We write this as E (  ) = 0

18. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-18 Expectations Expectations are means over all possible samples (think “super” Monte Carlo). Means are sums. Therefore, expectations follow the same algebraic rules as sums. See the Statistics Appendix for a formal definition of Expectations.

19. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-19 Algebra of Expectations k is a constant. E(k) = k E(kY) = kE(Y) E(k+Y) = k + E(Y) E(Y+X) = E(Y) + E(X) E(  Y i ) =  E(Y i ), where each Y i is a random variable.

20. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-20 Law of Iterated Expectations The expected value of the expected value of Y conditional on X is the expected value of Y. If we take expectations separately for each subpopulation (each value of X ), and then take the expectation of this expectation, we get back the expectation for the whole population.

21. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-21 Variances Population variances are also expectations.

22. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-22 Algebra of Variances One value of independent observations is that Cov ( Y i,Y j ) = 0, killing all the cross-terms in the variance of the sum.

23. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-23 Checking Understanding Consider the Data Generating Process (DGP): Y i =  +  i E(  i ) = 0 Var(  i ) =   2 Question: What are: E(Y i ) ? Var(Y i ) ?

24. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-24 Checking Understanding (cont.) Y i =  +  i E(  i ) = 0 Var(  i ) =   2 Note:  is a constant (we don’t observe it, but it’s still a constant). E(Y i ) = E(  +  i ) = E(  +  i ) =  +0 =  Var(Y i ) = Var (  +  i ) = Var(  i ) =   2 We will make (very) frequent use of this sort of calculation.

25. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-25 Algebra of Covariances Population Covariances are also expectations.

26. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-26 Algebra of Covariances (cont.)

27. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-27 Data Generating Processes In this class, we are trying to draw inferences from a sample (our data) back to an underlying process. We begin by making very concrete assumptions about the underlying process. These assumptions are called the “Data Generating Process.”

28. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-28 Data Generating Process (cont.) The Data Generating Process is our model of how the world really works. The DGP includes some parameter (usually called something like  ) that we would like to know. We don’t usually want to describe the data, we want to make predictions about what will happen next.

29. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-29 Data Generating Process (cont.) What does the DGP assume? – The relation between Y, X, and  – The mean, variance, and covariances of  – Cross-sample properties of X

30. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-30 Our Baseline DGP: Gauss–Markov Our benchmark DGP: Gauss–Markov X ’s fixed across samples We will refer to this DGP (very) frequently.

31. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-31 Data Generating Process Are the Gauss–Markov Assumptions true? “Models are to be used, not believed.” Where does the DGP come from? – Economic intuition – Inspection of the data – Convenience Later in the class, we will learn more complicated DGP’s. Every DGP involves dramatic simplification of reality.

32. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-32 Data Generating Process (cont.) It is very appropriate to be skeptical of the Gauss–Markov Assumptions. We will relax these assumptions as the class goes on. You want to be aware of the limitations of your assumptions so you can: – Fix them – Qualify your analysis when you can’t fix them

33. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-33 Data Generating Process (cont.) Making precise assumptions lets you identify precisely what aspects of the model are responsible for what components of your results. Making precise assumptions provides a clear basis for arguing about the limits of your analysis.

34. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-34 A Strategy for Inference The DGP tells us the assumed relationships between the data we observe and the underlying process of interest. Using the assumptions of the DGP and the algebra of expectations, variances, and covariances, we can derive key properties of our estimators, and search for estimators with desirable properties.

35. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-35 An Example:  g 1

36. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-36 An Example:  g 1 (cont.)

37. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-37 Linear Estimators (Chapter 3.1)  g 1 is unbiased. Can we generalize? We will focus on linear estimators Linear estimator: a weighted sum of the Y ’s Note: we will use hats to denote estimated quantities

38. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-38 Linear Estimators (cont.) Linear estimator: Example:  g 1 is a linear estimator.

39. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-39 Checking Understanding Linear Estimator: Question: Can you find weights w i to write  g 2 and  g 4 as linear estimators?

40. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-40 Checking Understanding (cont.) All of our “best guesses” are linear estimators!

41. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-41 Expectation of Linear Estimators (Chapter 3.2)

42. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-42 Expectation of Linear Estimator (cont.)

43. A linear estimator is unbiased if  w i X i = 1 Are  g 2,  g 3, and  g 4 unbiased?

44. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-44 Expectation of Linear Estimator (cont.) Similar calculations hold for  g 3. All 4 of our “best guesses” are unbiased.

45. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-45 Review In a sample, we know exactly the mean, variance, covariance, etc. We can calculate the sample statistics directly. We must infer the statistics for the underlying population. Means in populations are also called expectations.

46. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-46 Review (cont.) Algebra of Expectations  k is a constant.  E(k) = k  E(kY) = kE(Y)  E(k+Y) = k + E(Y)  E(Y+X) = E(Y) + E(X)  E(  Y i ) =  E(Y i ), where each Y i is a random variable.

47. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-47 Review (cont.) Algebra of Variances

48. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-48 Our Baseline DGP: Gauss–Markov Our benchmark DGP: Gauss–Markov Y =  X +  E(  i ) = 0 Var(  i ) =   2 Cov(  i,  j ) = 0, for i ≠ j X ’s fixed across samples We will refer to this DGP (very) frequently.

49. Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 4-49 Linear Estimators We will focus on linear estimators. Linear estimator: a weighted sum of the Y ’s

50. Expectation of Linear Estimators