Chapter 2 Ordinary Least Squares Copyright © 2011 Pearson Addison-Wesley. All rights reserved. Slides by Niels-Hugo Blunch Washington and Lee University.

Slides:



Advertisements
Similar presentations
Multiple Regression Analysis
Advertisements

CHAPTER 3: TWO VARIABLE REGRESSION MODEL: THE PROBLEM OF ESTIMATION
Specifying an Econometric Equation and Specification Error
Simple Linear Regression. Start by exploring the data Construct a scatterplot  Does a linear relationship between variables exist?  Is the relationship.
Introduction and Overview
The Use and Interpretation of the Constant Term
Choosing a Functional Form
Lecture 3 Cameron Kaplan
Chapter 10 Simple Regression.
9. SIMPLE LINEAR REGESSION AND CORRELATION
Chapter 12 Simple Regression
Econ Prof. Buckles1 Multiple Regression Analysis y =  0 +  1 x 1 +  2 x  k x k + u 1. Estimation.
The Simple Regression Model
Chapter 11 Multiple Regression.
Simple Linear Regression Analysis
Business Statistics - QBM117 Least squares regression.
Introduction to Regression Analysis, Chapter 13,
Ordinary Least Squares
Simple Linear Regression
Prepared by Robert F. Brooker, Ph.D. Copyright ©2004 by South-Western, a division of Thomson Learning. All rights reserved.Slide 1 Managerial Economics.
3. Multiple Regression Analysis: Estimation -Although bivariate linear regressions are sometimes useful, they are often unrealistic -SLR.4, that all factors.
McGraw-Hill/IrwinCopyright © 2009 by The McGraw-Hill Companies, Inc. All Rights Reserved. Simple Linear Regression Analysis Chapter 13.
Introduction to Linear Regression and Correlation Analysis
Statistics for Managers Using Microsoft Excel, 4e © 2004 Prentice-Hall, Inc. Chap 12-1 Chapter 12 Simple Linear Regression Statistics for Managers Using.
Chapter 11 Simple Regression
Copyright © Cengage Learning. All rights reserved.
5.1 Basic Estimation Techniques  The relationships we theoretically develop in the text can be estimated statistically using regression analysis,  Regression.
  What is Econometrics? Econometrics literally means “economic measurement” It is the quantitative measurement and analysis of actual economic and business.
1 Chapter 3: Examining Relationships 3.1Scatterplots 3.2Correlation 3.3Least-Squares Regression.
7.1 Multiple Regression More than one explanatory/independent variable This makes a slight change to the interpretation of the coefficients This changes.
Managerial Economics Demand Estimation. Scatter Diagram Regression Analysis.
Chapter 10 Hetero- skedasticity Copyright © 2011 Pearson Addison-Wesley. All rights reserved. Slides by Niels-Hugo Blunch Washington and Lee University.
Pure Serial Correlation
Ordinary Least Squares Estimation: A Primer Projectseminar Migration and the Labour Market, Meeting May 24, 2012 The linear regression model 1. A brief.
1 Chapter 12 Simple Linear Regression. 2 Chapter Outline  Simple Linear Regression Model  Least Squares Method  Coefficient of Determination  Model.
Copyright © 2005 by the McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Managerial Economics Thomas Maurice eighth edition Chapter 4.
1 Multiple Regression Analysis y =  0 +  1 x 1 +  2 x  k x k + u.
Y X 0 X and Y are not perfectly correlated. However, there is on average a positive relationship between Y and X X1X1 X2X2.
Roger B. Hammer Assistant Professor Department of Sociology Oregon State University Conducting Social Research Ordinary Least Squares Regression.
Chapter Three TWO-VARIABLEREGRESSION MODEL: THE PROBLEM OF ESTIMATION
© Copyright McGraw-Hill Correlation and Regression CHAPTER 10.
Simple Linear Regression In the previous lectures, we only focus on one random variable. In many applications, we often work with a pair of variables.
Chapter 4 The Classical Model Copyright © 2011 Pearson Addison-Wesley. All rights reserved. Slides by Niels-Hugo Blunch Washington and Lee University.
Correlation & Regression Analysis
1 Prof. Dr. Rainer Stachuletz Multiple Regression Analysis y =  0 +  1 x 1 +  2 x  k x k + u 1. Estimation.
Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 3 Association: Contingency, Correlation, and Regression Section 3.3 Predicting the Outcome.
Residuals Recall that the vertical distances from the points to the least-squares regression line are as small as possible.  Because those vertical distances.
Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Simple Linear Regression Analysis Chapter 13.
Simple Linear Regression The Coefficients of Correlation and Determination Two Quantitative Variables x variable – independent variable or explanatory.
Regression Analysis: A statistical procedure used to find relations among a set of variables B. Klinkenberg G
1 AAEC 4302 ADVANCED STATISTICAL METHODS IN AGRICULTURAL RESEARCH Part II: Theory and Estimation of Regression Models Chapter 5: Simple Regression Theory.
Chapter 13 Linear Regression and Correlation. Our Objectives  Draw a scatter diagram.  Understand and interpret the terms dependent and independent.
Chapter 4 Demand Estimation
Multiple Regression Analysis: Estimation
Correlation and Simple Linear Regression
Micro Economics in a Global Economy
Chapter 5: The Simple Regression Model
Ordinary Least Squares (OLS) Regression
Correlation and Simple Linear Regression
Managerial Economics in a Global Economy
Chapter 4 Demand Estimation
Chapter 6: MULTIPLE REGRESSION ANALYSIS
Correlation and Simple Linear Regression
Goodness of Fit The sum of squared deviations from the mean of a variable can be decomposed as follows: TSS = ESS + RSS This decomposition can be used.
M248: Analyzing data Block D UNIT D2 Regression.
Simple Linear Regression and Correlation
Product moment correlation
Correlation and Simple Linear Regression
Correlation and Simple Linear Regression
Presentation transcript:

Chapter 2 Ordinary Least Squares Copyright © 2011 Pearson Addison-Wesley. All rights reserved. Slides by Niels-Hugo Blunch Washington and Lee University

2-1 © 2011 Pearson Addison-Wesley. All rights reserved. Estimating Single-Independent- Variable Models with OLS Recall that the objective of regression analysis is to start from: (2.1) And, through the use of data, to get to: (2.2) Recall that equation 2.1 is purely theoretical, while equation (2.2) is it empirical counterpart How to move from (2.1) to (2.2)?

2-2 © 2011 Pearson Addison-Wesley. All rights reserved. Estimating Single-Independent- Variable Models with OLS (cont.) One of the most widely used methods is Ordinary Least Squares (OLS) OLS minimizes (i = 1, 2, …., N)(2.3) Or, the sum of squared deviations of the vertical distance between the residuals (i.e. the estimated error terms) and the estimated regression line We also denote this term the “Residual Sum of Squares” (RSS)

2-3 © 2011 Pearson Addison-Wesley. All rights reserved. Estimating Single-Independent- Variable Models with OLS (cont.) Similarly, OLS minimizes: Why use OLS? Relatively easy to use The goal of minimizing RSS is intuitively / theoretically appealing This basically says we want the estimated regression equation to be as close as possible to the observed data OLS estimates have a number of useful characteristics

2-4 © 2011 Pearson Addison-Wesley. All rights reserved. Estimating Single-Independent- Variable Models with OLS (cont.) OLS estimates have at least two useful characteristics: The sum of the residuals is exactly zero OLS can be shown to be the “best” estimator when certain specific conditions hold (we’ll get back to this in Chapter 4) –Ordinary Least Squares (OLS) is an estimator –A given produced by OLS is an estimate

Estimating Single-Independent- Variable Models with OLS (cont.) How does OLS work? First recall from (2.3) that OLS minimizes the sum of the squared residuals

2-6 © 2011 Pearson Addison-Wesley. All rights reserved. Estimating Single-Independent- Variable Models with OLS (cont.) Next, it can be shown (see Exercise 12) that the coefficients that ensure that for the case of just one independent variable are:

2-7 © 2011 Pearson Addison-Wesley. All rights reserved. Estimating Multivariate Regression Models with OLS In the “real world” one explanatory variable is not enough The general multivariate regression model with K independent variables is: Y i = β 0 + β 1 X 1i + β 2 X 2i β K X Ki + ε i (i = 1,2,…,N) Biggest difference with single-explanatory variable regression model is in the interpretation of the slope coefficients –Now a slope coefficient indicates the change in the dependent variable associated with a one-unit increase in the explanatory variable holding the other explanatory variables constant

2-8 © 2011 Pearson Addison-Wesley. All rights reserved. Estimating Multivariate Regression Models with OLS (cont.) Omitted (and relevant!) variables are therefore not held constant The intercept term, β 0, is the value of Y when all the Xs and the error term equal zero Nevertheless, the underlying principle of minimizing the summed squared residuals remains the same

2-9 © 2011 Pearson Addison-Wesley. All rights reserved. Example: financial aid awards at a liberal arts college Dependent variable: FINAID i : financial aid (measured in dollars of grant) awarded to the ith applicant

2-10 © 2011 Pearson Addison-Wesley. All rights reserved. Example: financial aid awards at a liberal arts college Theoretical Model: (2.9) (2.10) where: –PARENT i : The amount (in dollars) that the parents of the ith student are judged able to contribute to college expenses –HSRANK i : The ith student’s GPA rank in high school, measured as a percentage (i.e. between 0 and 100)

2-11 © 2011 Pearson Addison-Wesley. All rights reserved. Example: financial aid awards at a liberal arts college (cont.) Estimate model using the data in Table 2.2 to get: (2.11) Interpretation of the slope coefficients? –Graphical interpretation in Figures 2.1 and 2.2

2-12 © 2011 Pearson Addison-Wesley. All rights reserved. Figure 2.1 Financial Aid as a Function of Parents’ Ability to Pay

2-13 © 2011 Pearson Addison-Wesley. All rights reserved. Figure 2.2 Financial Aid as a Function of High School Rank

2-14 © 2011 Pearson Addison-Wesley. All rights reserved. Total, Explained, and Residual Sums of Squares (2.12) (2.13) TSS = ESS + RSS This is usually called the decomposition of variance

2-15 © 2011 Pearson Addison-Wesley. All rights reserved. Figure 2.3 Decomposition of the Variance in Y

Total, Explained, and Residual Sums of Squares

2-17 © 2011 Pearson Addison-Wesley. All rights reserved. Describing the Overall Fit of the Estimated Model The simplest commonly used measure of overall fit is the coefficient of determination, R 2 : (2.14) Since OLS selects the coefficient estimates that minimizes RSS, OLS provides the largest possible R 2 (within the class of linear models)

2-18 © 2011 Pearson Addison-Wesley. All rights reserved. Figure 2.4 Illustration of Case Where R 2 = 0

2-19 © 2011 Pearson Addison-Wesley. All rights reserved. Figure 2.5 Illustration of Case Where R 2 =.95

2-20 © 2011 Pearson Addison-Wesley. All rights reserved. Figure 2.6 Illustration of Case Where R 2 = 1

2-21 © 2011 Pearson Addison-Wesley. All rights reserved. The Simple Correlation Coefficient, r This is a measure related to R 2 r measures the strength and direction of the linear relationship between two variables: – r = +1: the two variables are perfectly positively correlated –r = –1: the two variables are perfectly negatively correlated –r = 0: the two variables are totally uncorrelated

2-22 © 2011 Pearson Addison-Wesley. All rights reserved. The adjusted coefficient of determination A major problem with R 2 is that it can never decrease if another independent variable is added An alternative to R 2 that addresses this issue is the adjusted R 2 or R 2 : (2.15) Where N – K – 1 = degrees of freedom

2-23 © 2011 Pearson Addison-Wesley. All rights reserved. The adjusted coefficient of determination (cont.) So, R 2 measures the share of the variation of Y around its mean that is explained by the regression equation, adjusted for degrees of freedom R 2 can be used to compare the fits of regressions with the same dependent variable and different numbers of independent variables As a result, most researchers automatically use instead of R 2 when evaluating the fit of their estimated regressions equations

2-24 © 2011 Pearson Addison-Wesley. All rights reserved. Evaluating the Quality of a Regression Equation Checkpoints here include the following: 1. Is the equation supported by sound theory? 2. How well does the estimated regression fit the data? 3. Is the data set reasonably large and accurate? 4. Is OLS the best estimator to be used for this equation? 5. How well do the estimated coefficients correspond to the expectations developed by the researcher before the data were collected? 6. Are all the obviously important variables included in the equation? 7. Has the most theoretically logical functional form been used? 8. Does the regression appear to be free of major econometric problems? *These numbers roughly correspond to the relevant chapters in the book

2-25 © 2011 Pearson Addison-Wesley. All rights reserved. Table 2.1a The Calculation of Estimated Regression Coefficients for the Weight/Height Example

2-26 © 2011 Pearson Addison-Wesley. All rights reserved. Table 2.1b The Calculation of Estimated Regression Coefficients for the Weight/Height Example

2-27 © 2011 Pearson Addison-Wesley. All rights reserved. Table 2.2a Data for the Financial Aid Example

2-28 © 2011 Pearson Addison-Wesley. All rights reserved. Table 2.2b Data for the Financial Aid Example

2-29 © 2011 Pearson Addison-Wesley. All rights reserved. Table 2.2c Data for the Financial Aid Example

2-30 © 2011 Pearson Addison-Wesley. All rights reserved. Table 2.2d Data for the Financial Aid Example

2-31 © 2011 Pearson Addison-Wesley. All rights reserved. Key Terms from Chapter 2 Ordinary Least Squares (OLS) Interpretation of a multivariate regression coefficient Total sums of squares Explained sums of squares Residual sums of squares Coefficient of determination, R 2 Simple correlation coefficient, r Degrees of freedom Adjusted coefficient of determination, R 2