I271B QUANTITATIVE METHODS Regression and Diagnostics.

Slides:



Advertisements
Similar presentations
11-1 Empirical Models Many problems in engineering and science involve exploring the relationships between two or more variables. Regression analysis.
Advertisements

Forecasting Using the Simple Linear Regression Model and Correlation
Inference for Regression
6-1 Introduction To Empirical Models 6-1 Introduction To Empirical Models.
Regression Analysis Simple Regression. y = mx + b y = a + bx.
Ch11 Curve Fitting Dr. Deshi Ye
Objectives (BPS chapter 24)
Simple Linear Regression
© 2010 Pearson Prentice Hall. All rights reserved Least Squares Regression Models.
PSY 307 – Statistics for the Behavioral Sciences
Chapter 10 Simple Regression.
Chapter Topics Types of Regression Models
Linear Regression and Correlation Analysis
Chapter 11 Multiple Regression.
Topic 3: Regression.
© 2000 Prentice-Hall, Inc. Chap Forecasting Using the Simple Linear Regression Model and Correlation.
Statistics 350 Lecture 17. Today Last Day: Introduction to Multiple Linear Regression Model Today: More Chapter 6.
Correlation and Regression Analysis
Copyright ©2006 Brooks/Cole, a division of Thomson Learning, Inc. More About Regression Chapter 14.
Simple Linear Regression Analysis
Introduction to Regression
Ordinary Least Squares
Correlation & Regression
Objectives of Multiple Regression
Introduction to Linear Regression and Correlation Analysis
Inference for regression - Simple linear regression
Chapter 11 Simple Regression
Correlation and Linear Regression
Copyright © 2013, 2010 and 2007 Pearson Education, Inc. Chapter Inference on the Least-Squares Regression Model and Multiple Regression 14.
1 G Lect 10a G Lecture 10a Revisited Example: Okazaki’s inferences from a survey Inferences on correlation Correlation: Power and effect.
OPIM 303-Lecture #8 Jose M. Cruz Assistant Professor.
+ Chapter 12: Inference for Regression Inference for Linear Regression.
Introduction to Linear Regression
Random Regressors and Moment Based Estimation Prepared by Vera Tabakova, East Carolina University.
Elementary Statistics Correlation and Regression.
Examining Relationships in Quantitative Research
Multiple Regression and Model Building Chapter 15 Copyright © 2014 by The McGraw-Hill Companies, Inc. All rights reserved.McGraw-Hill/Irwin.
Chapter 14 Inference for Regression AP Statistics 14.1 – Inference about the Model 14.2 – Predictions and Conditions.
1 Lecture 4 Main Tasks Today 1. Review of Lecture 3 2. Accuracy of the LS estimators 3. Significance Tests of the Parameters 4. Confidence Interval 5.
Inference for Regression Simple Linear Regression IPS Chapter 10.1 © 2009 W.H. Freeman and Company.
Inference for Regression Chapter 14. Linear Regression We can use least squares regression to estimate the linear relationship between two quantitative.
Lesson Multiple Regression Models. Objectives Obtain the correlation matrix Use technology to find a multiple regression equation Interpret the.
Copyright ©2011 Brooks/Cole, Cengage Learning Inference about Simple Regression Chapter 14 1.
I231B QUANTITATIVE METHODS ANOVA continued and Intro to Regression.
Correlation & Regression Analysis
Lesson 14 - R Chapter 14 Review. Objectives Summarize the chapter Define the vocabulary used Complete all objectives Successfully answer any of the review.
Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Simple Linear Regression Analysis Chapter 13.
1 1 Slide The Simple Linear Regression Model n Simple Linear Regression Model y =  0 +  1 x +  n Simple Linear Regression Equation E( y ) =  0 + 
Quantitative Methods. Bivariate Regression (OLS) We’ll start with OLS regression. Stands for  Ordinary Least Squares Regression. Relatively basic multivariate.
Regression Analysis Presentation 13. Regression In Chapter 15, we looked at associations between two categorical variables. We will now focus on relationships.
Chapter 12: Correlation and Linear Regression 1.
Bivariate Regression. Bivariate Regression analyzes the relationship between two variables. Bivariate Regression analyzes the relationship between two.
Lecture #25 Tuesday, November 15, 2016 Textbook: 14.1 and 14.3
Multiple Regression and Model Building
Fundamentals of regression analysis
Correlation and Simple Linear Regression
I271B Quantitative Methods
CHAPTER 29: Multiple Regression*
Multiple Regression Models
Simple Linear Regression
Correlation and Simple Linear Regression
I271b Quantitative Methods
CHAPTER 12 More About Regression
Simple Linear Regression and Correlation
Product moment correlation
Chapter 14 Inference for Regression
Introduction to Regression
Correlation and Simple Linear Regression
Correlation and Simple Linear Regression
Presentation transcript:

I271B QUANTITATIVE METHODS Regression and Diagnostics

Regression versus Correlation 2 Correlation makes no assumption about one whether one variable is dependent on the other– only a measure of general association Regression attempts to describe a dependent nature of one or more explanatory variables on a single dependent variable. Assumes one-way causal link between X and Y. Thus, correlation is a measure of the strength of a relationship -1 to 1, while regression measures the exact nature of that relationship (e.g., the specific slope which is the change in Y given a change in X)

Basic Linear Model 3 Yi = b0 + b 1 xi + ei. X (and X-axis) is our independent variable(s) Y (and Y-axis) is our dependent variable b 0 is a constant (y-intercept) b 1 is the slope (change in Y given a one- unit change in X) e is the error term (residuals)

Basic Linear Function 4

Slope 5 But...what happens if B is negative?

Statistical Inference Using Least Squares 6 We obtain a sample statistic, b, which estimates the population parameter. We also have the standard error for b Uses standard t-distribution with n-2 degrees of freedom for hypothesis testing. Y i = b 0 + b 1 x i + e i.

Why Least Squares? 7 For any Y and X, there is one and only one line of best fit. The least squares regression equation minimizes the possible error between our observed values of Y and our predicted values of Y (often called y-hat).

Data points and Regression 8 regress/regress.html regress/regress.html

Multivariate Regression Control Variables Alternate Predictor Variables Nested Models 9

Model 1 Control Variable 1 Control Variable 2 Model 2 Control Variable 1 Control Variable 2 Explanatory Variable 1 Model 3 Control Variable 1 Control Variable 2 Explanatory Variable 1 Explanatory Variable 2 10

Regression Diagnostics 11

Lab #4 Stating Hypothesis Interpreting Hypotheses  Terminology  Appropriate statistics and conventions Effect Size (revisited)  Cohen’s d and the.2,.5,.8 interpretation values  See also: for a very nice lecture and discussion of the different types of effect size calculationshttp://web.uccs.edu/lbecker/Psy590/es.htm 12

13 Multicollinearity Occurs when an IV is very highly correlated with one or more other IV’s  Caused by many things (including variables computed by other variables in same equation, using different operationalizations of same concept, etc) Consequences  For OLS regression, it does not violate assumptions, but  Standard Errors will be much, much larger than normal when there is multicollinearity (confidence intervals become wider, t-statistics become smaller) We often use VIF (variance inflation factors) scores to detect multicollinearity  Generally, VIF of 5-10 is problematic, higher values considered problematic Solving the problem  Typically, regressing each IV on the other IV’s is a way to find the problem variable(s).

14 Heteroskedasticity OLS regression assumes that the variance of the error term is constant. If the error does not have a constant variance, then it is heteroskedastic. Where it comes from  Error may really change as an IV increases  Measurement error  Underspecified model

15 Heteroskedasticity (continued) Consequences  We still get unbiased parameter estimates, but our line may not be the best fit.  Why? Because OLS gives more ‘weight’ to the cases that might actually have the most error from the predicted line. Detecting it  We have to look at the residuals (difference between observed responses from the predicted responses)  First, use a residual versus fitted values plot (in STATA, rvfplot) or the residuals versus predicted values plot, which is a plot of the residuals versus one of the independent variables.  We should see an even band across the 0 point (the line), indicating that our error is roughly equal.  If we are still concerned, we can run a test such as the Breusch-Pagan/Cook-Weisberg Test for Heteroskedasticity. It tests the null hypothesis that the error variances are all EQUAL, and the alternative hypothesis that there is some difference. Thus, if it is significant then we reject the null hypothesis and we have a problem of heteroskedasticity.