OPIM 303-Lecture #8 Jose M. Cruz Assistant Professor

Session 8 - Overview Simple Regression Model Determining the best fit “Goodness of Fit” –R 2 –Confidence Intervals –Hypothesis tests –Residual Analysis

Purpose of Regression Analysis Regression analysis is used primarily to model causality and provide prediction –Predicts the value of a dependent (response) variable based on the value of at least one independent (explanatory) variable –Explains the effect of the independent variables on the dependent variable

Types of Regression Models Positive Linear Relationship Negative Linear Relationship Relationship NOT Linear No Relationship

Simple Linear Regression Model Relationship between variables is described by a linear function The change of one variable causes the change in the other variable A dependency of one variable on the other

Population Regression Line (conditional mean) Population Linear Regression average value (conditional mean) Population regression line is a straight line that describes the dependence of the average value (conditional mean) of one variable on the other Population Y intercept Population Slope Coefficient Random Error Dependent (Response) Variable Independent (Explanatory) Variable

Population Linear Regression (continued) = Random Error Y X (Observed Value of Y) = Observed Value of Y (Conditional Mean)

estimate Sample regression line provides an estimate of the population regression line as well as a predicted value of Y Sample Linear Regression Sample Y Intercept Sample Slope Coefficient Residual Sample Regression Line (Fitted Regression Line, Predicted Value)

Sample Linear Regression and are obtained by finding the values of and that minimizes the sum of the squared residuals estimate provides an estimate of estimate provides and estimate of (continued)

Sample Linear Regression (continued) Y X Observed Value

Interpretation of the Slope and the Intercept is the average value of Y when the value of X is zero. measures the change in the average value of Y as a result of a one-unit change in X.

estimated is the estimated average value of Y when the value of X is zero. estimated is the estimated change in the average value of Y as a result of a one-unit change in X. (continued) Interpretation of the Slope and the Intercept

Simple Linear Regression: Example You want to examine the linear dependency of the annual sales of produce stores on their size in square footage. Sample data for seven stores were obtained. Find the equation of the straight line that fits the data best. Annual Store Square Sales Feet($1000) 1 1,726 3, ,542 3, ,816 6, ,555 9, ,292 3, ,208 5, ,313 3,760

Scatter Diagram: Example Excel Output

Equation for the Sample Regression Line: Example From Excel Printout:

Excel Output Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations7 ANOVA dfSSMSF Significance F Regression Residual Total Coefficient s Standard Errort StatP-valueLower 95%Upper 95% Intercept X Variable

Graph of the Sample Regression Line: Example Y i = X i 

Interpretation of Results: Example The slope of means that for each increase of one unit in X, we predict the average of Y to increase by an estimated units. The model estimates that for each increase of one square foot in the size of the store, the expected annual sales are predicted to increase by $1487.

How Good is the regression? R 2 Residual Plots Analysis of Variance Confidence Intervals Hypothesis (t) tests

Coefficient of Correlation Measures the strength of the linear relationship between two quantitative variables

The Coefficient of Determination Denoted by R 2 Measures the proportion of variation in Y that is explained by the independent variable X in the regression model

Coefficients of Determination (r 2 ) and Correlation (r) r 2 = 1, r 2 =.8,r 2 = 0, Y Y i =b 0 +b 1 X i X ^ Y Y i =b 0 +b 1 X i X ^ Y Y i =b 0 +b 1 X i X ^ Y Y i =b 0 +b 1 X i X ^ r = +1 r = -1 r = +0.9r = 0

Linear Regression Assumptions 1.Linearity 2.Normality –Y values are normally distributed for each X –Probability distribution of error is normal 2.Homoscedasticity (Constant Variance) 3.Independence of Errors

Residual Analysis Purposes –Examine linearity –Evaluate violations of assumptions Graphical Analysis of Residuals –Plot residuals vs. X i, Y i and time

Residual Analysis for Linearity Not Linear Linear X e e X Y X Y X

Y values are normally distributed around the regression line. For each X value, the “spread” or variance around the regression line is the same. Variation of Errors around the Regression Line X1X1 X2X2 X Y f(e) Sample Regression Line

Residual Analysis for Homoscedasticity Heteroscedasticity Homoscedasticity SR X X Y X X Y

Residual Analysis:Excel Output for Produce Stores Example Excel Output

Residual Analysis for Independence Not Independent Independent e e Time Residual is plotted against time to detect any autocorrelation No Particular PatternCyclical Pattern Graphical Approach

The ANOVA Table in Excel ANOVA dfSSMSF Significance F RegressionpSSR MSR =SSR/p MSR/MSE P-value of the F Test Residualsn-p-1SSE MSE =SSE/(n-p-1) Totaln-1SST

Measures of Variation The Sum of Squares: Example Excel Output for Produce Stores

Measures of Variation: Produce Store Example Excel Output for Produce Stores r 2 =.94 94% of the variation in annual sales can be explained by the variability in the size of the store as measured by square footage

Inference about the Slope: t Test t test for a population slope –Is there a linear dependency of Y on X ? Null and alternative hypotheses –H 0 :  1 = 0(no linear dependency) –H 1 :  1  0(linear dependency) Test statistic –

Example: Produce Store Data for Seven Stores: Estimated Regression Equation: The slope of this model is Is square footage of the store affecting its annual sales?  Annual Store Square Sales Feet($000) 1 1,726 3, ,542 3, ,816 6, ,555 9, ,292 3, ,208 5, ,313 3,760 Y i = X i

Inferences about the Slope: t Test Example H 0 :  1 = 0 H 1 :  1  0  .05 df  = 5 Critical Value(s): Test Statistic: Decision: Conclusion: There is evidence that square footage affects annual sales. t Reject.025 From Excel Printout Reject H 0

Inferences about the Slope: Confidence Interval Example Confidence Interval Estimate of the Slope: Excel Printout for Produce Stores At 95% level of confidence, the confidence interval for the slope is (1.062, 1.911). Does not include 0. Conclusion: There is a significant linear dependency of annual sales on the size of the store.

Confidence Intervals for Estimators Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations7 ANOVA dfSSMSF Significance F Regression Residual Total Coefficient s Standard Errort StatP-valueLower 95%Upper 95% Intercept X Variable