Linear Regression

 The following is a scatterplot of total fat versus protein for 30 items on the Burger King menu:  The model won’t be perfect, regardless of the line we draw. Some points will be above the line and some will be below.

 The estimate made from a model is the predicted value (denoted as ).  The difference between the observed value and its associated predicted value is called the residual.  A negative residual means the predicted value’s too big (an overestimate).  A positive residual means the predicted value’s too small (an underestimate).

 Some residuals are positive, others are negative, and, on average, they cancel each other out.  So, we can’t assess how well the line fits by adding up all the residuals.  Similar to what we did with deviations, we square the residuals and add the squares.  The smaller the sum, the better the fit.  The line of best fit is the line for which the sum of the squared residuals is smallest.

 In Algebra a straight line can be written as:  In Statistics we use a slightly different notation:  We write to emphasize that the points that satisfy this equation are just our predicted values, not the actual data values.  The coefficient b 1 is the slope, which tells us how rapidly y changes with respect to x.  The coefficient b 0 is the intercept, which tells where the line hits (intercepts) the y -axis.

 In our model, we have a slope ( b 1 ): ◦ The slope is built from the correlation and the standard deviations: ◦ Our slope is always in units of y per unit of x.  In our model, we also have an intercept ( b 0 ). ◦ The intercept is built from the means and the slope: ◦ Our intercept is always in units of y.

 The regression line for the Burger King data fits the data well: ◦ The equation is ◦ The predicted fat content for a BK Broiler chicken sandwich is (30) = 35.9 grams of fat.

 The linear model assumes that the relationship between the two variables is a perfect straight line. The residuals are the part of the data that hasn’t been modeled. Residual = Data – Model Or, in symbols, Residuals help us to see whether the model makes sense.

 When a regression model is appropriate, nothing interesting should be left behind.  After we fit a regression model, we usually plot the residuals in the hope of finding…nothing.  The residuals for the BK menu regression look appropriately boring:

 The variation in the residuals is the key to assessing how well the model fits.  In the BK menu items example, total fat has a standard deviation of 16.4 grams. The standard deviation of the residuals is 9.2 grams.

 If the correlation were 1.0 and the model predicted the fat values perfectly, the residuals would all be zero and have no variation.  As it is, the correlation is 0.83—not perfection.  However, we did see that the model residuals had less variation than total fat alone.  We can determine how much of the variation is accounted for by the model and how much is left in the residuals.  The squared correlation, r 2, gives the fraction of the data’s variance accounted for by the model.

 For the BK model, r 2 = = 0.69, so 31% of the variability in total fat has been left in the residuals.  All regression analyses include this statistic, although by tradition, it is written R 2 (pronounced “R-squared”). An R 2 of 0 means that none of the variance in the data is in the model; all of it is still in the residuals.  When interpreting a regression model you need to Tell what R 2 means. ◦ In the BK example, according to our linear model, 69% of the variation in total fat is accounted for by variation in the protein content.

 R 2 is always between 0% and 100%. What makes a “good” R 2 value depends on the kind of data you are analyzing and on what you want to do with it.  Along with the slope and intercept for a regression, you should always report R 2 so that readers can judge for themselves how successful the regression is at fitting the data.

 Don’t fit a straight line to a nonlinear relationship.  Beware of extraordinary points ( y -values that stand off from the linear pattern or extreme x -values).  Don’t invert the regression. To swap the predictor-response roles of the variables, we must fit a new regression equation.  Don’t extrapolate beyond the data—the linear model may no longer hold outside of the range of the data.  Don’t infer that x causes y just because there is a good linear model for their relationship— association is not causation.  Don’t choose a model based on R 2 alone.

 Page 216 – 223  Problem # 1, 3, 5, 13, 15, 27, 39, 41 (omit part e and f), 45, 47.