Introduction to Regression

Figure 17.7 (p. 564) Predicting the variance in academic performance from IQ and SAT scores. The overlap between IQ and academic performance indicates that 40% of the variance in academic performance can be predicted from IQ scores. Similarly, 30% of the variance in academic performance can be predicted from SAT scores. However, IQ and SAT also overlap, so that SAT scores contribute an additional prediction of only 10% beyond what is already predicted by IQ. Predicting the variance in academic performance from IQ and SAT scores. The overlap between IQ and academic performance indicates that 40% of the variance in academic performance can be predicted from IQ scores. Similarly, 30% of the variance in academic performance can be predicted from SAT scores. However, IQ and SAT also overlap, so that SAT scores contribute an additional prediction of only 10% beyond what is already predicted by IQ.

1. We are investigating only linear relationships. 2. For each x value, y is a random variable having a normal (bell-shaped) distribution. All of these y distributions have the same variance. Also, for a given value of x, the distribution of y-values has a mean that lies on the regression line. (Results are not seriously affected if departures from normal distributions and equal variances are not too extreme.)

The regression equation is obtained by first finding the error (or distance) between the actual data points and the predicted values on the line. Each error is then squared to make the values consistently positive. The goal of regression is to find the equation that produces the smallest total amount of squared error. Thus, the regression equation produces the “best fitting” line for the data points.

The regression equation is defined by the slope constant, b = SP/SSX, and the Y-intercept, a = MY  bMX, producing a linear equation of the form Y = bX + a. The equation can be used to compute a predicted Y value for each of the X values in the data.

The simple concept is that each new variable provides more information and allows for more accurate predictions. Having two predictors in the equation will produce more accurate predictions (less error and smaller residuals) than can be obtained using either predictor by itself.

Figure 17.1 (p. 550) Hypothetical data showing the relationship between SAT scores and GPA with a regression line drawn through the data points. The regression line defines a precise, one-to-one relationship between each X value (SAT score) and its corresponding Y value (GPA).

Figure 17.2 (p. 551) Relationship between total cost and number of hours playing tennis. The tennis club charges a $25 membership fee plus $5 per hour. The relationship is described by a linear equation: Total cost = $5 (number of hours) + $25 Y = bX + a

Figure 17.3 (p. 553) The distance between the actual data point (Y) and the predicted point on the line (Ŷ) is defined as Y – Ŷ. The goal of regression is to find the equation for the line that minimizes these distances.

Figure 17. 4 (p. 555) The scatterplot for the data in Example 17 Figure 17.4 (p. 555) The scatterplot for the data in Example 17.1 is shown with the best-fitting straight line. The predicted Y values (Ŷ) are on the regression line. Unless the correlation is perfect (+1.00 or – 1.00), there will be some error between the actual Y values and the predicted Y values. The larger the correlation is, the less the error will be.

Figure 17.5 (p. 558) (a) Scatter plot showing data points that perfectly fit the regression equation Ŷ = 1.6X – 2. Note that the correlation is r = 1.00. (b) Scatter plot for the data from Example 17.1. Notice that there is error between the actual data points and the predicted Y values of the regression line.

Figure 17.6 (p. 563) The partitioning of SS and df for analysis of regression. The variability in the original Y scores (both SSY and dfY) is partitioned into two components: (a) the variability that is explained by the regression equation, and (b) the residual variability.

Table 17.1 (p. 563) A summary table showing the results from an analysis of regression.

Figure 17.7 (p. 564) Predicting the variance in academic performance from IQ and SAT scores. The overlap between IQ and academic performance indicates that 40% of the variance in academic performance can be predicted from IQ scores. Similarly, 30% of the variance in academic performance can be predicted from SAT scores. However, IQ and SAT also overlap, so that SAT scores contribute an additional prediction of only 10% beyond what is already predicted by IQ. Predicting the variance in academic performance from IQ and SAT scores. The overlap between IQ and academic performance indicates that 40% of the variance in academic performance can be predicted from IQ scores. Similarly, 30% of the variance in academic performance can be predicted from SAT scores. However, IQ and SAT also overlap, so that SAT scores contribute an additional prediction of only 10% beyond what is already predicted by IQ.

Table 17.2 (p. 566) Hypothetical data consisting of three scores for each person. Two of the scores, X1 and X2, are used to predict the Y score for each individual.

Table 17.3 (p. 567) The predicted Y values and the residuals for the data in Table 17.2 The predicted Y values were obtained using the values of X1 and X2 in the multiple-regression equation for each individual.

A significant F-ratio indicates that the regression equation predicts a significant portion (more than just chance) of the variance in the Y scores. What are the weights of each of the IVs? (look at the betas)