Statistics 303 Chapter 10 Least Squares Regression Analysis

Regression Review (you may want to look at Chapter 2) –We have seen that we can examine the relationship between two numeric variables Negative RelationshipPositive Relationship

Regression –______________ or r: measures the direction and strength of the linear relationship between two numeric variables r =0.43 r = r = 0.04 r = 0.21 r = 0.76

Regression Relationships between two ____________ variables –General form y = a + bx a is the intercept b is the slope –The statistical model for simple linear regression for the population is: y i =  0 +  1 x i +  i –When we estimate this model using the data (finding the least-squares line) we obtain the estimate:

Regression Relationships between two numeric variables Horsepower = *weight (Equation of the line.) Slope: y-value or response (horsepower) when line crosses the y-axis. This can be computed as b 1 = r(s y /s x ). Intercept: increase in response for a unit increase in explanatory variable. This can be computed as So if weight increases by one pound, horsepower increases by 0.04 units (on average).

Regression: Residuals The _______________ (e i ) are the differences between each point and the estimated line: Example: Relationship between energy consumption and household income (in thousands of dollars) The residuals are these differences between the points and the line. This says that for a given point, Data = Fit + Residual The least squares regression line is the one that minimizes these distances (squared).

Regression: Least Squares Given the previous example, there are many, many lines that could be used to fit the data: We want the one that best fits the data. This is the one that minimizes the squared residuals as in the last slide. We use the computer to calculate the equation of this line, but we want to be able to interpret the output of the computer.

Regression: Computer Output Computer Output: Equation of the line: energy = (income) Standard error for the slope (used in calculating the t-statistic and for confidence intervals of the slope)

Regression: Computer Output Computer Output: We can also get the confidence interval for the slope. This gives us an idea of the range of values the true slope are most likely to be. This confidence interval is calculated using the following formula:

Regression: Confidence Intervals for the Mean Response We can obtain a confidence interval for the mean response  y for a specific x* we are interested in. We can also obtain ________________________ (or limits) for all the possible confidence intervals. Example: Skinfold thickness to predict Body Density Confidence Bands Confidence Intervals for a specific x*.

Regression: Prediction Interval for a Future Observation We can also obtain a _____________________ for a future observation for a specific x* we are interested in. There are corresponding prediction bands (or limits) giving all possible prediction intervals Example: Skinfold thickness to predict Body Density Prediction Bands Prediction Intervals for a specific x*.

Regression: Simple Linear The techniques that have been described are useful when there is a linear relationship and the variance is relatively constant. Often we plot the residuals to see if this is the case. Scatterplot Residual Plot: Skinfold and Density Beginning and Current Salary Vehicle Weight and m.p.g.

Regression: Example “An educational foundation awards scholarships to high-school graduates to help them with expenses during their first year at a major university. The foundation would like to consider a student for a scholarship only if that student will earn a grade point average (GPA) of at least 2.80 (on a 4.0-point scale) the first year at the university. Since the scholarship is awarded before the student enters the university, the first-year GPA must be predicted. Each applicant for a scholarship must take an achievement test, the result of which is used to predict his or her first-year GPA. The director of the foundation has access to the records of the 2,000 students who applied for scholarships during the past five years and completed their first year of college. A simple random sample of 60 students was selected from these records.” (from Graybill, Iyer and Burdick, Applied Statistics, 1998).

Regression: Example Data, plot: GPA Test ScoreGPA Test ScoreGPA Test Score

Regression: Example We examine a plot of the residuals: There are no obvious patterns: we can use the simple linear regression techniques.

Regression: Example Computer Output: There is statistically significant evidence that the true slope is different from 0. The equation of the least squares regression line is GPA = (SCORE)

Regression: Example Suppose we now want to predict the GPA for students with scores of 0.67, 0.73, 0.74, 0.79, and 0.83 on the test. The prediction bands (or limits) give a general picture.

Regression: Example Specific estimates with prediction intervals are seen as follows: LowerUpper PredictedPrediction scoreGPALimit