1. Correlation and Regression
Chapter 9
Correlation and Regression
Larson/Farber 4th ed.

2. Chapter Outline 9.1 Correlation 9.2 Linear Regression
9.3 Measures of Regression and Prediction Intervals
9.4 Multiple Regression
Larson/Farber 4th ed.

3. Section 9.1
Correlation
Larson/Farber 4th ed.

4. Section 9.1 Objectives
Introduce linear correlation, independent and dependent variables, and the types of correlation
Find a correlation coefficient
Test a population correlation coefficient ρ using a table
Perform a hypothesis test for a population correlation coefficient ρ
Distinguish between correlation and causation
Larson/Farber 4th ed.

5. Correlation Correlation A relationship between two variables.
The data can be represented by ordered pairs (x, y)
x is the independent (or explanatory) variable
y is the dependent (or response) variable
Larson/Farber 4th ed.

6. Correlation
A scatter plot can be used to determine whether a linear (straight line) correlation exists between two variables. x 2 4 –2
– 4 y 6
Example: x 1 2 3 4 5 y
– 4
– 2
– 1
Larson/Farber 4th ed.

7. Types of Correlation As x increases, y tends to decrease.
As x increases, y tends to increase.
Negative Linear Correlation
Positive Linear Correlation x y x y
No Correlation
Nonlinear Correlation
Larson/Farber 4th ed.

8. Example: Constructing a Scatter Plot
A marketing manager conducted a study to determine whether there is a linear relationship between money spent on advertising and company sales. The data are shown in the table. Display the data in a scatter plot and determine whether there appears to be a positive or negative linear correlation or no linear correlation.
Advertising expenses, ($1000), x
Company sales ($1000), y
2.4
225
1.6
184
2.0
220
2.6
240
1.4
180
186
2.2
215
Larson/Farber 4th ed.

9. Solution: Constructing a Scatter Plotx y
Advertising expenses
(in thousands of dollars)
Company sales
Appears to be a positive linear correlation. As the advertising expenses increase, the sales tend to increase.
Larson/Farber 4th ed.

10. Example: Constructing a Scatter Plot Using Technology
Old Faithful, located in Yellowstone National Park, is the world’s most famous geyser. The duration (in minutes) of several of Old Faithful’s eruptions and the times (in minutes) until the next eruption are shown in the table. Using a TI-83/84, display the data in a scatter plot. Determine the type of correlation.
Duration x
Time, y
1.8 56
3.78 79
1.82 58
3.83 85
1.9 62
3.88 80
1.93
4.1 89
1.98 57
4.27 90
2.05
4.3
2.13 60
4.43
2.3
4.47 86
2.37 61
4.53
2.82 73
4.55
3.13 76
4.6 92
3.27 77
4.63 91
3.65
Larson/Farber 4th ed.

11. Solution: Constructing a Scatter Plot Using Technology
Enter the x-values into list L1 and the y-values into list L2.
Use Stat Plot to construct the scatter plot.
STAT > Edit…
STATPLOT 1 5 50
100
From the scatter plot, it appears that the variables have a positive linear correlation.
Larson/Farber 4th ed.

12. Correlation Coefficient
A measure of the strength and the direction of a linear relationship between two variables.
The symbol r represents the sample correlation coefficient.
A formula for r is
The population correlation coefficient is represented by ρ (rho).
n is the number of data pairs
Larson/Farber 4th ed.

13. Correlation Coefficient
The range of the correlation coefficient is -1 to 1. -1 1
If r = -1 there is a perfect negative correlation
If r is close to 0 there is no linear correlation
If r = 1 there is a perfect positive correlation
Larson/Farber 4th ed.

14. Linear Correlation Strong negative correlationx y x y
r = 0.91
r = 0.88
Strong negative correlation
Strong positive correlation x y x y
r = 0.42
r = 0.07
Weak positive correlation
Nonlinear Correlation
Larson/Farber 4th ed.

15. Calculating a Correlation Coefficient
In Words In Symbols
Find the sum of the x-values.
Find the sum of the y-values.
Multiply each x-value by its corresponding y-value and find the sum.
Larson/Farber 4th ed.

16. Calculating a Correlation Coefficient
In Words In Symbols
Square each x-value and find the sum.
Square each y-value and find the sum.
Use these five sums to calculate the correlation coefficient.
Larson/Farber 4th ed.

17. Example: Finding the Correlation Coefficient
Calculate the correlation coefficient for the advertising expenditures and company sales data. What can you conclude?
Advertising expenses, ($1000), x
Company sales ($1000), y
2.4
225
1.6
184
2.0
220
2.6
240
1.4
180
186
2.2
215
Larson/Farber 4th ed.

18. Solution: Finding the Correlation Coefficientx y xy x2 y2
2.4
225
1.6
184
2.0
220
2.6
240
1.4
180
186
2.2
215
540
5.76
50,625
294.4
2.56
33,856
440 4
48,400
624
6.76
57,600
252
1.96
32,400
294.4
2.56
33,856
372 4
34,596
473
4.84
46,225
Σx = 15.8
Σy = 1634
Σxy =
Σx2 = 32.44
Σy2 = 337,558
Larson/Farber 4th ed.

19. Solution: Finding the Correlation Coefficient
Σx = 15.8
Σy = 1634
Σxy =
Σx2 = 32.44
Σy2 = 337,558
r ≈ suggests a strong positive linear correlation. As the amount spent on advertising increases, the company sales also increase.
Larson/Farber 4th ed.

20. Example: Using Technology to Find a Correlation Coefficient
Use a technology tool to calculate the correlation coefficient for the Old Faithful data. What can you conclude?
Duration x
Time, y
1.8 56
3.78 79
1.82 58
3.83 85
1.9 62
3.88 80
1.93
4.1 89
1.98 57
4.27 90
2.05
4.3
2.13 60
4.43
2.3
4.47 86
2.37 61
4.53
2.82 73
4.55
3.13 76
4.6 92
3.27 77
4.63 91
3.65
Larson/Farber 4th ed.

21. Solution: Using Technology to Find a Correlation Coefficient
To calculate r, you must first enter the DiagnosticOn command found in the Catalog menu
STAT > Calc
r ≈ suggests a strong positive correlation.
Larson/Farber 4th ed.

22. Using a Table to Test a Population Correlation Coefficient ρ
Once the sample correlation coefficient r has been calculated, we need to determine whether there is enough evidence to decide that the population correlation coefficient ρ is significant at a specified level of significance.
Use Table 11 in Appendix B.
If |r| is greater than the critical value, there is enough evidence to decide that the correlation coefficient ρ is significant.
Larson/Farber 4th ed.

23. Using a Table to Test a Population Correlation Coefficient ρ
Determine whether ρ is significant for five pairs of data (n = 5) at a level of significance of α = 0.01.
If |r| > 0.959, the correlation is significant. Otherwise, there is not enough evidence to conclude that the correlation is significant.
level of significance
Number of pairs of data in sample
Larson/Farber 4th ed.

24. Using a Table to Test a Population Correlation Coefficient ρ
In Words In Symbols
Determine the number of pairs of data in the sample.
Specify the level of significance.
Find the critical value.
Determine n.
Identify .
Use Table 11 in Appendix B.
Larson/Farber 4th ed.

25. Using a Table to Test a Population Correlation Coefficient ρ
In Words In Symbols
Decide if the correlation is significant.
Interpret the decision in the context of the original claim.
If |r| > critical value, the correlation is significant. Otherwise, there is not enough evidence to support that the correlation is significant.
Larson/Farber 4th ed.

26. Example: Using a Table to Test a Population Correlation Coefficient ρ
Using the Old Faithful data, you used 25 pairs of data to find r ≈ Is the correlation coefficient significant? Use α = 0.05.
Duration x
Time, y
1.8 56
3.78 79
1.82 58
3.83 85
1.9 62
3.88 80
1.93
4.1 89
1.98 57
4.27 90
2.05
4.3
2.13 60
4.43
2.3
4.47 86
2.37 61
4.53
2.82 73
4.55
3.13 76
4.6 92
3.27 77
4.63 91
3.65
Larson/Farber 4th ed.

27. Solution: Using a Table to Test a Population Correlation Coefficient ρ
There is enough evidence at the 5% level of significance to conclude that there is a significant linear correlation between the duration of Old Faithful’s eruptions and the time between eruptions.
Larson/Farber 4th ed.

28. Hypothesis Testing for a Population Correlation Coefficient ρ
A hypothesis test can also be used to determine whether the sample correlation coefficient r provides enough evidence to conclude that the population correlation coefficient ρ is significant at a specified level of significance.
A hypothesis test can be one-tailed or two-tailed.
Larson/Farber 4th ed.

29. Hypothesis Testing for a Population Correlation Coefficient ρ
Left-tailed test
Right-tailed test
Two-tailed test
H0: ρ  0 (no significant negative correlation) Ha: ρ < 0 (significant negative correlation)
H0: ρ  0 (no significant positive correlation) Ha: ρ > 0 (significant positive correlation)
H0: ρ = 0 (no significant correlation) Ha: ρ  0 (significant correlation)
Larson/Farber 4th ed.

30. The t-Test for the Correlation Coefficient
Can be used to test whether the correlation between two variables is significant.
The test statistic is r
The standardized test statistic
follows a t-distribution with d.f. = n – 2.
In this text, only two-tailed hypothesis tests for ρ are considered.
Larson/Farber 4th ed.

31. Using the t-Test for ρ In Words In Symbols
State the null and alternative hypothesis.
Specify the level of significance.
Identify the degrees of freedom.
Determine the critical value(s) and rejection region(s).
State H0 and Ha.
Identify .
d.f. = n – 2.
Use Table 5 in Appendix B.
Larson/Farber 4th ed.

32. Using the t-Test for ρ In Words In Symbols
Find the standardized test statistic.
Make a decision to reject or fail to reject the null hypothesis.
Interpret the decision in the context of the original claim.
If t is in the rejection region, reject H0. Otherwise fail to reject H0.
Larson/Farber 4th ed.

33. Example: t-Test for a Correlation Coefficient
Previously you calculated r ≈ Test the significance of this correlation coefficient. Use α = 0.05.
Advertising expenses, ($1000), x
Company sales ($1000), y
2.4
225
1.6
184
2.0
220
2.6
240
1.4
180
186
2.2
215
Larson/Farber 4th ed.

34. Solution: t-Test for a Correlation Coefficient
H0:
Ha:
 
d.f. =
Rejection Region:
ρ = 0
ρ ≠ 0
Test Statistic:
0.05
8 – 2 = 6
Decision:
Reject H0
At the 5% level of significance, there is enough evidence to conclude that there is a significant linear correlation between advertising expenses and company sales. t
-2.447
0.025
2.447
-2.447
2.447
5.478
Larson/Farber 4th ed.

35. Correlation and Causation
The fact that two variables are strongly correlated does not in itself imply a cause-and-effect relationship between the variables.
If there is a significant correlation between two variables, you should consider the following possibilities.
Is there a direct cause-and-effect relationship between the variables?
Does x cause y?
Larson/Farber 4th ed.

36. Correlation and Causation
Is there a reverse cause-and-effect relationship between the variables?
Does y cause x?
Is it possible that the relationship between the variables can be caused by a third variable or by a combination of several other variables?
Is it possible that the relationship between two variables may be a coincidence?
Larson/Farber 4th ed.

37. Section 9.1 Summary
Introduced linear correlation, independent and dependent variables and the types of correlation
Found a correlation coefficient
Tested a population correlation coefficient ρ using a table
Performed a hypothesis test for a population correlation coefficient ρ
Distinguished between correlation and causation
Larson/Farber 4th ed.

38. Section 9.2
Linear Regression
Larson/Farber 4th ed.

39. Section 9.2 Objectives Find the equation of a regression line
Predict y-values using a regression equation
Larson/Farber 4th ed.

40. Regression lines
After verifying that the linear correlation between two variables is significant, next we determine the equation of the line that best models the data (regression line).
Can be used to predict the value of y for a given value of x. x y
Larson/Farber 4th ed.

41. Residuals
Residual
The difference between the observed y-value and the predicted y-value for a given x-value on the line.
For a given x-value,
di = (observed y-value) – (predicted y-value) x y
}d1
}d2
d3{
d4{
}d5
d6{
Predicted y-value
Observed y-value
Larson/Farber 4th ed.

42. Regression Line Regression line (line of best fit)
The line for which the sum of the squares of the residuals is a minimum.
The equation of a regression line for an independent variable x and a dependent variable y is
ŷ = mx + b
y-intercept
Predicted y-value for a given x-value
Slope
Larson/Farber 4th ed.

43. The Equation of a Regression Line
ŷ = mx + b where
is the mean of the y-values in the data
is the mean of the x-values in the data
The regression line always passes through the point
Larson/Farber 4th ed.

44. Example: Finding the Equation of a Regression Line
Find the equation of the regression line for the advertising expenditures and company sales data.
Advertising expenses, ($1000), x
Company sales ($1000), y
2.4
225
1.6
184
2.0
220
2.6
240
1.4
180
186
2.2
215
Larson/Farber 4th ed.

45. Solution: Finding the Equation of a Regression Line
Recall from section 9.1: x y xy x2 y2
2.4
225
1.6
184
2.0
220
2.6
240
1.4
180
186
2.2
215
540
5.76
50,625
294.4
2.56
33,856
440 4
48,400
624
6.76
57,600
252
1.96
32,400
294.4
2.56
33,856
372 4
34,596
473
4.84
46,225
Σx = 15.8
Σy = 1634
Σxy =
Σx2 = 32.44
Σy2 = 337,558
Larson/Farber 4th ed.

46. Solution: Finding the Equation of a Regression Line
Σx = 15.8
Σy = 1634
Σxy =
Σx2 = 32.44
Σy2 = 337,558
Equation of the regression line
Larson/Farber 4th ed.

47. Solution: Finding the Equation of a Regression Line
To sketch the regression line, use any two x-values within the range of the data and calculate the corresponding y-values from the regression line. x
Advertising expenses
(in thousands of dollars)
Company sales y
Larson/Farber 4th ed.

48. Example: Using Technology to Find a Regression Equation
Use a technology tool to find the equation of the regression line for the Old Faithful data.
Duration x
Time, y
1.8 56
3.78 79
1.82 58
3.83 85
1.9 62
3.88 80
1.93
4.1 89
1.98 57
4.27 90
2.05
4.3
2.13 60
4.43
2.3
4.47 86
2.37 61
4.53
2.82 73
4.55
3.13 76
4.6 92
3.27 77
4.63 91
3.65
Larson/Farber 4th ed.

49. Solution: Using Technology to Find a Regression Equation
100 50 1 5
Larson/Farber 4th ed.

50. Example: Predicting y-Values Using Regression Equations
The regression equation for the advertising expenses (in thousands of dollars) and company sales (in thousands of dollars) data is ŷ = x Use this equation to predict the expected company sales for the following advertising expenses. (Recall from section 9.1 that x and y have a significant linear correlation.)
1.5 thousand dollars
1.8 thousand dollars
2.5 thousand dollars
Larson/Farber 4th ed.

51. Solution: Predicting y-Values Using Regression Equations
ŷ = x
1.5 thousand dollars
ŷ =50.729(1.5) ≈
When the advertising expenses are $1500, the company sales are about $180,155.
1.8 thousand dollars
ŷ =50.729(1.8) ≈
When the advertising expenses are $1800, the company sales are about $195,373.
Larson/Farber 4th ed.

52. Solution: Predicting y-Values Using Regression Equations
2.5 thousand dollars
ŷ =50.729(2.5) ≈
When the advertising expenses are $2500, the company sales are about $230,884.
Prediction values are meaningful only for x-values in (or close to) the range of the data. The x-values in the original data set range from 1.4 to 2.6. So, it would
not be appropriate to use the regression line to predict
company sales for advertising expenditures such as 0.5 ($500) or 5.0 ($5000).
Larson/Farber 4th ed.

53. Section 9.2 Summary Found the equation of a regression line
Predicted y-values using a regression equation
Larson/Farber 4th ed.

54. Measures of Regression and Prediction Intervals
Section 9.3
Measures of Regression and Prediction Intervals
Larson/Farber 4th ed.

55. Section 9.3 Objectives
Interpret the three types of variation about a regression line
Find and interpret the coefficient of determination
Find and interpret the standard error of the estimate for a regression line
Construct and interpret a prediction interval for y
Larson/Farber 4th ed.

56. Variation About a Regression Line
Three types of variation about a regression line
Total variation
Explained variation
Unexplained variation
To find the total variation, you must first calculate
The total deviation
The explained deviation
The unexplained deviation
Larson/Farber 4th ed.

57. Variation About a Regression Line
Total Deviation =
Explained Deviation =
Unexplained Deviation = y
(xi, yi)
Unexplained deviation
Total deviation
Explained deviation
(xi, ŷi)
(xi, yi) x
Larson/Farber 4th ed.

58. Variation About a Regression Line
Total variation
The sum of the squares of the differences between the y-value of each ordered pair and the mean of y.
Explained variation
The sum of the squares of the differences between each predicted y-value and the mean of y.
Total variation =
Explained variation =
Larson/Farber 4th ed.

59. Variation About a Regression Line
Unexplained variation
The sum of the squares of the differences between the y-value of each ordered pair and each corresponding predicted y-value.
Unexplained variation =
The sum of the explained and unexplained variation is equal to the total variation.
Total variation = Explained variation + Unexplained variation
Larson/Farber 4th ed.

60. Coefficient of Determination
The ratio of the explained variation to the total variation.
Denoted by r2
Larson/Farber 4th ed.

61. Example: Coefficient of Determination
The correlation coefficient for the advertising expenses and company sales data as calculated in Section 9.1 is r ≈ Find the coefficient of determination. What does this tell you about the explained variation of the data about the regression line? About the unexplained variation?
Solution:
About 83.4% of the variation in the company sales can be explained by the variation in the advertising expenditures. About 16.9% of the variation is unexplained.
Larson/Farber 4th ed.

62. The Standard Error of Estimate
The standard deviation of the observed yi -values about the predicted ŷ-value for a given xi -value.
Denoted by se.
The closer the observed y-values are to the predicted y-values, the smaller the standard error of estimate will be.
n is the number of ordered pairs in the data set
Larson/Farber 4th ed.

63. The Standard Error of Estimate
In Words In Symbols
Make a table that includes the column heading shown.
Use the regression equation to calculate the predicted y-values.
Calculate the sum of the squares of the differences between each observed y-value and the corresponding predicted y-value.
Find the standard error of estimate.
Larson/Farber 4th ed.

64. Example: Standard Error of Estimate
The regression equation for the advertising expenses and company sales data as calculated in section 9.2 is ŷ = x Find the standard error of estimate.
Solution:
Use a table to calculate the sum of the squared differences of each observed y-value and the corresponding predicted y-value.
Larson/Farber 4th ed.

65. Solution: Standard Error of Estimatex y
ŷ i
(yi – ŷ i)2
2.4
225
225.81
(225 – )2 =
1.6
184
185.23
(184 – )2 =
2.0
220
205.52
(220 – )2 =
2.6
240
235.96
(240 – )2 =
1.4
180
175.08
(180 – )2 =
186
(186 – )2 =
2.2
215
215.66
(215 – )2 =
Σ =
unexplained variation
Larson/Farber 4th ed.

66. Solution: Standard Error of Estimate
n = 8, Σ(yi – ŷ i)2 =
The standard error of estimate of the company sales for a specific advertising expense is about $10.29.
Larson/Farber 4th ed.

67. Prediction Intervals
Two variables have a bivariate normal distribution if for any fixed value of x, the corresponding values of y are normally distributed and for any fixed values of y, the corresponding x-values are normally distributed.
Larson/Farber 4th ed.

68. Prediction Intervals
A prediction interval can be constructed for the true value of y.
Given a linear regression equation ŷ = mx + b and x0, a specific value of x, a c-prediction interval for y is
ŷ – E < y < ŷ + E where
The point estimate is ŷ and the margin of error is E. The probability that the prediction interval contains y is c.
Larson/Farber 4th ed.

69. Constructing a Prediction Interval for y for a Specific Value of x
In Words In Symbols
Identify the number of ordered pairs in the data set n and the degrees of freedom.
Use the regression equation and the given x-value to find the point estimate ŷ.
Find the critical value tc that corresponds to the given level of confidence c.
d.f. = n – 2
Use Table 5 in Appendix B.
Larson/Farber 4th ed.

70. Constructing a Prediction Interval for y for a Specific Value of x
In Words In Symbols
Find the standard error of estimate se.
Find the margin of error E.
Find the left and right endpoints and form the prediction interval.
Left endpoint: ŷ – E Right endpoint: ŷ + E
Interval: ŷ – E < y < ŷ + E
Larson/Farber 4th ed.

71. Example: Constructing a Prediction Interval
Construct a 95% prediction interval for the company sales when the advertising expenses are $2100. What can you conclude? Recall, n = 8, ŷ = x , se =
Solution:
Point estimate:
ŷ = (2.1) ≈
Critical value:
d.f. = n –2 = 8 – 2 = tc = 2.447
Larson/Farber 4th ed.

72. Solution: Constructing a Prediction Interval
Left Endpoint: ŷ – E
Right Endpoint: ŷ + E – ≈ ≈
< y <
You can be 95% confident that when advertising expenses are $2100, the company sales will be between $183,735 and $237,449.
Larson/Farber 4th ed.

73. Section 9.3 Summary
Interpreted the three types of variation about a regression line
Found and interpreted the coefficient of determination
Found and interpreted the standard error of the estimate for a regression line
Constructed and interpreted a prediction interval for y
Larson/Farber 4th ed.

74. Section 9.4
Multiple Regression
Larson/Farber 4th ed.

75. Section 9.4 Objectives
Use technology to find a multiple regression equation, the standard error of estimate and the coefficient of determination
Use a multiple regression equation to predict y-values
Larson/Farber 4th ed.

76. Multiple Regression Equation
In many instances, a better prediction can be found for a dependent (response) variable by using more than one independent (explanatory) variable.
For example, a more accurate prediction for the company sales discussed in previous sections might be made by considering the number of employees on the sales staff as well as the advertising expenses.
Larson/Farber 4th ed.

77. Multiple Regression Equation
ŷ = b + m1x1 + m2x2 + m3x3 + … + mkxk
x1, x2, x3,…, xk are independent variables
b is the y-intercept
y is the dependent variable
* Because the mathematics associated with this concept is complicated, technology is generally used to calculate the multiple regression equation.
Larson/Farber 4th ed.

78. Example: Finding a Multiple Regression Equation
A researcher wants to determine how employee salaries at a certain company are related to the length of employment, previous experience, and education. The researcher selects eight employees from the company and obtains the data shown on the next slide. Use Minitab to find a multiple regression equation that models the data.
Larson/Farber 4th ed.

79. Example: Finding a Multiple Regression Equation
Employee
Salary, y
Employment (yrs), x1
Experience (yrs), x2
Education (yrs), x3 A
57,310 10 2 16 B
57,380 5 6 C
54,135 3 1 12 D
56,985 14 E
58,715 8 F
60,620 20 G
59,200 4 18 H
60,320 17
Larson/Farber 4th ed.

80. Solution: Finding a Multiple Regression Equation
Enter the y-values in C1 and the x1-, x2-, and x3-values in C2, C3 and C4 respectively.
Select “Regression > Regression…” from the Stat menu.
Use the salaries as the response variable and the remaining data as the predictors.
Larson/Farber 4th ed.

81. Solution: Finding a Multiple Regression Equation
The regression equation is ŷ = 49, x x x3
Larson/Farber 4th ed.

82. Predicting y-Values
After finding the equation of the multiple regression line, you can use the equation to predict y-values over the range of the data.
To predict y-values, substitute the given value for each independent variable into the equation, then calculate ŷ.
Larson/Farber 4th ed.

83. Example: Predicting y-Values
Use the regression equation ŷ = 49, x x x3 to predict an employee’s salary given 12 years of current employment, 5 years of experience, and 16 years of education.
Solution:
ŷ = 49, (12) + 228(5) + 267(16)
= 59,544
The employee’s predicted salary is $59,544.
Larson/Farber 4th ed.

84. Section 9.4 Summary
Used technology to find a multiple regression equation, the standard error of estimate and the coefficient of determination
Used a multiple regression equation to predict y-values
Larson/Farber 4th ed.