1. Regression and Correlation

2. Scatter Diagram
a plot of paired data to determine or show a relationship between two variables

3. When using x values to predict y values:
Call x the explanatory variable
Call y the response variable

4. Paired Data

5. Scatter Diagram

6. Questions Arising Can we find a relationship between x and y?
How strong is the relationship?

7. When there appears to be a linear relationship between x and y:
attempt to “fit” a line to the scatter diagram.

8. Linear Correlation
The general trend of the points seems to follow a straight line segment.

9. Linear Correlation

10. Non-Linear Correlation

11. No Linear Correlation

12. High Linear Correlation
Points lie close to a straight line.

13. High Linear Correlation

14. Moderate Linear Correlation

15. Low Linear Correlation

16. Perfect Linear Correlation

17. The Sample Correlation Coefficient, r
A measurement of the strength of the linear association between two variables
Also called the Pearson product-moment correlation coefficient

18. Positive Linear Correlation
High values of x are paired with high values of y and low values of x are paired with low values of y.

19. Negative Linear Correlation
High values of x are paired with low values of y and low values of x are paired with high values of y.

20. Little or No Linear Correlation
Both high and low values of x are sometimes paired with high values of y and sometimes with low values of y.

21. Positive Correlation y x

22. Negative Correlation y x

23. Little or No Linear Correlationy x

24. What type of correlation is expected?
Height and weight
Mileage on tires and remaining tread
IQ and height
Years of driving experience and insurance rates

25. Calculating the Correlation Coefficient, r

26. Linear correlation coefficient

27. If r = 0, scatter diagram might look like:y x

28. If r = +1, all points lie on the least squares liney x

29. If r = –1, all points lie on the least squares liney x

30. – 1 < r < 0 y x

31. 0 < r < 1 y x

32. To Compute r: Complete a table, with columns listing x, y, x2, y2, xy
Compute SSxy, SSx, and SSy
Use the formula:

33. Find the Correlation Coefficient

34. Calculations:

35. The Correlation Coefficient,

36. Warning
The correlation coefficient ( r) measures the strength of the relationship between two variables.
Just because two variables are related does not imply that there is a cause-and-effect relationship between them.

37. The Least Squares Line
The sum of the squares of the vertical distances from the points to the line is made as small as possible.

38. Least Squares Criterion
The sum of the squares of the vertical distances from the points to the line is made as small as possible.

39. Equation of the Least Squares Line
= a + bx
a = the y-intercept
b = the slope

40. Finding the slope

41. Finding the y-intercept

42. A relationship between correlation coefficient, r, and the slope, b, of the least squares line:

43. Find the Least Squares Line

44. Finding the slope

45. Finding the y-intercept

46. The equation of the least squares line is:
= a + bx
= x

47. Graphing the least squares line
Using two values in the range of x, compute two corresponding y values.
Plot these points.
Join the points with a straight line.

48. The following point will always be on the least squares line:

49. Graphing y = x
Use (8.3, 16.9)
(average of the x’s, the average of the y’s)
Try x = 5.
Compute y: y = (5)= 11.3

50. Sketching the Line Using the Points (8.3, 16.9) and (5, 11.3)

51. Using the Equation of the Least Squares Line to Make Predictions
Choose a value for x (within the range of x values).
Substitute the selected x in the least squares equation.
Determine corresponding value of y.

52. Predict the time to make a trip of 14 miles
Equation of least squares line:
y = x
Substitute x = 14:
y = (14)
y = 26.6
According to the least squares equation, a trip of 14 miles would take 26.6 minutes.

53. Interpolation
Using the least squares line to predict values for x values that fall between the points in the scatter diagram

54. Prediction beyond the range of observations
Extrapolation
Prediction beyond the range of observations

55. A statistic related to r:
the coefficient of determination = r2

56. Coefficient of Determination
a measure of the proportion of the variation in y that is explained by the regression line using x as the predicting variable

57. Formula for Coefficient of Determination

58. Interpretation of r2
If r = , then what percent of the variation in minutes (y) is explained by the linear relationship with x, miles traveled?
What percent is explained by other causes?

59. Interpretation of r2 If r = 0.9753643, then r2 = .9513355
Approximately 95 percent of the variation in minutes (y) is explained by the linear relationship with x, miles traveled.
Less than five percent is explained by other causes.

60. Testing the Correlation Coefficient
Determining whether a value of the sample correlation coefficient, r, is far enough from zero to indicate correlation in the population.

61. The Population Correlation Coefficient
 = Greek letter “rho”

62. H0: x and y are not correlated, so  = 0.
Hypotheses to Test Rho
Assume that both variables x and y are normally distributed.
To test if the (x, y) values are correlated in the population, set up the null hypothesis that they are not correlated:
H0: x and y are not correlated, so  = 0.

63. If you believe  is positive, use a right-tailed test. H1:  > 0

64. If you believe  is negative,
H0:  = 0
If you believe  is negative,
use a left-tailed test.
H1:  < 0

65. If you believe  is not equal to zero,
H0:  = 0
If you believe  is not equal to zero,
use a two-tailed test.
H1:   0

66. Convert r to a Student’s t Distribution

67. A researcher wishes to determine (at 5% level of significance) if there is a positive correlation between x, the number of hours per week a child watches television and y, the cholesterol measurement for the child. Assume that both x and y are normally distributed.

68. Correlation Between Hours of Television and Cholesterol
Suppose that a sample of x and y values for 25 children showed the correlation coefficient, r to be 0.42.
Use a right-tailed test.
The null hypothesis: H0:  = 0
The alternate hypothesis: H1:  > 0
 = 0.05

69. Convert the sample statistic r = 0.42 to t using n = 25

70. Find critical t value for right-tailed test with  = 0.05
Use Table 6.
d.f. = = 23.
t = 1.714
2.22 > 1.714
Reject the null hypothesis.
Conclude that there is a positive correlation between the variables.

71. P Value Approach Use Table 6 in Appendix II, d.f. = 23
Our t value =2.22 is between and
This gives P between and
Since we would reject H0 for any   P, we reject H0 for  = 0.05.
We conclude that there is a positive correlation between the variables.

72. Conclusion
We conclude that there is a positive correlation between the number of hours spent watching television and the cholesterol measurement.

73. Note
Even though a significance test indicates the existence of a correlation between x and y in the population, it does not signify a cause-and-effect relationship.

74. Standard Error of Estimate
A method for measuring the spread of a set of points about the least squares line

75. The Residual
y – yp
= difference between the y value of a data point on the scatter diagram
and
the y value of the point on the least-squares line with the same x value

76. The Residual
difference between the y value of a data point and the y value of the point on the line with the same x value

77. Standard Error of Estimate

78. Standard Error of Estimate
The number of points must be greater that or equal to three.
If n = 2, the line is a perfect fit and there is no need to compute Se.
The nearer the points are to the least squares line, the smaller Se will be.
The larger Se is, the more scattered the points are.

79. Calculating Formula for Se

80. Calculating Formula for Se
Use caution in rounding.
Uses quantities also used to determine the least squares line.

81. Find Se

82. Finding the Standard Error of Estimate

83. Finding Se

84. Finding Se

85. Finding Se

86. Confidence Interval for y
Least squares line gives a predicted y value, yp, for a given x.
Least squares line estimates the true y value.
True y value is given by: y =  + x + 
 = y intercept
 = slope
 = random error

87. For a Specific x, a c Confidence Interval for y

88. For a Specific x, a c Confidence Interval for y

89. For a Specific x, a c Confidence Interval for y

90. For a Specific x, a c Confidence Interval for y

91. For a Specific x, a c Confidence Interval for y

92. Find a 95% confidence interval for the number of minutes for a trip of eight miles

93. The least squares line and prediction, :
= a + bx
= x
For x = 8, = (8) = 16.4

94. For x = 8, a c Confidence Interval for y

95. Finding E

96. Finding E

97. For x = 8, a 95% Confidence Interval for y

98. we are 95% sure that the trip will take between 11.3 and 21.5 minutes.
For x = 8 miles
we are 95% sure that the trip will take between 11.3 and 21.5 minutes.

99. Confidence Interval for y at a Specific x Uses:
The values of E increase as x is chosen further from the mean of the x values.
Confidence interval for y becomes wider for values of x further from the mean.

100. Try not to use the least squares line to predict y values for x values beyond the data extremes of the sample x distribution.

101. Testing the Slope
 = slope of the population based least squares line.
b = slope of the sample based least squares line.

102. To test the slope: Use H0: The population slope = zero,  = 0
H1 may be  > 0 or  < 0 or   0
Convert b to a Student’s t distribution:

103. Standard Error for b

104. Test the Slope

105. We have: The least squares line: y = 2.8 + 1.7x Slope = b = 1.7
Se  1.85
SSx  115.4
We suspect the slope  is positive.

106. Hypothesis Test H0:  = 0 H1:  > 0 Use 1% level of significance.
Convert the sample test statistic b = 1.7 to a t value.

107. t value
For d.f. = = 5 and  ´ = 0.01, critical value of t =
From Table 6, we note that P <
Since we would reject H0 for any   P, we reject H0 for  = 0.01.
We conclude that  is positive.

108. Confidence Intervals for the Slope 
We wish to estimate the slope of the population-based least squares line.

109. Confidence Intervals for the Slope
 = slope of the population based least squares line.
b = slope of the sample based least squares line.

110. To determine a confidence interval for  :
Convert b to a Student’s t distribution:

111. A c Confidence Interval for 

112. b – E <  < b + E

113. Find a 95% Confidence Interval for 

114. We have: The least squares line: y = 2.8 + 1.7x Slope = b = 1.7
Se  1.85
SSx  115.4
c = 95% = 0.95
d.f. = n - 2 = = 5
t0.75 = 2.571

115. b – E <  < b + E

116. Conclusion:
We are 95% confident that the true slope of the regression line is between 1.26 and 2.14.

117. Multiple Regression
More than a single random variable is used in the computation of predictions.

118. Common formula for linear relationships among more than two variables:
y = b0 + b1x1 + b2x2 + … + bkxk
y = response variable
x1 , x2 , … , xk = explanatory variables, variables on which predictions will be based
b0 , b1, b2, … , bk = coefficients obtained from least squares criterion

119. Multiple regression models are analyzed by computer programs such as:
Minitab
Excel
SPSS

120. A collection of random variables with a number of properties
Regression Model
A collection of random variables with a number of properties

121. Properties of a Regression Model
One variable is identified as response variable. All other variables are explanatory variables.
For any application there will be a collection of numerical values for each variable.

122. Properties of a Regression Model
Using numerical data values, least squares criterion the least-squares equation (regression equation) can be constructed.
Usually includes a measure of “goodness of fit” of the regression equation to the data values.

123. Properties of a Regression Model
Allows us to supply given values of explanatory variables in order to predict corresponding value of the response variable.
A c% confidence interval can be constructed for least-squares criterion.

124. “Goodness of Fit” of Least-Squares Regression Equation
May be measured by coefficient of multiple determination, r2