1. Measures of relationship Dr. Omar Al Jadaan

2. Agenda Correlation – Need – meaning, simple linear regression – analysis – prediction

3. Correlation Correlation is a statistical measurement of the relationship between two variables. Possible correlations range from +1 to –1. A zero correlation indicates that there is no relationship between the variables. A correlation of –1 indicates a perfect negative correlation, meaning that as one variable goes up, the other goes down. A correlation of +1 indicates a perfect positive correlation, meaning that both variables move in the same direction together.

4. Correlation Why we need correlation? To discover the interaction patterns between the dependent variable and infer the mathematical model of the relation.

5. Linear regression simple linear regression is the least squares estimator of a linear regression model with a single predictor variable. In other words, simple linear regression fits a straight line through the set of n points in such a way that makes the sum of squared residuals of the model (that is, vertical distances between the points of the data set and the fitted line) as small as possible.least squares linear regression modelpredictor variableresiduals The adjective simple refers to the fact that this regression is one of the simplest in statistics. The fitted line has the slope equal to the correlation between y and x corrected by the ratio of standard deviations of these variables. The intercept of the fitted line is such that it passes through the center of mass (x, y) of the data points.correlation

6. The purpose of least-squares method is to find the equation of the straight line that fits the data in the sense of least squares. Assumption of regression: – Normality of errors (with zero mean of each value) – variation around the line of regression is constant for all the values of x (this means that the errors vary by the same amount for small x as for large x. – The errors are independent for all values of x. – The relationship between x, y is postulated to be linear.

7. The linear model y=β 0 +β 1 x+ε to calculate the estimate of β 0, β 1 we have to calculate

8. Example (study hours and score) studied hours xscored on test observed yx-mean(x)squary-mean(y)(x-mean(x))*(y-mean(y))predicted yy-predicted ysquar 1078-0.80.64-0.90.72 77.590910.40909130.167356 15834.217.644.117.22 85.77273-2.7727277.688014 875-2.87.84-3.910.92 74.318180.68181860.464877 777-3.814.44-1.97.22 72.681824.318182218.6467 13802.24.841.12.42 82.5-2.56.249998 15854.217.646.125.62 85.77273-0.7727270.597107 20959.284.6416.1148.12 93.954551.0454551.092976 1083-0.80.644.1-3.28 77.590915.409091329.25827 565-5.833.64-13.980.62 69.40909-4.4090919.44008 568-5.833.64-10.963.22 69.40909-1.409091.985536 mean10.878.9 4.312E-0685.59091SSE SSxx215.6 this might be zero this is the minimum value you can get Ssxy352.8 b11.636363636 b061.22727273

9. Exercise The following table shows the prominent product sales in millions for the years 1998- 2003. Assuming the trend continues in 2004, predict the sales in 2004 YearYear codedSales in millions 1998171.3 1999259.5 2000351.9 2001441.1 2002524.9 2003617.5

10. Solution Sales = 82.7-11.0 year coded Sales= 82.7-11.0 (7) = 5.7 million From the line equation we can conclude that each year we pass the sales decreases 11 million.

11. Inference about the slope of the regression line Purpose of the test is to determine whether the given value is reasonable for the slope of the population regression line (H 0 : β 1 =c). The test H 0 : β 1 =0 is a test to determine whether a straight line should be fit to data. If he null hypothesis is not rejected then the straight line does not model the relationship between x and y.

12. Assumptions: The regression model is y=β 0 +β 1 x+ε and β 1 is the slope of the model. To test the null hypothesis that β 1 equals some value, say c, we divide the difference (β 1 - c ) by the standard error of β 1

13. The following test H 0 : β 1 =0, H a : β 1  0 Student test t with n-2 degree of freedom, where SE(β 1 ) is the standard error of β 1

14. Example The following table shows the systolic blood pressure readings with weights for 10 newly diagnosed patients with high blood pressure. PatientSystolic (y)weight(x) (pound) 1145210 2155245 3160260 4155230 5130175 6140185 7135230 8165249 9150200 10130190 We would like to test that the systolic blood pressure increases one point for each pound that the patient increases

15. Solution CoefficientsStandard Errort StatP-value Intercept72.5689312519.45984043.7291637430.005794692 X Variable 10.3400693130.0887776053.8305754450.005013816 The regression equation is systolic = 72.56893 +0340069 weight The statistic is computed as follow. C=1, β 1 = 0.34007 and the standard error of β 1 = 0.08878

16. We calculate At α=0.05, the t values with 8 degree of freedom are  2.306. The data would refute the null hypothesis, Each additional pound would increase the systolic blood pressure by less than 1. The T value (3.83) shown in the table along with the two-tailed p-value (0.005) is for the null hypothesis H 0 : β 1 =0, H a : β 1  0

17. The coefficient of Correlation The coefficient correlation is used to measure the strength of the linear relationship between two random variables. A measure very much related to the slope of regression line is the Pearson correlation coefficient.

18. The value of r will be in the range of -1 to +1 If the point fall on the straight line with a positive slope then r=+1, If the point fall on the straight line with a negative slope then r=-1, If the point from the shotgun pattern, r=0.

19. Example The correlations for the systolic blood pressure with weight is as follow As you can see the linear relation is negative SystolicWeight Systolic1 Weight0.8044641

20. The coefficient of determination The coefficient of determination is used to measure the strength of the linear relationship between dependent variables. Assumptions : the analysis of variance (ANOVA) for simple linear regression may be represented as follows Sourced.f.Sum of squares Mean of squares F-value Explained variation 1SSRMSR=SSR/1F=MSR/MSE Unexplained variation N-2SSEMSE=SSE/(n-2) TotalN-1SS(total)

21. The symbol r2 is used to represent the ration SSR/SS(total) and is called the coefficient of determination. Which measures the proportion of variation in y that explained by x. coefficient of determination can be called explained variation, regression variation, unexplained variation, residual variation.

22. Example The following table explains the contraceptive prevalence (x) and the fertility rate (y) CountryContraceptive (x)Fertility (y) Thailand692.3 Costa Rica713.5 Turkey623.4 Mexico554 Zimbabwe465.4 Jordan355.5 Gana146 Pakistan135 Sudan104.8 Nigeria75.7

23. SUMMARY OUTPUT Regression Statistics Multiple R0.810919794 R Square0.657590913 Adjusted R Square0.614789777 Standard Error0.748909931 Observations10 ANOVA dfSSMSFSignificance F Regression18.617071323 15.363865920.004420408 Residual84.4869286770.560866085 Total913.104 CoefficientsStandard Errort StatP-valueLower 95%Upper 95%Lower 95.0% Upper 95.0% Intercept6.0157408660.44047637413.657351957.95434E-075.0000005277.0314812045.0000005277.031481 contraceptive (x)-0.03810840.009722332-3.9196767620.004420408-0.060528138-0.015688661-0.060528138-0.01569

24. The coefficient of determination is shown as R Square= 65.8% It may computed as The interpretation is that about 65.8% of the variation in fertility rates is explained by the variation in contraceptive prevalence

25. Using the model for estimation and prediction The estimated regression equation y=β 0 +β 1 x+ε can be used to predict the value of y for some value of x. also the same equation can be used to estimate the mean values of ys. Example – Suppose you would like to know the estimate systolic blood pressure of a patient weighted 250 pound. Simply substitute the of the weight in the equation systolic = 72.56893 +0340069*(250)=157.6

26. We would expect the prediction interval to be wider than the confidence interval, that is the interval estimate of the expected value of y will be narrower that the prediction interval for the same value of x and confidence interval.

27. A (1-α)100% prediction interval for an individual new value of y at x=x 0 is Where y ^ =b 0 +b 1 x, the t value is based on (n-2) degree of freedom and is referred as the estimated standard error of the regression model, n is the sample size, x 0 is the fixed value of x,

28. A (1-α)100% confidence interval for the mean value of y at x=x 0 is

29. Example The following table shows the results of an experiment conducted on 15 diabetic patients, the independent variable x was hemoglobin A1C value, taken after 3 months of taking the fasting blood glucose value each morning of the three months period and averaging the values. The later values was the dependent value y. We wish to set a 95% prediction interval for average glucose reading of a diabetic who has hemoglobin A1C value of 7.0 as well as 95% confidence interval for all diabetics with hemoglobin A1C value of 7.0.

30. Patientx, Hemoglobin y, average fasting blood sugar over 3 month period 16.1120 26.8146 36.5125 47.1135 57.4140 65.8115 78145 88.3147 98150 105.5110 1110160 127.7145 139155 1411170 155.5118

31. SUMMARY OUTPUT Regression Statistics Multiple R0.94747 R Square0.897699 Adjusted R Square0.88983 Standard Error5.8809 Observations15 ANOVA dfSSMSFSignificance F Regression13945.328514 114.07633668.33726E-08 Residual13449.604819234.58498609 Total144394.933333 Coefficient sStandard Errort StatP-valueLower 95%Upper 95%Lower 95.0%Upper 95.0% Intercept60.552387.4757014998.0998929911.95158E-0644.40211176.7026533444.40211176.70265334 X Variable 110.405630.97425021410.680652448.33726E-088.30088830512.510367558.30088830512.51036755

32. Where y ^ =60.6+10.4(7.0)=133.39 The 95% confidence interval is The prediction interval is

33. We are 95% confident that a diabetic with a hemoglobin A1C value of 7 had a fasting blood sugar over the past 3 months that average between 120.23 and 146.56. We are 95% confident that diabetics with hemoglobin A1A value of 7.0 had an average fasting blood sugar over the past 3 months between 129.93 and 136.85

34. Exerscises 1.Give the deterministic equation for the line passing through the following pair of points: a) (1,1.5) and (3,8.5) b) (0,1) and (2,-3) c) (0,3.1) and (1,4.8)

35. Reference This lecture prepared from Advanced statistics demystified “MCGrawHill” Dr. Larry J. Stephens