Regression [Example 3.] As a motivating example, suppose we are modelling sales data over time. SALES TIME We seek the straight line “Y = m X + c” that best approximates the data. By “best” in this case, we mean the line which minimizes the sum of squares of vertical deviations of points from the line: SS =  ( Y i - [ mX i + c ] ) 2. Setting the partial derivatives of SS with respect to m and c to zero leads to the “normal equations”  Y = m  X + n.c, where n = # points  X.Y= m  X 2 + c  X. Let 1990 correspond to Year 0. X.X X X.Y Y Y.Y X Y Y = m X + c m X i + c YiYi 0 XiXi Time Sales

Example 3 - Workings. The normal equations are: 30 = 15 m + 6 c => 150 = 75 m + 30 c 87 = 55 m + 15 c 174 = 110 m + 30 c =>24 = 35 m=>30 = 15 (24 / 35) + 6 c=> c = 23/7 Thus the regression line of Y on X is Y = (24/35) X + (23/7) and to plot the line we need two points, so X = 0 => Y = 23/7and X = 5 => Y = (24/35) /7 = 47/7. It is easy to see that ( X, Y ) satisfies the normal equations, so that the regression line of Y on X passes through the “Center of Gravity” of the data. By expanding terms, we also get  ( Y i - Y ) 2 =  ( Y i - [ m X i + c ] ) 2 +  ( [ m X i + c ] - Y ) 2 Total SumErrorSumRegression Sum of Squaresof Squaresof Squares SST = SSE + SSR In regression, we refer to the X variable as the independent variable and Y as the dependent variable. X Y YiYi mX i +C Y X Y

Correlation The coefficient of determination r 2 ( which takes values in the range 0 to 1) is a measure of the proportion of the total variation that is associated with the regression process: r 2 = SSR/ SST = 1 - SSE / SST. The coefficient of correlation r ( which takes values in the range -1 to +1 ) is more commonly used as a measure of the degree to which a mathematical relationship exists between X and Y. It can be calculated from the formula: r =  ( X - X ) ( Y - Y )   ( X - X ) 2 ( Y - Y ) 2 = n  X Y -  X  Y  { n  X 2 - (  X ) 2 } { n  Y 2 - (  Y ) 2 } Example. In our case r = {6(87) - (15)(30)}/  { 6(55) - (15) 2 } { 6(160) - (30) 2 } = r = - 1r = + 1r = 0

Collinearity If the value of the correlation coefficient is greater than 0.9 or less than - 0.9, we would take this to mean that there is a mathematical relationship between the variables. This does not imply that a cause-and-effect relationship exists. Consider a country with a slowly changing population size, where a certain political party retains a relatively stable percentage of the poll in elections. Let X = Number of people that vote for the party in an election Y = Number of people that die due to a given disease in a year Z = Population size. Then, the correlation coefficient between X and Y is likely to be close to 1, indicating that there is a mathematical relationship between them (i.e.) X is a function of Z and Y is a function of Z also. It would clearly be silly to suggest that the incidence of the disease is caused by the number of people that vote for the given political party. This is known as the problem of collinearity. Spotting hidden dependencies between distributions can be difficult. Statistical experimentation can only be used to disprove hypotheses, or to lend evidence to support the view that reputed relationships between variables may be valid. Thus, the fact that we observe a high correlation coefficient between deaths due to heart failure in a given year with the number of cigarettes consumed twenty years earlier does not establish a cause-and- effect relationship. However, this result may be of value in directing biological research in a particular direction.