Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and l Chapter 9 l Simple Linear Regression 9.1 Simple Linear Regression 9.2 Scatter Diagram 9.3 Graphical Method for Determining Regression 9.4 Least Square Method 9.5 Correlation Coefficient and Coefficient Determination 9.6 Test of Significance

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Introduction to Regression  Regression is a statistical procedure for establishing the relationship between 2 or more variables.  This is done by fitting a linear equation to the observed data.  The regression line is then used by the researcher to see the trend and make prediction of values for the data.  There are 2 types of relationship: Simple ( 2 variables) Multiple (more than 2 variables)

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Simple Linear Regression  It involve relationship analysis between two variables (One independent variable and one dependent variable).  Its model use an equation that describes a dependent variable (Y) in terms of an independent variable (X) plus random error ε.  Random error, is the difference of data point from the deterministic value.  This regression line is estimated from the data collected by fitting a straight line to the data set and getting the equation of the straight line, where, = intercept of the line with the Y-axis = slope of the line = random error

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Simple Linear Regression  Example of independent and dependent variables. 1) A nutritionist studying weight loss programs might wants to find out if reducing intake of carbohydrate can help a person reduce weight. X is the carbohydrate intake (independent variable). Y is the weight (dependent variable). 2) An entrepreneur might want to know whether increasing the cost of packaging his new product will have an effect on the sales volume. X is the cost (independent variable) Y is sales volume (dependent variable)

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Scatter Diagram  A scatter plot is a graph or ordered pairs (x,y).  The purpose of scatter plot – to describe the nature of the relationships between independent variable, X and dependent variable, Y in visual way.  The independent variable, x is plotted on the horizontal axis and the dependent variable, y is plotted on the vertical axis.

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Scatter Diagram  A linear regression can be develop by freehand plot of the data.  Example: The given table contains values for 2 variables, X and Y. Plot the given data and make a freehand estimated regression line. X Y

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Scatter Diagram

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Scatter Diagram

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Least Square Method  Produces a straight line that minimizes the sum of square differences between the point and the line (determine values for and that ensure a best fit for the estimated regression line to the sample data points).  Involve some calculation procedure.  However, as we are using SPSS output, no calculation is necessary.

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Least Square Method

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Least Square Method  Conducting simple linear regression through SPSS. The data below represent scores obtained by ten primary school students before and after they were taken on a tour to the museum (which is supposed to increase their interest in history) Fit a linear regression model with “before” as the explanatory variable and “after” as the dependent variable. Predict the score a student would obtain “after” if he scored 60 marks “before”. Before,x After, y

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Least Square Method  Analysis 1 2 3

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and If a student scored 60 marks in “before”, he would obtain in “after” 9.4 Least Square Method  Output

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Correlation Coefficient and Coefficient of Determination  Correlation measures the strength of a linear relationship between the two variables.  Also known as Pearson’s product moment coefficient of correlation.  The symbol for the sample coefficient of correlation is r.  Values of r close to 1 strong positive linear relationship between x and y. close to -1 strong negative linear relationship between x and y. close to 0 little or no linear relationship between x and y.

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Correlation Coefficient and Coefficient of Determination  Positive linear relationship E(y)E(y)E(y)E(y) x Intercept  0 Regression line Slope  1 is positive

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Correlation Coefficient and Coefficient of Determination  Negative linear relationship E(y)E(y)E(y)E(y) x  0  0Intercept Regression line Slope  1 is negative

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Correlation Coefficient and Coefficient of Determination  No relationship E(y)E(y)E(y)E(y) x  0  0Intercept Regression line Slope  1 is 0

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Correlation Coefficient and Coefficient of Determination  The coefficient of determination is a measure of the variation of the dependent variable (Y) that is explained by the regression line and the independent variable (X).  If r = 0.90, then = It means that 81% of the variation in the dependent variable (Y) is accounted for by the variations in the independent variable (X).  The rest of the variation, 0.19 or 19%, is unexplained and called the coefficient of nondetermination.  Formula for the coefficient of nondetermination is 1-

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Correlation Coefficient and Coefficient of Determination  Output Strong positive linear relationship 88.7% variation in X is explained by Y

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Test of Significance  Simple linear regression involves two estimated parameters which are β0 and β1.  Test of hypothesis is used in order to know whether independent variable is significant to dependent variable (whether X provides information in predicting Y).  The t-test or analysis of variance (ANOVA) method is an approach to test the significance of the regression.  Basically, two test are commonly used: t Test F Test (ANOVA)

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Test of Significance t-test Compare P-value (refer to Coefficient table) with α Reject if P-value < α If we reject there is a significant relationship between variable X and Y. (NO RELATIONSHIP) (THERE IS RELATIONSHIP)

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Test of Significance F-test (ANOVA) Compare P-value (refer to ANOVA table) with α Reject if P-value < α If we reject there is a significant relationship between variable X and Y. (NO RELATIONSHIP) (THERE IS RELATIONSHIP)

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Test of Significance  Construction of ANOVA table Source of variation Sum of squares Degree of freedom Mean square f test Regression SSR1MSR =SSR/1 f = MSR/MSE Error SSEn-2MSE =SSE/n-2 Total SSTn-1

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Exercise  The following table gives information on lists of the midterm, X, and final exam, Y, scores for seven students in a statistics class.  Find the least squares regression line.  Explain the values of r and.  Predict the final exam scores the student will get if he/she got 60 marks for midterm test.  Do the data support the existence of a linear relationship between midterm and final exam? Test using α = X Y

Irwin/McGraw-Hill © Andrew F. Siegel, 1997 and Exercise  The manufacturer of Cardio Glide exercise equipment wants to study the relationship between the number of months since the glide was purchased and the length of time the equipment was used last week. Determine the regression equation. At, test whether there is a linear relationship between the variables