1. M248: Analyzing data
Block D
UNIT D2
Regression

2. Block D Contents UNIT D2: Regression Section 1: Regression models
Section 2: Fitting a linear regression model
Terms to know and use
Unit D2 Exercises

3. Section 1: Regression models
A scatterplot is a graphical representation that shows the relationship between two variables.
A regression model describe the relationship between the two variables while at the same time allowing for some deviation from the line/curve.
In a scatterplot of real observations, the dots might not lie on a straight line, and so the regression model that represent the data could be a line or a curve.

4. Section 1: Regression models
Linear relationship between the two variables
There is an overall downward trend but there is no suggestion about any relationship between the two variables
Strong relationship between the two variables but it is not linear
Check Examples 1.1, 1.2, 1.3 and 1.4

5. Section 1: Regression models
A variable that ‘explains’ another variable is called an explanatory variable.
The variable that ‘responds’ to the value of the explanatory variable is called the response variable.
Notice that, the explanatory variable plotted along the x-axis in the scatterplot, and the response variable along the y-axis. This is standard practice.
Solve activities 1.1, 1.2 and 1.3

6. Section 1: Regression models
In regression, it is customary to regard the explanatory variable as non-random and the response variable as a random variable.
The general regression model includes a function that defines the line or curve about which the points in a scatterplot are scattered, and a term which models the scatter, that is, the variation in the response variable.

7. Section 1: Regression models
If the response variable is denoted by Y and the explanatory variable by x, then the general regression model can be written
Here h(.) represents some functions and the ‘s are independent random variables with zero mean.
The random terms account for the scatter around the straight line (curve).

8. Section 1: Regression models
The most important case of the regression model is the linear regression model.
A linear regression model is a regression model where the relationship between Y and x is linear.
Sometimes non-linear regression models can be reduced to linear regression models by a transformation of the data. This case will be discussed later.

9. Section 1: Regression models
The linear regression model
If Y is the response variable and x is the explanatory variable, then the linear regression model can be written
The terms are independent random variables with zero mean and constant variance.
The line is called the regression line.
Read examples 1.5, 1.6 and 1.8
Solve activity 1.4

10. Section 2: Fitting a linear regression model
Section 2.1: The principle of least squares
which line is best describe the data ?

11. Section 2: Fitting a linear regression model
The traditional criterion for estimating the best fitting line to data is the principle of least squares.
In general, if the curve y=h(x) is fitted to data points then the residual for the point is the difference between the observed value and the value given by the fitted curve:
When using the principle of least squares to choose a line or curve, the sum of the squares of residual is minimized

12. Section 2: Fitting a linear regression model
Observed value

13. Section 2: Fitting a linear regression model
The sum of squares of the residual is called the residual sum of squares and is given by
When fitting a straight line , this reduces to
A small sum indicates a good fit, while a large sum indicates a poor fit.

14. Section 2: Fitting a linear regression model
We are interesting in the values of that minimize the residual sum of squares.
The minimizing values of are called the least squares estimates of the parameters of the regression line, and are denoted by
Refer to chapter 3 of Computer Book D for the rest of the work in this subsection.
Read example 2.1

15. Section 2: Fitting a linear regression model
Section 2.2: The least square line through the origin
Suppose that a scatterplot of data points , i=1,2,…,n, suggests that an appropriate regression model is of the form
where the ‘s are independent with zero mean and variance . Then the least square estimate is given by
The equation of the least squares line through the origin is
Read example 2.2 and Solve activity 2.1

16. Section 2: Fitting a linear regression model
Section 2.3: The least square line
Suppose that a scatterplot of data points , i=1,2,…,n, suggests that an appropriate regression model is of the form
where the ‘s are independent with zero mean and variance . Then the squares estimate of the slope of the regression line is
where and

17. Section 2: Fitting a linear regression model
The least squares estimate of the constant term is given by The equation of the least squares line is Note: The least squares line passes through , the centroid of the data.

18. Section 2: Fitting a linear regression model
Although, you might use the following formulas during calculations:
Solve Activities 2.2 and 2.3
Read Examples 2.3 and 2.4

19. Terms to know and use Explanatory variables Regression line
Least square line
Response variable
Regression model
Regression curve
Linear regression model

20. Unit D2 Exercises M248 Exercise Booklet
Solve the following exercises:
Exercise 65 …………………………………… Page 19