1. 1/18/2019
ST3131, Lecture 1

2. Chapter 1. Introduction Questions: Q1. What is Regression Analysis?
Q2. How to Do Regression Analysis?
Q3. Where to Use Regression Analysis?
1/18/2019
ST3131, Lecture 1

3. Q1: What is Regression Analysis?
A useful tool for finding Functional Relationships among Variables based on data, and using this relationship for further analysis of the data.
Functional Relationships:
Mathematical Formulas or Equations connects
a response variable & several predictor variables.
Variables include Quantitative (Numerical) Variables ( e.g )
2 Types: Continuous(e.g ) and Discrete (e.g ).
Binary Variable: takes only 2 values, 0 and 1, say.
And Qualitative (Non-numerical) Variables:
e.g., Neighborhood type ( ), Blood type ( )
1/18/2019
ST3131, Lecture 1

4. Exercise
Page 18, Problem 1.1 (b),(d),(f) and (h): Quantitative or Qualitative Variables? If Latter, state the possible categories:
(b). # of Children in a family
(d). Race
(f). Fuel Consumption
(h). Political party preference
1/18/2019
ST3131, Lecture 1

5. A General Regression Model
Response Variable.: Y
Predictor Variables.:
Measurement Error:
f : unknown regression function
1/18/2019
ST3131, Lecture 1

6. Parametric Regression Models
Parametric Regression Models : f is a ( ) function
Type 1: Linear Regression Model: f is a ( ) function:
: Regression Parameters /Coefficients
Type 2: Nonlinear Regression Model: f is a ( ) function in ( ):
1/18/2019
ST3131, Lecture 1

7. Exercise Linear Regression / Nonlinear Regression Model? (a). (b).
1/18/2019
ST3131, Lecture 1

8. Types of Parametric Regression Models
Regression Type
Conditions/Definitions
Univariate (Multivariate)
One (two or more Quantitative) Response variables
Simple (Multiple)
Only one (two or more) Predictor variables
Linear /Nonlinear
Response is Linear/Nonlinear in Parameters
Logistic
Response variable is Qualitative
Analysis of Variance (Covariance)
All (Some) Predictors are Qualitative variables
1/18/2019
ST3131, Lecture 1

9. An Example Questions of Interest
 A company markets and repairs small computers. How fast (Time, response) an electronic component (Computer Unit, predictor variable) can be repaired is very important to the efficiency of the company. The Variables in this example are:
Time and Units
Questions of Interest
What is the relationship between the length of a service call (Time) and the number of electronic components (Computer Units)?
In general, how long will it take to repair k computer units?
1/18/2019
ST3131, Lecture 1

10. Computer Repair Data Table 2.5 Page 27
Pre-Data Analysis
Units
Min’s 1 23 6 97 2 29 7
109 3 49 8
119 4 64 9
149 74
145 5 87 10
154 96
166
To see How  the Time is related with computer Units, we can draw  a plot of Time  against computer Units.
From the plot, we can see the simple relationship between Time and Units.
This will suggest what kind of model is good to fit the data.
1/18/2019
ST3131, Lecture 1

11. Pre-Data Analysis Scatter Plot (Time vs Units) Some Simple Conclusions
Time is Linearly related with computer Units.
Time is Increasing with Number of Units.
The Linearity is NOT exactly, Measurement Errors exist.
Thus Linear Regression Model can be used for the relation between Time and computer Units
1/18/2019
ST3131, Lecture 1

12. Simple Linear Regression Model
called Linear Regression Intercept
called Linear Regression Slope
called Regression Parameters or Coefficients 
=I-th Measurement Error
n=# of observations
where
X=Units, called Independent,  Explanatory or Predictor variable
the i-th observation
Y=Time, called Dependent or Response variable
1/18/2019
ST3131, Lecture 1

13. Least Squares Method
Least Squares Method is often used to fit the above Linear Regression Model:
Find   to Minimize 
Least: Minimization
Squares : Sum of Squared residuals
The solution is called Least Squares Estimator of , denoted as
1/18/2019
ST3131, Lecture 1

14. Simple Linear Regression Fit
The Fitted Equation
Some Conclusions
Least Squares fit gives the LS-Estimates
The left plot shows that the simple linear regression model is good for the data.
The fitted line is increasing with increasing Units.
1/18/2019
ST3131, Lecture 1

15. The Resulting Model & Prediction
The resulting model is
Where , read as “Y hat”, stands for the estimation at X.
That is “Minutes= * Units”.
Prediction:
X=1, Y= *1=19.67,
X=5, Y= *5=81.71, etc.
Interpretation: it takes about minutes to repair 1 computer unit; about minutes to repair 5 computer units.
1/18/2019
ST3131, Lecture 1

16. Q2:How to Do Regression Analysis?
In summary, we can list the steps in Regression Analysis as:
Step 1. State the Problem of Interest
Step 2. Select Potentially Relevant Variables
Step 3. Collect Relevant Data
Step 4. Specify a Model
Step 5. Choose a Fitting Method
Step 6. Fit the Chosen Model
Step 7. Check the Resulting Model
Step 8. Apply the Resulting Model for Prediction
1/18/2019
ST3131, Lecture 1

17. Step1: State the Problem of Interest
A Key step in Regression Analysis. Different statement of the problem will lead to different choice of response and predictor variables.
Example: Suppose we wish to determine if or not an employer is discriminating against a given group of employees,say women. Data on Salary, Qualifications, and Sex are available.
Example 1: For question “ On average, are women paid less than equally qualified men?”, we choose Salary as Response, Qualification and Sex as Predictors.
Example 2: For question “ On average, are women more qualified than equally paid men?”, we choose Qualification as Response, Salary and Sex as Predictors.
1/18/2019
ST3131, Lecture 1

18. Exercise
Page 19, Problem 1.3 (a), (b), (c ). Which variable can be used as Response, Which can be used as Predictors? Why?
(a). Number of cylinders, gasoline consumption of cars
(b). SAT scores, grade point average, college admission
(c ). Supply and demand of certain goods
1/18/2019
ST3131, Lecture 1

19. Step 2:Select Potentially Relevant Variables
Depends on the Problem of Interest. Usually, Y denotes Response variable, X1, X2, …,Xp denote Predictor variables.
For the question “ if the price of a single house in a given geographical area is high?”,
response: price of a single house(Y),
predictor variables may include:
area of the lot(X1), area of the house(X2),
age of the house(X3), number of bathrooms(X4),
type of neighborhood(X5), style of the house(X6),
amount of real estate taxes(X7) etc.
1/18/2019
ST3131, Lecture 1

20. Step 3: Data Collection
Data are collected for the chosen response and predictor variables. The collected data are usually recorded in the following form:
Observation
No.
Response Y
Predictors
X1 X2 …. Xp 1 2 … n Y1 Y2 …. Yn
x11, x12,… x1p
x21, x22,…, x2p
……………….
xn1, xn2,…,xnp
Each column lists observations for a variable,
Each row list observations for all predictor variables in a case.
1/18/2019
ST3131, Lecture 1

21. Step 4: Model Specification
Usually specified by the experts in the area of study
May be specified based on the Initial Analysis of the data
In our level, the variables are given, and the model is often given, mostly Linear and Parametric.
1/18/2019
ST3131, Lecture 1

22. Step 5: Method of Fitting
In our case, we mainly focus on Least Squares Method.
People may use other methods such as Weighted Least Squares, Maximum likelihood method, Ridge Regression, Principal Components method etc. from different views and different contexts.
1/18/2019
ST3131, Lecture 1

23. Step 6:Fitted Model and Prediction
When the parameters are estimated , we got the fitted regression equation or formula, e.g., the Estimated Linear Regression Equation is:
: Fitted value when (X1,..,Xp) is a data point, n Fitted
Values are:
: Predicted Value when (X1,..,Xp) is not a data point.
1/18/2019
ST3131, Lecture 1

24. Step 7: Model Checking
To check if the assumptions for the regression model is valid or not. This is Regression Diagnostics problem. The details will be given in Chapter 4.
1/18/2019
ST3131, Lecture 1

25. Q3: Where to Use Regression Analysis?
Regression Analysis is one of the most widely used statistical tools.
It has extensive applications in many subject areas, including:
Agricultural Sciences
Industrial and Labor Relations
History
Government
Environmental Sciences
Military, Economics, Financial etc.
For details, read Section 1.3, Page 3-7.
1/18/2019
ST3131, Lecture 1

26. Read Chapter 1, Sections 1-4
Reading Assignments
Read Chapter 1, Sections 1-4
Read Chapter 2, Sections 1-4. Thinking about the following questions:
a). What is SLR model?
b). How to find the LS estimates of the parameters?
c). How to compute the LS estimates manually?
1/18/2019
ST3131, Lecture 1