...Relax... 9/21/2018 ST3131, Lecture 3 ST5213 Semester II, 2000/2001 HJGHK 9/21/2018 ST3131, Lecture 3

Lecture 3 Main Tasks Today! Review of Lecture 2 Statistical Inferences of the LS-Estimators a). Properties of the LS-estimators b). Accuracy of the LS-estimators c). Significance Tests of the Parameters 9/21/2018 ST3131, Lecture 3

Review of Lecture 2 Standardization of Y: we have =0 and =1. Covariance / Correlation of Y and X : Properties of Cov(Y,X) / [Cor(Y,X)] (1). Symmetric (2). Scale-dependent /[invariant] (3). Take values in the real line/[in [-1,1]] (4). Measure of direction of the linear relationship between Y and X (5). Not Robust statistics( affected by outliers) 9/21/2018 ST3131, Lecture 3

Review of Lecture 2 (cont.) For linear regression equation Y= + X + , Cov(Y,X) or Cor(Y,X) Linear Relationship between Y and X Positive Positive(Y increases as X increases) zero Uncorrelated (Y does not change as X changes) negative Negative (Y decreases as X increases) Cor(Y,X) Strength of Linear Relationship between Y and X Close to –1 or 1 Very strong Close to -.5 or .5 Strong Close to 0 Very weak (near linearly uncorrelated) 9/21/2018 ST3131, Lecture 3

i 1 2 … n …. Review of Lecture 2 (cont.) Total Note that Var(Y)=sum of squared deviations of Y divided by (n-1) Var(X)=sum of squared deviations of X divided by (n-1) Cov(Y,X)=sum of products of deviations of Y and X divided by (n-1). 9/21/2018 ST3131, Lecture 3

Review of Lecture 2 (cont.) The Least Squares Estimators of and : They are linear combinations of the observations . That is (Linearity) (Linearity) 9/21/2018 ST3131, Lecture 3

Review of Lecture 2 (cont.) Linear Regression Line: Fitted Values: Predicted Values Linear Regression Line For Computer Repair Data 9/21/2018 ST3131, Lecture 3

Linear Estimator and Its Property Linear Estimator = a linear combination of observations i.e., where c’s are any constants. Notice: and are linear estimators of and . Property: when T= and are independently normally distributed, we have 9/21/2018 ST3131, Lecture 3

Application to SLR Models Simple Linear Regression Model: Assumptions: (1) Linearity between Y and X (2) Normality for measurement errors (3) Measurement Errors are independently identically distributed (iid) Using the property of linear estimators, we have 9/21/2018 ST3131, Lecture 3

Application to Parametric Estimators of SLR Model Using the property of linear estimators, we can show that (Unbiased) (Normality) (Unbiased) (Normality) 9/21/2018 ST3131, Lecture 3

Properties of the LS-Estimators 1.) Linearity 2). Normality 3). Unbiased 4). Best Linear Unbiased Estimators. That is , they have smallest variances among all unbiased linear estimators. Let and be any other Unbiased Linear Estimators of and respectively. Then 9/21/2018 ST3131, Lecture 3

Accuracy of the LS-Estimates Noise Variance is usually unknown. It has a natural estimator using residuals, given by SSE=Sum of Squared Errors (Residuals) n-2= degrees of freedom of SSE=Sample size- # of Coefficients Using the estimated noise variance, we can obtain the measures of the accuracy (Standard Errors, s.e.) of the estimators and 9/21/2018 ST3131, Lecture 3

Significance Test of Parameters In the SLR model, we often want to know if (1) “ Y and X are linearly uncorrelated” (X has no effect on Y) ( ) (2) “ the linear regression line passes the origin” (no intercept is needed).( ) A hypothesis test has two hypotheses: H0 (Null) and H1 ( Alternative). A general test about the slope (intercept ) can be written as versus (a) versus (b) where ( or ) is a predetermined constant (belief of the experts). To test (1) [or (2)] is equivalent to test =0 [or =0] 9/21/2018 ST3131, Lecture 3

Test Statistics and 2-Sided Tests To test the hypotheses, we need construct test statistics. Test statistic can usually be constructed as: For example, to test (1) and (2) , we have test statistics respectively Which have t-distributions with n-2 degrees of freedom (dfs). Let be the Upper - percentile of t-distribution with n-2 dfs. Then the level two- sided test for test (1) or (2) is can be found in the Appendix (Table A.2, Page 339) 9/21/2018 ST3131, Lecture 3

P-value of the Test In another way, we can compute the p-value of the test statistic to perform the test: If the p-value< , then the test is level significant. The smaller the p-value, the higher significant level the test is. In general, we say 1) when p-value<.05, the test is significant; 2) when p-value < .01, the test is very significant; 3) when p-value>.20, the test is not significant. Example For computer repair data, to test Since 6.948>2.18, the test is significant at 5%-level significance. The <.005. So the test is highly significant 9/21/2018 ST3131, Lecture 3

One-Sided Tests: (a) ( (b) ) (c) ( (d) ) 9/21/2018 ST3131, Lecture 3

Outputs for the computer repair data using Splus 9/21/2018 ST3131, Lecture 3

Results using Minitab Results for: P027.txt Regression Analysis: Minutes versus Units The regression equation is Minutes = 4.16 + 15.5 Units Predictor Coef SE Coef T P-value Constant 4.162 3.355 1.24 0.239 Units 15.5088 0.5050 30.71 0.000 S = 5.392 R-Sq = 98.7% R-Sq(adj) = 98.6% Note that the T-values calculated here are for testing`` the true parameters are zeros”. 9/21/2018 ST3131, Lecture 3

A Test Using Correlation Coefficient Since =0 is equivalent to Cor(Y,X)=0. So the following two tests are equivalent to each other: (3) (4) where denotes the true correlation between Y and X. To test (4), we can use the following test statistic We can also compute to perform the test 9/21/2018 ST3131, Lecture 3

Example: For the computer repair data, since Cor(Y,X)=.996, we have Which is much larger than . So the correlation is highly Significantly different from 0. That is X has high impact on Y. We reject H0 of test (4) or (3) at 5%-significant level. The corresponding p-value much less than .0001. So the significance is very high. 9/21/2018 ST3131, Lecture 3

Review Sections 2.6-2.9 of Chapter 2. Reading Assignment Review Sections 2.6-2.9 of Chapter 2. Read Sections 2.10-2.11 of Chapter 2. Consider problems: a) What are special SLR models? b) Why need we study the special SLR models? c) How to do inference about the special SLR models? 9/21/2018 ST3131, Lecture 3