1 732G21/732A35/732G28

 732G21 Sambandsmodeller One semester=Regr.analysis+ + analysis of variance (teacher: Lotta Hallberg) 732G28 Regression methods Half of semester=Regr. analysis 732A35 Linear statistical models Almost one semester=Regr. Analysis+ + analysis of variance (teacher: Lotta Hallberg) 732G21/732A35/732G282

 Course language: English, but you may use Swedish  We use It’s learning (accessed via Student portal) (show…)  9 Lectures  8 Labs (computer). Deadlines, around 5 days after lab ends  8 Lessons=I solve problems on the whiteboard + lab discussion  One written final exam  Course book: Kutner, M.H., Nachtsheim, C.J., Neter, J. and Li, W. Applied Linear Statistical Models with Student Data CD, 5th Edition, ISBN G21/732A35/732G283

 Linear statistical models are widely used in ◦ Business ◦ Economics ◦ Engineering ◦ Social, biological sciences ◦ Etc Example: A database contains price of houses sold in Linköping in 2009, their age, size, other parameters. ◦ Given parameters of a new house  determine its approximate market price  Determine reasonable price bounds 732G21/732A35/732G284

 Analysis of databases  Observations (records, cases) in rows  Variables in columns ◦ Explanatory variables (predictors, inputs) X i ◦ Response Y, we assume Y=f(X 1,…,X n ) In this lecture, models with only one explanatory variable 732G21/732A35/732G285 NoArea (X 1 )Age (X 2 )Price (Y) ,530, ,800,000 …………

 Real data can seldom be presented as Y=βX (observation errors, missing inputs etc) 732G21/732A35/732G286 Example: Age and salary for a sample of eight persons from a company. AgeSalary Scatterplot

 Presented relation is almost linear  Linear regression analysis: find a linear finction as close as possible to the data 732G21/732A35/732G287

 For each X, there is a probability distribution P(Y=y|X=x) of Y  The aim is to find a regression function E(Y|X=x) 732G21/732A35/732G288

Construction of regression models  Selection of prediction variables (variance reduction)  Functional form (from theory, approximation)  Domain of the model Software  MINITAB  SAS  SPSS  Matlab  Excel 732G21/732A35/732G289

Formal statement  Y i is i th response value  β 0 β 1 model parameters, regression parameters (intercept, slope)  X i is i th predictor value  is i.i.d. random vars with expectation zero and variance σ 2 732G21/732A35/732G2810

Features (show…)  All Y i and Y j are uncorrelated Meaning of regression parameters  β 0 response value at X=0  β 1 change in EY per unit increase in X 732G21/732A35/732G2811

Given data set Method of least squares:  Observed response Y i  Estimated response  Deviation  Regression fit is good when all deviations are minimized (see pict) -> minimimize sum of squares 732G21/732A35/732G2812

 How to find minimum of Q? Estimators of β 0 and β 1  732G21/732A35/732G2813

Exercise (For salary data, MINITAB): 1. Make scatterplot (Scatterplot…, with, without regression lien) 2. Perform regression using ”Regression…” 3. Perform regression using ”Fitted line plot..” 4. Calculate coefficients by hand 732G21/732A35/732G2814

732G21/732A35/732G2815

Gauss-Markov theorem  Estimators b 0 and b 1 are unbiased and have minimum variance among all unbiased estimators  Unbiased  bias=Eb 0 -β 0 =0  Eb 0 =β 0  Analogously, Eb 1 =β 1 Show illustration… 732G21/732A35/732G2816

 Mean (expected response)  Point estimator of mean response (fitted value) Residuals 732G21/732A35/732G2817

 Plot of residuals (obtain it with MINITAB) 732G21/732A35/732G2818

 Properties of residuals 1. (because ) 2. is minimum possible 3. (because of 1) 4., (can be shown) 5. Regression line always goes through 732G21/732A35/732G2819

 Estimate of variance of single population (sample variance)  In regression, we compute s 2 using residuals (look at residual plot) 732G21/732A35/732G2820

 Why divided by n-2? Because E(MSE)=σ 2  Important: In general, unbiased d - degrees of freedom, number of model parameteres Example: Compute residuals, SSE, MSE, find it in MINITAB output 732G21/732A35/732G2821

 Minitab ◦ Graph → Scatterplot ◦ Stat → Regression ◦ Stat->Fitted Line Plot 732G21/732A35/732G2822

 Course book, Ch. 1 up to page G21/732A35/732G2823