Managerial Economics & Decision Sciences Department intro to linear regression  underlying concepts for the linear regression  interpret linear regression results  business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II ▌ the linear regression model week 2 week 1 week 3

© 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II readings ► statistics & econometrics ► (MSN)  test hypotheses for linear regression parameters  build, run and interpret results for linear regression learning objectives  visualize data through graphs/diagrams  run a linear regression  perform tests for linear regression parameters ►  Chapter 3 ► (CS)  Autoparts session two the linear regression model business analytics II Developed for

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page1 Autoparts ► In the Autoparts case we are comparing sales of one company, “Skokie Auto”, with those of other 24 comparable companies. While all companies have similar locations within a mall Skokie Auto’s view from the street is obstructed. This might in turn lead to lower sales. ► The first approach would be to set up a simple hypothesis that, indeed, Skokie Auto’s sales are below the mean sales of other comparable companies. We might or might nor reject this hypothesis… ► Notice how in this first approach we are referring to “comparable” companies…But, so far, the only criterion used to consider a company comparable to Skokie Auto was the similar end-cap location.  For this first approach to be meaningful we should be able to argue that the only factor that affects sales is the visibility of a store in an end-cap location  Data on comparable companies include also the population (in number of people) and average income (in $ per year) within a three mile radius  It is likely that sales are affected by location, population and income… How do we incorporate the last two variables (if at all) in the analysis? session two data analytics: from data to model … and beyond

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page2 Autoparts ► As a starting point: let’s visualize the data in two separate scatter diagrams. Is there any relation between sales and these variables ? session two data analytics: from data to model … and beyond Figure 1. Sales, population (left diagram) and income (right) for comparable companies twoway scatter sales pop3mi twoway scatter sales inc3mi The twoway command prompts STATA that a two-variable ( y, x ) graph is plotted. Next you have to indicate what type of graph to draw: scatter, line, etc. Finally you have to indicate the y -variable and the x -variable. You can plot several pairs (y,x) on the same graph by separating the details through parentheses: twoway (scatter sales pop3mi) (scatter sales inc3mi) graph type y -variable x -variable

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page3 Autoparts session two data analytics: from data to model … and beyond ► Visually it looks like sales change with the level of population and income variables. A simplistic approach would be to draw a line/curve through the “clouds of points”… twoway (scatter sales pop3mi) (lfit sales pop3mi) twoway (scatter sales inc3mi) (lfit sales inc3mi) You can use the lfit type of graph to draw the best linear fit through a “cloud of points”. As with the scatter type you have to specify the y and x variables. Figure 2. Sales, population (left diagram) and income (right) for comparable companies: linear fit graph

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page4 Autoparts session two data analytics: from data to model … and beyond key issue quantifying relations between variables ► In order to perform inference analysis with testable hypotheses on the direction and strength of relations between variables we are interested in a formal way to quantify these relations. key concept regression analysis ► Regression analysis is a statistical technique based on a model relating the statistics of a variable y to other given variables x 1, x 2, …, x k in a causal way. For the mean of y it is written in a compact way as E [ y | x 1, x 2, …, x k ]  h ( x 1, x 2,…, x k ) where  y is the dependent variable  x 1, x 2, …, x k represent the independent variables (could be one or more)  h (  ) represents the causal relation between y and the x-variables ► The linear regression model assumes the following linear functional relation between the true mean E [ y | x 1, x 2, …, x k ]   0   1 · x 1  …   k · x k ► Running the linear regression model means to find, based on the available sample, estimates b 0, b 1, …, b k of true parameters  0,  1, …  k

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page5 Autoparts session two data analytics: from data to model … and beyond 1 perform regression regress yvar xvar 1 xvar 2 … xvar k generate yhat  _b[_cons]  _b[ xvar 1 ]* xvar 1  …  _b[ xvar k ]* xvar k 2 generate the linearly fitted data twoway (scatter yvar xvar 1 ) (connected yhat xvar 1, sort) 3 visualize the data (for one xvar )  In the first step we perform the regression which provides the estimates for the linear coefficients.  The estimates are stored, and available for use in subsequent calculations, as _b[_cons], _b[ xvar 1 ], …, _b[ xvar k ]  We generate then the fitted values of yvar for each available observation of xvar (s). We usually call this fitted value as yhat and its calculation is fairly intuitive: use the estimated coefficients and “plug” the values for xvar (s). Remark on the twoway graph. Notice that we introduced a new type of graph: connected. This is really a scatter sub-type in which the “ dots” are connected through a line. An important detail is to add sort as an option which will basically sort ascending the variables on the x axis (otherwise the graph will be drawn using the observation in the order they are provided in the sample – try running the same command above without the sort option). Another issue here: you can remove the “marker” along the connected graph by including the option msymbol(i) as in: twoway (scatter yvar xvar 1 ) (connected yhat xvar 1, sort msymbol(i))

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page6 Autoparts session two data analytics: from data to model … and beyond sales | Coef. Std. Err. t P>|t| [95% Conf. Interval] pop3mi | _cons | regress sales pop3mi generate yhat1  _b[_cons]  _b[pop3mi]*pop3mi twoway (scatter sales pop3mi) (connected yhat1 pop3mi, sort) regress sales inc3mi generate yhat2  _b[_cons]  _b[inc3mi]*inc3mi twoway (scatter sales inc3mi) (connected yhat2 inc3mi, sort) sales | Coef. Std. Err. t P>|t| [95% Conf. Interval] inc3mi | _cons | Figure 3. Regression results and graphical representation (pop3mi) yhat1    pop3mi interceptslope yhat2    inc3mi interceptslope Figure 4. Regression results and graphical representation (inc3mi)

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page7 The Linear Regression: Concepts session two ► What does the regression model allow us to do/estimate?  A regression model allows the estimation of the mean of some variable, the y  variable, controlling for, i.e. given, the values of other variables, the x variables.  A regression model also allows us to estimate how changes in the value of one or more x -variables are associated with changes in the y -variable. Note the term “ associated with.” Regressions do not automatically tell us about causality. Sorting out causality, known as the identification problem, will be a central issue in this course, but we must first learn a lot before we get there. ► How is the data organized in a regression model?  Data used in regression models are laid out in a spreadsheet. There are n rows and k  1 columns We call n the “sample size”. Each column contains a variable: there are k independent x  variables, and one dependent variable, the y  variable  It is not necessary for y to appear in the first column of your spreadsheet and, in general, it may appear in any column. When you run your regression, you will specify which variable is your y- variable. The other columns contain the explanatory variables x 1, x 2, … x k  Your regression may not (and, in many applications, should not) include all or even most of the x  variables that appear in your data. We will spend a lot of time discussing how you determine which x  variables to include in the model

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page8session two ► How is the regression model specified?  We believe ( assume ) that the true mean of y depends on the values of x 1, x 2, …, x k and let’s call this true mean as E [ y | x 1 x 2 … x k ]. Further, we believe ( assume ) that the true mean of y is different for different values of the x ’s and depends on the x ’s in a linear way E [ y | x 1 x 2 … x k ]   0   1 ·x 1   2 ·x 2  …   k ·x k 00 11 E [ y | x 1 x 2 … x k ]   0   1 ·x  As a simple example let’s consider a one- variable linear regression E [ y | x ]   0   1 ·x In the diagram on the left,  0 is the intercept while  1 is the slope of the regression line. y x Figure 5. y -variable population level (true) mean relationship to x -variable The Linear Regression: Concepts

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page9session two ► What is the difference between the true mean (unknown) and observed dependent variable?  The central assumption in the linear regression model is that the true mean (population mean) can be expressed as E [ y | x 1 x 2 … x k ]   0   1 ·x 1   2 ·x 2  …   k ·x k however we never observe all the possible values of y for each value of the explanatory variables, thus we write y   0   1 ·x 1   2 ·x 2  …   k ·x k    This is an example of a data generating process. It describes the real world process that generates the observations in the sample  The term  is called the error term and it contains all other factors that affect the y -variable but we are unable to identify or we are not aware of. The term ε is also sometimes called the idiosyncratic or random or stochastic part of the equation  The term  0   1 ·x 1   2 ·x 2  …   k ·x k is called the deterministic part of the regression and it is the component of sales that is attributable to known factors The Linear Regression: Concepts

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page10session two 00 00 y x y x  In the diagrams the red line is again the one-variable linear regression E [ y | x ]   0   1 ·x Notice in the top diagram that for each value of the explanatory variable x the y variable can take several possible values. In the bottom diagram, we see that for each x the explained variable y is distributed around the mean E [ y | x ]. Of course for each value of x, y has a different mean according to E [ y | x ]   0   1 ·x.  In the bottom diagram the distribution of y for each given x is normal around the mean. This is the normality assumption in the linear regression.  In both diagrams the distribution of y is the theoretical – population level - distribution of y. But this is never observed in practice but rather a sample from this distribution is observed. Figure 6. y -variable population level (true) mean relationship to x -variable The Linear Regression: Concepts

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page11session two ► What do we estimate in the regression model?  The top diagram shows the population level (true) relation between y and x. The parameters of the relation are the  0 (the intercept) and  1 (the slope). The “dots” representing all the possible values for y given each level of x.  The bottom diagram shows the sample level (estimated) relation between y and x. The parameters of the relation are the b 0 (the intercept) and b 1 (the slope). The blue “dots” represent the observed values for y given each level of x.  Since in a sample we only observe one value of y per value of x we can only obtain an estimate of the true relationship : Est.E [ y | x ]  b 0  b 1 ·x  The estimated relation is obtained through a procedure called ordinary least squares (OLS); coefficients b 0 and b 1 are the sample estimates of true parameters  0 and  1 respectively. 00 y 00 y population (true) relationship E [ y | x ]   0   1 ·x sample (estimated) relationship Est.E [ y | x ]  b 0  b 1 ·x b0b0 slope  1 slope b 1 Figure 7. Population level and sample mean The Linear Regression: Concepts

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page12session two ► Why using the (linear) regression model?  Linear regression is a powerful tool. In addition to its widespread direct use in management, many other analytic tools, such as ANOVA (analysis of variance), correlation, two-sample t -tests, and conjoint analysis are either special cases of or closely related to regression  It is the most popular and the easiest model to work with and often a good approximation to other, more complex, statistical models For now, we will suppose that no other aspect of the distribution of y (e.g., the variance of y ) depends on the x ’s. This makes it easier to estimate the linear regression (we will return to this assumption later in the course). The Linear Regression: Concepts

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page13 The Linear Regression: Interpretation session two ► As notation E [ sales | pop3mi inc3mi ] represent the true mean sales at considered locations with a given level of income and population in their surrounding communities. ► We assume that E[ sales | pop3mi inc3mi ] can be expressed as a linear function of income and population E [ sales | pop3mi inc3mi ]   0   1 ·pop   2 ·inc This is our regression equation: sales is the dependent (predicted) variable and, inc and pop are the independent (predictor) variables. We can write the regression in the data generating fashion as sales i   0   1 ·pop i   2 ·pnc i   I with i  1,2, …, n Since the main task is to estimate the parameters (coefficients)  0,  1 and  2 we should discuss them first:   0 is called the constant. It measures the true mean sales when pop  inc  0; this does not always have a sensible interpretation, inasmuch as it makes no sense to think of a population with 0 income   2 is called the coefficient on population. It measures how much true mean sales increases if population increases by exactly 1 person and income stays the same, i.e. holding income constant   1 is called the coefficient on income. It measures how much true mean sales increases if income increases by exactly $1 and population stays the same, i.e. holding population constant

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page14 The Linear Regression: Interpretation session two ► Estimating the regression model ■ The population level equation is E [ sales | pop3mi inc3mi ]   0   1 ·pop   2 ·inc ■ The estimated regression equation is Est. E [ sales | pop3mi inc3mi ]  b 0  b 1 ·pop  b 2 ·inc where  b 0 estimates  0, which is the average value of sales when inc = pop = 0  b 1 estimates  1, which is the change in mean sales when population increases by 1 person and income remains unchanged  b 2 estimates  2, which is the change in mean sales when income increases by $1 and population remains unchanged  Est. E [ sales | pop3mi inc3mi ] is the estimated mean sales: it estimates the true mean sales for given income and population

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page15 The Linear Regression: Interpretation session two regress sales pop3mi inc3mi generate yhat  _b[_cons]  _b[pop3mi]*pop3mi  _b[inc3mi]*inc3mi Source | SS df MS Number of obs = F(2, 21) = 2.56 Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = sales | Coef. Std. Err. t P>|t| [95% Conf. Interval] pop3mi | inc3mi | _cons | Figure 8. Regression results of sales on population and income estimated coefficients ( b 1, b 2 and b 0 ) p -values for two-tail tests of H 0 (  1 ):  1 = 0reject H 0 at  = 0.05 H 0 (  2 ):  2 = 0 cannot reject H 0 at  = 0.05 H 0 (  0 ):  0 = 0reject H 0 at  = 0.05 confidence intervals for the true parameters

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page16 The Linear Regression: Interpretation session two sales | Coef. Std. Err. t P>|t| [95% Conf. Interval] pop3mi | inc3mi | _cons | Figure 9. Regression results of sales on population and income ► What exactly are the ttest and p  value about? Are they related? hypothesis test decision H 0 :  2  0 H a :  2  0 The degrees of freedom are df  n  k  1 If p–value <  reject H 0 otherwise cannot reject H 0 at the significance level  _b[inc3mi] _se[inc3mi] ► The calculated results: scalar t_test_inc3mi  (_b[inc3mi]  0)/_se[inc3mi] display t_test_inc3mi display 2*ttail(21,abs(t_test_inc3mi)) This is very important: STATA will provide implicitly the results for this pair of hypotheses. It’s a two-tail test!

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page17 The Linear Regression: Interpretation session two sales | Coef. Std. Err. t P>|t| [95% Conf. Interval] pop3mi | inc3mi | _cons | Figure 10. Regression results of sales on population and income ► What about the confidence intervals? _b[inc3mi] _se[inc3mi] ► Since the results of the regression are basically estimates of true parameters it means we can at best provide confidence intervals for the value of the true parameters. STATA will provide, by default, a 95% confidence interval for each parameter. How exactly are they calculated? The “old” theory still applies (the confidence interval is centered in the sample-based estimate): with probability 1   ► The calculated results: scalar t_value_inc3mi  invttail(21,0.025) display _b[inc3mi]  _se[inc3mi]*t_value_inc3mi display _b[inc3mi]  _se[inc3mi]*t_value_inc3mi

Managerial Economics & Decision Sciences Department session two the linear regression model business analytics II Developed for intro to linear regression ◄ underlying concepts for the linear regression ◄ interpret linear regression results ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page18 Autoparts: The decission session two ► Using the estimated coefficients we can write the estimated model as: Est.mean sales   · population  · income  If Skokie Auto can provide reliable information about the population and income variables, as they pertain to its store, we can derive what would the implied mean sales be based on the general relation above (assumed to hold for all the Autoparts stores)  The logic is fairly simple: we assumed that there are two factors that drive sales volume, in particular population and income. Then we estimated, based on the sample available, how sales depend on the two factors. Finally, with this relation available we would use the particular population and income values corresponding to Skokie Auto to infer what the sales would have been if the changes in the landscape had not been taking place. If the inferred value for sales is significantly higher than $1,883 then Skokie Auto has a case against the bank.  We used the phrase “significantly higher” – what do we mean by that? How can we test whether the obtained estimate of sales based on the model is significantly different (higher or lower) than a specific value?