1. Managerial Economics & Decision Sciences Department hypotheses  tests  confidence intervals  business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II ▌ statistical models: hypotheses, tests & confidence intervals week 1 week 2 week 3

2. © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II readings ► statistics ► (MSN)  null and alternative hypotheses  testing a hypothesis  p  value  test significance level and test power: type I and type II errors  confidence intervals: construction and interpretation learning objectives  load, modify and save data  basic statistical tools and graphics  perform tests and build confidence intervals ►  Chapter 2 ► (CS)  Height, Marriages and Preferences session one statistical models: hypotheses, tests & confidence intervals business analytics II Developed for

3. Managerial Economics & Decision Sciences Department session one statistical models: hypotheses, tests & confidence intervals business analytics II Developed for a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page1 Why Taller Wife Couples Are So Rare ► The article “ Why It's So Rare for a Wife to Be Taller Than Her Husband ” asserts that the most common married couples have the husband five to six inches taller, and a small minority of couples have the wife taller. For 2009, based on a sample of 4,600 married couples, the distribution of the difference (in inches) between husband’s and wife’s height is shown in the diagram below. session one a first look at statistical hypotheses and tests Figure 1. Height difference for married couples difference between husband’s and wife’s height in inches quiz Suppose it’s year 2009 and you encounter a pair of two Americans (male and female). You estimate the height difference between the two (male height – female height) to be negative 2 inches, i.e. the female is taller than the male. ► Do you think the pair is actually a husband  wife couple? Assume that the sample distribution is the same for the whole population of all married couples.

4. Managerial Economics & Decision Sciences Department session one statistical models: hypotheses, tests & confidence intervals business analytics II Developed for a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page2 Answer Common sense indicates that for married couples it is very rare to see the woman 2 inches taller than the man (based on the provided distribution the likelihood of this situation is about 0.9218%). According to the this reasoning, you would probably conclude that it is very unlikely that the pair is a husband  wife couple. session one Why Taller Wife Couples Are So Rare a first look at statistical hypotheses and tests Figure 2. Tail or “extreme” values more extreme height differences than the observed one ► In fact the likelihood of obtaining even more (extreme) negative height differences than the currently observed height difference is 1.9722%. This likelihood (called the “ p  value ”) gives you a measure of how far in the “tail” of the distribution your currently observed value is. ► The lower this likelihood is the farther the observed value is in the tail of the distribution. An indication that the observed value is unlikely to “come from” the considered distribution.

5. Managerial Economics & Decision Sciences Department business analytics II Developed for a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page3 ► Summary of approach:  the hypothesis was stated as “the pair is married” (notice that data/information was available only for married couples) which allows you to reason further…  …“if the hypothesis is true then the observed height difference should not be extremely unlikely under the provided distribution for married couples”  you have to set the level at which “extremely unlikely” is indicative that the observed height difference does not come from the provided distribution for married couples and thus reject the hypothesis beyond any reasonable doubt. session one Why Taller Wife Couples Are So Rare hypothesis test decision a first look at statistical hypotheses and tests quiz If the only information available is the distribution of height difference for married couples, could the hypothesis “ the pair is not married ” be tested? Answer No. This hypothesis cannot be tested in the absence of information about the height difference for non  married persons since there is no way to find any evidence against the hypothesis “ the pair is not married ”. ► Notice how the hypothesis is formulated in a way that allows you to look for “credible” or “beyond any reasonable doubt” evidence against the hypothesis. If this evidence is found then you can reject the hypothesis. session one statistical models: hypotheses, tests & confidence intervals

6. Managerial Economics & Decision ► business analytics II Developed for a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page4 key concept : statistical testing components  a hypothesis providing a statement about a variable with assumed (known) distribution  a statistical test providing quantitative evidence used to potentially reject the hypothesis  a rule prescribing how the result of the statistical test is used to accept/reject the stated hypothesis session one Why Taller Wife Couples Are So Rare a first look at statistical hypotheses and tests test value comes from assumed underlying distributionun hypothesis is true likely test value is not farther into the tail of assumed distribution ( high p  value ) likely test value is not farther into the tail of assumed distribution ( high p  value ) logical implication flow test value does not come from assumed underlying distribution hypothesis is not true likely test value is farther into the tail of assumed distribution ( low p  value ) likely test value is farther into the tail of assumed distribution ( low p  value ) logical implication flow ► Once the hypothesis is stated and the underlying distribution is determined/assumed the following logical implications are considered: Figure 3. Hypothesis testing and p  value session one statistical models: hypotheses, tests & confidence intervals

7. Managerial Economics & Decision ► business analytics II Developed for a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page5 quiz Suppose it’s year 2009 and you encounter a pair of two Americans (male and female). You estimate the height difference between the two (male height – female height) to be positive 8 inches, i.e. the male is taller than the female. session one Why Taller Wife Couples Are So Rare a first look at statistical hypotheses and tests ► Do you think the pair is actually a husband  wife couple? Again, assume that the sample distribution is the same for the whole population of all married couples. Figure 4. Tail or “extreme” values height differences lower than the observed one Answer The likelihood to observe height differences less than 8 inches is 71.1852% and the likelihood to observe height differences larger than 8 inches is 28.8148%. It does not appear to have any credible evidence against the hypothesis that the pair is a husband  wife couple. height differences higher than the observed one session one statistical models: hypotheses, tests & confidence intervals

8. Managerial Economics & Decision ► business analytics II Developed for a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page6 ► A somewhat different way to use the distribution is to derive an interval centered in the mean height difference (approximately 6 inches) such that a proportion 1   of married couple will have the height difference within this interval. session one Why Taller Wife Couples Are So Rare a first look at statistical hypotheses and tests ► Say   10%; we need the lower bound ( lb ) and upper bound ( ub ) such that the sum of the frequencies (“area under the curve”) between lb and ub is exactly 90%. The added constraint is that the interval is centered in the mean of the distribution. ► For lb  0 and ub  12 we get the area under the curve between lb and ub approximately 90.3334%. Figure 5. “Confidence interval” for   10% 90% of married couples will exhibit height differences between 0 inches and 12 inches area  1 –  session one statistical models: hypotheses, tests & confidence intervals

9. Managerial Economics & Decision ► business analytics II Developed for a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page7 ► The average age at enrollment time for Kellogg’s Part  Time MBA 2014 Entrants is about 28.8 years. Total number of entrants is in the range 500  600. session one MBA Demographics formalizing hypothesis testing and confidence intervals Figure 6. Part  Time MBA 2014 Entrants ► The age distribution for 500 of Kellogg’s Part  Time MBA 2015 Entrants is shown below. For the moment let’s refrain from calculating directly the average age and female percentage. ( Details in the file MBADemo.dta ) Figure 7. Age Distribution Part  Time MBA 2015 Entrants session one statistical models: hypotheses, tests & confidence intervals

10. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page8 ► A random sample of size 40 was extracted from the population of 500 students. For full disclosure and later use, the procedure to extract the random sample is described below: session one MBA Demographics formalizing hypothesis testing and confidence intervals 1 shuffle the initial dataset generate random  runiform() sort random sample 40, count 2 choose the first 40 observations from the shuffled dataset summarize 3 visualize the summary statistics In the first step we add a new column called “random” in which we have random numbers between 0 and 1. At this point the dataset in the memory has the initial 500 observations thus there will be 500 random numbers added. When we sort the dataset by column “random” we basically rearrange (randomly) the observations in the data set. Finally, we pick the first 40 observations and since the initial observations we randomly shuffled, we can say that the sample we obtain is a random sample (i.e., we did not “cherry  pick”.) session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

11. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page9session one MBA Demographics formalizing hypothesis testing and confidence intervals Variable | Obs Mean Std. Dev. Min Max ---------+---------------------------------------------------------- Age | 40 29.5125 1.824504 25.9 35.1 Gender | 40.325.4743416 0 1 Figure 9. Sample Summary Statistics IndexAgeGender 127.41 230.61 328.21 ……… 3829.30 3930.70 4029.50 4128.60 ……… 49828.10 49927.30 50033.40 Figure 8. Part  Time MBA 2015 Entrants – Population (left) and randomly generated sample (right) IndexAgeGender 1832.20 43230.81 11228.60 ……… 37028.50 174300 38832.80 generate random  runiform() sort random sample 40, count drop random sample mean sample std. deviation sample size session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

12. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page10session one MBA Demographics formalizing hypothesis testing and confidence intervals key issue inference at population level ► The first key problem we want to address is the following: based solely on the available sample, can we infer anything about the average age at the population level relative to a pre-set benchmark X 0 ? ► We will answer the above question using the framework introduced earlier:  a statement about E [ X ]  a value that can be compared to an assumed/implied distribution under the stated hypothesis  the decision rule evaluates how far is the test into the tail(s) of assumed/implied distribution hypothesis test decision Remark In our “controlled environment” we have access to the population dataset from which we extracted the sample. This is for pedagogical purposes as we will perform the inference on some statistics based solely on the available sample and then evaluate our analysis by “revealing” the statistics at the population level. session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

13. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page11session one MBA Demographics formalizing hypothesis testing and confidence intervals hypothesis test decision Remark The pre-set benchmark can be any number you wish, however the choice for X 0 is a meaningful one. ► Consider the Part-Time MBA 2014 Entrants’ average age as X 0 = 28.8 and our hypothesis to be that the average (mean) age E [ X ] for Part-Time MBA 2015 Entrants is less than 28.8 years.  “less than” E [ X ]  X 0 key concept hypothesis ► The hypothesis is a statement relating the true mean E [ X ] with a pre-set benchmark X 0. The three typical hypotheses are:  “less than” E [ X ]  X 0 (H: the true mean is less than X 0 )  “greater than” E [ X ]  X 0 (H: the true mean is greater than X 0 )  “different than” E [ X ]  X 0 (H: the true mean is different than X 0 ) session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

14. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page12session one MBA Demographics formalizing hypothesis testing and confidence intervals hypothesis test decision key concept null and alternative hypotheses ► The hypothesis we want to test is called the null hypothesis ( H 0 ) and the opposite of the null hypothesis is called the alternative hypothesis ( H a ). Remark The null and alternative hypotheses together should be MECE (i.e., mutually exclusive and collectively exhaustive): ► The purpose of the testing step is to evaluate how strong is the evidence against the null hypothesis, i.e. try to find credible evidence (beyond any reasonable doubt) to reject the null hypothesis. ► This is the only way in which whatever decision we make “reject H 0 ” or “cannot reject H 0 ” we can say that the decision was made beyond any reasonable doubt. H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

15. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page13session one MBA Demographics formalizing hypothesis testing and confidence intervals ► A statement relating the true mean E [ X ] with a pre-set benchmark X 0. hypothesis test decision key concept sample estimator ► A statistics calculated at the population level (e.g., mean, standard deviation) can be estimated by the corresponding statistics based on an available sample with n observations extracted from the population. Remark A simple example: the sample average is an estimator for the true mean E [ X ]. ► The sample average of 29.51 years is the sample-based estimator for mean age of Part-Time MBA 2015 Entrants (a population of size 500). quiz Using the sample-based average X as an estimator for E [ X ] it seems we find evidence against the hypothesis that E [ X ]  X 0. However we do not have information about the distribution of age for the Part-Time MBA 2015 Entrants to gauge how far is 29.51 from 28.8. What distribution should we use? Answer (Not obvious at all.) There is a theoretical result that provides the distribution of the ratio that can be used in most parameter testing cases. session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

16. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page14session one MBA Demographics formalizing hypothesis testing and confidence intervals ► A statement relating the true mean E [ X ] with a pre-set benchmark X 0. hypothesis test decision key concept distribution of ttest statistics ► Suppose that we obtain a sample with n observation for a variable X with assumed mean X 0. Then Remark About notations: X is the sample mean of X and s X is the sample standard error of the mean. For this case, the t distribution has n – 1 degrees of freedom. ► The t distribution - a few important properties:  has a zero mean and is symmetric around its mean, thus its density is always centered in zero  has one parameter “degrees of freedom” ( df ) that affects its shape, in particular how thick are the tails (affecting the variance only) Figure 10. Density function for the t distribution  You can generate the density function for the t distribution using the STATA reserved function tden( df, x ), range(a b): twoway function tden( df, x ), range(  4 4) session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

17. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page15session one MBA Demographics formalizing hypothesis testing and confidence intervals ► A statement relating the true mean E [ X ] with a pre-set benchmark X 0. hypothesis test decision ► Provides calculated ttest that, given the hypothesis is true, comes from a t distribution key concept the p  value ► The p  value is the probability that a t  distributed variable would be further into the tail(s) than the calculated ttest. ttest Figure 11. Left tail p  value 1 – ttail( df, ttest ) Figure 12. Right tail p  value ttail( df, ttest ) |ttest| –|ttest|ttest Figure 13. Two tail p  value 1 – ttail( df,  |ttest |) ttail( df, |ttest |) p  value  Pr[ t  ttest ] area to the left of ttest p  value  Pr[ t  ttest ] area to the right of ttest p  value  Pr[ t  |ttest |] + Pr[ t  |ttest |] area to the left of  |ttest| and right of |ttest| session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

18. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page16session one MBA Demographics formalizing hypothesis testing and confidence intervals ► A statement relating the true mean E [ X ] with a pre-set benchmark X 0. hypothesis test decision ► Provides calculated ttest that, given the hypothesis is true, comes from a t distribution key concept significance level ► The significance level is the threshold for p  value below which you considered p  value small enough to indicate that the calculated ttest is too far into the tail of the t distribution and thus evidence against the stated hypothesis. Remark The common notation for significance level is the Greek letter  (0,1). The significance level is usually expressed as percentage (e.g., 5%, 10%, etc.) session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

19. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page17session one MBA Demographics formalizing hypothesis testing and confidence intervals ► A statement relating the true mean E [ X ] with a pre-set benchmark X 0. hypothesis test decision ► Provides the calculated ttest that, given the hypothesis is true, comes from a t distribution ► Evaluates whether the calculated ttest is too far into the tail of the t distribution to represent evidence against the stated hypothesis. hypotheses pair calculated ttest and distribution calculated p–value accept/reject decision Figure 14. Hypothesis testing H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 if p–value <  reject H 0 otherwise cannot reject H 0 at the significance level  if p–value <  reject H 0 otherwise cannot reject H 0 at the significance level  if p–value <  reject H 0 otherwise cannot reject H 0 at the significance level  session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

20. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page18session one MBA Demographics formalizing hypothesis testing and confidence intervals ► Regardless of which of the previous three pairs of null/alternative hypotheses you test, you’ll start with the ttest : ttest Age  28.8 Figure 15. Results of ttest command You have to specify the variable for which you conduct the test (Age) and what is the assumed mean (here we are using the mean of the population, but you can use any other value if that is the benchmark against which you test the sample mean.) X  29.51 s X  0.29 calculated ttest ► Here’s the ttest calculated “manually”: ► Notice that: X0X0 Variable | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- Age | 40 29.5125.2884794 1.824504 28.929 30.096 ------------------------------------------------------------------------------ mean  mean(Age) t  2.4698 Ho: mean  28.8 degrees of freedom  39 session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

21. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page19session one MBA Demographics formalizing hypothesis testing and confidence intervals ► A very negative ttest means this value is very much into the left tail of the t distribution. This implies that the area to the left of ttest is small too. ► The p  value is the probability that a t distributed variable takes even more extreme values than the calculated ttest : p  value  Pr[ t  ttest ] ► Graphically this is the area to the left of calculated ttest. ► The likelihood of rejecting H 0 increases as the p  value is smaller. This is an indication of ttest being too far into the tail of a t distribution  this should be an unlikely outcome if indeed ttest is t distributed. ttest p  value is the area to the left of ttest You can calculate p  value as 1 – ttail( df, ttest ) Figure 16. p  value for H 0 : E [ X ]  X 0 ► Consider: H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 Note that a X significantly lower than X 0 (implying ttest  0) might raise some concerns over H 0 being true. session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

22. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page20session one MBA Demographics formalizing hypothesis testing and confidence intervals ► A very positive ttest means this value is very much into the right tail of the t distribution. This implies that the area to the right of ttest is small too. ► The p  value is the probability that a t distributed variable takes even more extreme values than the calculated ttest : p  value  Pr[ t  ttest ] ► Graphically this is the area to the right of calculated ttest. ► The likelihood of rejecting H 0 increases as the p  value is smaller. This is an indication of ttest being too far into the tail of a t distribution  this should be an unlikely outcome if indeed ttest is t distributed. ► Consider: H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 Note that a X significantly higher than X 0 (implying ttest  0) might raise some concerns over H 0 being true. ttest p  value is the area to the right of ttest You can calculate p  value as ttail( df, ttest ) Figure 17. p  value for H 0 : E [ X ]  X 0 session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

23. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page21session one MBA Demographics formalizing hypothesis testing and confidence intervals ► A very negative or very positive ttest means this value is very much into the left or the right tail of the t distribution. This implies that the area to the left of  | ttest | plus the are to the right of | ttest | is small too. ► The p  value is the probability that a t distributed variable takes even more extreme values than the calculated ttest : p  value  Pr[ t   | ttest| ]  Pr[ t  | ttest| ] ► Graphically this is the area to the left of  | ttest | plus the area to right of | ttest|. ► The likelihood of rejecting H 0 increases as the p  value is smaller. This is an indication of ttest being too far into one of the tails of a t distribution – this should be an unlikely outcome if indeed ttest is t distributed. ► Consider: H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 Note that a X significantly different than X 0 (implying either ttest  0 or ttest  0) might raise some concerns over H 0 being true. |ttest| p  value is the sum of You can calculate p  value as 2*ttail( df,abs( ttest )) Figure 18. p  value for H 0 : E [ X ]  X 0 –|ttest| area to the left of –|ttest| area to the right of |ttest| session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

24. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page22session one MBA Demographics formalizing hypothesis testing and confidence intervals key concept type I and type II errors ► The analysis so far indicates that, regardless of the choice of null and alternative hypotheses, the smaller the corresponding p  value the more evidence we have against the null (i.e., more likely to reject H 0 ). ► But how small should p  value be in order to be sure we can reject H 0 ? The key point to grasp is here is that you can never be sure that the null is false. Thus you have to set a threshold (  ) such that you are comfortable rejecting H 0 whenever the calculated p–value is below this threshold. ► What  should you choose? ► The choice of  should be made independently of calculating the p  value, the significance level is really a subjective choice made by the analyst. However, in choosing the significance level  keep this in mind: type I error a large  means it is easier to reject the null H 0 (i.e., you will reject H 0 for a wider range of values for the p–value ). You are likely to reject H 0 when actually H 0 is true type II error a small  means it is harder to reject the null H 0 (i.e., you will reject H 0 for a narrower range of values for the p–value ). You are likely to accept H 0 when actually H 0 is false session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

25. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page23session one MBA Demographics formalizing hypothesis testing and confidence intervals ► Back to the output of our previous ttest : ttest Age  28.8 Figure 19. Results of ttest command Notice that the STATA output provides the p  value for three alternative hypotheses. It is analyst’s job to choose the correct alternative H a. This is not difficult since the choice of the alternative is the opposite of the null that the analyst has stated from the beginning. calculated ttest X0X0 p–value  0.991 p–value  0.018 p–value  0.009 Variable | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- Age | 40 29.5125.2884794 1.824504 28.929 30.096 ------------------------------------------------------------------------------ mean = mean(Age) t = 2.4698 Ho: mean = 28.8 degrees of freedom = 39 Ha: mean 28.8 Pr(T |t|) = 0.0180 Pr(T > t) = 0.0090 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

26. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page24session one MBA Demographics formalizing hypothesis testing and confidence intervals Figure 20. Graphical interpretation for results of ttest command 2.47 – 2.472.47 You can calculate the corresponding p–value using : 1 – ttail(39,2.47) 2*ttail(39,2.47) ttail(39,2.47) mean = mean(Age) t = 2.4698 Ho: mean = 28.8 degrees of freedom = 39 Ha: mean 28.8 Pr(T |t|) = 0.0180 Pr(T > t) = 0.0090 p–value  0.991 p–value  0.018 p–value  0.009 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

27. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page25session one MBA Demographics formalizing hypothesis testing and confidence intervals quiz Which null hypothesis would you be able to reject? Answer The accept/reject decision depends on the chosen significance level . Say   0.05 (5.00%) then Figure 21. Results of ttest command mean = mean(Age) t = 2.4698 Ho: mean = 28.8 degrees of freedom = 39 Ha: mean 28.8 Pr(T |t|) = 0.0180 Pr(T > t) = 0.0090 p–value  0.991 p–value  0.018 p–value  0.009 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 H 0 : E [ X ]  X 0 H a : E [ X ]  X 0 cannot reject null at 5% significance level as p  value   reject null at 5% significance level as p  value   session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

28. Managerial Economics & Decision ► business analytics II Developed for a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄ © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page26session one MBA Demographics formalizing hypothesis testing and confidence intervals key concept confidence intervals ► The interval, centered in the sample-based average, that would contain the true mean with a given probability (confidence level) is referred to as a confidence interval. Remark The sample provides an estimator X for E [ X ], and we analyzed a way to test various relations between E [ X ] and a pre-set benchmark X 0. However the tests do not provide an idea on how far apart are X and E [ X ]. The aim is to be able to make statements like: “with a given probability the interval [ lb, ub ] contains the true mean E [ X ]”. key issue inference at population level ► The second key problem we want to address is the following: based solely on the available sample, can we infer anything about the range of the average age at the population level ? ► We already know that: ► Then for any number x we can calculate then the following probability:  area between  x and x under the t distribution density session one statistical models: hypotheses, tests & confidence intervals

29. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page27session one MBA Demographics formalizing hypothesis testing and confidence intervals This probability is the area between – x and x under the t distribution density. Using the symmetry of the t distribution it’s easy to see that this area could be calculated as 1  (area to the left of  x )  (area to the right of x ) that is 1  (1  ttail( df,  x ))  (ttail( df, x ))  1  2*ttail( df, x ) We can restate the “ Indirect ” problem as  find x such that 1  (area to the left of  x )  (area to the right of x )  1  A But (area to the left of  x )  (area to the right of x ) and so the “ Indirect ” problem becomes:  find x such that 1  2  (area to the right of x )  1  A Finally:  find x such that area to the right of x is A /2 22 2 1  ttail(39,  2) (0.02625) ttail(39,2) (0.02625) area between  2 and 2 is 0.9475 Figure 22. “ Direct ” problem: Graphical solution xx x area to the left of  x area between  x and x is 1  A Figure 23. “ Indirect ” problem: Graphical solution area to the right of x “Direct” problem ► For a given x calculate Pr[  x  ttest  x ] df = 39 x = 2 “Indirect” problem ► For a given A  (0,1) find x such that Pr[  x  ttest  x ]  1  A session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

30. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page28session one MBA Demographics formalizing hypothesis testing and confidence intervals ► The “ Indirect ” problem was reduced to solve for x such that area to the right of x is A /2 As a reminder, the STATA function ttail( df, x ) provides the area to the right of x. Quite conveniently, STATA provides also the inverse function invttail( df, area ) that calculates the x for which the area to the right of x is exactly area. ► Using the STATA reserved function invttail( df, area ) with area  A /2  0.025 we get the solution x  2.0226909 as invttail(39,0.025)  2.022 area  0.025 area between  x and x is 1  A  0.95 Figure 24. “ Indirect ” problem: Graphical solution area  0.025 2.022 –invttail(39,0.025) invttail(39,0.025) ► We can summarize the setup for the “ Indirect ” problem as follows: for a given  (0,1) solve for x such that Pr[  x  ttest  x ]  1  . ► The solution is x  invttail( df,  /2). You will find the standardized notation t df,  /2  invttail( df,  /2) thus x  t df,  /2. ► Thus we know that  x  ttest  x with probability 1   that is with probability 1   session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

31. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page29session one MBA Demographics formalizing hypothesis testing and confidence intervals ► With a bit of algebra we can show that the statement is equivalent to with probability 1   ► This is exactly the confidence interval we were looking for. Since we already run the ttest command we do have all information necessary to derive the interval: Figure 25. Results of ttest command X  29.51 s X  0.29 Variable | Obs Mean Std. Err. Std. Dev. [95% Conf. Interval] ---------+-------------------------------------------------------------------- Age | 40 29.5125.2884794 1.824504 28.929 30.096 ------------------------------------------------------------------------------ mean  mean(Age) t  2.4698 Ho: mean  28.8 degrees of freedom  39 ► For   5.00% we get t df,  /2  invttail(39,0.025)  2.0226909 and so the interval is lb = 29.51  0.29  2.023  28.92333 and ub = 29.51  0.29  2.023  30.09667 The interval (28.92333, 30.09667) is an approximation as we used rounded values to simplify calculations. session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄

32. Managerial Economics & Decision ► business analytics II Developed for © 2016 kellogg school of management | managerial economics and decision sciences department | business analytics II | page30session one MBA Demographics formalizing hypothesis testing and confidence intervals We can obtain directly the confidence interval for the mean using: ci means var, level(1  ) Remark. For var you have to specify the name of the variable exactly as defined in the data set (e.g., Age) while for level you should specify it as 95 when   5%, 90 when   10%, etc. Variable | Obs Mean Std. Err. [95% Conf. Interval] ---------+--------------------------------------------------------------- Age | 40 29.5125.2884794 28.929 30.096 Figure 26. Results of ci command for   5% X  29.51 s X  0.29 confidence interval always centered in the mean 1    95 ci means Age, level(95) variable name quiz Suppose we construct the confidence interval for   10%. Is this confidence interval narrow/wider than the confidence interval constructed for   5%? Answer Narrower! session one statistical models: hypotheses, tests & confidence intervals a first look at statistical hypotheses and tests ◄ formalizing hypothesis testing and confidence intervals ◄ confidence intervals ◄