Inference We want to know how often students in a medium-size college go to the mall in a given year. We interview an SRS of n = 10. If we interviewed lots of SRSs, the “average sample frequency of visits” would be centered around the true “average population frequency of visits.”

Inference Suppose that instead we interviewed an SRS of n = 400. Our estimates will be more reliable because estimates from other SRSs would be similar … that is, our estimates would be less variable.

Because we didn’t have money for 16 separate samples, we actually only collected data from the first sample, whose sample mean is = Is the true number actually 50? Is the difference between 50 and purely a fluke? Does this result exclude 50 as a possibility?

The Central Limit Theorem says that if the entire population has a mean  and a standard deviation , then in repeated samples of size n the sample mean approximately follows a Normal distribution

The first sample had a mean = and a standard deviation = nStrd Dev of = = Sample A = /sqrt(10) Sample B = /sqrt(400)

We know that 95% of all observations fall within ± two standard deviations of the mean. Likewise, 95% of all sample means fall within ± two standard deviations of the observed sample mean. So, for 1900 out of 2,000 samples, the interval will contain the true population mean.

2 x (0.0513)

Now there are two possibilities. Either 1. the true population mean is contained in the interval 2. or this is one of those 5% of samples whose interval does not contain the true value. ( , )

C is typically set at 95%, but it’s sometimes chosen to be 90% or 99%. STATA Exercise 1

z* = 1.96 if C=95%-z*= if C=95%

So don’t use 2 when constructing a 95% CI: use 1.96.

If the margin of error is too large… Reduce  –  is determined by the population: a population with a lot of variability will increase the chance that a sample contain observations very far from the true mean. – This is easier to say than to do.

16 samples. The  of the population increases from 1 to 4, increasing the spread of the sample and the likelihood of getting  wrong.

If the margin of error is too large… Increase the sample size (larger n)

If the margin of error is too large… Be less confident of your estimate … Use a lower confidence level (make C smaller, hence a smaller z*)

If the margin of error is too large… “We’re 99% sure that the President will receive 51.5% of the votes, with a ±5% margin of error.” “We’re 95% sure that the President will receive 51.5% of the votes, with a ±3% margin of error.” “We’re 90% sure that the President will receive 51.5% of the votes, with a ±1% margin of error.”

Cautions 1. Is it an SRS? 2. Is the data unbiased (or do we know the bias)? 3. Are there no outliers that influence the sample mean? 4. Is n large? If not, is the underlying population Normally distributed? 5. Do you know the true  ? Theorems of mathematical statistics are true; statistical methods are effective only when used with skill.

Cautions FALSE: “The probability that the true mean falls within is 95%” – This is false because either the interval contains the true population mean (which is not a random variable), with Pr=1, or it doesn’t, with Pr=0. TRUE: “The probability that the interval is one of the ones that contain the true mean is 95%”

Tests of Significance

Making claims about the population parameters In our sample, we observed a mean of visits to the mall per year. – Assuming that the true population mean is 50, how likely is it that we observe a sample mean as small as , or even smaller? – if the true population mean were 45, how likely is it that we observe a sample mean as large as , or even larger?

Making claims about the population parameters

x if the true population mean were 45, how likely is it that we observe a sample mean as large as ? Pr=0.68% If the true population mean were 50, how likely is it that we observe a sample mean at least as small as ? Pr=49% Making claims about the population parameters

We found that if the true mean is 45, the Pr of observing a sample mean as large as is 0.68%. Either – we’ve observed a very rare event (our sample is really unusual) – the true mean is not 45. There’s another number that makes the observed sample more likely. Making claims about the population parameters A sample outcome that would be extreme if a hypothesis were true is evidence that they hypothesis is not true.

H 0 :  =45 H a :  45 This is a two- sided alternative hypothesis These are hypothesis about the population.

Test Statistics A test statistic measures compatibility between the null hypothesis and the data. The z-score can be used as a test statistic because we can compare it against 1.96, the z-score that delimits a 0.95 area under the Normal curve. – 1.96 is called the appropriate “critical value”.

Test Statistics The Student’s t Distribution is used when n is small. It approximates the Standard Normal, z- distribution as n gets large.

Test Statistics We know that 95% of all values are between 2 standard deviations of the mean. That is, 95% of all values are between the z- score of 1.96 and the z-score of So if we get a sample outcome whose z- score is greater than 1.96 (in absolute value), we know that it it is unlikely to belong to the population of which the null hypothesis is a parameter.

Suppose – n = 110 –  = 26.4 – x = 8.1 – H 0 :  = 0 – H a :   0

Exercise A company makes cellphones using components from two countries: Ecuador and Canadaguay. Here are data on days of cellphone durability. Your retail shop buys 100 cellphones because the manufacturer claims they were made in Ecuador. On average, they stop working after 279 days of use. Is this difference (279 days versus 300 days) significant? Is it a fluke or does it mean something? # days till broken  Ecuador Canadaguay10050

Exercise The null hypothesis is that the phone typically lasts 300 days. Alternatively, it’s a lower quality phone. The z-score can tell us how far this observation is from the mean. Look up in table A the probability of observing a z-score as small as this or smaller.

Exercise Suppose the parameters were, instead Now, is this difference (279 days versus 300 days) significant? Is it a fluke or does it mean something? # days till broken  Ecuador300200

Exercise Suppose average durability of the 100 cellphones was, instead, 90 days. Now, is this difference (90 days versus 300 days) significant? Is it a fluke or does it mean something? # days till broken  Ecuador300200

We found that if the true mean is 45, the Pr of observing a sample mean as large as is 0.68%. Notice that here H 0 :  = 45 H a :  > 45 This is a one- sided alternative hypothesis

Look this up in Table D, 20-1 degrees of freedom. We have to use the Student’s t because n is small.

Tests for Population Mean 1. State the hypothesis 2. Calculate the test statistics 3. Find the P-value 4. State your conclusion in the context of your specific setting

C = 1-  for two-sided tests

 = x = H 0 :  = 0.86 n = 3 Look in Table D for the z-score on a two-tailed 1% significance level (look in the column) for df = 3-1. Is it smaller (in absolute value) than ?

 = x = H 0 :  = 0.86 n = 3 The 99% CI is ( , ) cii , level(99) Look up the t* for df=3-1, upper tail probability 0.005

P-values versus a fixed  If the z-score is , the corresponding p- value is The p-value is the smallest level of  at which the data are significant. Remember that C = 1-  for two-sided tests, and that bigger Confidence means wider CI. “The smallest level of  ” then mean the largest C and widest CI that will still contain the hypothesized value.

H 0 :  x p-value

If the P-value is larger than the chosen significance level , we say that the statistic is not significant. If the P-value is smaller than the chosen significance level , we say that the statistic is not significant.

Using Significance Tests

. tabstat guess grade diff if position<8 stats | guess grade diff mean | tabstat guess grade diff if position>=8 stats | guess grade diff mean | Is it true that, on average, people who finish earlier tend to do better? (Notice causality is not determined).

Significance Tests H 0 is our hypothesis: how plausible is it, given the data, our statistic, and its sampling variation? – If a priori H 0 seems true, very small p-values will be needed to convince people that H 0 are wrong. A small p-value means that your estimated statistic is so far from H 0 that it’s unlikely that your statistics is derived from a population where H 0 is true. H0H0

Significance Tests H 0 is our hypothesis: what are the consequences of rejecting H 0. – If rejecting H 0 led to huge changes in our behavior, with large costs, we’ll need to be very convinced. H0H0

Significance Tests Decide on a significance level, . – Remember  = 1 - C, where C is the confidence level Check if the P-value is below your pre- decided significance level. H 0 :  x p-value H 0 :  x p-value

Significance Tests Check for the practical significance (the actual size of the number) of a statistic that is statistically significant. Do exploratory data analysis. – Check for outliers. – Check for the Normality of the data. Report confidence intervals. Excel and icosahedron exercise 1

Power and Inference as a Decision

A more powerful test will be more likely to reject a false H 0 in favor of true alternatives.