AP Statistics Chapter 11 Notes

Significance Test & Hypothesis Significance test: a formal procedure for comparing observed data with a hypothesis whose truth we want to assess. Significance test: a formal procedure for comparing observed data with a hypothesis whose truth we want to assess. Hypothesis: a statement about a population parameter. Hypothesis: a statement about a population parameter.

Null (H o ) and Alternative (H a ) Hypotheses The null hypothesis is the statement being tested in a significance test. The null hypothesis is the statement being tested in a significance test. Usually a statement of “no effect”, “no difference”, or no change from historical values. Usually a statement of “no effect”, “no difference”, or no change from historical values. The significance test is designed to assess the strength of evidence against the null hypothesis. The significance test is designed to assess the strength of evidence against the null hypothesis. The alternative hypothesis is the claim about the population that we are trying to find evidence for. The alternative hypothesis is the claim about the population that we are trying to find evidence for.

Example: One-sided test Administrators suspect that the weight of the high school male students is increasing. They take an SRS of male seniors and weigh them. A large study conducted years ago found that the average male senior weighed 163 lbs. Administrators suspect that the weight of the high school male students is increasing. They take an SRS of male seniors and weigh them. A large study conducted years ago found that the average male senior weighed 163 lbs. What are the null and alternative hypotheses? What are the null and alternative hypotheses? H o : μ = 163 lbs. H o : μ = 163 lbs. H a : μ > 163 lbs. H a : μ > 163 lbs.

Example: Two-sided test How well do students like block scheduling? Students were given satisfaction surveys about the traditional and block schedules and the block score was subtracted from the traditional score. How well do students like block scheduling? Students were given satisfaction surveys about the traditional and block schedules and the block score was subtracted from the traditional score. What are the null and alternative hypotheses? What are the null and alternative hypotheses? H o : μ = 0 H o : μ = 0 H a : μ ≠ 0 H a : μ ≠ 0 *You must pick the type of test you want to do before you look at the data.* *You must pick the type of test you want to do before you look at the data.* Be sure to define the parameter. Be sure to define the parameter.

Conditions for Significance Tests SRS SRS Normality (of the sampling distribution) Normality (of the sampling distribution) For means: For means: 1. population is Normal or 1. population is Normal or 2. Central Limit Theorem (n > 30) or 2. Central Limit Theorem (n > 30) or 3. sample data is free from outliers or strong skew 3. sample data is free from outliers or strong skew For proportions: For proportions: np > 10, n(1 - p) > 10 np > 10, n(1 - p) > 10 Independence (N > 10n) Independence (N > 10n)

Test Statistic Compares the parameter stated in H o with the estimate obtained from the sample. Compares the parameter stated in H o with the estimate obtained from the sample. Estimates that are far from the parameter give evidence against H o. Estimates that are far from the parameter give evidence against H o. For now we’ll us the z-test. For now we’ll us the z-test.

P-Value Assuming that H 0 is true, the probablility that the observed outcome (or a more extreme outcome) would occur is called the p-value of the test. Assuming that H 0 is true, the probablility that the observed outcome (or a more extreme outcome) would occur is called the p-value of the test. Small p-value = strong evidence against H 0. Small p-value = strong evidence against H 0. How small does the p-value need to be? How small does the p-value need to be? We compare it with a significance level (α – level) chosen beforehand. We compare it with a significance level (α – level) chosen beforehand. Most commonly α =.05 Most commonly α =.05

P-value continued If the p-value is as small or smaller than α, then the data are “statistically significant at level α”. If the p-value is as small or smaller than α, then the data are “statistically significant at level α”. Ex: α =.05 Ex: α =.05 If the p-value is <.05, then there is less than a 5% chance of obtaining this particular sample estimate if H 0 is true. If the p-value is <.05, then there is less than a 5% chance of obtaining this particular sample estimate if H 0 is true. Therefore we reject the null hypothesis. Therefore we reject the null hypothesis. If the p-value is >.05, our result is not that unlikely to occur. If the p-value is >.05, our result is not that unlikely to occur. Therefore we fail to reject the null hypothesis. Therefore we fail to reject the null hypothesis. If done by hand, the p-value must be doubled when performing a 2-sided test. The calculator will already display this doubled p- value if you choose the 2-sided option. If done by hand, the p-value must be doubled when performing a 2-sided test. The calculator will already display this doubled p- value if you choose the 2-sided option.

Confidence vs. Significance Performing a level α 2-sided significance test is the same as performing a 1 – α confidence interval and seeing if μ 0 falls outside of the interval. Performing a level α 2-sided significance test is the same as performing a 1 – α confidence interval and seeing if μ 0 falls outside of the interval. e.g. If a 99% CI estimated a mean to be (4.27, 5.12), then a significance test testing the null hypothesis H 0 : µ = 4 would be significant at α =.01. e.g. If a 99% CI estimated a mean to be (4.27, 5.12), then a significance test testing the null hypothesis H 0 : µ = 4 would be significant at α =.01.

Reminders about Significance Tests 1. Don’t place too much importance on “statistically significant”. 1. Don’t place too much importance on “statistically significant”. Smaller p-value = stronger evidence against H 0 Smaller p-value = stronger evidence against H 0 2.Statistical significance is not the same as practical importance. 2.Statistical significance is not the same as practical importance. 3. Don’t automatically use a test…examine the data and check the conditions. 3. Don’t automatically use a test…examine the data and check the conditions. 4. Statistical inference is not valid for badly- produced data. 4. Statistical inference is not valid for badly- produced data.

Mistakes in significance testing Type I error: Type I error: Reject H 0 when H 0 is actually true. Reject H 0 when H 0 is actually true. Type II Error: Type II Error: Fail to reject H 0 when H 0 is actually false. Fail to reject H 0 when H 0 is actually false.

Errors Continued

Errors continued The significance level α is the probability of making a Type I error. The significance level α is the probability of making a Type I error. Power: The probability that a fixed level α significance test will reject H 0 when a particular alternative value of the parameter is true. Power: The probability that a fixed level α significance test will reject H 0 when a particular alternative value of the parameter is true. Ways to increase the power. Ways to increase the power. Increase α Increase α Decrease σ Decrease σ Increase n Increase n