Section 8.2 - Testing a Proportion Objectives: To understand the logic of a significance test for a proportion To know the meanings of Type I and Type II errors and how to reduce their probability To know the meaning of the power of a test and how to increase it.

Section 8.2 - Testing a Proportion Review: We want to make a decision based on data by comparing the results from a sample (a sample proportion) to some predetermined standard (an hypothesis about the population proportion) . These kinds of decisions are called tests of significance or hypothesis tests.

Section 8.2 - Testing a Proportion Types of Errors The goal of a test of significance is to evaluate a claim about a population proportion. Assume that the population proportion is equal to some standard (p = p0). This assumption is called the null hypothesis. Use sample data to compute a sample proportion. Compute a test statistic (convert the sample proportion to a z-score). Compute a P-value and compare it to the significance level , or compare the test statistic to the critical value (based on ). If P-value < , or if |z| > |z*|, reject the null hypothesis (H0: p = p0). If P-value > , or if |z| < |z*|, fail to reject the null hypothesis. Remember that a test of significance is based on data from a sample, and is therefore subject to sampling variability. There are two errors (incorrect decisions) that are possible. What are they?

Section 8.2 - Testing a Proportion Types of Errors If the null hypothesis (H0: p = p0) is true, and we make a mistake and reject it, we have made a Type I Error. If the null hypothesis (H0: p = p0) is false, and we make a mistake and fail to reject it, we have made a Type II Error. Clearly, we want to minimize both types of errors, if possible. Which error is more serious?

Section 8.2 - Testing a Proportion Types of Errors Jury Trial Defendant is Actually Innocent Guilty Jury’s Decision Not Guilty Correct Error Worse error

Section 8.2 - Testing a Proportion Types of Errors Jury Trial Defendant is Actually Innocent Guilty Jury’s Decision Not Guilty Correct Error Worse error Significance Testing Null Hypothesis is Actually True False Your Decision Don’t Reject H0 Correct Type II Error Reject H0 Type I Error

Section 8.2 - Testing a Proportion Example: Miguel & Kevin Miguel & Kevin’s test statistic was large in absolute value: z = -3.16. They concluded that spinning a penny was not fair. These are the possibilities: The null hypothesis is true and a rare event occurred. The null hypothesis is false, and that’s why their result was so far from p. The sampling process was biased, so their result is questionable. If we rule out the last possibility, since z = -3.16, the decision will be to reject the null hypothesis. However, we could be making a Type I error - rejecting H0 when in fact it is true. Note that making a Type I error does not mean that you did anything wrong - it’s just bad luck.

Section 8.2 - Testing a Proportion Example: Jenny & Maya Jenny & Maya’s test statistic was small in absolute value: z = -0.95. They concluded that their result was consistent with the idea that spinning a penny is fair. These are the possibilities: The null hypothesis is true; that’s why the test statistic was so small. The null hypothesis is false, and it was just by chance that their result was so close to p. The sampling process was biased, so their result is questionable. If we rule out the last possibility, since z = -0.95 the conclusion is that we cannot reject the null hypothesis. However, we could be making a Type II error - failing to reject H0 when it is false. Note that making a Type II error does not mean that you did anything wrong - it’s just bad luck.

Section 8.2 - Testing a Proportion Type I Error Suppose the null hypothesis is true. What is the chance of (incorrectly) rejecting H0 and making a Type I error?

Section 8.2 - Testing a Proportion Type I Error Suppose the null hypothesis is true. What is the chance of (incorrectly) rejecting H0 and making a Type I error? The only way to make a Type I error is to get a rare event from the sample. For example, if the significance level is 0.05 ( = 0.05), you would make a Type I error if z > 1.96 or z < -1.96. This happens 5% of the time. That is, P(z ≤ -1.96 or z ≥ 1.96) = 0.05. If the null hypothesis is true, the probability of a Type I error (the probability of a rare event) is equal to the significance level, , of the test. To lower the chance of a Type I error, you should use a lower level of significance, , with larger critical values, z*. A lower level of significance makes it harder to reject H0

Section 8.2 - Testing a Proportion Types of Errors You commit a Type I Error if you incorrectly reject the null hypothesis when it is true. You commit a Type II Error if you incorrectly fail to reject the null hypothesis when it is false.

Section 8.2 - Testing a Proportion Power of a Test The power of a test is the probability of rejecting the null hypothesis. When the null hypothesis is false When the null hypothesis is true, you can’t make a Type II error.

Section 8.2 - Testing a Proportion Power of a Test If Jenny & Maya had spun more pennies, they would have collected more evidence and gotten a different result. (Spinning pennies, as Miguel & Kevin determined, and as we saw in our class, is not fair.) Their sample size was too small (failure to reject H0 is the result of insufficient evidence). As a result, they made a Type II error (failure to reject H0 when it is false) Their test did not have enough power to be able to detect that p ≠ p0

Section 8.2 - Testing a Proportion Power of a Test If Jenny and Maya had spun more pennies, they would have collected more evidence and gotten a different result. (Spinning pennies, as we saw in our class, is not fair.) The power of a test is the probability of rejecting the null hypothesis.

Section 8.2 - Testing a Proportion Power of a Test The power of a test is the probability of rejecting the null hypothesis. If the null hypothesis is false, we want to reject it. If we fail to reject it, we make a Type II error. Power = 1 - probability of a Type II error If the probability of a Type II error is small, the power of the test is large.

Section 8.2 - Testing a Proportion Types of Error and Power Type I Error When the null hypothesis is true and you reject it, you have made a Type I error. Type II Error When the null hypothesis is false and you fail to reject it, you have made a Type II error. Power Power is the probability of rejecting the null hypothesis.

Section 8.2 - Testing a Proportion Type I Error When the null hypothesis is true and you reject it, you have made a Type I error. The probability of making a Type I error is equal to the significance level, , of the test. To decrease the probability of a Type I error, make  smaller. Changing the sample size n has no effect on the probability of a Type I error. Decreasing  makes it harder to reject H0 If the null hypothesis is false, you can’t make a Type I error.

Section 8.2 - Testing a Proportion Type II Error When the null hypothesis is false and you fail to reject it, you have made a Type II error. To decrease the probability of making a Type II error, take a larger sample (increase n), or increase the significance level . Increasing  makes it easier to reject H0 If the null hypothesis is true, you can’t make a Type II error.

Section 8.2 - Testing a Proportion Power Power is the probability of rejecting the null hypothesis. When the null hypothesis is false, you want to reject it and therefore you want the power to be large. To increase power, either increase the sample size n or increase the significance level . (Collect more data, or adjust  so that it is easier to reject H0)