1. Section 8.2 - Testing a Proportion
Objectives:
To understand the logic of a significance test for a proportion
To know the meanings of Type I and Type II errors and how to reduce their probability
To know the meaning of the power of a test and how to increase it.

2. Section 8.2 - Testing a Proportion
Review:
We want to make a decision based on data by comparing the results from a sample (a sample proportion) to some predetermined standard (an hypothesis about the population proportion) .
These kinds of decisions are called tests of significance or hypothesis tests.

3. Section 8.2 - Testing a Proportion
Types of Errors
The goal of a test of significance is to evaluate a claim about a population proportion.
Assume that the population proportion is equal to some standard (p = p0). This assumption is called the null hypothesis.
Use sample data to compute a sample proportion. Compute a test statistic (convert the sample proportion to a z-score).
Compute a P-value and compare it to the significance level , or compare the test statistic to the critical value (based on ).
If P-value < , or if |z| > |z*|, reject the null hypothesis (H0: p = p0).
If P-value > , or if |z| < |z*|, fail to reject the null hypothesis.
Remember that a test of significance is based on data from a sample, and is therefore subject to sampling variability. There are two errors (incorrect decisions) that are possible. What are they?

4. Section 8.2 - Testing a Proportion
Types of Errors
If the null hypothesis (H0: p = p0) is true, and we make a mistake and reject it, we have made a Type I Error.
If the null hypothesis (H0: p = p0) is false, and we make a mistake and fail to reject it, we have made a Type II Error.
Clearly, we want to minimize both types of errors, if possible.
Which error is more serious?

5. Section 8.2 - Testing a Proportion
Types of Errors
Jury Trial
Defendant is Actually
Innocent
Guilty
Jury’s Decision
Not Guilty
Correct
Error
Worse error

6. Section 8.2 - Testing a Proportion
Types of Errors
Jury Trial
Defendant is Actually
Innocent
Guilty
Jury’s Decision
Not Guilty
Correct
Error
Worse error
Significance Testing
Null Hypothesis is Actually
True
False
Your Decision
Don’t Reject H0
Correct
Type II Error
Reject H0
Type I Error

7. Section 8.2 - Testing a Proportion
Example: Miguel & Kevin
Miguel & Kevin’s test statistic was large in absolute value: z = They concluded that spinning a penny was not fair.
These are the possibilities:
The null hypothesis is true and a rare event occurred.
The null hypothesis is false, and that’s why their result was so far from p.
The sampling process was biased, so their result is questionable.
If we rule out the last possibility, since z = -3.16, the decision will be to reject the null hypothesis.
However, we could be making a Type I error - rejecting H0 when in fact it is true.
Note that making a Type I error does not mean that you did anything wrong - it’s just bad luck.

8. Section 8.2 - Testing a Proportion
Example: Jenny & Maya
Jenny & Maya’s test statistic was small in absolute value: z = They concluded that their result was consistent with the idea that spinning a penny is fair.
These are the possibilities:
The null hypothesis is true; that’s why the test statistic was so small.
The null hypothesis is false, and it was just by chance that their result was so close to p.
The sampling process was biased, so their result is questionable.
If we rule out the last possibility, since z = the conclusion is that we cannot reject the null hypothesis.
However, we could be making a Type II error - failing to reject H0 when it is false.
Note that making a Type II error does not mean that you did anything wrong - it’s just bad luck.

9. Section 8.2 - Testing a Proportion
Type I Error
Suppose the null hypothesis is true.
What is the chance of (incorrectly) rejecting H0 and making a Type I error?

10. Section 8.2 - Testing a Proportion
Type I Error
Suppose the null hypothesis is true.
What is the chance of (incorrectly) rejecting H0 and making a Type I error?
The only way to make a Type I error is to get a rare event from the sample.
For example, if the significance level is 0.05 ( = 0.05), you would make a Type I error if z > 1.96 or z < This happens 5% of the time.
That is, P(z ≤ or z ≥ 1.96) = 0.05.
If the null hypothesis is true, the probability of a Type I error (the probability of a rare event) is equal to the significance level, , of the test.
To lower the chance of a Type I error, you should use a lower level of significance, , with larger critical values, z*.
A lower level of significance makes it harder to reject H0

11. Section 8.2 - Testing a Proportion
Types of Errors
You commit a Type I Error if you incorrectly reject the null hypothesis when it is true.
You commit a Type II Error if you incorrectly fail to reject the null hypothesis when it is false.

12. Section 8.2 - Testing a Proportion
Power of a Test
The power of a test is the probability of rejecting the null hypothesis. When the null hypothesis is false
When the null hypothesis is true, you can’t make a Type II error.

13. Section 8.2 - Testing a Proportion
Power of a Test
If Jenny & Maya had spun more pennies, they would have collected more evidence and gotten a different result. (Spinning pennies, as Miguel & Kevin determined, and as we saw in our class, is not fair.)
Their sample size was too small (failure to reject H0 is the result of insufficient evidence).
As a result, they made a Type II error (failure to reject H0 when it is false)
Their test did not have enough power to be able to detect that p ≠ p0

14. Section 8.2 - Testing a Proportion
Power of a Test
If Jenny and Maya had spun more pennies, they would have collected more evidence and gotten a different result. (Spinning pennies, as we saw in our class, is not fair.)
The power of a test is the probability of rejecting the null hypothesis.

15. Section 8.2 - Testing a Proportion
Power of a Test
The power of a test is the probability of rejecting the null hypothesis.
If the null hypothesis is false, we want to reject it.
If we fail to reject it, we make a Type II error.
Power = 1 - probability of a Type II error
If the probability of a Type II error is small, the power of the test is large.

16. Section 8.2 - Testing a Proportion
Types of Error and Power
Type I Error
When the null hypothesis is true and you reject it, you have made a Type I error.
Type II Error
When the null hypothesis is false and you fail to reject it, you have made a Type II error.
Power
Power is the probability of rejecting the null hypothesis.

17. Section 8.2 - Testing a Proportion
Type I Error
When the null hypothesis is true and you reject it, you have made a Type I error.
The probability of making a Type I error is equal to the significance level, , of the test.
To decrease the probability of a Type I error, make  smaller.
Changing the sample size n has no effect on the probability of a Type I error.
Decreasing  makes it harder to reject H0
If the null hypothesis is false, you can’t make a Type I error.

18. Section 8.2 - Testing a Proportion
Type II Error
When the null hypothesis is false and you fail to reject it, you have made a Type II error.
To decrease the probability of making a Type II error, take a larger sample (increase n), or increase the significance level .
Increasing  makes it easier to reject H0
If the null hypothesis is true, you can’t make a Type II error.

19. Section 8.2 - Testing a Proportion
Power
Power is the probability of rejecting the null hypothesis.
When the null hypothesis is false, you want to reject it and therefore you want the power to be large.
To increase power, either increase the sample size n or increase the significance level .
(Collect more data, or adjust  so that it is easier to reject H0)