1. Unit 9.1B: Testing a Claim Significance Tests: The Basics Using α-levels Type I and Type II Errors The Power of a Test 1

2. Objectives 9.1B: Interpret the p-value in context Use the p-value and alpha to make a decision Describe and interpret Type I and Type II errors in context Describe how to change the Power of a test 2

3. Jenn is a golfer who would like to improve her play. She borrows a 7-iron from a friend. Based on years of experience, Jenn has established that she hits the ball on average 175 yards with a standard deviation of 15 yards when she uses her old 7-iron. Jenn is hoping that the new 7-iron will improve her consistency and therefore goes to the driving range and hits 50 shots with the new 7-iron. 3

4. a) Interpret the P-value in context. b) Do the data provide convincing evidence against the null? (explain!) 4

5. P-valueDecisionConclusion SmallReject HoHa (in context) LargeFail to reject Ho Not enough evidence to reject Ho (in context) If the P-value is small, then it is unlikely that the observed statistic could have happened due to chance alone when Ho is true. Therefore we have good evidence against the null. 5

6. Statistical Significance: When are the results of a study statistically significant? When the P-value is lower than our α-level. What is an α-level? 6

7. The α-level (known as the significance level) is a pre-determined, fixed value that we regard as decisive. 7

8. For example, if we choose α = 0.05 we are saying that we require our statistical results to be so rare that they would happen less than 5% of the time due to chance alone when Ho is true! If we choose α = 0.01 then we want our evidence to be even stronger! In this case, results that would occur only 1% of the time due to chance alone!

9. Significant does not mean important. It simply means not likely to happen just by chance. 9

10. Mrs. McDonald would like to see if students prefer cookies made with regular vanilla versus those made with Mexican vanilla. She randomly selects 50 students and has each student try both type of cookies in random order. 34 of the 50 preferred the cookies with Mexican vanilla. Mrs. McD performed a significance test using Ho: p = 0.5 versus Ha: p > 0.5 where p = true proportion of students who prefer Mexican vanilla. The P-value = 0.045. 10

11. What conclusion can we make: a)At α = 0.05? b)At α = 0.01? 11

12. 12 If the Null is true I reject the null based on my sample This is an error I fail to reject the null based on my sample This a correct decision

13. 13 If the Null is true I reject the null based on my sample This is an error I fail to reject the null based on my sample This a correct decision If the Null is false I reject the null based on my sample This is a correct decision I fail to reject the null based on my sample This is an error

14. True state of the universe Ho is True Ho is False Decision we make based on the sample Reject Ho Type I error Correct Decision Fail to reject Ho Correct Decision Type II error 14

15. 15

16. True state of the universe Ho is TrueHo is False Decision we make based on the sample Reject Ho Type I error Correct Decision Fail to reject Ho Correct Decision Type II error P(Type I error) = α P(Type II error) = β But α is also the significance level….and we choose that ourselves! 16

17. The power of a statistical test is the probability that the test will reject the null hypothesis when the null hypothesis is false (i.e. the probability of not committing a Type II error) Power = 1 − β 17

18. Power = 1 − β Power = Probability of NOT making a Type II error A higher β means the less able your test is to detect Ha when Ha is actually true. 18

19. How do you get higher power? (By getting smaller β!)  Increase your sample size  Increase α (uh, oh….)  Collect data in a way that reduces variability (stratification, blocking)  Move the hypothesized value farther from the true value of the parameter…. 19

20. 20 XKCD Jan 26, 2015

21. 21