Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 21 More About Tests

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Zero In on the Null To perform a hypothesis test, the null must be a statement about the value of a _______________________. How do we choose the null hypothesis? The appropriate null arises directly from the context of the problem—it is not dictated by the ______, but instead by the ______________. To write a null hypothesis, you can’t just choose any parameter value you like. The null must relate to the question at hand—it is ______________________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Zero In on the Null (cont.) There is a temptation to state your claim as the ____________________. However, you cannot ________ a null hypothesis _______. So, it makes more sense to use what you want to show as the ________________. This way, when you _________________, you are left with what you want to show.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide How to Think About P-Values A P-value is a conditional probability—the probability of the ________________ given that the __________________________. The P-value is NOT the probability that the null hypothesis _________. It’s not even the conditional probability that null hypothesis is true given ___________. Be careful to interpret the P-value correctly.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Alpha Levels We can define “rare event” arbitrarily by setting a _____________ for our P-value. If our P-value falls __________________, we’ll reject H 0. We call such results ____________ ______________. The threshold is called an _______________, denoted by ___.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Alpha Levels (cont.) Common alpha levels are __________________. You have the option—almost the obligation—to consider your alpha level carefully and ______ ___________________________. The alpha level is also called the _____________ __________. When we reject the null hypothesis, we say that the test is ______________________

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Alpha Levels (cont.) What can you say if the P-value does not fall below  ? You should say that “The data have ________ _____________________________________ ___________________________________. Don’t say that you ______________________ ________________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Alpha Levels (cont.) The P-value gives the reader ______________________ than just stating that you reject or fail to reject the null. In fact, by providing a P-value to the reader, you allow that person to _________________________________ ___________________. What you consider to be ________________________ might not be the same as what someone else considers ____________________________. There is more than one _______________ that can be used, but each test will give only one _____________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide What Not to Say About Significance What do we mean when we say that a test is ________________________? All we mean is that the test statistic had a P- value _________ than our alpha level. Don’t be lulled into thinking that statistical significance carries with it any sense of ________ _____________________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide What Not to Say About Significance (cont.) For large samples, even small, unimportant (“insignificant”) deviations from the null hypothesis can be _______________________. On the other hand, if the sample is not large enough, even large, financially or scientifically “significant” differences may not be _____________________. It’s good practice to report the ______________________ between the observed statistic value and the null hypothesis value (in the data units) along with the ________ on which we base statistical significance.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Critical Values Again When making a confidence interval, we’ve found a ________________ to correspond to our selected confidence level. Prior to the use of technology, P-values were difficult to find, and it was easier to select a few _________________________ and learn the corresponding critical values for the Normal model.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Critical Values Again (cont.) Rather than looking up your z-score in the table, you could just check it directly against these _________________. Any z-score larger in magnitude than a particular critical value leads us to ____________. Any z-score smaller in magnitude than a particular critical value leads us to __________ _____________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Critical Values Again (cont.) When the alternative is one-sided, the critical value puts all of  on one side: When the alternative is two-sided, the critical value splits  equally into two tails:

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Critical Values Again (cont.) Here are the traditional critical values from the Normal model:  1-sided2-sided

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Confidence Intervals and Hypothesis Tests Confidence intervals and hypothesis tests are built from the same ______________. They have the same ______________ and _______________. You can approximate a _________________ by examining a ___________________. Just ask whether the null hypothesis value is _____________ with a confidence interval for the parameter at the corresponding _________ ____________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Confidence Intervals and Hypothesis Tests (cont.) Because confidence intervals are two-sided, they correspond to ________ tests. In general, a confidence interval with a confidence level of C% corresponds to a two-sided hypothesis test with an  -level of ______________

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Confidence Intervals and Hypothesis Tests (cont.) The relationship between confidence intervals and __________ hypothesis tests is a little more complicated. A confidence interval with a confidence level of C% corresponds to a one-sided hypothesis test with an  -level of _________________

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Just Checking In the last chapter we encountered a bank that wondered if it could get more customers to make payments on delinquent balance3s by sending them a video tape urging them to set up a payment plan. Well, the bank just got back the results on their test of the video tape strategy. A 90% confidence interval for the success rate is (0.29, 0.45). Their old send-a-letter method has worked 30% of the time. Can you reject the null hypothesis that the proportion is still 30% at α = 0.05? Explain. Given the confidence interval the bank found and that sending a video tape costs $9.60 more than sending a letter, what would you recommend that they do? Should they scrap the video tape strategy?

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Assignment P. 491 #1-4, 6-9

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 21 More About Tests (2)

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Making Errors Here’s some shocking news for you: nobody’s perfect. Even with lots of evidence we can still make the wrong decision. When we perform a hypothesis test, we can make mistakes in ____ ways: I. The null hypothesis is true, but we mistakenly ____________________ II. The null hypothesis is false, but we ____________________________

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Making Errors (cont.) Which type of error is more serious depends on the situation at hand. In other words, the gravity of the error is _________________________ Here’s an illustration of the four situations in a hypothesis test:

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Making Errors (cont.) How often will a Type I error occur? Since a Type I error is rejecting a true null hypothesis, the probability of a Type I error is ____________ When H 0 is false and we reject it, we have done _____________. A test’s ability to detect a false hypothesis is called ________________________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Making Errors (cont.) When H 0 is false and we fail to reject it, we have made _______________. We assign the letter ___ to the probability of this mistake. It’s harder to assess the value of  because we don’t know what the value of the ___________ really is. There is no single value for  --we can think of a whole collection of  ’s, one for each incorrect ____________________

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Making Errors (cont.) One way to focus our attention on a particular  is to think about the ______________ Ask “How big a difference would matter?” We could reduce  for all alternative parameter values by ______________ . This would reduce  but _________ the chance of a Type I error. This tension between Type I and Type II errors is inevitable. The only way to reduce both types of errors is to _______________. Otherwise, we just wind up trading off one kind of error against the other.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Power The power of a test is the probability that it _____________________________________. When the power is high, we can be confident that we’ve looked hard enough at the situation. The power of a test is ________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Power (cont.) Whenever a study fails to reject its null hypothesis, the test’s _____ comes into question. When we calculate power, we imagine that the null hypothesis is _______. The value of the power depends on __________ ______________________________________. The distance between the null hypothesis value, p 0, and the truth, p, is called the _________________. Power depends directly on effect size.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide A Picture Worth a Thousand Words The larger the effect size, the easier it should be to see it. Obtaining a larger sample size decreases the probability of a _________, so it increases the _________. It also makes sense that the more we’re willing to accept a Type I error, the less likely we will be to make a ____________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide A Picture Worth a Thousand Words (cont.) This diagram shows the relationship between these concepts:

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Reducing Both Type I and Type II Error The previous figure seems to show that if we reduce Type I error, we must automatically increase Type II error. But, we can reduce both types of error by _______________________________. How do we make the curves narrower? ______________________________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Reducing Both Type I and Type II Error (cont.) This figure has means that are just as far apart as in the previous figure, but the sample sizes are ________, the standard deviations are ________, and the error rates are ___________:

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide Reducing Both Type I and Type II Error (cont.) Original comparison of errors: Comparison of errors with a larger sample size:

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide What Can Go Wrong? Don’t interpret the P-value as the probability that ________________. The P-value is about _______, not the ____________. It’s the probability of the _____ given that __________, not the other way around. Don’t believe too strongly in arbitrary ______________. It’s better to report your __________ and a _____________________ so that the reader can make her/his own decision.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide What Can Go Wrong? (cont.) Don’t confuse __________ and _____________ significance. Just because a test is __________ significant doesn’t mean that it is significant in ________. And, ____________ can impact your decision about a null hypothesis, making you miss an important difference or find an “insignificant” difference. Don’t forget that in spite of all your care, ___________________________________.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Slide What have we learned? There’s a lot more to hypothesis testing than a simple yes/no decision. And, we’ve learned about the two kinds of errors we might make and seen why in the end we’re never sure we’ve made the right decision.

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Just Checking Remember our bank that’s sending out video tapes to try to get customers to make payments on delinquent loans? It is looking for evidence that the costlier video tape strategy produces a higher success rate than the letters it has been sending. Explain what a Type I error is in this context, and what the consequences would be to the bank. What’s a Type II error in the bank experiment context, and what would the consequences be? If the video tape strategy really works well, actually getting 60% of the people to pay off their balances after receiving the tape, would the power of the test be higher or lower compared to a 32% pay off rate? Slide

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Assignment P. 492 #11, 13, 16, 18, 19, 21, 24, 26, 27 Slide

Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Assignment P. 492 #12, 14, 15, 17, 20, 22, 23, 25, 28 Slide