Introduction to Hypothesis Math 3680 Lecture #6 Introduction to Hypothesis Testing Lecture #6

We illustrate the method of hypothesis testing with an example We illustrate the method of hypothesis testing with an example. Throughout this example, the lingo of hypothesis testing will be introduced. We will use this terminology throughout the rest of the chapter. Example: Historically, an average of 1,808 people have entered a certain store each day. In hopes of increasing the number of customers, a designer is hired to refurbish the store's entrance. After the redesign, a simple random sample of 45 days is taken. In this sample, an average of 1,895 people enter the store daily, with a sample SD of 187. Does it appear that the redesign of the store's entrance was effective? Lecture #6

“Solution”: There are two possibilities: The increase in the average number of customers may be plausibly attributed to a run of good luck, or The average increased after the redesign. We cannot give a decisive answer to this question, since both of these explanations are possible. The first option, which attributes the increase in the number of customers to mere chance, is called the null hypothesis. The other option is called the alternative hypothesis. We need to assess which option is more plausible. Lecture #6

Under certain extremes, the choice is fairly obvious. If 10,000 people were now entering the story daily, then it can be confidently concluded that, after the redesign, the store had more customers. In this case, we would reject the null hypothesis. On the other hand, if the store averaged only 1,809 people entering daily, then such an increase is reasonably attributable to chance. In this case, we would retain (or fail to reject) the null hypothesis. Lecture #6

Somewhere between these two extremes will be a certain cut-off point called the critical value. Above this critical value, it will be more plausible to think that the average number of customers increased after the redesign. Below this critical value, it will be more plausible to think that the increase is simply attributable to chance. Critical value 1808 1809 10,000 Lecture #6

Therefore, the question may be rephrased as follows: Is the sample average of 1,895 close enough to 1,808 to be consistent with the assumption that the population mean is still 1,808? This question leads to two hypotheses: H0 : The average is still 1,808. Ha : The average is greater than 1,808. (Why greater?) Lecture #6

Ha : The average is greater than 1,808. H0 : The average is still 1,808. Ha : The average is greater than 1,808. Critical value 1808 Decision: Retain H0 Reject H0 Lecture #6

Example: A dime is flipped 100 times to determine if it is fair. State the null and alternative hypotheses. Lecture #6

H0 : p = 0.5 Ha : p ≠ 0.5 Critical value Critical value p = 0.5 Reject H0 Retain H0 Reject H0 Lecture #6

Essential difference between probability and statistics: Probability: A fair dime is flipped 100 times. Find the probability that it lands heads 55 times. Statistics: A dime is flipped 100 times to determine if it is fair. 50% 50% 1 ? ? ? ? ? 1 - p p 1 1 1 Lecture #6

State the null and alternative hypotheses. Example: Many staff members in teaching hospitals are hired in July. Because of these inexperienced staff members, is it more dangerous to be admitted to a teaching hospital between July and September than it is during the rest of the year? State the null and alternative hypotheses. Lecture #6

Example: A dime is suspected of landing heads more often than tails Example: A dime is suspected of landing heads more often than tails. It is flipped 100 times to determine if it is fair. State the null and alternative hypotheses, and draw the corresponding figure. Lecture #6

The probability of a Type I error is denoted by a. No matter where we set the critical value, we will occasionally make a mistake. By chance, a fair coin may land heads more times than the critical value. This situation is called a Type I error, meaning that we would decide to reject the null hypothesis even though the coin was fair. The probability of a Type I error is denoted by a. Lecture #6

The probability of a Type II error is denoted by b. Another type of error could occur if the coin that favors heads actually lands heads less often than the critical value. This is called a Type II error, meaning that we fail to reject the null hypothesis even though the coin is imbalanced. The probability of a Type II error is denoted by b. We define the power of the test to be 1 - b. This is the probability of correctly rejecting the null hypothesis when the null hypothesis is false. Lecture #6

The null hypothesis is true but we reject it. Type I Error: The null hypothesis is true but we reject it. P(reject H0| H0 is true) = α Type II Error: The null hypothesis is false but we retain it. P(retain H0| H0 is false) = β Power: P(reject H0| H0 is false) = 1- β Lecture #6

THE WAY IT IS H0 true Ha true Decide to retain H0 THE WAY WE THINK IT IS Type II Error Decide to reject H0 Type I Error Lecture #6

State in words what a Type I error means. Repeat for a Type II error. Example: Many staff members in teaching hospitals are hired in July. Because of these inexperienced staff members, is it more dangerous to be admitted to a teaching hospital between July and September than it is during the rest of the year? State in words what a Type I error means. Repeat for a Type II error. Lecture #6

State in words what a Type I error means. Repeat for a Type II error. Example: Historically, an average of 1,808 people have entered a certain store each day. In the hopes of increasing the number of customers, a designer is hired to refurbish the store's entrance. After the redesign, a simple random sample of 45 days is taken. In this sample, an average of 1,895 people enter the store daily, with a sample standard deviation of 187. State in words what a Type I error means. Repeat for a Type II error. Lecture #6

Basic question: “How do we pick the critical value Basic question: “How do we pick the critical value?” Typically, the value of a is prescribed and the critical values are determined by a. We will discuss this in more depth later. For now, to develop our intuition, we’ll take the critical values as prescribed and compute a and b from these critical values. Lecture #6

Example: A scientific article compared two different methods for the analysis of ampicillin dosages. It was thought that the first method generally returned a higher amount than the second method. In one series of experiments, pairs of tablets were analyzed by the two methods. The data below give the percentages of claimed amount of ampicillin found by the two methods in 15 pairs of tablets. State the null and alternative hypotheses. 97.2 97.2 79.2 74.0 96.8 95.8 105.8 97.8 76.0 75.0 99.2 98.0 99.5 96.2 69.5 67.5 99.2 99.0 100.0 101.8 23.5 21.2 91.0 100.2 93.8 88.0 95.2 94.8 72.0 67.5 Lecture #6

Ha : p > 0.5 (compound hypothesis) Solution #1 (in words): H0 : If there is a difference, the probability that the first method returns a higher amount is 50%. Ha : If there is a difference, the probability that the first method returns a higher amount is greater than 50%. Solution #2 (more precise): Let p be the population proportion of pairs in which the first method returns a higher amount. H0 : p = 0.5 (simple hypothesis) Ha : p > 0.5 (compound hypothesis) Lecture #6

Example: For this particular data set, there are 14 pairs in which there is a measurable difference. Let K denote the number of pairs in which the first method returned a higher amount. Suppose we make the decision rule Retain H0 if K  9, and reject H0 if K  10. Compute a. 97.2 97.2 79.2 74.0 96.8 95.8 105.8 97.8 76.0 75.0 99.2 98.0 99.5 96.2 69.5 67.5 99.2 99.0 100.0 101.8 23.5 21.2 91.0 100.2 93.8 88.0 95.2 94.8 72.0 67.5 Lecture #6

Solution: This is the risk of an error in case that H0 is true. a = P(H0 is rejected | H0 is true) = P(K  10 | p = 0.5) Lecture #6

Example: Suppose we make the decision rule retain H0 if K  9 reject H0 if K  10 Compute b if (a) p = 0.7 (b) p = 0.9 Lecture #6

Solution: (a) This is the risk of an error in case that p = 0.7, so that H0 is false. b = b (0.7) = P(H0 is retained | p = 0.7) = P(K  9 | p = 0.7) Lecture #6

Solution: (b) This is the risk of an error in case that p = 0.9, so that H0 is false. b = b (0.9) = P(H0 is retained | p = 0.9) = P(K  9 | p = 0.9) Lecture #6

Notice that b is a function of p. (Why is this function decreasing?) Lecture #6

Power = 1 - b. Lecture #6

This lengthy calculation may also be facilitated with Note. This lengthy calculation may be facilitated by using page 505 from the back of the book: This lengthy calculation may also be facilitated with EXCEL: =BINOMDIST(9,14,0.7,1) TI-83: binomcdf(14,0.7,9) Lecture #6

Example: For this particular data set, there are 14 pairs in which there is a measurable difference. Let K denote the number of pairs in which the first method returned a higher amount. Suppose we make the decision rule Retain H0 if K  10, and reject H0 if K  11. Compute a. 97.2 97.2 79.2 74.0 96.8 95.8 105.8 97.8 76.0 75.0 99.2 98.0 99.5 96.2 69.5 67.5 99.2 99.0 100.0 101.8 23.5 21.2 91.0 100.2 93.8 88.0 95.2 94.8 72.0 67.5 Lecture #6

Solution: This is the risk of an error in case that H0 is true. a = P(H0 is rejected | H0 is true) = P(K  11 | p = 0.5) Practically, why did this decrease? Lecture #6

From the back of the book, 1 - 0.9713 = 0.0287 Note. This lengthy calculation may be facilitated by using page 505 from the back of the book: From the back of the book, 1 - 0.9713 = 0.0287 From Excel, use =1 - BINOMDIST(10,14,0.5,1) Lecture #6

Unfortunately, decreasing a also has the effect of increasing b. (Why?) Lecture #6

In statistical parlance, the test has become less powerful. Lecture #6

Note: In a perfect world, we would have a = b = 0, but this can only happen in trivial cases. For any realistic scenario of hypothesis testing, decreasing a will increase b, and vice versa. In practice, we set the significance level a in advance, usually at a fairly small number (a = 0.05 is typical) . We then compute b for this level of a. (Ideally, we would like to construct a test that makes b as small as possible. This topic will be considered in future statistics courses.) Lecture #6

In these problems, the critical regions were prescribed and a was computed. In practice, a is chosen and the critical value is computed to match this value of a. Lecture #6