Hypothesis Testing “Teach A Level Maths” Statistics 2 Hypothesis Testing © Christine Crisp

Hypothesis Testing Suppose there is a new drug treatment which we hope will be better than the existing one. To test whether it is better, we could set up a trial involving a certain number of people. We could then see how long it takes people to get well with the new drug and compare it with results for the old one. Statistics has a big part to play in making decisions of this type and this presentation introduces an important theory that is widely used. However, sampling involves random effects and we need to know whether the apparently good results are really due to improvements in treatment.

Hypothesis Testing e.g. 1. In a trial of 20 patients with a new drug, the condition of 15 was greatly improved. On the older drug, 60% reported the same improvement. Is there evidence that the new drug is more effective than the old one? The situation can be modelled with the Binomial distribution. We let p, the probability of success, be the probability that a patient improved with the new drug. We can’t just say that the drug is clearly better because 75% of patients improved compared with 60% before. We need to find out what the probability is that 15 patients improve even if the new drug is no more effective than the old one.

Hypothesis Testing We write We set up a hypothesis ( a theory ) that the new drug is not more effective and only reject this if we have significant evidence against it. H 0 is called the null hypothesis. ( I think of it as “no change”. ) We want to test if the new drug is better than the old one so we also have H 1 is the alternative hypothesis. The null hypothesis is using the value of p from data for the old drug.

Hypothesis Testing Let X be the random variable ”the number of patients that improve” We now want to test the hypothesis. There were 20 patients in the trial so n = 20. Also, So, We want to find out how likely, or unlikely it is that, with the old drug, 15 or more patients will improve. We say that we reject the null hypothesis at the 5% level of significance. We will only reject H 0 if this probability is less than 0·05.

Hypothesis Testing Assuming the null hypothesis we have n = 20 and p = 0·6 : There is a probability of 0·1256 ( more than 12% ) that 15 or more patients will improve even with the old drug. We do not reject the null hypothesis. Test at 5% level of significance.

Hypothesis Testing We have to consider because we are comparing with the end 5% of the distribution. Let’s see how many patients would need to improve on the new drug before we could reject H 0 and accept that the new drug is better than the old.

Hypothesis Testing ( Near, but still greater than 5% ) So, if 17 patients improved we would have our evidence. H 0 would be rejected.

Hypothesis Testing The critical region for the test consists of the values of the random variable, X, that would cause us to reject the null hypothesis. Critical Region In this example, the critical region is 17, 18, 19 and 20.

Hypothesis Testing e.g. 1. In a trial of 20 patients with a new drug, the condition of 15 was greatly improved. On the older drug, 60% reported the same improvement. Is there evidence that the new drug is more effective than the old one? Let X be the random variable ”the number of patients that improve” There is a probability of 0·1256 ( more than 12% ) that 15 or more patients will improve even with the old drug. We do not reject the null hypothesis. There is insufficient evidence to suggest the new drug is better. Test at 5% level of significance. Solution:

Hypothesis Testing 1-tailed and 2-tailed tests In our example the alternative hypothesis, H 1, was given by This is called a 1 -tailed test because we are only considering values of p greater than 0·6. If our alternative hypothesis were we would have a 2 -tailed test. We would also have a 1 -tailed test if we had. For a 2-tailed test, the percentage for the significance is split into 2, one half at each end of the distribution.

Hypothesis Testing e.g. 2. In a trial, 16 seeds are sown and only 11 germinate. Use a 10% significance level to test the supplier’s claim that 85% germinate. Find the critical region for the test. Let X be the random variable ”the number of seeds that germinate” There is a probability of 0·0791 ( less than 10% ) that 11 or fewer seeds will germinate. We reject the null hypothesis at the 10% level of significance and conclude that the germination rate is below 85%. Test at 10% level of significance. Solution: To test the supplier’s claim, the alternative hypothesis is that fewer than 85% germinate. This is again a 1-tailed test but this time we need to test the bottom end of the distribution.

Hypothesis Testing The probability of 12 or fewer germinating is 0·2101 ( 21·01% ), so the critical region for the test is 0, 1, 2,... 10, 11.

Hypothesis Testing e.g. 3. A team captain thinks the coin thrown at the start of a match is biased. Find the critical region for the number of heads in 30 tosses using a 5% significance level. Solution: Let X be the random variable “ number of heads ”. So, The null hypothesis is that the coin is fair, so We don’t know whether there might be more or fewer heads than on a fair coin so the alternative hypothesis is We have a 2 -tailed test, so the 5% must be split into 2·5% at each end of the distribution.

Hypothesis Testing We want to find out how many heads would convince us that the coin is biased. We refer to tables to get an idea of the number we want and then check by doing the calculation. We would not reject H 0 with 20 heads because, although we are using a 5% significance level, we are looking for values in the top or bottom 2·5% of the distribution.

Hypothesis Testing We now try 21 : This gives 2·14% which is less than 2·5% so with 21 or more heads we reject H 0 in favour of H 1. However, we are doing a 2 -tailed test so we must also consider the bottom end of the distribution. However, with p = 0·5, the Binomial is symmetric, so without further calculations, we know that would also cause us to reject H 0. With a value of p other than 0·5, the Binomial distribution is not symmetric so we would need to do the calculation.

Hypothesis Testing The values of X where we would accept H 0 form the acceptance region, so in this example they are 10, 11, The critical region has 2 parts: 0, 1, 2,... 9, and 21, 22,

Hypothesis Testing SUMMARY  To carry out a hypothesis test on a Binomial model we define a random variable write down the distribution with n equal to the number of trials and using p for the probability of success state the null (H 0 ) and alternative (H 1 ) hypotheses using for a 1 -tailed test and  for a 2 -tailed test write down the significance level of the test calculate the cumulative probability reject H 0 if the probability is less than the significance level ( or, for a 2 -tailed test, less than half the total significance level ). continued

Hypothesis Testing SUMMARY  The critical region consists of the values of the random variable that cause H 0 to be rejected.  A 1 -tailed test is used if we are testing a claim that the probability is higher or lower than defined in H 0. If there is no indication, we use a 2 -tailed test.  For a 2 -tailed test, the significance level is split into 2 halves so, for example, 5% becomes 2·5% at each end.

Hypothesis Testing Exercise 1. In a survey it was found that 35% of members of a political party supported candidate D to be leader. A month later, D claimed his support had increased. A 2 nd survey of 20 members showed that 11 supported him. Would you accept D’s claim at the 5% significance level? 2.Hyacinth bulbs are sold to a retailer in packs of 100 which claim to have equal numbers of bulbs producing blue and pink flowers. A random sample of 25 produces 17 blue flowers. Test at the 10% level of significance whether the retailer has a right to complain that there are not equal numbers of each colour. Find the critical region for the test.

Hypothesis Testing Solution: 1. Let X be the r.v.” number supporting D” Test at 5% significance level. We do not reject H 0 and so the candidate cannot claim greater support. At a 10% level of significance we would reject H 0 and conclude that D’s support had increased.

Hypothesis Testing 11 is outside the critical region at 5% significance but inside it at 10%.

Hypothesis Testing 2.Let X be the r.v. “ number of blue flowers” Test at 10% significance level. ( This is a 2 -tailed test so we check against 5% each end of the distribution. ) We do not reject H 0. There is insufficient evidence at the 10% level of significance to claim that there are more of one colour than the other.

Hypothesis Testing 18 blue flowers would fall within the critical region. The value 0·0539 is so close to 0·05 that we are sure the critical region starts at 18. However, we should check. Since this is a 2 -tailed test the critical region is in 2 parts. With p = 0·5, the Binomial distribution is symmetrical, so the critical region is 0, 1,... 7 and 18, 19,... 25

Hypothesis Testing

The following slides contain repeats of information on earlier slides, shown without colour, so that they can be printed and photocopied.

Hypothesis Testing SUMMARY  To carry out a hypothesis test on a Binomial model we define a random variable write down the distribution with n equal to the number of trials and using p for the probability of success state the null (H 0 ) and alternative (H 1 ) hypotheses using for a 1 -tailed test and  for a 2 -tailed test write down the significance level of the test calculate the cumulative probability reject H 0 if the probability is less than the significance level ( or, for a 2 -tailed test, less than half the total significance level ). continued

Hypothesis Testing  The critical region consists of the values of the random variable that cause H 0 to be rejected.  A 1 -tailed test is used if we are testing a claim that the probability is higher or lower than defined in H 0. If there is no indication, we use a 2 -tailed test.  For a 2 -tailed test, the significance level is split into 2 halves so, for example, 5% becomes 2·5% at each end.

Hypothesis Testing e.g. 1. In a trial of 20 patients with a new drug, the condition of 15 was greatly improved. On the older drug, 60% reported the same improvement. Is there evidence that the new drug is more effective than the old one? Let X be the random variable ”the number of patients that improve” There is a probability of 0·1256 ( more than 12% ) that 15 or more patients will improve even with the old drug. We do not reject the null hypothesis. Test at 5% level of significance. Solution:

Hypothesis Testing e.g. 2. In a trial, 16 seeds are sown and only 11 germinate. Use a 10% significance level to test the supplier’s claim that 85% germinate. Find the critical region for the test. Let X be the random variable ”the number of seeds that germinate” There is a probability of 0·0791 ( less than 10% ) that 11 or fewer seeds will germinate. Test at 10% level of significance. Solution: To test the supplier’s claim, the alternative hypothesis is that fewer than 85% germinate. This is again a 1-tailed test but this time we need to test the bottom end of the distribution.

Hypothesis Testing The Autograph illustration is as follows: The probability of 12 or fewer germinating is 0·2101 ( 21·01% ), so the critical region for the test is 0, 1, 2,... 10, 11. We reject the null hypothesis at the 10% level of significance and conclude that the germination rate is below 85%.

Hypothesis Testing e.g. 3. A team captain thinks the coin tossed at the start of a match is biased. Find the critical region for the number of heads in 30 tosses using a 5% significance level. Solution: Let X be the random variable “ number of heads ”. So, The null hypothesis is that the coin is fair, so We don’t know whether there might be more or fewer heads than on a fair coin so the alternative hypothesis is We have a 2-tail test, so the 5% must be split into 2·5% at each end of the distribution.

Hypothesis Testing We want to find out how many heads would convince us that the coin is biased. A reasonable guess would be 20 out of 30. We would not reject H 0 with 20 heads because we are looking for values in the top or bottom 2·5% of the distribution.

Hypothesis Testing We now try 21 : This gives 2·14% which is less than 2·5% so with 21 or more heads we reject H 0 in favour of H 1. However, we are doing a 2 -tailed test so we must also consider the bottom end of the distribution. However, with p = 0·5, the Binomial is symmetric, so without further calculations, we know that would also cause us to reject H 0. With a value of p other than 0·5, the Binomial distribution is not symmetric so we would need to do the calculation.

Hypothesis Testing The values of X where we would accept H 0 form the acceptance region, so in this example they are 10, 11, The critical region has 2 parts: 0, 1, 2,... 9, and 21, 22,