Power of a test

power The power of a test (against a specific alternative value) Is a tests ability to detect a false hypothesis Is the probability that the test will correctly reject a false null hypothesis In practice, we carry out the test in hope of showing that the null hypothesis is false, so high power is important

H 0 True H 0 False Reject Fail to reject Type I Correct Type II Power Suppose H 0 is true – what if we decide to fail to reject it? Suppose H 0 is false – what if we decide to reject it? Suppose H 0 is true – what if we decide to reject it? Suppose H 0 is false – what if we decide to fail to reject it? We correctly reject a false H 0 !

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. A random sample of 100 people are vaccinated and then exposed to the flu. Is this claim too high? Use =.05. What are the hypotheses? H 0 : p =.7 H a : p <.7

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. A random sample of 100 people are vaccinated and then exposed to the flu. Is this claim too high? Use =.05. Find p and p. p =.7 p =.0458

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. A random sample of 100 people are vaccinated and then exposed to the flu. Is this claim too high? Use =.05. What is the probability of committing a Type I error? =.05

.7 H 0 : p =.7 H a : p <.7 =.05 For what values of the sample proportion would you reject the null hypothesis? Invnorm(.05,.7,.0458) =.625 =.05 p? So if we get p-hat=.625 or less, we would reject H 0.

H 0 : p =.7 H a : p <.7 We reject H 0 and decide that p<.7. Suppose that p a is 0.6. What is the probability of committing a Type II error? Where did this number come from? I selected a number that was less than =.05 Reject What is a type II error? = ? How can we find this area? What is the standard deviation of this curve? Normalcdf(.625,,.6,.0458) =.293 failing to reject H 0 when the alternative is true

What is the power of the test? Power = = =.05 =.293 What is the definition of power? The probability that the test correctly rejects H 0 Power = ? Power - the probability that the test correctly rejects H0, if p =.6, is.707 Is power a conditional probability?

Suppose we select.55 as the alternative proportion (p). a)What is the probability of the type II error? b) What is the power of the test? = normalcdf(.625,,.55,.0458) = =.05 Power = =.949 What happened to the power of the test when the difference |p 0 – p a | is increased?

Suppose we select.65 as the alternative proportion (p). a) What is the probability of the type II error? b) What is the power of the test? Power = = =.05 = normalcdf(.625,,.65,.0458) =.707 What happened to the power when the difference |p 0 -p a | is decreased? Power

Suppose that we change alpha to 10%. Using p a =.6, what would happen to the probability of a type II error and the power of the test?.7.6 =.05 =.1 Power BUT also increased The probability of the type II error ( ) decrease and power increased, BUT the probability of a type I error also increased. =.1836 Power =.8164

What happens to,, & power when the sample size is increased? Reject H 0 Fail to Reject H 0 p0p0 papa P(type II) decreases when n increases Power Power increases when n increases

p0p0 papa Reject H 0 Fail to Reject H 0 Power = 1 - APPLET

Recap: What affects the power of a test? As |p 0 – p a | increases, power increases As increases, power increases As n increases, power increases

Facts: The researcher is free to determine the value of. The experimenter cannot control, since it is dependent on the alternate value. The ideal situation is to have as small as possible and power close to 1. (Power >.8) powerAs increases, power increases. (But also the chance of a type I error has increased!) sample sizeBest way to increase power, without increasing, is to increase the sample size

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. In a test, vaccinated people were exposed to the flu. Is this claim too high? Identify the decision: a) You decide that the proportion of vaccinated people who do not get the flu is less than 70% when it really is not. Type I Error

A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. In a test, vaccinated people were exposed to the flu. Is this claim too high? Identify the decision: b) You decide that the proportion of vaccinated people who do not get the flu is less than 70% when it really is. Correct – Power!!