Supplement 10: The Power of a Test *The ppt is a joint effort: Mr DAI Liang discussed the power of a test with Dr. Ka-fu Wong on 16 April 2007; Ka-fu explained the problem; Liang drafted the ppt; Ka-fu revised it. Use it at your own risks. Comments, if any, should be sent to

Imagine this scenario  At a starry night, we noticed a bright spot in the sky.  From the map, we knew that it is a system of binary stars. However, our naked eyes could not tell.  So, we borrowed a telescope and looked at it again.

If it is actually two stars…  Given the same telescope (or naked eyes), the larger the distance between the two stars, the more likely we can distinguish between them.  Given the same two stars, the larger the Magnification Coefficient (MC) of our telescope, the more likely we can distinguish between the two stars.

The idea of “power”!!  The power a test is a measure of the ability of a test in distinguishing between two possible values of the parameter of interest.  In the context of hypothesis testing, the power of the test is the ability of the test in telling us the null is wrong when the true parameter is different from the null.  Conditional (given) a test, such ability depends on how far apart the null is from the true parameter.  The larger the difference between the truth and the null, the larger the power of a given test.

Analogy between statistical test and telescope Statistical testTelescope (Star gazing) Power of a test in distinguishing between two values of parameter. Magnification coefficient (MC) of your telescope The larger the difference between the truth and the null, the larger the power. The larger the distance between the two stars, the more likely we can distinguish between them using the same telescope. Given the same truth and null, the larger is the power, the more accurate is the test. Given the same two stars, the larger the MC of our telescope, the more likely we can distinguish the two stars.

Exercise (Problem 10.42)  A random sample of 802 supermarket shoppers had 378 shoppers that preferred generic brand items if the price was lower.  Test at the 10% level the null hypothesis that at least one-half of all shoppers preferred generic brand items against the alternative that the population proportion is less than one-half.  Find the power of a 10% level test if, in fact, 45% of the supermarket shoppers are able to state the correct price of an item immediately after putting it into the cart.

The null hypothesis  The hypotheses:  Null hypothesis (H 0 ):  0.5,  Alternative (H 1 ):  < 0.5  Variance of the sample proportion p under H 0 is:  0.5*(1-0.5)/802=  Level of significance = 0.1

When is H 0 rejected?  Reject H 0 if sample proportion p is too small.  At the level of significance (=0.1), z =  Upper limit of rejection is 0.5+ z*[std. dev. under H 0 ] =  Therefore, H 0 is rejected when p   =0.5  =0 p z Standardized to standard normal: z=(p-  )/std(p)  =0.1 Rejection region p=378/802 = Null rejected.

If the real proportion is 0.45  H 0 is false since  = 0.45 ≠ 0.5  The power is Pr(rejecting the null |  =0.45) = Pr(p  |  =0.45)  Variance under  = 0.45 is : .45*(1-.45)/802=  Pr(p  |  =0.45) =Pr(Z  ( )/sqrt( )) =  Thus, is the probability that H 0 (  0.5 ) is correctly rejected when the truth is  = 0.45.

- END - Supplement 9: AnExample of Switching Null and Alternative Hypothesis An Example of Switching Null and Alternative Hypothesis A simulated example of a binary star, where two bodies with similar mass orbit around a common barycenter in elliptic orbits. (From