Monday, November 11 Statistical Power

Monday, November 12 Statistical Power It teaches you about the importance of effect size,  (gamma).

Monday, November 12 Statistical Power It teaches you about the importance of effect size,  (gamma). It helps put the risk of Type I error,  (alpha) into perspective.

Monday, November 12 Statistical Power It teaches you about the importance of effect size,  (gamma). It helps put the risk of Type I error,  (alpha) into perspective. It helps you appreciate the value of the sample size, N.

Monday, November 12 Statistical Power It teaches you about the importance of effect size,  (gamma). It helps put the risk of Type I error,  (alpha) into perspective. It helps you appreciate the value of the sample size, N. It simply makes you a better person.

Monday, November 12 Statistical Power It teaches you about the importance of effect size,  (gamma). It helps put the risk of Type I error,  (alpha) into perspective. It helps you appreciate the value of the sample size, N. It simply makes you a better person.  =  x f (N)

Let’s sample from HSB universe of N=300 where we know the population values.

What causes you to decide on a value of  under H 1 ?

The alternative hypothesis should be the point at which we say, “aha, it’s important enough to pay attention to”!

What causes you to decide on a value of  under H 1 ? The alternative hypothesis should be the point at which we say, “aha, it’s important enough to pay attention to”! Would you pay attention to Stanley Kaplan raising SAT’s by 10 points? 20 points? 30 points? …

What causes you to decide on a value of  under H 1 ? The alternative hypothesis should be the point at which we say, “aha, it’s important enough to pay attention to”! Would you pay attention to Stanley Kaplan raising SAT’s by 10 points? 20 points? 30 points? … So you set the value of alternative hypothesis at a point that you care about -- a value of practical significance.

What causes you to decide on a value of  under H 1 ? The alternative hypothesis should be the point at which we say, “aha, it’s important enough to pay attention to”! Would you pay attention to Stanley Kaplan raising SAT’s by 10 points? 20 points? 30 points? … So you set the value of alternative hypothesis at a point that you care about -- a value of practical significance. Power analysis says, if the effect is of that magnitude, what is the risk that I will fail to detect it, by failing to reject the null hypothesis.

If I really care about a 5-point difference, then this is bad news.

“Reality” H 0 TrueH 0 False Decision Reject H 0 Don’t Reject H 0 Yeah! Type I Error 

“Reality” H 0 TrueH 0 False Decision Reject H 0 Don’t Reject H 0 Yeah! Type I Error  Yeah! Type II Error 

1 -  The ability to avoid Type II error (fail to reject H 0 that should be rejected).

Ordinarily, one is well advised to take the largest sample that is practical and then determine if this sample has adequate power for detecting a difference large enough to be of interest. Researchers often strive for power  80 with  =.05. More often, however, one finds that power is low even for detecting differences large enough to be of practical importance.

Problem You develop a new measure of social efficacy for adolescent girls, with 24 items on a 3-point scale. The scale seems to have  = 18, and  = 16. You are asked to evaluate a new program to promote social efficacy in adolescent girls, and want to use your scale. You sample 16, but alas find that the sample mean of 22 does not allow you to reject the null hypothesis at  =.05. You’re really really frustrated because you think that a 4-point gain is meaningful. What should your next steps be?

 =  x f (N)  =  N 1/2  = 4/16 =.25 N = 16  = 1.0

 =  x f (N)  =  N 1/2  = 4/16 =.25 N = 16  = 1.0 What would it take for power =.80? N = (  /  ) 2 N = (2.8 /.25) 2 =

What can you do to increase power? Increase n

What can you do to increase power? Increase n Decrease measurement error

What can you do to increase power? Increase n Decrease measurement error Increase , say, from.05 to.10 (or fiddle with tails*)

What can you do to increase power? Increase n Decrease measurement error Increase , say, from.05 to.10 (or fiddle with tails*) *not advised

What can you do to increase power? Increase n Decrease measurement error Increase , say, from.05 to.10 (or fiddle with tails*) Increase the magnitude of the effect *not advised

You are a better person because now you appreciate this better!