Design Considerations: Independent Samples v. Repeated Measures Independent samples design is the canonical situation of control v. experimental treatment, randomly assigned. Even non-scientists feel comfortable with this, except for ethical dilemmas. Repeated measures is efficient when you want to compare one or more variables (e.g., responses to questions on a questionnaire) against each other. Repeated measures is also tactically useful when there is a high degree of individual variation in the measure of interest. You can use each unit / person as its own “control” to look at before-after effects without worrying about individual differences. Concern: contamination by the measurement itself.

Monday, November 12 Statistical Power If you are interested in testing for the statistical significance of an effect that is of a particular magnitude of practical significance, what should the sample size be?

It teaches you about the importance of effect size. Why Statistical Power? It teaches you about the importance of effect size.  = d x f (N)

It teaches you about the importance of effect size. Statistical Power It teaches you about the importance of effect size. It helps put the risk of Type I error,  (alpha) into perspective.  = d x f (N)

It teaches you about the importance of effect size. Statistical Power It teaches you about the importance of effect size. It helps put the risk of Type I error,  (alpha) into perspective. It helps you appreciate the value of the sample size, N.  = d x f (N)

It teaches you about the importance of effect size. Statistical Power It teaches you about the importance of effect size. It helps put the risk of Type I error,  (alpha) into perspective. It helps you appreciate the value of the sample size, N. It simply makes you a better person.  = d x f (N)

“Reality” H0 True H0 False Type I Error  Reject H0 Yeah! Decision Yeah! Don’t Reject H0 Yeah!

“Reality” H0 True H0 False Yeah! Type I Error  Reject H0 Yeah! Decision Yeah! Type II Error  Don’t Reject H0 Yeah!

Statistical Power 1 -  The ability to avoid Type II error (fail to reject H0 that should be rejected).

σ = 100 σX= 100/12.81 = 7.81

Ordinarily, one is well advised to take the largest sample that is practical and then determine if this sample has adequate power for detecting a difference large enough to be of interest. Researchers often strive for power  80 with  = .05. More often, however, one finds that power is low even for detecting differences large enough to be of practical importance.

Problem You develop a new measure of social efficacy for adolescent girls, with 24 items on a 3-point scale. The scale seems to have  = 18, and  = 16. You are asked to evaluate a new program to promote social efficacy in adolescent girls, and want to use your scale. You sample 16, but alas find that the sample mean of 22 does not allow you to reject the null hypothesis at =.05. You’re really really frustrated because you think that a 4-point difference is meaningful. What should your next steps be?

What would it take for power = .80?  = d x f (N)  = d N 1/2 d = 4/16 = .25 N = 16  = 1.0 What would it take for power = .80? N = ( / d )2 N = (2.8 / .25)2 = 125.44

What would it take for power = .80?  = d x f (N)  = d (N-1) 1/2  = ρ (N-1) 1/2 d = .5 N = 21  = 2.24 What would it take for power = .80? N = ( / d )2 +1 N = (2.8 / .5)2 +1 = 32.36

What would it take for power = .90?  = d x f (N)  = d (N-1) 1/2  = ρ (N-1) 1/2 d = .5 N = 21  = 2.24 What would it take for power = .90? N = ( / d )2 +1 N = (3.25/ .5)2 +1 = 43.25

What can you do to increase power? Increase n

What can you do to increase power? Increase n Decrease measurement error

What can you do to increase power? Increase n Decrease measurement error Increase , say, from .05 to .10 (or fiddle with tails*)

What can you do to increase power? Increase n Decrease measurement error Increase , say, from .05 to .10 (or fiddle with tails*) *not advised

What can you do to increase power? Increase n Decrease measurement error Increase , say, from .05 to .10 (or fiddle with tails*) Increase the magnitude of the effect *not advised

In an independent samples t-test where the effect size d= In an independent samples t-test where the effect size d=.5 and N=100, what is the expected power? (see p. 299)

In an independent samples t-test where the effect size d= In an independent samples t-test where the effect size d=.5 and N=100, what is the expected power? (see p. 299)