Sample Power

How big should my sample be? Optimum size will depend on the purpose of the study, the precision required, and the costs of increased precision

How precise should this be? Sample Power How precise should this be? Sample mean Population mean

Sample Power - introduction Null Hypothesis is true: No difference Null Hypothesis is false: There is difference Null Hypothesis is accepted Correct Decision 95% probability Wrong Decision Null Hypothesis is rejected 5% probability Type I error

Sample Power Probability that a sample of given size will allow us to correctly reject the null hypothesis. Null Hypothesis is true: No difference Null Hypothesis is false: There is difference Wrong Decision Null Hypothesis is accepted Type II error 20% probability Null Hypothesis is rejected Correct decision Sample Power 80% probability

Example: Probability of rejecting a null hypothesis We have a sample: Mean 104, SD=15 N=50 Is this sample is coming from a Population A: Mean 100, SD=15 Null Hypothesis???? Alternative Hypothesis???? Solution ???? Conclusions ????? Actually, the sample is coming from population B Mean 105

If Population A was generating 1000 samples of 50 Population mean 5% rejection area mean = 104, p=.03 Population mean = 100, SD = 15, n=50, 1,000 samples were drawn.

Sample is actually coming from Population B: Null hypothesis is false Samples from population A Samples from population B Power Type II error Population mean Population A mean = 100, SD = 15, n=50, 1,000 samples were drawn. Population B mean = 105, SD = 15, n=50, 1,000 samples were drawn.

If Population A was generating 1000 samples of 20 Population mean 5% rejection area mean = 104, p=.06 Population mean = 100, SD = 15, n=20, 1,000 samples were drawn.

Sample is actually coming from Population B: Null hypothesis is false Samples from population A Samples from population B Power Type II error Population mean Population A mean = 100, SD = 15, n=20, 1,000 samples were drawn. Population B mean = 105, SD = 15, n=20, 1,000 samples were drawn.

Summary Power is the probability of correctly rejecting the null hypothesis Power is related to the mean of the population that the sample is actually coming from Power is related to the SEM (SD & sample size) larger the sample size, greater the power Power is related to the chosen level of significance

IMPLICATION Given: WE CAN COMPUTE THE REQUIRED SAMPLE SIZE DESIRED POWER, POPULATION MEAN, POPULATION SD, AND LEVEL OF SIGNIFICANCE, WE CAN COMPUTE THE REQUIRED SAMPLE SIZE

Summary Power is the probability of correctly rejecting the null hypothesis Power is related to the mean of the population that the sample is actually coming from Power is related to the SEM (SD & sample size) larger the sample size, greater the power Power is related to the chosen level of significance (usually p=.05)

Example sample size calculation I have two groups that I wish to compare. The mean of group 1 is 50. The mean of group 2 is 55. SD of both groups is 10. What should be my sample size so that I can conclude that the difference is statistically significant? Probability of correctly detecting this difference should be high (usually 80%) SOLUTION: http://www.stat.uiowa.edu/~rlenth/Power/

Power Analysis Example Exercise 1: I wish to estimate the behavior problem scores of children age 12-14 who had school disciplinary problems. This mean is estimated to be 115. I wish to be able to say that this sample of children could not have been generated by a normative population (mean = 100, SD = 15). What should my sample size be?

Using effect size: Exercise 2 Hypothesis: Children who experienced highly authoritarian parenting are expected to be less empathetic in their dating relationships than children who did not experience high levels of authoritarian parenting because …. What should be your sample size???? What is the IV and what is the DV? What would be your expectation about the size of the effect? Small, medium, or large? How does that translate to the difference between two groups?

Effect Size 0.2 or less: Small effect 0.3-0.4: Moderate effect 0.5-0.6: Large effect 0.7 or greater: Very large effect (almost impossible in Social Sciences)

Power Analysis Example Exercise 3: I wish to compare the mean behavior problems scores of children age 12-14 who have intact or disrupted families. I expect the score in disrupted families to be higher and to me, this score should be at least 0.3 SD higher than the intact families to be meaningful. What should my sample size be?

Power Analysis Example Exercise 4 I would like to test if students with high self esteem give more autonomous decisions in choosing their colleges than students with low self esteem. I expect that the difference between low and high self esteem groups will be small (0.2 SD). What should my sample size be, so that I can detect this small difference with statistical significance at p=0.05 level with 85% power?

Power Analysis Example Exercise 5 I would like to estimate the effects of authoritarian parenting on the level of religious prejudice of adolescents. I expect that this correlation will be about 0.2. What should my sample size be, so that I can detect this modest correlation with statistical significance at p=0.05 level with 85% power?

Power Analysis FINAL Example Exercise 6 [YOUR PARAGRAPH ON SAMPLE POWER] This research is about the effects of interparental violence on the level of intimacy in the dating relationships of college students. It is expected that there is a moderate negative correlation between the level of overt interparental conflict and ability of the college students to maintain intimacy in their dating relationships. Previous studies found this correlation to be around 0.25 (Smith, 1999; Miller, 2001). What should the sample size be, so that this correlation can be detected with statistical significance at p=0.05 level with 80% power?