Intro to Statistics for the Behavioral Sciences PSYC 1900 Lecture 11: Power

Power When you find a significant result, it implies that the null hypothesis is unlikely to be correct. When you find a significant result, it implies that the null hypothesis is unlikely to be correct. It does not imply that you will find the same result every time you conduct the experiment. It does not imply that you will find the same result every time you conduct the experiment. Though, on average, you should find differences in the same direction as the one you originally found. Though, on average, you should find differences in the same direction as the one you originally found. Type II errors lead to failures to reject the null when it is incorrect. Type II errors lead to failures to reject the null when it is incorrect.

Type II Errors and Power Power is the probability of correctly rejecting a false null hypothesis. Power is the probability of correctly rejecting a false null hypothesis. Power = (1-  ) Power = (1-  ) Why do we care about power? Why do we care about power? It tells us about the sensitivity of our experiments. It tells us about the sensitivity of our experiments. Allows us to determine needed sample sizes to detect effects of different sizes. Allows us to determine needed sample sizes to detect effects of different sizes.

Type II Errors and Power If we have two distributions, the control group and the treatment group, we can define the errors and power by examining their overlap. If we have two distributions, the control group and the treatment group, we can define the errors and power by examining their overlap. The region to the right of the critical value represents alpha (assuming a one-tailed test). The region to the right of the critical value represents alpha (assuming a one-tailed test). The second distribution represents the sampling distribution of the mean when the null is false (H1). The second distribution represents the sampling distribution of the mean when the null is false (H1). If part of the sampling distribution for H1 falls to the left of the critical value in H0, this represents the probability of a type II error. If part of the sampling distribution for H1 falls to the left of the critical value in H0, this represents the probability of a type II error. Power is the probability represented by the area under H1 that exceeds the critical value. Power is the probability represented by the area under H1 that exceeds the critical value.

Factors That Affect Power Alpha Alpha As alpha increases, power decreases. As alpha increases, power decreases. The Difference Between Means The Difference Between Means Increasing differences are associated with a higher probability of detection (greater power) Increasing differences are associated with a higher probability of detection (greater power) Sample Size Sample Size Increasing sample sizes reduce the standard error and thus increase power. Increasing sample sizes reduce the standard error and thus increase power.

Effect Size Again To calculate power (approximately) we will use a measure of effect size: To calculate power (approximately) we will use a measure of effect size: Gamma is the measure of the degree to which the two means differ relative to the sd of the parent population. Gamma is the measure of the degree to which the two means differ relative to the sd of the parent population.

Effect Size Again How to calculate gamma? How to calculate gamma? If we need to determine power before conducting an experiment, we can estimate gamma based on past research. If we need to determine power before conducting an experiment, we can estimate gamma based on past research. We can also estimate it based on theory using Cohen’s conventions for small, medium, and large effects. We can also estimate it based on theory using Cohen’s conventions for small, medium, and large effects. Or, we can run a small pilot study. Or, we can run a small pilot study. Once we have gamma, we need to determine delta, which will be a different function of sample size depending on the specific test we will conduct. Once we have gamma, we need to determine delta, which will be a different function of sample size depending on the specific test we will conduct.

Power for One Sample t-Tests Here, the function is: Here, the function is: So, if a group of 49 people taking an herbal supplement has an IQ of 108, the power with which we can detect this is: So, if a group of 49 people taking an herbal supplement has an IQ of 108, the power with which we can detect this is:

Power for One Sample t-Tests Once we have delta, we turn to the power tables and find the power for a two tailed test at the appropriate alpha level. Once we have delta, we turn to the power tables and find the power for a two tailed test at the appropriate alpha level. Here, power=.96 Here, power=.96 What does this mean? What does this mean?

Determining Sample Size Sometimes, we wish to know what size sample we would need to obtain a given level of power. Sometimes, we wish to know what size sample we would need to obtain a given level of power. We usually don’t have control over mean differences or conventional alpha levels. We usually don’t have control over mean differences or conventional alpha levels. In this case, we simply set delta to the desired level and solve for n. In this case, we simply set delta to the desired level and solve for n.

N for One Sample t-Tests Here, the function is: Here, the function is: We would need 28 people in the sample to have power of.80 in the previous experiment. We would need 28 people in the sample to have power of.80 in the previous experiment.

Power for Independent Sample t-Tests Here, the function is: Here, the function is: To determine sample size: To determine sample size:

Let’s Look at An Example