Elementary Statistical Methods André L. Souza, Ph.D. The University of Alabama Lecture 22 Statistical Power

Type II Error occurs when you fail to reject the null hypothesis when, in fact, you should have rejected it The opposite of a Type II Error is the correct decision of rejecting the null hypothesis when in fact it should have been rejected Think about a pregnancy test.

What is Power? In statistics, power is the probability of rejecting the null hypothesis (i.e., finding a significant different) when the null hypothesis is false In other words, power is the probability of being correct about your decision Example: o How many dates you think you should go to before the first kiss? Guys mean = 1.38 sd = 0.2 Guys mean = 1.38 sd = 0.2 Girls mean = 2.92 sd = 1.5 Girls mean = 2.92 sd = 1.5

What if you want to replicate? Population A μ = 1.38 σ = 0.2 Population A μ = 1.38 σ = 0.2 Population B μ = 2.92 σ = 1.5 Population B μ = 2.92 σ = 1.5 What’s the probability that my result will be successful? This is what power is all about. Let’s simulate this

Simulation To calculate power, you need to have some idea of the mean and standard deviation of the population of girls and boys. Our best guess are the means and standard deviations of the study mentioned previously o Guys (mean = 1.38, standard deviation = 0.2) o Girls (mean = 2.92, standard deviation = 1.5) Simulation o We draw 20 guys from Population A and 20 girls from Population B o Run a t-test comparing the two samples o Save the t o Repeat this procedure 10,000 times o What percentage of the time will I reject the null hypothesis

Simulation This is the result of this sampling process Note that even though 92.7% of the time we “rejected the null”, in 7.3% of them we failed to reject (Type II Error) Thus, the power of this experiment, given the parameter estimates and sample size, is.927

Factors affecting power Power is a function of several variables It is a function of α (the probability of Type I error) It is a function of the true alternative hypothesis It is a function of sample size

Power as a function of α If you increase your α, the cutoff point moves to the left, decreasing β This will increase power But will also increase the probability of Type I error

Power as a function of H 1 If the distance between H 1 and H 0 increases, then the power also increases Type II error probability is still large The chances of finding a difference depend on how large the difference is

Power ~ of n and σ 2 This relationship is subtle You need to recall that we are dealing with sampling distributions This is the most direct way to manipulate your experiment to achieve a desirable level of power

Elementary Statistical Methods André L. Souza, Ph.D. The University of Alabama Lecture 32 Thank you!