Power and Error What is it?
Hypothesis Testing Remember that tests of significance are to assess the evidence the null hypothesis When statisticians make decisions based off the strength of evidence against a null hypothesis they also must also worry about the possiblity of making a wrong decision.
Types of wrong decisions TYPE 1 ERROR: if we reject the null hypothesis when it is actually true. TYPE 2 ERROR: if we fail to reject the null when if fact the Alternative is true.
Type I error Therefore: α is the probability of type I error!! Is set by the statistician! Remember: P-value: probability of a null hypothesis being true given then sample data. α= probability that the test will reject the null hypothesis Therefore: α is the probability of type I error!!
Type II Error This is a much harder calculation! This assumes that the alternative is true but we failed to reject the null. To calculate this error we need to know what the alternative parameter actually is… this is rarely known! High type II error implies that the test is not sensitive enough to detect the alternative.
POWER The probability at a fixed significance level that we will reject the null when the particular alternative value is true! POWER= 1-type II error. P-values describes the probability of a situation given the null is true. Power describes the probability given a particular alternative value is true!
fail to reject Ho reject Ho power type II error
and the power increases! fail to reject Ho reject Ho What if we increased alpha? power type II error and the power increases!
fail to reject Ho reject Ho What if we increased sample size? Well, the standard deviation would get smaller. power type II error What happened to power? To a type II error?
What if the alternative (truth?) were further from Ho? fail to reject Ho reject Ho What if the alternative (truth?) were further from Ho? power type II error power is near 1 type II error is near 0
So it’s a balancing act. What is most important in the context of the problem? Which error would be most costly or dangerous?
To raise the power of a test (that is, to increase the probability of correctly rejecting Ho if it is NOT true) increase sample size decrease variation move alternative further from Ho increase alpha (prob. of type I error)