Statistical Significance Victor I. Piercey February 9, 2010

Several studies in the 1990’s suggested that eating pizza reduces cancer risk. When these studies are conducted, how are these conclusions reached? The answer lies in the concept of statistical significance.

What type of quantity would we measure in one of these studies? Formulate the null hypothesis H 0 How would we set up a test statistic? What quantity measures the probability for a measured value of our test statistic? Before doing any work with the data, we must decide on a “tolerance level” : How low must our P-value be before we reject the null hypothesis?

This tolerance level, denoted α, is called the significance level. For example, if we set α = 0.05 we require that the data give evidence against H 0 so strong that it would happen no more than 5% of the time (in the long run). Caution! The significance level α must be decided before studying the data!

If the P-value is smaller than α, we say that the data are statistically significant at level α. In this example, if P < α, we would say that eating pizza has a statistical significant effect on cancer risk. If our study yields P = 0.023, then the result is statistically significant at the α = 0.05 level, but not the α =0.01 level. Typically, we use α = Caution: Statistical significance is not the same as practical Significance!

Interpreting Results If P < α, we reject the null hypothesis H 0. Otherwise, we fail to reject H 0. So if α =0.05 and our study yields P = 0.023, have a slice!!

Homework Page 701, and Page 704,