MA 102 Statistical Controversies Wednesday, April 17, 2002 Today: Finish up confidence intervals Discuss significance tests Reading (for Friday) : Chapter 22- What is a Test of Significance? Exercises: 22.1, 22.3, 22.8, 22.13, 22.15, 22.21

What is a significance test? An example from Test #2 Cocaine Relapse? YesNo Desipramine1014 Lithium186 Placebo204 Is there a significant difference between Desipramine and the placebo? Ditto for Lithium and the placebo?

Example Our null hypothesis is that there is no significant difference between the placebo (relapse rate of 20/24 = 83%) and Desipramine (relapse rate of 10/24 = 42%). Could this difference have happened by chance? SD =  ((.83)(.17) / 24) = 7.7% Hence the standard score for Decipramine is (42% - 83%) / 7.7% = 5.3 This is a VERY SIGNIFICANT difference!

P-Value The P-value is the probability that a sample effect would occur by chance only. The smaller the P-value, the more significant the result! In the case of Desipramine, a standard score of 5.3 corresponds to a P-value of somewhere around.01% (see Table B in the text), which is very tiny.

Example continued Let’s test if the Lithium result is significantly different from the placebo: Lithium’s standard score is (75% - 83%) / 7.7% = Looking at Table B, this corresponds to a P-value of about 15.7%, i.e., there is about a 16% chance that this result could have occurred simply by chance. This is not considered to be significant (P > 5%, e.g.)

Summary The null hypothesis H 0 is the assumption that no effect has occurred as a result of treatment. The alternate hypothesis H a is that an effect has occurred. We use standard scores to assess the probability (the P-value) that the measured effect could have occurred only by chance. The result is said to be statistically significant at the 5% level (for example) if P = 5%.