Section 10.3 Making Sense of Statistical Significance February 15th 2013

Choosing a level of significance (alpha level) How plausible is H0? Depending on H0 plausibility, you may choose a smaller alpha. If H0 is very plausible, you will need to have collect “more” evidence to reject it.

What are the consequences of rejecting H0? If rejecting H0 would costs lots of money possibly cost lives costs jobs then alpha is usually very small

Fishing for significance Let’s say were trying to find a connection between eating habits and intelligence. Choose 40 foods, and assign people to increase the amount of the foods they eat, and see if there are any foods that make people smarter. Of the 40 foods, we find that peeps and green beans make you smarter with alpha=.05. Is this a problem?

Section 10.4.1 Inference as Decision February 15th 2013

When inference is used to make a decision… Either you reject H0 or you fail to reject H0. You can reject H0 correctly You can fail to reject H0 correctly You reject H0 incorrectly (Type I error) You can fail to reject H0 incorrectly (Type II error)

Fail to Reject H0 Reject H0 H0 is true Correct Type I error H0 is false Type II error

H0: No Wolf Ha: Wolf People don’t believe People do believe No Wolf Accept H0 Accept Ha H0: No Wolf Ha: Wolf People don’t believe People do believe No Wolf Correct Type I error Wolf Type II error H0 is true Ha is true

Potato Chips Example The salt content of the chips should have a mean of 2 mg with a standard deviation of .1 mg. When deciding whether to accept or reject a batch of potato chips, a company looks at the salt content of 50 chips. If the salt content is too far away from the mean, it will reject the batch.

What range values are acceptable? The company will check a 50 chip sample. If our alpha is .05, the acceptable range is the same as the 95% confidence interval:

Accept or reject? We understand with normal variation and everything working normally, we will get a sodium value between 1.9723 mg and 2.0277 mg 95% of the time.

Accept or reject? We understand with normal variation and everything working normally, we will get a sodium value between 1.9723 mg and 2.0277 mg 95% of the time. This means the 5% of the time you will reject a batch of chips that are fine. When we reject the batch (and H0) incorrectly we have committed a Type I error.

Accept H0 Reject H0 Reject H0 95% Confidence Interval 1.9723 2.0277

Significance and Type I Error The significance level α of any fixed level test is the probability of a Type I error. That is, α is the probability that the test will reject the null hypothesis H0 when H0 is in fact true.

Probability of the Type II error What if there really is a difference in the overall saltiness of the potato chips, would always we see a “significant” result?

Accept H0 Reject H0 95% Confidence Interval 1.9723 2.0277 2.05

Section 10.4.2 Power

What is Power? Power is a test of sensitivity. Your statistical test may be able to detect differences, but how well does it detect difference of a pre-determined nature? The Power procedure allows to state the probability of our procedure to catch the differences.

Power Procedure Begin by stating your H0 and Ha as usual. Find the z* or t* that would allow you to reject H0. Find the x-bar that matches up with the z* or t*. Assuming that you have a particular true mean, what is the probability that you would be to still reject the H0?

Power Example: Example 10.23 Can a 6-hour study program increase your score on SAT? A team of researchers is planning as study to examine this question. Based on the result of a previous study, they are willing to assume that the change has σ=50. Research would like significance at the .05 level.

Power Example: Example 10.23 A change of 50 points would be considered important, and the researchers would like to have a reasonable chance of detecting a change is this large or larger. Is 25 subjects a large enough sample for this project?

Step 1: State your hypothesis Ha: µ>0 Where µ represents the change is in the SAT score.

Step 2: Find the z* value and find the data value We'll set α=.05, invNorm(.95) gives us a z*=1.645. What is the lowest x-bar would show significance? Summary: If we had a study with n=25 and x-bar>16.45, we would have significance.

Step 3: Chance at importance We stated that gains of 50 points would be considered "important". We state this as the alternative µ=50. The power against the alternative µ=50 increase is the probability that H0 is rejected when µ=50. Restated: What the area from 16.45 to ∞ under a normal curve centered at µ=50.

Step 3 normalcdf(16.45,1E99,50,50/√(25))=.9996 Summary: because the power is so high, there is a great chance of finding a significance when the real increase is 50.

Increase Power by… increase alpha increase sample size

Exercises 10.71-10.77 odd, 10.79-10.89