1. Making Sense of Statistical Significance Inference as Decision
Section 9.1(re-visited)
Making Sense of Statistical Significance
Inference as Decision

2. Warm-up
The one-sample t statistic for testing H0: μ = 0 and Ha: μ > 0 from a sample of n = 15 observations has the value t = 1.82
What are the degrees of freedom for this statistic?
Between what two values does the P-value of the test fall?
Is the value t = 1.82 significant at the 5% level? Is it significant at the 1% level?

3. Practical Applications
In practice, statistical tests are used for marketing, research, and the pharmaceutical industry.
The decisions we make as statisticians must have practical significance. This means that it must be worthwhile to use the information we find significant.

4. Points to Keep in Mind
If you are going to make a decision based on a statistical test, choose α in advance.
When choosing α, ask these questions:
Does H0 represent an assumption that people have believed for years? If so, then strong evidence (small α) is needed to persuade them.
What are the consequences of rejecting H0? Costly changes will require strong evidence.

5. Statistical Significance vs. Practical Significance
Statistical significance is based on the hypothesis test.
A large sample size will almost always show that small deviations are significant.
Why?
Practical significance means the data isn’t convincing enough to make a change.

6. Example of statistical significance that is not practical
Suppose we are testing a new antibacterial cream, “Formulation NS” on a small cut made on the inner forearm. We know from previous research that with n medication, the mean healing time (defined as the time for the scab to fall off) is 7.6 days, with a standard deviation of 1.4 days. The claim we want to test here is that Formulation NS speeds healing. We will use a 5% significance level.
We cut 25 volunteer college students and apply Formulation NS to the wound. The mean healing time for these subjects is x-bar = 7.1 days. We will assume that σ = 1.4 days.

7. Solution We find that the data is statistically significant.
However, it does not appear that the effect is all that great. Is it practical to use this treatment if it only reduces the amount of time you have a scab by about a day?

8. Cautions PROCEED WITH…..
Watch out for badly designed surveys or experiments!
Statistical inference cannot correct for basic flaws in design.
Always plot the data (if it’s given to you) and look for outliers or other deviations from a consistent pattern.

9. Type I and Type II Errors
Sometimes our decision (reject or fail to reject H0) will be wrong.
We could reject H0 when we shouldn’t have.
We could fail to reject H0 when we should have.

10. Type I and Type II Errors
H0 is True
H0 is False
Reject H0
Type I Error 
Fail to Reject H0
Type II Error

11. In words… Type I Error: Reject H0 when H0 is actually true.
Type II Error: Fail to reject H0 when H0 is actually false.

12. Why do we care about errors?
If a potato chip factory rejects bags of chips that statistically fail to meet a salt value, they lose money if the batch is really ok.
On the other hand, if they fail to reject a batch that has too much salt, they will have unhappy customers.
Type I
Type II

13. Probability of Type I error
The probability of a Type I error occurring is equal to alpha.

14. Find Probability of Type I error
The mean salt content of a certain type of potato chips is supposed to be 2.0mg. The salt content of these chips varies normally with standard deviation σ = 0.1mg. From each batch produced, an inspector takes a sample of 50 chips and measures the salt content of each chip. The inspector rejects the entire batch if the sample mean salt content is significantly different from 2mg at the 5% significance level.

15. I’ve Got the Power! Power is good!
Power is the probability that a fixed α level significance test will reject H0 if Ha is true.
Power of a test = 1 – P(Type II Error)
Power can increase by having a larger n. More and more often, statisticians are looking at the power of a study along with confidence intervals and significance tests

16. Try this out for size Have HIV Do not have HIV Test is HIV + 26 381
1235
H0: A person has HIV.
What is the alternative hypothesis?
What is a Type I error? What is a Type II error?
What is the probability of a Type I error? Type 2 error? Power?

17. Significance Tests: The Basics
Error Probabilities
The potato-chip producer wonders whether the significance test of H0 : p = 0.08 versus Ha : p > 0.08 based on a random sample of 500 potatoes has enough power to detect a shipment with, say, 11% blemished potatoes. In this case, a particular Type II error is to fail to reject H0 : p = 0.08 when p = 0.11.
Significance Tests: The Basics
What if p = 0.11?
Earlier, we decided to reject H0 at α = 0.05 if our sample yielded a sample proportion to the right of the green line.
Since we reject H0 at α= 0.05 if our sample yields a proportion > , we’d correctly reject the shipment about 75% of the time.
The power of a test against any alternative is 1 minus the probability of a Type II error for that alternative; that is, power = 1 - β.
Power and Type II Error

18. Homework
Chapter 9 #