Section 9.1(re-visited)  Making Sense of Statistical Significance  Inference as Decision

Warm-up  The one-sample t statistic for testing H 0 : μ = 0 and H a : μ > 0 from a sample of n = 15 observations has the value t = 1.82  What are the degrees of freedom for this statistic?  Between what two values does the P-value of the test fall?  Is the value t = 1.82 significant at the 5% level? Is it significant at the 1% level?

Practical Applications  In practice, statistical tests are used for marketing, research, and the pharmaceutical industry.  The decisions we make as statisticians must have practical significance. This means that it must be worthwhile to use the information we find significant.

Points to Keep in Mind  If you are going to make a decision based on a statistical test, choose α in advance.  When choosing α, ask these questions:  Does H 0 represent an assumption that people have believed for years? If so, then strong evidence (small α) is needed to persuade them.  What are the consequences of rejecting H 0 ? Costly changes will require strong evidence.

Statistical Significance vs. Practical Significance  Statistical significance is based on the hypothesis test.  A large sample size will almost always show that small deviations are significant.  Why?  Practical significance means the data isn’t convincing enough to make a change.

Example of statistical significance that is not practical  Suppose we are testing a new antibacterial cream, “Formulation NS” on a small cut made on the inner forearm. We know from previous research that with n medication, the mean healing time (defined as the time for the scab to fall off) is 7.6 days, with a standard deviation of 1.4 days. The claim we want to test here is that Formulation NS speeds healing. We will use a 5% significance level.  We cut 25 volunteer college students and apply Formulation NS to the wound. The mean healing time for these subjects is x-bar = 7.1 days. We will assume that σ = 1.4 days.

Solution  We find that the data is statistically significant.  However, it does not appear that the effect is all that great. Is it practical to use this treatment if it only reduces the amount of time you have a scab by about a day?

Cautions  Watch out for badly designed surveys or experiments!  Statistical inference cannot correct for basic flaws in design.  Always plot the data (if it’s given to you) and look for outliers or other deviations from a consistent pattern. PROCEED WITH…..

Type I and Type II Errors  Sometimes our decision (reject or fail to reject H 0 ) will be wrong.  We could reject H 0 when we shouldn’t have.  We could fail to reject H 0 when we should have.

Type I and Type II Errors H 0 is True H 0 is False Reject H 0 Type I Error Fail to Reject H 0 Type II Error

In words…  Type I Error: Reject H 0 when H 0 is actually true.  Type II Error: Fail to reject H 0 when H 0 is actually false.

Why do we care about errors?  If a potato chip factory rejects bags of chips that statistically fail to meet a salt value, they lose money if the batch is really ok.  On the other hand, if they fail to reject a batch that has too much salt, they will have unhappy customers.

Probability of Type I error  The probability of a Type I error occurring is equal to alpha.

Find Probability of Type I error  The mean salt content of a certain type of potato chips is supposed to be 2.0mg. The salt content of these chips varies normally with standard deviation σ = 0.1mg. From each batch produced, an inspector takes a sample of 50 chips and measures the salt content of each chip. The inspector rejects the entire batch if the sample mean salt content is significantly different from 2mg at the 5% significance level.

I’ve Got the Power!  Power is good!  Power is the probability that a fixed α level significance test will reject H 0 if H a is true.  Power of a test = 1 – P(Type II Error)  Power can increase by having a larger n. More and more often, statisticians are looking at the power of a study along with confidence intervals and significance tests

Try this out for size Have HIV Do not have HIV Test is HIV Test is HIV  What is the alternative hypothesis?  What is a Type I error? What is a Type II error?  What is the probability of a Type I error? Type 2 error? Power? H 0 : A person has HIV.

 Error Probabilities The potato-chip producer wonders whether the significance test of H 0 : p = 0.08 versus H a : p > 0.08 based on a random sample of 500 potatoes has enough power to detect a shipment with, say, 11% blemished potatoes. In this case, a particular Type II error is to fail to reject H 0 : p = 0.08 when p = Significance Tests: The Basics Earlier, we decided to reject H 0 at α = 0.05 if our sample yielded a sample proportion to the right of the green line. What if p = 0.11? Since we reject H 0 at α= 0.05 if our sample yields a proportion > , we’d correctly reject the shipment about 75% of the time. The power of a test against any alternative is 1 minus the probability of a Type II error for that alternative; that is, power = 1 - β. Power and Type II Error

Homework Chapter 9 #