Significance Toolbox 1) Identify the population of interest (What is the topic of discussion?) and parameter (mean, standard deviation, probability) you want to draw conclusions about. State the null and alternative hypotheses. 2) Choose the appropriate inference procedure (type of test) and verify conditions (what kind of information is given about population/sample. Is there an SRS? If not, we may be able to perform the test because of the finite number of observations – central limit theorem but generalizations may not be necessarily true especially if the distribution is severely nonnormal). 3) If the conditions are met, carry out the inference procedure (find the mean, deviation, and P-value). 4) Interpret your results (Is the information statistically significant?).

One-sample z statistic H 0 : µ = µ 0 The test uses H a : µ > µ 0 is P(Z ≥ z) H a : µ < µ 0 is P(Z ≤ z) H a : µ ≠ µ 0 is 2P(Z ≥ |z|) View graphs on page 573.

Fixed Significant Level for Z tests for Population Mean The outcome of a test is significant at level alpha if P-value ≤. Once we have computed the z test statistic, reject H 0 at significant level against a one sided alternative when H a : µ>µ 0 if z ≥ z* and H a : µ < µ 0 if z ≤ - z* Reject H 0 at significant level alpha against a two-sided alternative H a : µ≠µ 0 if |z| ≥ z*

10.3 Making Sense of Statistical Significance

Choosing a level,  Standard: the level of significance gives a clear statement of the degree of evidence provided by the sample against the null hypothesis. Best practice: Decide on a significance level prior to testing. If the result satisfies the level, reject the null. If the result fails the level, find the null acceptable (fail to reject).

If we have a fixed significance level, we should ask how much evidence is required to reject H 0. If H 0 represents an assumption people have believed for years, strong evidence (small ) is needed. If rejecting H 0 for H a means making expensive changeover (products), strong evidence must show sales will soar.

Significant vs Insignificant There is no sharp border between significant and insignificant only increasingly strong evidence as the P-value decreases. When a null hypothesis can be rejected (5% or 1% level), there is good evidence that an effect is present. To keep statistical significance in its place, pay close attention to the actual data and the P-value.

Statistical inference is not always valid Surveys and experiments that are designed badly will produce invalid results. Outliers in the data and testing a hypothesis on the same data that suggested the hypothesis invalidates the test. Since tests of significance and confidence intervals are based on the laws of probability, randomization in sampling or experimentation ensures these laws apply.

Assignment Exercises 10.44, 10.57, 10.58, 10.62, 10.64

10.4 Inference as Decision Reminders: Tests of significances assess the strength of evidence ______ (for/against) the null hypothesis. Measurement: P-value which is the probability computed under the assumption that null hypothesis is ______ (true/false). The alternative hypothesis helps us to see what outcomes count ______ (for/against) the null hypothesis.

Strength  Decision A significance level chosen in advance points to the outcome of the test as a decision. If the result is significant, we reject the null hypothesis in favor of the alternate. If the result is not significant, we fail to reject the null (null hypothesis is acceptable). Making the decision to either fail to reject (acceptable) or reject results should be left to the user but at times the final decision is stated during the interpretation.

Acceptance Sampling A decision or action must be made as an end result of inference. Failing to reject (Acceptable) or rejecting the end product.

Type I and II Errors - In tests of significance - H 0 - the null hypothesis - H a - the alternative hypothesis - However, when dealing with Type I and Type II errors, these hypotheses will represent accepting one decision and rejecting the other. - Now … - H 0 should be considered the initial hypothesis - H a the secondary hypothesis

Type I Error We have been calculating this type of error all along. If we reject H 0 (acceptable H a ) when in fact H 0 is true.

Type II Error If we find that the H 0 is acceptable (reject H a ) when in fact H a is true.

Quick Comparison H 0 TrueH a True Reject H 0 Type 1 ErrorCorrect Decision Fail to reject H 0 (acceptable) Correct Decision Type 2 Error Truth about the Population Decision based on sample

Cancer Scenario H o : “We suspect that you have cancer” H o is TrueH a is True Reject the null: “You don’t have cancer!” Diagnosis: Cancer Type I Error Diagnosis: No Cancer Correct Decision Fail to reject the null: “You have cancer!” Diagnosis: Cancer Correct Decision Diagnosis: No Cancer Type II Error

Significance and Type I Error The significance level of any fixed level test is the probability of a Type I error. is the probability that the test will reject the null hypothesis H 0 when H 0 is in fact true. Example 10.68, Page 598

Example You have an SRS of size n = 9 from a normal distribution with  = 1. You wish to test H 0 : µ = 0 H a : µ > 0. You decide to reject H 0 if x > 0 and to accept H 0 otherwise.

Power A significance test measures the ability to detect an alternative hypothesis. The power against a specific alternative is the probability that the test will reject H 0 when the alternative is true. Calculate the power of a specific alternative: subtract the probability of the Type II error for the alternative from 1. Class example Accept that the mean, H 0, will be less than or equal to % of the time; however, the mean should be greater than % of the time (100% %).

Power continued Power works best for fixed significance levels. Larger sample sizes will increase the power for a fixed significant level.

Increase the Power If the strength of evidence required for rejection is too low, increase the significance level. Consider an alternative farther away from µ 0. Increase the sample size. Decrease the standard deviation, .

Assignment (Work due on Monday, 3/28) Exercises 10.67, 10.69, and 10.81

Scenarios OJ Simpson – Guilty man goes free. Ho: OJ is innocent Ha: OJ is guilty Found not guilty – Type 2 Guilty man goes free. Movie: A Time to Kill – Ho: Father is innocent Ha: Father is guilty Found to be innocent –Type 2 To Kill a Mockingbird – Send an innocent man to jail Ho: Tom Robinson is innocent Ha: Tom Robinson is guilty Found guilty – Type 1 The Green Mile