Introduction to Inference Tests of Significance

Errors in the justice system Actual truth Guilty Not guilty Correct decision Type I error Guilty Jury decision Not guilty Type II error Correct decision

“No innocent man is jailed” justice system Actual truth Guilty Not guilty Type I error Guilty smaller Jury decision Not guilty Type II error larger

“No guilty man goes free” justice system Actual truth Guilty Not guilty Type I error Guilty larger Jury decision Not guilty Type II error smaller

Errors in the justice system Actual truth Guilty Not guilty (Ha true) (H0 true) Correct decision Type I error Guilty (reject H0) Jury decision Not guilty Type II error Correct decision (fail to reject H0)

Type I and Type II example Water samples are taken from water used for cooling as it is being discharged from a power plant into a river. It has been determined that as long as the mean temperature of the discharged water is at most 150oF, there will be no negative effects on the river's ecosystem. To investigate whether the plant is in compliance with regulations that prohibit a mean discharge water temperature above 150o, 50 water samples will be taken at randomly selected times, and the temperature of each sample recorded. The resulting data will be used to test the hypotheses Ho: m = 150o versus Ha: m > 150o.

Type I and Type II example Type I error: We think the water temperature is greater than 150o, but actually the temperature is equal to 150o. Consequence: We falsely accuse the plant of producing water too hot and harming the environment when nothing wrongful was done.

Type I and Type II example Type II error: We think the water temperature is equal to 150o, but actually the temperature is greater than 150o. Consequence: We believe the plant’s water is a normal temperature, when actually they are harming the environment.

Type I and Type II example The significance level (a) is also the probability of committing a Type I error. As the probability of committing a Type I error goes down, the probability of committing a Type II error goes up

Type I and Type II errors If we believe Ha when in fact H0 is true, this is a type I error. If we believe H0 when in fact Ha is true, this is a type II error. Type I error: if we reject H0 and it’s a mistake. Type II error: if we fail to reject H0 and it’s a mistake. APPLET

Type I and Type II example A distributor of handheld calculators receives very large shipments of calculators from a manufacturer. It is too costly and time consuming to inspect all incoming calculators, so when each shipment arrives, a sample is selected for inspection. Information from the sample is then used to test Ho: p = .02 versus Ha: p < .02, where p is the true proportion of defective calculators in the shipment. If the null hypothesis is rejected, the distributor accepts the shipment of calculators. If the null hypothesis cannot be rejected, the entire shipment of calculators is returned to the manufacturer due to inferior quality. (A shipment is defined to be of inferior quality if it contains 2% or more defectives.)

Type I and Type II example Type I error: We think the proportion of defective calculators is less than 2%, but it’s actually 2% (or more). Consequence: Accept shipment that has too many defective calculators so potential loss in revenue.

Type I and Type II example Type II error: We think the proportion of defective calculators is 2%, but it’s actually less than 2%. Consequence: Return shipment thinking there are too many defective calculators, but the shipment is ok.

Type I and Type II example Calculator manufacturer wants to avoid Type II error. Choose a = .10 Distributor wants to avoid Type I error. Choose a = .01

Concept of Power Definition? Power is the capability of accomplishing something… The power of a test of significance is…

Power Example In a power generating plant, pressure in a certain line is supposed to maintain an average of 100 psi over any 4 - hour period. If the average pressure exceeds 103 psi for a 4 - hour period, serious complications can evolve. During a given 4 - hour period, thirty random measurements are to be taken. The standard deviation for these measurements is 4 psi (graph of data is reasonably normal), test Ho: m = 100 psi versus the alternative “new” hypothesis m = 103 psi. Test at the alpha level of .01. Calculate a type II error and the power of this test. In context of the problem, explain what the power means.

Type I error and a a is the probability that we think the mean pressure is above 100 psi, but actually the mean pressure is 100 psi (or less)

Type I error and a

Type II error and b

Type II error and b b is the probability that we think the mean pressure is 100 psi, but actually the pressure is greater than 100 psi.

Power?

For a sample size of 30, there is a For a sample size of 30, there is a .9495 probability that this test of significance will correctly detect if the pressure is above 100 psi.

Concept of Power The power of a test of significance is the probability that the null hypothesis will be correctly rejected. Because the true value of m is unknown, we cannot know what the power is for m, but we are able to examine “what if” scenarios to provide important information. Power = 1 – b

Concept of Power Ways to increase Power: Increase sample size (which also decreases the standard deviation) Choose a larger alpha Pick an Alternative hypothesis that is further away from the Null hypothesis

Concept of Power So… Alpha (α) and Beta (β) move in opposite directions Beta (β) and Power move in opposite directions Alpha (α) and Power move in the same direction

Concept of Power Recall that the significance level (α), is the probability of committing a Type I error. The probability of committing a Type II error is known as β (beta) As alpha goes up, beta goes down, and vice versa

Concept of Power The power of a test of significance is the probability that the null hypothesis will be correctly rejected. The jury finds the defendant guilty and he/she is actually guilty In other words, it is the probability of coming to a correct, accurate conclusion Power = 1 – b

Concept of Power Ways to increase Power: Increase sample size (which also decreases the standard deviation) Choose a larger alpha Pick an Alternative hypothesis that is further away from the Null hypothesis

Concept of Power So… Alpha (α) and Beta (β) move in opposite directions Beta (β) and Power move in opposite directions Alpha (α) and Power move in the same direction