What’s in the Bag? Lecture 4 Section Wed, Sep 7, 2005

How Strong is the Evidence? Rather than give an accept/reject answer, we may ask a different question: Rather than give an accept/reject answer, we may ask a different question: How strong is the evidence against H 0 ? How strong is the evidence against H 0 ?

The p-value p-value – The likelihood of getting by chance, if H 0 is true, a value at least as extreme as the one observed. p-value – The likelihood of getting by chance, if H 0 is true, a value at least as extreme as the one observed.

Two Bags If the selected token is worth $50, what is the p- value? If the selected token is worth $50, what is the p- value?

Two Bags Bag A Bag B

Two Bags Bag A Bag B At least as extreme as 50

Two Bags Bag A Bag B p-value = 2/20 = 0.10 At least as extreme as 50

Two More Bags Bag F Bag E

Two More Bags If the selected token is worth $8, what is the p- value? If the selected token is worth $8, what is the p- value? First, what is the direction of extreme? First, what is the direction of extreme? Which values are at least as extreme as 8? Which values are at least as extreme as 8? Bag E

Two More Bags Bag E Bag F At least as extreme as 8

Two More Bags Bag E Bag F p-value = 12/30 = 0.40

The p-value If the null hypothesis is true, then If the null hypothesis is true, then The observed data is likely to be close to what the null hypothesis predicts. The observed data is likely to be close to what the null hypothesis predicts. Therefore, Therefore, Large discrepancies are less likely than small discrepancies. Large discrepancies are less likely than small discrepancies. The p-value measures the likelihood of a discrepancy at least as large as the one observed. The p-value measures the likelihood of a discrepancy at least as large as the one observed.

Interpretation of the p-value 01 Smaller p-values Larger p-values p-values

Interpretation of the p-value 01 More extreme values Less extreme values p-values

Interpretation of the p-value 01 Larger discrepancy Smaller discrepancy p-values

Interpretation of the p-value 01 Stronger evidence against H 0 Weaker evidence against H 0 p-values

Interpretation of the p-value 01 Statistically more significant Statistically less significant p-values

Two Explanations For any discrepancy between the evidence and what is predicted by the null hypothesis, there are always two explanations: For any discrepancy between the evidence and what is predicted by the null hypothesis, there are always two explanations: The discrepancy occurred by chance (and H 0 is true). The discrepancy occurred by chance (and H 0 is true). The discrepancy occurred because for a specific reason other than chance (and H 0 is false). The discrepancy occurred because for a specific reason other than chance (and H 0 is false). Given the evidence, which explanation is more believable? Given the evidence, which explanation is more believable?

Two Explanations The null hypothesis gets the benefit of the doubt. The null hypothesis gets the benefit of the doubt. Therefore, Therefore, If the discrepancy is small (p-value is large), then we go with the first explanation and accept H 0. If the discrepancy is small (p-value is large), then we go with the first explanation and accept H 0. If the discrepancy is large (p-value is small), then we go with the second explanation and reject H 0. If the discrepancy is large (p-value is small), then we go with the second explanation and reject H 0.

Example Consider the Intelligent Design hypothesis vs. the Evolution hypothesis. Consider the Intelligent Design hypothesis vs. the Evolution hypothesis. Which one assumes the “chance” explanation? Which one assumes the “chance” explanation? Which one assumes a non-random, directed mechanism as the explanation? Which one assumes a non-random, directed mechanism as the explanation? Which would be the null hypothesis? Which would be the null hypothesis? Which would have the burden of proof? Which would have the burden of proof?

Let’s Do It! Let’s do it! 1.9, p. 32 – p-value for a One-Sided Rejection Region to the Left. Let’s do it! 1.9, p. 32 – p-value for a One-Sided Rejection Region to the Left. Let’s do it! 1.10, p. 34 – p-value for a Two-Sided Rejection Region. Let’s do it! 1.10, p. 34 – p-value for a Two-Sided Rejection Region. Example 1.9, p. 36 – Can You Picture the p- value? Example 1.9, p. 36 – Can You Picture the p- value?

Let’s Do It! Let’s do it! 1.7, p. 26 – Chromium Supplements. Let’s do it! 1.7, p. 26 – Chromium Supplements. Let’s do it! 1.8, p. 27 – Three Studies. Let’s do it! 1.8, p. 27 – Three Studies. Let’s do it! 1.11, p. 39 – Machine A or Machine B? Let’s do it! 1.11, p. 39 – Machine A or Machine B?