Last Time Binomial Distribution Political Polls Hypothesis Testing Excel Computation Political Polls Strength of evidence Hypothesis Testing Yes – No Questions

Administrative Matter Midterm I, coming Tuesday, Feb. 24 (will say more later)

Reading In Textbook Approximate Reading for Today’s Material: Pages 488-491, 317-318 Approximate Reading for Next Class: Pages 261-262, 9-14, 270-276, 30-34

Haircut? Why? Website: http://www.time.com/time/health/article/0,8599,1733719,00.html

Haircut?

Hypothesis Testing Example: Suppose surgery cures (a certain type of) cancer 60% of time Q: is eating apricot pits a more effective cure?

(i.e. proportion of people cured) Hypothesis Testing E.g. Pits vs. Surgery Let p be “cure rate” of pits (i.e. proportion of people cured)

Hypothesis Testing E.g. Pits vs. Surgery Let p be “cure rate” of pits (H0 & H1? New method needs to “prove it’s worth” so put burden of proof on it)

Hypothesis Testing E.g. Pits vs. Surgery Let p be “cure rate” of pits H0: p < 0.6 vs. H1: p ≥ 0.6 Recall cure rate of surgery (competing treatment)

Hypothesis Testing E.g. Pits vs. Surgery Let p be “cure rate” of pits H0: p < 0.6 vs. H1: p ≥ 0.6 (OK to be sure of “at least as good”, since pits nicer than surgery)

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose observe X = 11, out of 15 were cured by pits

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose observe X = 11, out of 15 were cured by pits I.e.: “best guess about p” is:

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose observe X = 11, out of 15 were cured by pits I.e.: “best guess about p” is:

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose observe X = 11, out of 15 were cured by pits I.e.: “best guess about p” is: Looks Better?

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose observe X = 11, out of 15 were cured by pits I.e.: “best guess about p” is: But is it conclusive?

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose observe X = 11, out of 15 were cured by pits I.e.: “best guess about p” is: But is it conclusive? Or just due to sampling variation?

Hypothesis Testing Approach: Define “p-value” =

Hypothesis Testing Approach: Define “p-value” = “observed significance level”

Hypothesis Testing Approach: Define “p-value” = “observed significance level” = “significance probability”

Hypothesis Testing Approach: Define “p-value” = “observed significance level” = “significance probability” = P[seeing something as unusual as 11 | H0 is true]

Hypothesis Testing “p-value” = “observed significance level” = P[seeing something as unusual as 11 | H0 is true]

Hypothesis Testing “p-value” = “observed significance level” = P[seeing something as unusual as 11 | H0 is true] Note: for

Hypothesis Testing “p-value” = “observed significance level” = P[seeing something as unusual as 11 | H0 is true] Note: for could use “X/n = 0.733”

Hypothesis Testing “p-value” = “observed significance level” = P[seeing something as unusual as 11 | H0 is true] Note: for could use “X/n = 0.733”, but this depends too much on n

(look at example illustrating this) Hypothesis Testing “p-value” = “observed significance level” = P[seeing something as unusual as 11 | H0 is true] Note: for could use “X/n = 0.733”, but this depends too much on n (look at example illustrating this)

Class Example 4 For X ~ Bi(n,0.6): n P(X/n = 0.6) P(X/n >= 0.6) 5 0.346 0.317 10 0.251 0.367 30 0.147 0.422 100 0.081 0.457 300 0.047 0.475 1000 0.026 0.486 3000 0.015 0.492 10000 0.008 0.496

Class Example 4 For X ~ Bi(n,0.6): Computed using Excel: n P(X/n = 0.6) P(X/n >= 0.6) 5 0.346 0.317 10 0.251 0.367 30 0.147 0.422 100 0.081 0.457 300 0.047 0.475 1000 0.026 0.486 3000 0.015 0.492 10000 0.008 0.496 For X ~ Bi(n,0.6): Computed using Excel: http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg4.xls

Class Example 4 For X ~ Bi(n,0.6): Note: these go to 0, even at “most likely value” n P(X/n = 0.6) P(X/n >= 0.6) 5 0.346 0.317 10 0.251 0.367 30 0.147 0.422 100 0.081 0.457 300 0.047 0.475 1000 0.026 0.486 3000 0.015 0.492 10000 0.008 0.496

Class Example 4 For X ~ Bi(n,0.6): Note: these go to 0, even at “most likely value” So “small” is not conclusive n P(X/n = 0.6) P(X/n >= 0.6) 5 0.346 0.317 10 0.251 0.367 30 0.147 0.422 100 0.081 0.457 300 0.047 0.475 1000 0.026 0.486 3000 0.015 0.492 10000 0.008 0.496

Class Example 4 For X ~ Bi(n,0.6): But for these “small” is conclusive P(X/n = 0.6) P(X/n >= 0.6) 5 0.346 0.317 10 0.251 0.367 30 0.147 0.422 100 0.081 0.457 300 0.047 0.475 1000 0.026 0.486 3000 0.015 0.492 10000 0.008 0.496

Class Example 4 For X ~ Bi(n,0.6): But for these “small” is conclusive (so use range, not value) n P(X/n = 0.6) P(X/n >= 0.6) 5 0.346 0.317 10 0.251 0.367 30 0.147 0.422 100 0.081 0.457 300 0.047 0.475 1000 0.026 0.486 3000 0.015 0.492 10000 0.008 0.496

Hypothesis Testing “p-value” = “observed significance level” = P[seeing 11 or more unusual | H0 is true]

Hypothesis Testing “p-value” = “observed significance level” = P[seeing 11 or more unusual | H0 is true] So use: = P[X ≥ 11 | H0 is true]

Hypothesis Testing “p-value” = P[X ≥ 11 | H0 is true]

Hypothesis Testing “p-value” = P[X ≥ 11 | H0 is true] What to use here?

Hypothesis Testing “p-value” = P[X ≥ 11 | H0 is true] What to use here? Recall: H0: p < 0.6

Hypothesis Testing “p-value” = P[X ≥ 11 | H0 is true] What to use here? Recall: H0: p < 0.6 How does P[X ≥ 11 | p] depend on p?

Hypothesis Testing How does P[X ≥ 11 | p] depend on p?

Hypothesis Testing How does P[X ≥ 11 | p] depend on p? Calculated in Class EG 4b: http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg4.xls p P(X >= 11|p) 0.2 0.000 0.3 0.001 0.4 0.009 0.5 0.059 0.6 0.217 0.7 0.515 0.8 0.836

Hypothesis Testing How does P[X ≥ 11 | p] depend on p? Bigger assumed p goes with Bigger Probability i.e. less conclusive p P(X >= 11|p) 0.2 0.000 0.3 0.001 0.4 0.009 0.5 0.059 0.6 0.217 0.7 0.515 0.8 0.836

Hypothesis Testing “p-value” = P[X ≥ 11 | H0 is true] = = P[X ≥ 11 | p < 0.6]

Hypothesis Testing “p-value” = P[X ≥ 11 | H0 is true] = = P[X ≥ 11 | p < 0.6] So, to be “sure” of conclusion, use largest available value of P[X ≥ 11 | p]

Hypothesis Testing “p-value” = P[X ≥ 11 | H0 is true] = = P[X ≥ 11 | p < 0.6] So, to be “sure” of conclusion, use largest available value of P[X ≥ 11 | p] Thus, define: “p-value” = P[X ≥ 11 | p = 0.6]

Hypothesis Testing “p-value” = P[X ≥ 11 | H0 is true] = = P[X ≥ 11 | p < 0.6] So, to be “sure” of conclusion, use largest available value of P[X ≥ 11 | p] Thus, define: “p-value” = P[X ≥ 11 | p = 0.6] (since “=” gives safest result)

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6]

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] Generally: use = P[seeing something as unusual as X = 11 | H0 is true]

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] Generally: use = P[seeing something as unusual as X = 11 | H0 is true] Here use boundary between H0 & H1

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] Generally: use = P[seeing something as unusual as X = 11 | H0 is true] Here use boundary between H0 & H1 (above e.g. p = 0.6)

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] Now calculate numerical value

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] Now calculate numerical value (already done above, Class EG 4)

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 Now calculate numerical value (already done above, Class EG 4)

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 Now calculate numerical value (already done above, Class EG 4) How to interpret?

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 Intuition: p-value reflects chance of error when H0 is rejected

(i.e. when conclusion is made) Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 Intuition: p-value reflects chance of error when H0 is rejected (i.e. when conclusion is made)

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 Intuition: p-value reflects chance of error when H0 is rejected (i.e. when conclusion is made) (based on available evidence)

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 Intuition: p-value reflects chance of error when H0 is rejected (i.e. when conclusion is made) (based on available evidence) When p-value is small, it is safe to make a firm conclusion

Hypothesis Testing For small p-value, safe to make firm conclusion

Hypothesis Testing For small p-value, safe to make firm conclusion How small?

Hypothesis Testing For small p-value, safe to make firm conclusion How small? Approach 1: Traditional (& legal) cutoff

Hypothesis Testing For small p-value, safe to make firm conclusion How small? Approach 1: Traditional (& legal) cutoff Called here “Yes-No”:

Hypothesis Testing For small p-value, safe to make firm conclusion How small? Approach 1: Traditional (& legal) cutoff Called here “Yes-No”: Reject H0 when p-value < 0.05

(just an agreed upon value, Hypothesis Testing For small p-value, safe to make firm conclusion How small? Approach 1: Traditional (& legal) cutoff Called here “Yes-No”: Reject H0 when p-value < 0.05 (just an agreed upon value, but very widely used)

Hypothesis Testing For small p-value, safe to make firm conclusion How small? Approach 1: Traditional (& legal) cutoff Called here “Yes-No”: Reject H0 when p-value < 0.05 (but sometimes want different values, e.g. your airplane is safe to fly)

Hypothesis Testing Approach 1: “Yes-No” Reject H0 when p-value < 0.05

Hypothesis Testing Approach 1: “Yes-No” Reject H0 when p-value < 0.05 Terminology: say results are “statistically significant”, when this happens

Hypothesis Testing Approach 1: “Yes-No” Reject H0 when p-value < 0.05 Terminology: say results are “statistically significant”, when this happens Sometimes specify a value α Greek letter “alpha”

Hypothesis Testing Approach 1: “Yes-No” Reject H0 when p-value < 0.05 Terminology: say results are “statistically significant”, when this happens Sometimes specify a value α as the cutoff (different from 0.05)

Hypothesis Testing Approach 2: “Gray Level” Idea: allow “shades of conclusion”

Hypothesis Testing Approach 2: “Gray Level” Idea: allow “shades of conclusion” e.g. Do p-val = 0.049 and p-val = 0.051 represent very different levels of evidence?

Hypothesis Testing Approach 2: “Gray Level” Idea: allow “shades of conclusion” Use words describing strength of evidence: 0.1 < p-val: no evidence 0.01 < p-val < 0.1 marginal evidence p-val < 0.01 very strong evidence

Hypothesis Testing Approach 2: “Gray Level” Use words describing strength of evidence: 0.1 < p-val: no evidence 0.01 < p-val < 0.1 marginal evidence p-val < 0.01 very strong evidence

Hypothesis Testing Approach 2: “Gray Level” Use words describing strength of evidence: 0.1 < p-val: no evidence 0.01 < p-val < 0.1 marginal evidence p-val < 0.01 very strong evidence stronger when closer to 0.01

Hypothesis Testing Approach 2: “Gray Level” Use words describing strength of evidence: 0.1 < p-val: no evidence 0.01 < p-val < 0.1 marginal evidence p-val < 0.01 very strong evidence stronger when closer to 0.01 weaker when closer to 0.1

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 Bottom Line: Yes-No: can not reject H0, since 0.217 > 0.05 i.e. no firm evidence pits better than surgery Gray level: not much indicated

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 No firm evidence pits better than surgery Gray level: not much indicated

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 No firm evidence pits better than surgery Gray level: not much indicated Practical Issue: since 73% = observed rate for pits > 60% (surgery),

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 No firm evidence pits better than surgery Gray level: not much indicated Practical Issue: since 73% = observed rate for pits > 60% (surgery), may want to gather more data

Hypothesis Testing “p-value” = P[X ≥ 11 | p = 6] = 0.217 No firm evidence pits better than surgery Gray level: not much indicated Practical Issue: since 73% = observed rate for pits > 60% (surgery), may want to gather more data, might show value of pits

Research Corner Medical Imaging – Another Fun Example Cornea Data

Research Corner Medical Imaging – Another Fun Example Cornea Data Cornea = Outer surface of eye

Research Corner Medical Imaging – Another Fun Example Cornea Data Cornea = Outer surface of eye “Curvature” important to vision

Research Corner Medical Imaging – Another Fun Example Cornea Data Cornea = Outer surface of eye “Curvature” important to vision Study heat map showing curvature

Research Corner Cornea Data Heat map shows curvature Each image is one person

Research Corner Cornea Data Heat map shows curvature Each image is one person Understand “population variation”?

Research Corner Cornea Data Heat map shows curvature Each image is one person Understand “population variation”? (too messy for brain to summarize)

Research Corner Cornea Data Approach: Principal Component Analysis

Research Corner Cornea Data Approach: Principal Component Analysis Idea: follow “direction” in image space,

Research Corner Cornea Data Approach: Principal Component Analysis Idea: follow “direction” in image space, that highlights population features

Research Corner Cornea Data Population features

Research Corner Cornea Data Population features Overall curvature (hot – cold)

Research Corner Cornea Data Population features Overall curvature (hot – cold) With the rule astigmatism (figure 8 pattern)

Research Corner Cornea Data Population features Overall curvature (hot – cold) With the rule astigmatism (figure 8 pattern) Correlation

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 (cured by pits)

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 (cured by pits) (recall above saw 11 / 25 not conclusive, so now suppose stronger evidence)

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 So

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 So

(more conclusive than before) Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 So (more conclusive than before)

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 So (more conclusive than before) (how much stronger is the evidence?)

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 So p-value = P[ X ≥ 13 | p = 0.6]

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 So p-value = P[ X ≥ 13 | p = 0.6] = 0.027

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 So p-value = P[ X ≥ 13 | p = 0.6] = 0.027 Calculated similar to above: http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg4.xls

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 p-value = P[ X ≥ 13 | p = 0.6] = 0.027

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 p-value = P[ X ≥ 13 | p = 0.6] = 0.027 Conclusions: Yes-No: 0.027 < 0.05, so can reject H0 and make firm conclusion pits are better

Hypothesis Testing H0: p < 0.6 vs. H1: p ≥ 0.6 Now suppose X had been 13 out of 15 p-value = P[ X ≥ 13 | p = 0.6] = 0.027 Conclusions: Yes-No: 0.027 < 0.05, so can reject H0 and make firm conclusion pits are better Gray Level: Strong case, nearly very strong that pits are better

Hypothesis Testing In General: p-value = P[what was seen, or more conclusive | at boundary between H0 & H1]

Hypothesis Testing In General: p-value = P[what was seen, or more conclusive | at boundary between H0 & H1] (will use this throughout the course, well beyond Binomial distributions)

Hypothesis Testing HW C14: Answer from both gray-level and yes-no viewpoints: (a) A TV ad claims that less than 40% of people prefer Brand X. Suppose 7 out of 10 randomly selected people prefer Brand X. Should we dispute the claim? (p-value = 0.055)

Hypothesis Testing HW C14: Answer from both gray-level and yes-no viewpoints: (b) 80% of the sheet metal we buy from supplier A meets our specs. Supplier B sends us 12 shipments, and 11 meet our specs. Is it safe to say the quality of B is higher? (p-value = 0.275)

Warning Avoid the “Excel Twiddle Trap”

Warning Avoid the “Excel Twiddle Trap”, E.g. C14(a)

Warning Avoid the “Excel Twiddle Trap”, E.g. C14(a) Find what Excel needs:

Warning Avoid the “Excel Twiddle Trap”, E.g. C14(a) Find what Excel needs: Number_s: 7 Trials: 10 Probability_s: 0.4 Cumulative: true (plug in)

Warning Avoid the “Excel Twiddle Trap”, E.g. C14(a) Check given answer (0.055)

Warning Avoid the “Excel Twiddle Trap”, E.g. C14(a) Check given answer (0.055) Way off!

Warning Avoid the “Excel Twiddle Trap”, E.g. C14(a) Check given answer (0.055) Way off! Try “1 -” i.e. target (0.945)

Warning Avoid the “Excel Twiddle Trap”, E.g. C14(a) Check given answer (0.055) Way off! Try “1 -” i.e. target (0.945) Still off, how about the “> vs. ≥” issue?

Warning Avoid the “Excel Twiddle Trap”, E.g. C14(a) Check given answer (0.055) Way off! Try “1 -” i.e. target (0.945) Still off, how about the “> vs. ≥” issue? try replacing 7 by 6?

Warning Avoid the “Excel Twiddle Trap”, E.g. C14(a) Check given answer (0.055) Way off! Try “1 -” i.e. target (0.945) Still off, how about the “> vs. ≥” issue? try replacing 7 by 6? Yes!

Warning Avoid the “Excel Twiddle Trap”: Can solve HW OK

Warning Avoid the “Excel Twiddle Trap”: Can solve HW OK But not on exam No numerical answer given No interaction with Excel

Warning Avoid the “Excel Twiddle Trap”: Can solve HW OK But not on exam No numerical answer given No interaction with Excel Real Goal: Understanding Principles

And now for something completely different Lateral Thinking: What is the phrase?

And now for something completely different Lateral Thinking: What is the phrase? Card Shark

And now for something completely different Lateral Thinking: What is the phrase?

And now for something completely different Lateral Thinking: What is the phrase? Knight Mare

And now for something completely different Lateral Thinking: What is the phrase?

And now for something completely different Lateral Thinking: What is the phrase? Gator Aide

Hypothesis Testing In General: p-value = P[what was seen, or more conclusive | at boundary between H0 & H1]

Hypothesis Testing In General: p-value = P[what was seen, or more conclusive | at boundary between H0 & H1] Caution: more conclusive requires careful interpretation

Hypothesis Testing Caution: more conclusive requires careful interpretation

Hypothesis Testing Caution: more conclusive requires careful interpretation Reason: Need to decide between 1 - sided Hypotheses

Hypothesis Testing Caution: more conclusive requires careful interpretation Reason: Need to decide between 1 - sided Hypotheses, like H0 : p < vs. H1: p ≥ some given numerical value

Hypothesis Testing Caution: more conclusive requires careful interpretation Reason: Need to decide between 1 - sided Hypotheses, like H0 : p < vs. H1: p ≥ And 2 - sided Hypotheses

Hypothesis Testing Caution: more conclusive requires careful interpretation Reason: Need to decide between 1 - sided Hypotheses, like H0 : p < vs. H1: p ≥ And 2 - sided Hypotheses, like H0 : p = vs. H1: p ≠

Hypothesis Testing 2 - sided Hypotheses, like H0 : p = vs. H1: p ≠ Note: Can never have H1: p =

Hypothesis Testing 2 - sided Hypotheses, like H0 : p = vs. H1: p ≠ Note: Can never have H1: p = , since can’t tell for sure between and + 0.000001

(Recall: H1 has burden of proof) Hypothesis Testing 2 - sided Hypotheses, like H0 : p = vs. H1: p ≠ Note: Can never have H1: p = , since can’t tell for sure between and + 0.000001 (Recall: H1 has burden of proof)

Hypothesis Testing Caution: more conclusive requires careful interpretation 1 - sided Hypotheses & 2 - sided Hypotheses

(important choice will need to make a lot) Hypothesis Testing Caution: more conclusive requires careful interpretation 1 - sided Hypotheses & 2 - sided Hypotheses (important choice will need to make a lot)

Hypothesis Testing Caution: more conclusive requires careful interpretation 1 - sided Hypotheses & 2 - sided Hypotheses Useful Rule: set up 2-sided when problem uses words like “equal” or “different”

Hypothesis Testing e.g. a slot machine Gambling device

Hypothesis Testing e.g. a slot machine Gambling device Players put money in

(of quite a lot more money) Hypothesis Testing e.g. a slot machine Gambling device Players put money in With (small) probability, win a “jackpot” (of quite a lot more money)

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time”

(in real life, focus is on “return rate”) Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” (in real life, focus is on “return rate”)

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” (in real life, focus is on “return rate”) (since people enjoy fewer, but bigger jackpots)

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” (in real life, focus is on “return rate”) (since people enjoy fewer, but bigger jackpots) (but usually no signs, since return rate is < 0)

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any.

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Can I conclude sign is false?

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Can I conclude sign is false? (& thus have grounds for complaint, or is this a reasonable occurrence?)

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win]

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win] (usual approach: give unknowns a name, so can work with)

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win], let X = # wins in 10 plays

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win], let X = # wins in 10 plays Model: X ~ Bi(10, p)

(set up as H0, the point want to disprove) Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win], let X = # wins in 10 plays Model: X ~ Bi(10, p) (set up as H0, the point want to disprove)

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win], let X = # wins in 10 plays Model: X ~ Bi(10, p) Test: H0: p = 0.3 vs. H1: p ≠ 0.3

(“false” means don’t win 30% of time, Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win], let X = # wins in 10 plays Model: X ~ Bi(10, p) Test: H0: p = 0.3 vs. H1: p ≠ 0.3 (“false” means don’t win 30% of time, so go 2-sided)

Hypothesis Testing Aside (similar to above): Can never set up H0: p ≠ 0.3

Hypothesis Testing Aside (similar to above): Can never set up H0: p ≠ 0.3 And then prove that p = 0.3

Hypothesis Testing Aside (similar to above): Can never set up H0: p ≠ 0.3 And then prove that p = 0.3 Since can’t handle gray area of hypo test

Hypothesis Testing Aside (similar to above): Can never set up H0: p ≠ 0.3 And then prove that p = 0.3 Since can’t handle gray area of hypo test E.g. can’t distinguish from p = 0.30001

Hypothesis Testing Aside (similar to above): Can never set up H0: p ≠ 0.3 And then prove that p = 0.3 Since can’t handle gray area of hypo test E.g. can’t distinguish from p = 0.30001 Could always be “off a little bit”

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win], let X = # wins in 10 plays Model: X ~ Bi(10, p) Test: H0: p = 0.3 vs. H1: p ≠ 0.3

(now test & see how weird X = 0 is, for p = 0.3) Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win], let X = # wins in 10 plays Model: X ~ Bi(10, p) Test: H0: p = 0.3 vs. H1: p ≠ 0.3 (now test & see how weird X = 0 is, for p = 0.3)

Hypothesis Testing e.g. a slot machine bears a sign which says “Win 30% of the time” In 10 plays, I don’t win any. Conclude false? Let p = P[win], let X = # wins in 10 plays Model: X ~ Bi(10, p) Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = P[X = 0 or more conclusive | p = 0.3]

Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = P[X = 0 or more conclusive | p = 0.3]

(understand this by visualizing # line) Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = P[X = 0 or more conclusive | p = 0.3] (understand this by visualizing # line)

Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = P[X = 0 or more conclusive | p = 0.3] 0 1 2 3 4 5 6

Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = P[X = 0 or more conclusive | p = 0.3] 0 1 2 3 4 5 6 30% of 10, most likely when p = 0.3 i.e. least conclusive

Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = P[X = 0 or more conclusive | p = 0.3] 0 1 2 3 4 5 6 so more conclusive includes

Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = P[X = 0 or more conclusive | p = 0.3] 0 1 2 3 4 5 6 so more conclusive includes but since 2-sided, also include

Hypothesis Testing Generally how to calculate? 0 1 2 3 4 5 6

Hypothesis Testing Generally how to calculate? Observed Value 0 1 2 3 4 5 6

Hypothesis Testing Generally how to calculate? Observed Value Most Likely Value 0 1 2 3 4 5 6

Hypothesis Testing Generally how to calculate? Observed Value Most Likely Value 0 1 2 3 4 5 6 # spaces = 3

Hypothesis Testing Generally how to calculate? Observed Value Most Likely Value 0 1 2 3 4 5 6 # spaces = 3 so go 3 spaces in other direct’n

Hypothesis Testing Result: More conclusive means X ≤ 0 or X ≥ 6 0 1 2 3 4 5 6

Hypothesis Testing Result: More conclusive means X ≤ 0 or X ≥ 6 p-value = P[X = 0 or more conclusive | p = 0.3]

Hypothesis Testing Result: More conclusive means X ≤ 0 or X ≥ 6 p-value = P[X = 0 or more conclusive | p = 0.3] = P[X ≤ 0 or X ≥ 6 | p = 0.3]

Hypothesis Testing Result: More conclusive means X ≤ 0 or X ≥ 6 p-value = P[X = 0 or more conclusive | p = 0.3] = P[X ≤ 0 or X ≥ 6 | p = 0.3] = P[X ≤ 0] + (1 – P[X ≤ 5])

Hypothesis Testing Result: More conclusive means X ≤ 0 or X ≥ 6 p-value = P[X = 0 or more conclusive | p = 0.3] = P[X ≤ 0 or X ≥ 6 | p = 0.3] = P[X ≤ 0] + (1 – P[X ≤ 5]) = 0.076

Hypothesis Testing Result: More conclusive means X ≤ 0 or X ≥ 6 p-value = P[X = 0 or more conclusive | p = 0.3] = P[X ≤ 0 or X ≥ 6 | p = 0.3] = P[X ≤ 0] + (1 – P[X ≤ 5]) = 0.076 Excel result from: http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg4.xls

Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = 0.076

Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = 0.076 Yes-No Conclusion: 0.076 > 0.05, so not safe to conclude “P[win] = 0.3” sign is wrong, at level 0.05

(10 straight losses is reasonably likely) Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = 0.076 Yes-No Conclusion: 0.076 > 0.05, so not safe to conclude “P[win] = 0.3” sign is wrong, at level 0.05 (10 straight losses is reasonably likely)

Hypothesis Testing Test: H0: p = 0.3 vs. H1: p ≠ 0.3 p-value = 0.076 Yes-No Conclusion: 0.076 > 0.05, so not safe to conclude “P[win] = 0.3” sign is wrong, at level 0.05 Gray Level Conclusion: in “fuzzy zone”, some evidence, but not too strong