1. Water Test Take 1 cup from each sleeve –See numbers on bottom of cup –Numbers should be a # < 100 and 500 + that number –For small # (<100), 1’s place tells which bottle to pour –For large # (>500), 1’s place + 1 tells which bottle to pour –Labels do not matter. (Water has been transferred, etc) According to Evian’s website, their H 2 O has a: –“pleasant taste and silky texture” –Further, it is “exceptionally drinkable” Decide which is which, and give me the cup that you think is tap water. (One is Evian and 1 is tap.)

2. Water Test Experiment estimates overall ability of population to tell difference between tap and Evian water: –X i = 0 if person i can’t tell difference = 1 if person i can –p = x 1 + …+x n Note that this is different from the probability that an individual person can tell the difference.

3. Picture – idea for what p means Tasting Abilities Density of tasting ability -4-2024 0.0 0.1 0.2 0.3 0.4 water connoisseurs “all water tastes the same” average joe / jane

4. Picture – idea for what p means Tasting Abilities Density of tasting ability -4-2024 0.0 0.1 0.2 0.3 0.4 If a person’s ability is greater than a certain number (“threshold”), then the person can tell the difference. p is: ½ + area under curve to right of the “ability cutoff” (or 1 if that sum is greater than 1) Certain ability or threshold

5. Water Test Experiment estimates overall ability of population to tell difference between tap and Evian water: –X i = 0 if person i can’t tell difference = 1 if person i can –p = (x 1 + …+x n )/n Note that this is different from the probability that an individual person can tell the difference. How could we estimate this?

6. Hypothesis test Suppose we want to know if people can do better than “just guessing”? What’s the null hypothesis?

7. Type 1 and Type 2 Errors Truth H 0 True H A True Action Fail to Reject H 0 Reject H 0 correct Type 1 error Type 2 error Significance level =  =Pr(Making type 1 error) Power = 1–Pr(Making type 2 error)

8. In terms of our water example, suppose we repeated the experiment and sampled 50 new people Pr( Type 1 error ) = Pr( reject H 0 when mean is 50% ) = Pr( |Z| > z 0.025 ) = Pr( Z > 1.96 ) + Pr( Z < -1.96 ) = 0.05 =  When p is 0.5, then Z, the test statistic, has a standard normal distribution. Note that the test is designed to have type 1 error = 

9. Power = Pr( reject H 0 when p is not 0.5), For instance, suppose 0.75 is a difference from 0.5 that is important to detect. = Pr( reject H 0 when p is 0.75 ) = Pr( |(P-0.5)/sqrt((0.75*0.25)/35)| > 1.96) = Pr((P-0.5)/sqrt( (0.75*0.25)/35) > 1.96 ) + Pr((P-0.5)/sqrt( (0.75*0.25)/35) < -1.96 ) P ~N(0.75, (0.75*0.25)/35 ) by CLT, so (P-0.5)/sqrt( (0.75*0.25)/35) ~ N(3.42,1). This means, we want Pr( Z > 1.96-3.42) + Pr(Z -1.46 ) + Pr(Z < -5.38) = 0.93

10. Power calculations are an integral part of planning any experiment: Given: –a certain level of  –difference that is of interest –(need preliminary estimate of std dev of x’s that go into x when the test is about a mean) Compute required n in order for power to be at least 85% (or some other percentage...)

11. Power calculations are an integral part of planning any experiment: Bad News: Algebraically messy (but you should know how to do them) Good News: Minitab (or other statistical software) can be used to do them: Stat: Power and Sample Size… –Choose type of test you’re doing (“Z test” = large sample test for a mean) –Inputs: 1.required power 2.difference of interest –Output: Result = required sample size –Options: Change , one sided versus 2 sided tests

12. Picture for Power True p Power 0.5 0.2 0.4 0.6 0.8 1.0 “Pr(Reject H O when it’s false)” As n increases and/or  increases and/or std dev decreases, these curves become steeper 1.00.0