Fall 2002Biostat Statistical Inference - Confidence Intervals General (1 -  ) Confidence Intervals: a random interval that will include a fixed (but unknown) parameter (1 -  ) of the time CI for ,  assumed known:  confidence  wider interval  sample size  smaller SE  narrower interval

Fall 2002Biostat

Fall 2002Biostat Confidence Intervals Q: When we do not know the population parameter, how can we use the sample to estimate the population mean, and use our knowledge of probability to give a range of values consistent with the data? Parameter:  Estimate: Given a normal population, or large sample size, we can state: Note: this is not

Fall 2002Biostat Confidence Intervals We can do some rearranging: The interval is called a 95% confidence interval for . Interpretation: If we repeat this procedure 100 times, the interval constructed in this manner will include the true mean (  ) 95 times.

Fall 2002Biostat Confidence Intervals  known When  is known we can construct a confidence interval for the population mean, , for any given confidence level, (1 -  ). Instead of using 1.96 (as with 95% CI’s) we simply use a different constant that yields the right probability. So if we desire a (1 -  ) confidence interval we can derive it based on the statement That is, we find constantsand that have exactly (1 -  ) probability between them. A (1 -  ) Confidence Interval for the Population Mean or, equivalently,

Fall 2002Biostat Confidence Intervals  known - EXAMPLE Suppose gestational times are normally distributed with a standard deviation of 6 days. A sample of 30 second time mothers yield a mean pregnancy length of days. Construct a 90% confidence interval for the mean length of second pregnancies based on this sample.

Fall 2002Biostat Summary General (1 -  ) Confidence Intervals: (statistic + Q (  /2)  SE statistic, statistic + Q (1-  /2)  SE statistic ) CI for ,  assumed known  Z.  confidence  wider interval  sample size  smaller SE  narrower interval

Fall 2002Biostat Statistical Inference - Hypothesis Testing Null Hypothesis Alternative Hypothesis Significance level Statistically significant Critical value Acceptance / rejection region p-value power Types of errors: Type I (  ), Type II (  ) One-sided (one-tailed) test Two-sided (two-tailed) test

Fall 2002Biostat The ideas in hypothesis testing are based on deductive reasoning - we assume that some probability model is true and then ask “What are the chances that these observations came from that probability model?”.

Fall 2002Biostat Hypothesis Testing Motivation A coin is flipped 50 times. Of the 50 flips, 45 are heads. Do you think it is a fair coin? (what do we mean by a fair coin?) (what is your reasoning behind a yes or no answer?)

Fall 2002Biostat

Fall 2002Biostat Hypothesis Testing Motivation Among year-old males the mean serum cholesterol is 211 mg/ml with standard deviation of 46 mg/ml. In a sample of 25 hypertensive men we find a mean serum-cholesterol level ( ) of 220 mg/ml. Is this strong enough evidence to convince us that the mean for the population of hypertensive men differs from 211 mg/ml? Is the data consistent with that model? What if= 230 mg/ml? What if = 250 mg/ml? What if the sample was of 100 instead of 25?

Fall 2002Biostat Hypothesis Testing Define:  = population mean serum cholesterol for male hypertensives Hypotheses: 1.Null Hypothesis: Generally, the hypothesis that the unknown parameter equals a fixed value. H 0 :  = 211 mg/ml 2.Alternative Hypothesis: contradicts the null hypothesis. H A :   211 mg/ml Typically, the alternative hypothesis is the thing you are trying to prove. The null hypothesis is a “straw man”. There is a reason for this, as we will see in a moment.

Fall 2002Biostat Hypothesis Testing Decision / Action: Either H 0 or H A must be true. Based on the data we will choose one of these hypotheses. But what if we choose wrong?! What is the probability of that happening?  = significance level = P(reject H 0 | H 0 true) 1-  = power = P(reject H 0 | H A true) Let’s construct a procedure that will help us decide between the two hypotheses …

Fall 2002Biostat Hypothesis Testing ***Suppose H o is true *** Then the sample mean ( ) for cholesterol should be “near”  0 = 211 mg/ml. Since is normally distributed with std. error, this suggests that, if H 0 is true, then should be near 0. Distribution of Z if H 0 is true Notice that, if H 0 is true, then only 5% of the time.

Fall 2002Biostat Hypothesis Testing Suppose we decide to reject H 0 if |Z| > 1.96; then P[reject H 0 | H 0 true] = In other words, we would only make a type I error 5% of the time. More generally, suppose we want P[reject H 0 | H 0 true] =  ; then our rule should be reject H 0 if |Z| >. The type I error rate is . Given  and H 0 we can construct a test of H 0 with a specified significance level (we can’t eliminate the possibility of making an error, but we can control it). But remember, we start by assuming that H 0 is true - we haven’t proved it is true. Therefore, we usually say if |Z| > 1.96 then we reject H 0. if |Z| < 1.96 then we fail to reject H 0.

Fall 2002Biostat Hypothesis Testing Let’s review what we’ve just done …. 1)I made an assumption - the null hypothesis is true (therefore Z ~ N(0,1)) 2)I determined if the data was consistent with that assumption (is it plausible that my observed Z came from a N(0,1) distribution?) 3)If the data are not consistent with the assumption, I conclude that either H 0 is true and something unusual happened (with probability  )  or, H 0 is not true. This 3-step procedure is the basis of all hypothesis tests, but … different hypotheses  different details

Fall 2002Biostat Hypothesis Testing Cholesterol Example: Let  be the (true but unknown population) mean serum cholesterol level for male hypertensives. We observe =220 mg/ml Also, we are told that for the general population...  0 = mean serum cholesterol level for males = 211 mg/ml  =std. dev. of serum cholesterol for males = 46 mg/ml NULL HYPOTHESIS: mean for male hypertensives is the same as the general male population. ALTERNATIVE HYPOTHESIS: mean for male hypertensives is different than the mean for the general male population. H 0 :  =  0 = 211 mg/ml H A :    0 (   211 mg/ml)

Fall 2002Biostat Hypothesis Testing Cholesterol Example: Test H 0 with significance level . Under H 0 we know: Therefore, Reject H 0 if > 1.96 gives an  = 0.05 test. Reject H 0 if

Fall 2002Biostat Hypothesis Testing Cholesterol Example: TEST: Reject H 0 if In terms of Z... Reject H 0 if Z 1.96

Fall 2002Biostat Rejection region

Fall 2002Biostat Hypothesis Testing p-value: smallest possible  for which the observed sample would still reject H 0. probability of obtaining a result as extreme or more extreme than the actual sample (given H 0 true). NOTE: probability calculations are always based on a model.

Fall 2002Biostat Hypothesis Testing p-value: Cholesterol Example = 220 mg/mln = 25  = 46 mg/ml H 0 :  = 211 mg/ml H A :   211 mg/ml What value of  would cause us to reject H o ? Need to find the value of that corresponds to the observed (Z). The p-value is given by: 2 * P[ > 220] = 2 * P[Z > 0.978] =.33

Fall 2002Biostat Determination of Statistical Significance for Results from Hypothesis Tests Either of the following methods can be used to establish whether results from hypothesis tests are statistically significant: (1)The test statistic Z can be computed and compared with the critical value at an  level of.05. Specifically, if H 0 :  =  0 versus H 1 :    0 are being tested and |Z| > 1.96, then H 0 is rejected and the results are declared statistically significant (i.e., p <.05). Otherwise, H 0 is accepted and the results are declared not statistically significant (i.e., p .05). We refer to this approach as the critical-value method. (2)The exact p-value can be computed, and if p <.05, then H 0 is rejected and the results are declared statistically significant. Otherwise, if p .05 then H 0 is accepted and the results are declared not statistically significant. We will refer to this approach as the p-value method.

Fall 2002Biostat Guidelines for Judging the Significance of p-value (Rosner pg 200) If.01 < p <.05, then the results are significant. If.001 < p <.01, then the results are highly significant. If p <.001, then the results are very highly significant. If p >.05, then the results are considered not statistically significant (sometimes denoted by NS). However, if.05 < p <.10, then a trend toward statistically significance is sometimes noted.

Fall 2002Biostat Hypothesis Testing and Confidence Intervals Confidence Interval: “Plausible” values for  are given by If the null hypothesis value of  (namely,  0 ) is not in this interval, then it is not “plausible” and should be rejected. In fact, this relationship is exact – 1)any value for  that is not in an  -level confidence interval will be rejected by a  -level hypothesis test. 2)an  -level confidence interval for  consists of all possible values of  0 that would not be rejected in a  -level hypothesis test.

Fall 2002Biostat Hypothesis Testing “how many sides?” Depending on the alternative hypothesis a test may have a one-sided alternative or a two- sided alternative. Consider H 0 :  =  0 We can envision (at least) three possible alternatives H A :    0 (1) H A :  <  0 (2) H A :  >  0 (3) (1)is an example of a “two-sided alternative” (2) and (3) are examples of “one-sided alternatives” The distinction impacts Rejection regions p-value calculation

Fall 2002Biostat Hypothesis Testing “how many sides?” Cholesterol Example: Instead of the two-sided alternative considered earlier we may have only been interested in the alternative that hypertensives had a higher serum cholesterol. H 0 :  = 211 H A :  > 211 Given this, an  = 0.05 test would reject when We put all the probability on “one-side”. The p-value would be half of the previous, p-value = P[ > 220] =.163

Fall 2002Biostat

Fall 2002Biostat Hypothesis Testing Through this worked example we have seen the basic components to the statistical test of a scientific hypothesis. Summary 1.Identify H 0 and H A 2.Identify a test statistic 3.Determine a significance level,  = 0.05,  = Critical value determines rejection / acceptance region 5.p-value 6.Interpret the result

Fall 2002Biostat Hypothesis Testing Null Hypothesis Alternative Hypothesis Significance level Statistically significant Critical value Acceptance / rejection region p-value power Types of errors: Type I (  ), Type II (  ) One-sided (one-tailed) test Two-sided (two-tailed) test