Review: Large Sample Confidence Intervals 1-  confidence interval for a mean: x +/- z  /2 s/sqrt(n) 1-  confidence interval for a proportion: p +/- z  /2 p(1-p)/sqrt(n) 1-  confidence interval for the difference between two means: x 1 – x 2 +/- z  /2 sqrt(s 2 1 /n 1 +s 2 2 /n 2 ) n >30 or so for means, np and n(1-p) both > 5 for proportions

In General: Estimate (that is normally distributed via the Central Limit Theorem) +/- standard deviation Z  /2 of estimate ( ) This gives an interval: (Lower Bound, Upper Bound) Interpretation: This is a plausible range for the true value of the number that we’re estimating.  is a tuning parameter for level of plausibility: smaller  = more conservative estimate.

Large Sample Confidence Intervals 1-  confidence interval for difference between two proportions: p 1 -p 2 +/- z  /2 sqrt[(p 1 (1-p 1 )/n 1 )+(p 2 (1-p 2 )/n 2 )] np and n(1-p) > 5 for all p’s…

Designing an Experiment and Choosing a Sample Size Example: Compare the shrinkage in a tumor due to a “new” cancer treatment relative to standard treatment 100 patients randomly assigned to “new” treatment or standard treatment x inew = reduction in tumor size for person i under new treatment x jstd = reduction in tumor size for person j under std treatment x new and s 2 new x std and s 2 std Mean and sample variance of the changes in size for the new and standard treatments

Suppose the data are: x new = 25.3 s new = 2.0 x std = 24.8 s std = % Confidence Interval for difference: x 1 – x 2 +/- z  /2 sqrt(s 2 1 /n 1 +s 2 2 /n 2 ) = 0.5 +/ What can we conclude?

There’s no difference? Can’t see a difference? There’s a difference, but it’s too small to care about?

There is a difference between: Can’t see a difference There’s no difference Situation for Cancer example (In cancer experiment, we can assume we care about small differences.)

Can’t see a difference (that is big enough to care about) = wasted experiment AVOID / PREVENT THE WASTE AND ASSOCIATED TEARS USE SAMPLE SIZE PLANNING

Sample Size Planning Length of a 1-  level confidence interval is: “2 z  /2 std deviation of estimate” 2z  /2 s/sqrt(n) 2z  /2 p(1-p)/sqrt(n) 2z  /2 sqrt((s 2 1 /n 1 )+(s 2 2 /n 2 )) 2z  /2 sqrt[(p 1 (1-p 1 )/n 1 )+(p 2 (1-p 2 )/n 2 )]

1.Suppose we want a 95% confidence interval no wider than W units. 2.  is fixed. Assume a value for the standard deviation (or variance) of the estimator. 3.Solve for an n (or n 1 and n 2 ) so that the width is less than W units. 4.When there are two sample sizes (n 1 and n 2 ), we often assume that n 1 = n 2.

Cancer example Let W = 0.1. Want 95% CI for difference between means with width less than W. Suppose s 2 new = s 2 std = 6 (conservative guess) W > 2z  /2 sqrt((s 2 new /n 1 )+(s 2 std /n 2 )) 0.1 > 2(1.96)sqrt(6/n + 6/n) 0.1 > 3.92sqrt(12/n) 0.01 > ( )12/n n > (each group…) Book’s B = our W/2

Hypothesis testing and p-values (Chapter 9) We used confidence intervals in two ways: 1.To determine an interval of plausible values for the quantity that we estimate. Level of plausibility is determined by 1- . 90% (  =0.1) is less conservative than 95% (  =0.05) is less conservative than 99% (  =0.01)... 2.To see if a certain value is plausible in light of the data: If that value was not in the interval, it is not plausible (at certain level of confidence). Zero is a common certain value to test, but not the only one. Hypothesis tests address the second use directly

Example: Dietary Folate Data from the Framingham Heart Study n = 333 Elderly Men Mean = x = Std Dev = s = Can we conclude that the mean is greater than 300 at 5% significance? (same as 95% confidence)

Five Components of the Hypothesis test: 1. Null Hypothesis = “What we want to disprove” = “H 0 ” = “H not” = Mean dietary folate in the population represented by these data is <= 300. =  <= Alternative Hypothesis = “What we want to prove” = “H A ” = Mean dietary folate in the population represented by these data is > 300. =  > 300

3. Test Statistic To test about a mean with a large sample test, the statistic is z = (x –  )/(s/sqrt(n)) (i.e. How many standard deviations (of X) away from the hypothesized mean is the observed x?) 4. Significance Level of Test, Rejection Region, and P-value 5. Conclusion Reject H 0 and conclude H A if test stat is in rejection region. Otherwise, “fail to reject” (not same as concluding H 0 – can only cite a “lack of evidence” (think “innocent until proven guilty”) (Equivalently, reject H 0 if p-value is less than .) Next page

Significance Level:  =1% or 5% or 10%... (smaller is more conservative) (Significance = 1-Confidence) Rejection Region: –Reject if test statistic in rejection region. –Rejection region is set by: Assume H 0 is true “at the boundary”. Rejection region is set so that the probability of seeing the observed test statistic or something further from the null hypothesis is less than or equal to  P-value –Assume H 0 is true “at the boundary”. –P-value is the probability of seeing the observed test statistic or something further from the null hypothesis. –= “observed level of significance” Note that you reject if the p-value is less than . (Small p-values mean “more observed significance”)

Example: H 0 :  300 z (x-  )/(s/sqrt(n)) = (336.4 – 300)/(193.4/sqrt(333)) = 3.43 Significance level = 0.05 When H 0 is true, Z~N(0,1). As a result, the cutoff is z 0.05 = (Pr(Z>1.645) = 0.05.) P-value = Pr(Z>3.43 when true mean is 300) = Reject. Mean is greater than 300. Would you reject at significance level ?

Picture Test Statisistic Density Rejection region Distribution of Z = (X – 300)/(193.4/sqrt(333)) when true mean is 300. Area to right of =0.05 = sig level 3.43 Area to right of 3.43 = = p-value Test statistic Observed Test Statistic

One Sided versus Two Sided Tests Previous test was “one sided” since we’d only reject if the test statistic is far enough to “one side” (ie. If z > z 0.05 ) Two sided tests are more common (my opinion): H 0 :  =0, H A :  does not equal 0

Two Sided Tests (cntd) Test Statistic (large sample test of mean) z = (x –  )/(s/sqrt(n)) Rejection Region: reject H 0 at signficance level  if |z|>z  /2 i.e. if z>z  /2 or z<-z  /2 Note that this “doubles” p-values. See next example.

Example: H 0 :  =300, H A :  doesn’t equal 300 z=(x-  )/(s/sqrt(n)) = (336.4 – 300)/(193.4/sqrt(333)) = 3.43 Significance level = 0.05 When H 0 is true, Z~N(0,1). As a result, the cutoff is z =1.96. (Pr(|Z|>1.96)=2*Pr(Z>1.96)=0.05 P-value = Pr(|Z|>3.43 when true mean is 300) = Pr(Z>3.43) + Pr(Z<-3.43) = 2(0.0003)= Reject. Mean is not equal to 300. Would you reject at significance level ?

Picture 1.96 Test Statisistic Density Rejection region Distribution of Z = (X – 300)/(193.4/sqrt(333)) when true mean is Area to right of 3.43 = Test statistic Area to left of = Rejection region Sig level = area to right of area to the left of = 0.05=  Pvalue=0.0006=Pr(|Z|>3.43)

Power and Type 1 and Type 2 Errors Truth H 0 True H A True Action Fail to Reject H 0 Reject H 0 correct Type 1 error Type 2 error Significance level =  =Pr( Making type 1 error ) Power = 1–Pr( Making type 2 error )

Assuming H 0 is true, what’s the probability of making a type I error? H 0 is true means true mean is  0. This means that the test statistic has a N(0,1) distribution. Type I error means reject which means |test statistic| is greater than z  /2. This has probability .