Stat 512 – Lecture 12 Two sample comparisons (Ch. 7) Experiments revisited

Announcements No HW this week!  PP 10 due Thursday, preview next example  Readings (no starred sections) Final Exam schedule Thank you for feedback Midterms returned at end of class  Grades updated on blackboard  Exam solutions online Probably a sub on Thursday

What’s left to do? Practice the concepts  p-value, confidence  scope of conclusions New techniques for new situations  Minitab number crunching  Type and number of variables Keep thinking about whether a technique will apply to your project

Example 1: Newspaper Credibility Nationwide sample of 1,002 adults interviewed via telephone (under direction of Princeton Survey Research Assoc), 2002 Comparison to 1998 nationwide sample Variable of interest: Would you rate your local daily newspaper as largely believable?  Can consider 1998 vs as second variable

Example 1: Newspaper Credibility Two-way table Segmented bar graph Some evidence that believability of local paper was higher in 1998, but not a very large difference (.670 vs..634)

Example 1: Newspaper Credibility Want to compare a categorical variable (believability rating) between two populations Let     = difference in population proportion who rate daily paper as largely believable  H 0 :     = 0 (no change in proportion)  H a :     > 0 (more believable in 1998 Assume H 0 is true, could these sample proportions be coming from the same population proportion?

Example 1: Newspaper Credibility Estimate: difference in sample proportions Observed: =.036 Could a difference this large happen by chance?  If we repeatedly took two samples from the same population, how often would we observe a difference in the sample proportions at least this large?

Sampling Distribution of Mean:     Standard deviation  Variances add…  When null hypothesis is true:

Shape?  Normal if sample size is large…  At least 5 successes and 5 failures in each group Sampling Distribution of n 1  1 > 5, n 1 (1-  1 ) >5 and n 2  2 > 5, n 2 (1-  2 ) > 5

Example 1: Newspaper Credibility Large sample sizes √ Independent random samples√

Example 1: Newspaper Credibility  Difference in sample proportions   z values 

Summary The difference in the proportions is statistically significant (p-value =.051 ≈.05)  Difference in sample proportions observed is larger than we would expect from the random sampling process alone We believe a smaller proportion of the population would rate their daily newspaper as largely believable in 2002 than in 1998 We are 90% confident that the population proportions differ by.00 to.072 so this may not be considered a practically significant decrease. Cause and Effect? No Generalizable? Yes

Example 2: Letrozole and Cancer New England Journal of Medicine, 2003 Study of over 5,000 women, randomly assigned to letrozole or placebo  Do we have a random sample of letrozole users and a random sample of placebo users? Willing to generalize to all women with breast cancer?  Do we have random assignment? Hypotheses will be about “treatment effect” instead of about difference in population proportions

Example 2: Letrozole and Cancer Want to compare a categorical response variable (whether survived) between two explanatory variable groups Consider underlying treatment effect (increase in survival rate) from using letrozole  H 0 : true treatment effect = 0 (letrozole has no real effect on survival)  H a : true treatment effect > 0 (letrozole increases survival rate) Assume H 0 is true, could the difference between these two group proportions occur from the randomization process alone?

Example 2: Letrozole and Cancer LetrozolePlaceboTotal Disease-free survival Not disease-free survival Total Data are in direction conjectured, have a higher rate of survival in the letrozole group How often would we find this many of the successes in the letrozole group by chance (from the randomization process) alone?

Example 2: Letrozole and Cancer Using normal distribution Large sample sizes √ Randomization √

Example 2: Letrozole and Cancer Conclusions: Not only was p-value small, the group proportions were more than 7 standard deviations apart. We are 95% confident that the percentage of women surviving will be 4.3% to 7.7% higher if letrozole is used. Cause and effect? Yes Generalizable? Risky

Example 3: Body Temperatures Minitab commands depend on which format typed data in Statistical Analysis  Observational study, treating as independent random samples  Samples show slight tendency for higher body temperatures among women

Example 3: Body Temperatures Perhaps the population means are equal, and these sample means differ just based on random sampling variability H 0 :      H a :     ≠  (“differs”) Technical conditions  Normal populations (works ok if n 1, n 2 >20)  Large populations  Independent random samples

Example 3: Body Temperatures

Test statistic Example 3: Body Temperatures Result is statistically significant at 5% level but not 1% level. Moderate evidence that these sample means are further apart than we would expect from random sampling variability alone if the population means were equal. Conclude that the mean body temperature differs.

Example 4: Sleep Deprivation Case 2: Randomized Experiment When samples sizes are large or each group distribution is normal, the randomization distribution is well approximated by the t distribution  Pooled t test?

Example 4: Sleep Deprivation Case 2: Randomized Experiment Validity?

Example 4 Conclusions 1. Statistically significant 2. Cause and effect conclusion valid 3. Generalizing to larger population? Is it possible that we are making the wrong decision?  Yes, type I error…

Summary Type of study  Do you have (independent) random samples from two populations? OR Do you have a randomized experiment?  Same calculations, different conclusions Are the sample sizes large for you to use normal/t procedures?  With small sample sizes, use Fisher’s Exact Test (two-way table simulation) or randomization tests from before  With larger samples, get test statistic and confidence interval conveniently

Comments on Exam Average score around 81  Still a bit to do  Focus on big issues: Types of study and consequences on conclusions (e.g., cause and effect), validity of methods, interpretation of results (What is a p-value? What is a confidence interval?)  Solutions on line with commentary Estimated current course grade  Average around.85

For Thursday Preview Example 1, Lecture 13 PP 10 online Read Ch. 7, chi-square procedures