2. UWHC Scholarly Forum April 17, 2013 Ismor Fischer, Ph.D. UW Dept of Statistics, UW Dept of Biostatistics and Medical Informatics ifischer@wisc.edu

3. UWHC Scholarly Forum April 17, 2013 Ismor Fischer, Ph.D. UW Dept of Statistics, UW Dept of Biostatistics and Medical Informatics ifischer@wisc.edu All slides posted at http://www.stat.wisc.edu/~ifischer/UWHChttp://www.stat.wisc.edu/~ifischer/UWHC

4. Click on image for full.pdf article Links in article to access datasets

5. POPULATION Study Question: Has “Mean (i.e., average) Age at First Birth” of women in the U.S. changed since 2010 (25.4 yrs old)? Present Day: Assume “Mean Age at First Birth” follows a normal distribution (i.e., “bell curve”) in the population. “Statistical Inference”

6. ~ The Normal Distribution ~  symmetric about its mean  unimodal (i.e., one peak), with left and right “tails”  models many (but not all) naturally-occurring systems  useful mathematical properties… “population mean” “population standard deviation” 

7. ~ The Normal Distribution ~ “population mean” “population standard deviation”  symmetric about its mean  unimodal (i.e., one peak), with left and right “tails”  models many (but not all) naturally-occurring systems   useful mathematical properties…

8. ~ The Normal Distribution ~ “population standard deviation”  symmetric about its mean  unimodal (i.e., one peak), with left and right “tails”  models many (but not all) naturally-occurring systems Approximately 95% of the population values are contained between  – 2 σ and  + 2 σ. 95% is called the confidence level. 5% is called the significance level. 95% 2.5% ≈ 2 σ “population mean”   useful mathematical properties…

9. POPULATION “Null Hypothesis” via… “Hypothesis Testing” Study Question: Has “Mean (i.e., average) Age at First Birth” of women in the U.S. changed since 2010 (25.4 yrs old)? H 0 : pop mean age  = 25.4 (i.e., no change since 2010) “Statistical Inference” Present Day: Assume “Mean Age at First Birth” follows a normal distribution (i.e., “bell curve”) in the population.  cannot be found with 100% certainty, but can be estimated with high confidence (e.g., 95%).

10. Is the difference STATISTICALLY SIGNIFICANT, at the 5% level? POPULATION “Null Hypothesis” via… “Hypothesis Testing” Study Question: Has “Mean (i.e., average) Age at First Birth” of women in the U.S. changed since 2010 (25.4 yrs old)? FORM ULA x1x1 x4x4 x3x3 x2x2 x5x5 x 400 … etc… H 0 : pop mean age  = 25.4 (i.e., no change since 2010) sample mean age Do the data tend to support or refute the null hypothesis? “Statistical Inference” T-test Present Day: Assume “Mean Age at First Birth” follows a normal distribution (i.e., “bell curve”) in the population.

11. Samples, size n ~ The Normal Distribution ~ … etc… CENTRAL LIMIT THEOREM

12. Approximately 95% of the sample mean values are contained between and 95% 2.5% ≈ 2 σ ~ The Normal Distribution ~ Approximately 95% of the population values are contained between  – 2 σ and  + 2 σ. Approximately 95% of the intervals from to contain , and approx 5% do not.

13. Approximately 95% of the intervals from to contain , and approx 5% do not. 95% margin of error

14. PROBLEM! σ is unknown the vast majority of the time!PROBLEM! Present Day: Assume “Mean Age at First Birth” follows a normal distribution (i.e., “bell curve”) in the population. “Null Hypothesis” via… “Hypothesis Testing” H 0 : pop mean age  = 25.4 (i.e., no change since 2010) sample mean = 25.6 “Statistical Inference” POPULATION Study Question: Has “Mean (i.e., average) Age at First Birth” of women in the U.S. changed since 2010 (25.4 yrs old)? FORM ULA SAMPLE n = 400 ages x3x3 x2x2 x5x5 x 400 … etc… x1x1 x4x4 95% margin of error Approximately 95% of the intervals from to contain , and approx 5% do not.

15. Present Day: Assume “Mean Age at First Birth” follows a normal distribution (i.e., “bell curve”) in the population. POPULATION “Null Hypothesis” via… “Hypothesis Testing” Study Question: Has “Mean (i.e., average) Age at First Birth” of women in the U.S. changed since 2010 (25.4 yrs old)? FORM ULA SAMPLE n = 400 ages H 0 : pop mean age  = 25.4 (i.e., no change since 2010) sample mean = 25.6 “Statistical Inference” x3x3 x2x2 x5x5 x 400 … etc… x1x1 x4x4 sample standard deviation sample variance 95% margin of error = modified average of the squared deviations from the mean

16. Present Day: Assume “Mean Age at First Birth” follows a normal distribution (i.e., “bell curve”) in the population. POPULATION “Null Hypothesis” via… “Hypothesis Testing” Study Question: Has “Mean (i.e., average) Age at First Birth” of women in the U.S. changed since 2010 (25.4 yrs old)? FORM ULA SAMPLE n = 400 ages sample mean = 25.6 “Statistical Inference” x3x3 x2x2 x5x5 x 400 … etc… x1x1 x4x4 sample variance = 1.6 sample standard deviation = 0.16 95% margin of error H 0 : pop mean age  = 25.4 (i.e., no change since 2010)

17. Approximately 95% of the intervals from to contain , and approx 5% do not.

18. 25.7625.44 BASED ON OUR SAMPLE DATA, the true value of μ today is between 25.44 and 25.76 years, with 95% “confidence” (…akin to “probability”). = 0.16 95% margin of error = 0.16

19. 25.7625.44 BASED ON OUR SAMPLE DATA, the true value of μ today is between 25.44 and 25.76 years, with 95% “confidence” (…akin to “probability”). 95% CONFIDENCE INTERVAL FOR µ “P-VALUE” of our sample Very informally, the p-value of a sample is the probability (hence a number between 0 and 1) that it “agrees” with the null hypothesis. Hence a very small p-value indicates strong evidence against the null hypothesis. The smaller the p-value, the stronger the evidence, and the more “statistically significant” the finding. Two main ways to conduct a formal hypothesis test:

20. Very informally, the p-value of a sample is the probability (hence a number between 0 and 1) that it “agrees” with the null hypothesis. Hence a very small p-value indicates strong evidence against the null hypothesis. The smaller the p-value, the stronger the evidence, and the more “statistically significant” the finding. 25.7625.44 BASED ON OUR SAMPLE DATA, the true value of μ today is between 25.44 and 25.76 years, with 95% “confidence” (…akin to “probability”). 95% CONFIDENCE INTERVAL FOR µ Two main ways to conduct a formal hypothesis test: “P-VALUE” of our sample IF H 0 is true, then we would expect a random sample mean that is at least 0.2 years away from  = 25.4 (as ours was), to occur with probability 1.24%. FORMAL CONCLUSIONS:  The 95% confidence interval corresponding to our sample mean does not contain the “null value” of the population mean, μ = 25.4 years.  The p-value of our sample,.0124, is less than the predetermined α =.05 significance level. Based on our sample data, we may (moderately) reject the null hypothesis H 0 : μ = 25.4 in favor of the two-sided alternative hypothesis H A : μ ≠ 25.4, at the α =.05 significance level. INTERPRETATION: According to the results of this study, there exists a statistically significant difference between the mean ages at first birth in 2010 (25.4 years old) and today, at the 5% significance level. Moreover, the evidence from the sample data would suggest that the population mean age today is significantly older than in 2010, rather than significantly younger. FORMAL CONCLUSIONS:  The 95% confidence interval corresponding to our sample mean does not contain the “null value” of the population mean, μ = 25.4 years.  The p-value of our sample,.0124, is less than the predetermined α =.05 significance level. Based on our sample data, we may (moderately) reject the null hypothesis H 0 : μ = 25.4 in favor of the two-sided alternative hypothesis H A : μ ≠ 25.4, at the α =.05 significance level. INTERPRETATION: According to the results of this study, there exists a statistically significant difference between the mean ages at first birth in 2010 (25.4 years old) and today, at the 5% significance level. Moreover, the evidence from the sample data would suggest that the population mean age today is significantly older than in 2010, rather than significantly younger.

21. POPULATION “Null Hypothesis” via… “Hypothesis Testing” Study Question: Has “Mean (i.e., average) Age at First Birth” of women in the U.S. changed since 2010 (25.4 yrs old)? FORM ULA x1x1 x4x4 x3x3 x2x2 x5x5 x 400 … etc… H 0 : pop mean age  = 25.4 (i.e., no change since 2010) sample mean age Do the data tend to support or refute the null hypothesis? Is the difference STATISTICALLY SIGNIFICANT, at the 5% level? “Statistical Inference” Present Day: Assume “Mean Age at First Birth” follows a normal distribution (i.e., “bell curve”) in the population. T-test Two loose ends

22. POPULATION “Null Hypothesis” via… “Hypothesis Testing” Study Question: Has “Mean (i.e., average) Age at First Birth” of women in the U.S. changed since 2010 (25.4 yrs old)? H 0 : pop mean age  = 25.4 (i.e., no change since 2010) “Statistical Inference” Present Day: Assume “Mean Age at First Birth” follows a normal distribution (i.e., “bell curve”) in the population. T-test The reasonableness of the normality assumption is empirically verifiable, and in fact formally testable from the sample data. If violated (e.g., skewed) or inconclusive (e.g., small sample size), then “distribution-free” nonparametric tests can be used instead of the T-test. Examples: Sign Test, Wilcoxon Signed Rank Test (= Mann-Whitney Test) Two loose ends Check?

23. POPULATION “Null Hypothesis” via… “Hypothesis Testing” Study Question: Has “Mean (i.e., average) Age at First Birth” of women in the U.S. changed since 2010 (25.4 yrs old)? x1x1 x4x4 x3x3 x2x2 x5x5 x 400 … etc… H 0 : pop mean age  = 25.4 (i.e., no change since 2010) “Statistical Inference” Present Day: Assume “Mean Age at First Birth” follows a normal distribution (i.e., “bell curve”) in the population. T-test Two loose ends HOWEVER Sample size n partially depends on the power of the test, i.e., the desired probability of correctly rejecting a false null hypothesis. HOWEVER……

24. Approximately 95% of the intervals from to contain , and approx 5% do not. Samples, size n ~ The Normal Distribution ~ Approximately 95% of the population values are contained between  – 2 σ and  + 2 σ. … etc… “population standard deviation” 95% 2.5% ≈ 2 σ Approximately 95% of the sample mean values are contained between and “population mean”

25. Approximately 95% of the intervals from to contain , and approx 5% do not. Samples, size n ~ The Normal Distribution ~ Approximately 95% of the population values are contained between  – 2 s and  + 2 s. … etc… “population standard deviation” 95% 2.5% ≈ 2 σ Approximately 95% of the sample mean values are contained between and “population mean” …IF n is large,  30 traditionally. But if n is small… … this “T-score" increases (from ≈ 2 to a max of 12.706 for a 95% confidence level) as n decreases  larger margin of error  less power to reject.

26. If n is large, T-score ≈ 2. If n is small, T-score > 2.