1. POSC 202A: Lecture 9 Lecture: statistical significance.

2. Statistical Significance The fundamental question of statistical significance:

3. Statistical Significance The fundamental Question: How likely is the result we observe to be the product of chance? This question drives all of the tests we perform by allowing us to differentiate the systematic from the stochastic.

4. Statistical Significance Confidence Interval- An interval calculated from sample data that is guaranteed to capture the true population parameter. It tells us how large of an interval we need to create in order to capture the true population value in some fixed percentage of the intervals we draw.

6. Statistical Significance Think of it this way: We can draw a sample in order to estimate some statistic. If we repeat over and over we start to create a sampling distribution. To create a 95% CI we need to see how large of an interval around the statistic we need to create to satisfy the condition that in 95% of the samples drawn the interval we draw contains the true population parameter (value). Page 386 has a nice graph of this.

7. Statistical Significance Confidence Interval: An example: a 95% confidence interval is the range needed to capture the true population value in 95% of the intervals we draw from a population.

8. Statistical Significance Confidence Interval: The interval thus captures the variability inherent in using samples to draw inferences about a population. To estimate, we need an estimate of the population mean and the standard deviation. Reported in the form: Estimate Margin of Error Generally, the interval is: mean – (z*sd); mean+(z*sd)

9. Statistical Significance Confidence Interval: To estimate, we need an estimate of the population mean and the standard deviation. How do we calculate this for a sampling distribution? This is for proportions: Mean=p

10. Refresher: Thumb’s Rule Recall that with a normal distribution: Apportionment of area about the mean is +/- 1sd= 68% +/- 2sd= 95% +/- 3sd= 99.7% So, a 95% CI corresponds to +/- 2sd= 95%

11. Statistical Significance Lets create an example using a sample from Dear Abby, in which 400 women responded of whom 60% would rather just cuddle than have sex with their husbands. What do we want to know?

12. Statistical Significance Is 60% beyond what we would expect to see due to chance alone? Is this likely to be just random variation? What do we want to know?

13. Statistical Significance Lets create an example using a sample from Dear Abby, in which 400 women responded of whom 60% would rather just cuddle than have sex with their husbands. Sample mean=p.6 =p, so: S.D.=

14. Confidence Interval Our confidence interval is thus: +/- 2sd= 95 CI Or.6 +/- 2(.0245)=.6-.049 and.6+.049

15. Confidence Interval.6 +/- 2(.0245)=.6-.049 and.6+.049 Round off to.05.60.65.55 95% CI

16. Confidence Interval 95% of all samples will capture the true population parameter in the range between.55-.65.60.65.55 95% CI

17. Confidence Interval 95% of all samples will capture the true population parameter in the range between.55-.65 From this we conclude that we are 95% confident that between 55% and 65% of women prefer cuddling to sex.

18. Confidence Interval What would we have expected if they answered randomly?

19. Exercise A study of graduate placement finds that examining 50 graduates, 20% of students earn jobs at universities that are not teaching intensive. Construct a confidence interval that allows us to assess whether this result is too small to attribute to chance.

20. Statistical Significance How do our estimates change with the size of the population? Recall we found The population average (mean) stays the same regardless of sample size. But what of the SD?

21. Sample Size and Confidence Intervals Where the sample statistic is.5

23. Confidence Interval A shortcut for approximating the 95% CI for a proportion: More accurate as you get closer to an even split (i.e. 50%)

24. Confidence Interval.

25. Statistical Significance Recall--The fundamental Question: How likely is the result we observe to be the product of chance? This question drives all of the tests we perform by allowing us to differentiate the systematic from the stochastic. Significance testing is all about comparisons.

26. Statistical Significance Significance testing is all about comparisons. Is what we observe close or far from what we expect ? Is what we observe so far from what we expect that we cannot attribute what we see to chance alone.

27. Statistical Significance An example: Imagine we took a valid random sample of women’s preferences and got the same result as Dear Abby’s survey of women’s cuddling preferences (60% preferred cuddling to sex; 400 women responded). How would we conduct a significance test? We need to identify the appropriate comparison.

28. Statistical Significance How would we conduct a significance test? What if women randomly answered “snuggle” or “sex”?

29. Statistical Significance What we would expect if women just answered randomly? If so, we might expect 50% of respondents to prefer snuggling.

30. Statistical Significance What we would expect if women just answered randomly? If so, we might expect 50% of respondents to prefer snuggling. Then we need to know if what we observed is (60%) too large of a result to be attributable to chance alone. How can we determine this?

31. Statistical Significance How can we determine this? One way is to estimate a 95% confidence interval around our sample mean (60%) and see if it contains the result we would expect sue to chance alone (50%).

32. Confidence Interval Recall, we created the following interval earlier. We can simply look to see if.50 is within the interval constructed. Since it is not, then our sample result is statistically significantly different than chance..60.65.55 95% CI.50

33. Statistical Significance How can we determine this? A second way is to conduct a significance test by solving for areas under the normal curve. Recall we know how to find the likelihood that some event occurs using the formula:. This formula asks whether what we see is too far from chance to be attributed to chance alone.

34. Statistical Significance We simply calculate the number o standard units what we observe is from random chance. Then use the Z table to obtain the likelihood that we see a sample as large as 60% if the true value is 50%.

35. Statistical Significance Our Z table only goes to 3.4! Less than 1 time in 10,000 would we see a sample mean of 60% if the process were driven by chance alone.

36. Confidence Interval We could illustrate this process on the normal curve as well. The Q: How likely is it that we would see a result above 60%?.60.55.50 Z>.9998 Z<.0001