1. Chapter 15 Inference in Practice PSLS/2eChapter 151

2. Effective use of inferential methods requires more than knowing the facts. It requires understanding the reasoning behind the process.

3. z Procedures If we know standard deviation  before data collected, the confidence interval for  is: To test H 0 :  =  0, we use this statistic: These are called z procedures because they rely on critical values from the Z~N(0,1) density function PSLS/2e Chapter 153

4. Conditions for Z Procedures SRS 1.Data must resemble an SRS from the population Ask: “ where did the data come from? ” – Bad samples – Bad samples (see next slide) invalidate methods Normal Central Limit Theorem if 2.Population must be Normal …BUT…a fact known as the Central Limit Theorem tells us the sampling distribution of x-bar will be Normal even if the population is not Normal if the sample is “ large enough ” – In practice, z procedures are robust in large samples must be known 3.Population standard deviation  must be known before data are collected … Chapter 17 will introduce procedures that can be used when  is not known PSLS/2eChapter 154

5. Bad Samples Examples of Bad Samples Convenience samples - selecting members of the population that are easiest to reach – Example: sample of mall shoppers  teenagers and retired people will be over-represented Voluntary response samples - people who choose themselves by responding to a broad appeal – Example: online polls are useless scientifically (people who take the trouble to respond are not representative of the larger population) Under-coverage - some groups in the population are left out or underrepresented – Example: using telephone listing to select subjects (not everyone has a listed phone number If the data do not come from an SRS or a randomized experiment  conclusions are open to challenge. Always ask where the data came from. Always ask where the data came from. PSLS/2eChapter 155

6. Inference about µ610/20/2015Inference about µ6 Normality Assumption and the Central Limit Theorem Normality can be assumed when n is large because of the Central Limit Theorem Sample size less than 15: “Normality” can be assumed if data are symmetric, have a single peak and no outliers. If data are highly skewed, avoid z [and t] procedures. Sample size at least 15: Normality can be assumed unless data are strongly skewed or have outliers. Large samples n > 30 - 60: Normality can be assumed even for skewed distributions when the sample is large (n ≥ ~40)

8. Inference about µ810/20/2015Inference about µ8 Can Normality be assumed? Moderately sized dataset (n = 20) w/strong skew. Normality cannot be assumed Do NOT use z [or t] procedures

9. Inference about µ910/20/2015Inference about µ9 Can Normality be assumed? Extremely large data set (n ≈ 1000) The data has a strong positive skew But since sample is large, central limit theorem is strong and we can assume Normality. Do use z [or t] procedures.

10. Inference about µ1010/20/2015Inference about µ10 Can Normality be assumed? The distribution has no clear departures from Normality. Therefore, we can trust z [and t] procedures. n is moderate

11. Additional Caution: GIGO PSLS/2eChapter 1511 Garbage In, Garbage Out A study is only as good as the quality of the data CIs and P-values are valueless when the INFORMATION is of POOR QUALITY Example: Self-reported data can be inaccurate and biased

12. Additional Caution: P-values P-values (significance tests) are often misunderstood Even large differences can fail to be significant if the sample is small Statistical significance does NOT tell us whether a finding is important  statistical significance is NOT the same as practical significance P values are NOT the probability that H 0 is true; it is the probability the data came from a distribution in which H 0 is correct Failure to reject H 0 is NOT the same as accepting H0 Although  = 0.05 is a common cut-off, there is NO set border between “ significant ” and “ insignificant ” results, surely God loves P =.06 nearly as much as P =.05. PSLS/2eChapter 1512

13. Margin of Error (m) When estimating µ with C confidence, the margin of error: The margin of error = half the CI length  indicates the precision of the estimate z* and σ are immutable at a given level of confidence To increase precision, increase the sample size: ↑ n → ↓ m → ↑ precision PSLS/2eChapter 1513

14. Choosing a Sample Size PSLS/2eChapter 1514 To determine the sample size requirement to achieve margin of error m when estimating µ use:

15. Example: National Assessment of Educational Progress (NAEP) Math Scores PSLS/2eChapter 1515 NEAP math scores predict success following High School Suppose that we want to estimate a population mean NAEP scores with 90% confidence and want the margin of error to be no more than ±5 points We know the NEAP math scores have  = 60 What sample size will be required to enable us to create such an interval?

16. Example PSLS/2eChapter 1516 NAEP Quantitative Scores If you round down your margin of error will be bigger If you round up your margin of error will be smaller (a good thing).  Always round UP to next integer. Study 400 individuals so m no greater than 5. = 399.67

17. Example: Decrease margin of error m PSLS/2eChapter 1517 Now suppose we want to estimate the population mean NAEP scores with 90% confidence and want the margin of error not to exceed 3 points (recall that  = 60). What sample size will be required to enable us to create such an interval?

18. Case Study PSLS/2eChapter 15 18 NAEP Quantitative Scores Therefore resolve to study 1083 (so that the margin of error does not exceed 3 points. Note that lowering the margin of error to 3 points, required a much larger sample size!

19. The Relation between Confidence Level and CI length has already been covered PSLS/2eChapter 1519 90% Confidence Interval for µ CI length for 90% CI = 275 – 269 = 6 (margin of error = 3) CI length for confidence = 276 – 268 = 8 (margin of error = 4) The 95% CI is wider than the 90% CI. 95% Confidence Interval for µ ↑ confidence requires ↑ confidence interval length