1. The Right Questions about Statistics: How confidence intervals work Maths Learning Centre The University of Adelaide A confidence interval is designed to give a RANGE of possible answers for a “WHAT’S THE NUMBER?” question, using DATA from a sample. You calculate which numbers could have been likely to produce your data.

2. What is the median number of chapters in a novel? Randomly choose 12 books Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

3. Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20 Which possible medians might be likely to produce this data? For each particular median, we could do a hypothesis test to answer the question (in this case it’s called the sign test) … 1214162018101113151719212223

4. 1214162018101113151719212223 ? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 10. Is the median for all books 10? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

5. 1214162018101113151719212223 ? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 11. Is the median for all books 11? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

6. 1214162018101113151719212223 ? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 12. Is the median for all books 12? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

7. 1214162018101113151719212223 ? SIGN TEST P-value = 0.001 Under 0.05, so NO, the median is not 13. Is the median for all books 13? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

8. 1214162018101113151719212223 ? SIGN TEST P-value = 0.02 Under 0.05, so NO, the median is not 14. Is the median for all books 14? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

9. 1214162018101113151719212223 ? SIGN TEST P-value = 0.23 Over 0.05, so YES, the median could be 15. Is the median for all books 15? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

10. 1214162018101113151719212223 ? SIGN TEST P-value = 0.75 Over 0.05, so YES, the median could be 16. Is the median for all books 16? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

11. 1214162018101113151719212223 ? SIGN TEST P-value = 1.0 Over 0.05, so YES, the median could be 17. Is the median for all books 17? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

12. 1214162018101113151719212223 ? SIGN TEST P-value = 0.56 Over 0.05, so YES, the median could be 18. Is the median for all books 18? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

13. 1214162018101113151719212223 ? SIGN TEST P-value = 0.11 Over 0.05, so YES, the median could be 19. Is the median for all books 19? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

14. 1214162018101113151719212223 ? SIGN TEST P-value = 0.01 Under 0.05, so NO, the median is not 20. Is the median for all books 20? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

15. 1214162018101113151719212223 ? SIGN TEST P-value = 0.001 Under 0.05, so NO, the median is not 21. Is the median for all books 21? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

16. 1214162018101113151719212223 ? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 22. Is the median for all books 22? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

17. 1214162018101113151719212223 ? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 23. Is the median for all books 23? Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20

18. 1214162018101113151719212223 That is, all the medians that would give a p-value of over 0.05. “95% confidence interval” It shows all the medians that could be true based on hypothesis tests at the 5% level of significance. Book# Chapters 1 21 2 16 3 14 4 18 5 14 6 16 7 19 8 15 9 17 10 13 11 19 12 20 This is the (95%) confidence interval You could say it’s the medians I am “happy to believe” based on my data.

19. BookWeight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 317 10 314 11 217 12 379 What is the mean weight in a novel? Randomly choose 12 books Another example...

20. BookWeight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 317 10 314 11 217 12 379 Which possible means might be likely to produce this data? For each particular mean, we could do a hypothesis test to answer the question (in this case it’s called the t-test) … Another example... 250300350200400

21. 250300350200400 BookWeight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 317 10 314 11 217 12 379 ? T-TEST P-value = 0.00007 Under 0.05, so NO, the mean is not 200. Is the mean for all books 200?

22. 250300350200400 BookWeight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 317 10 314 11 217 12 379 ? T-TEST P-value = 0.00008 Under 0.05, so NO, the mean is not 201. Is the mean for all books 201?

23. 250300350200400 BookWeight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 317 10 314 11 217 12 379 ? T-TEST P-value = 0.00009 Under 0.05, so NO, the mean is not 202. Is the mean for all books 202?

24. 250300350200400 BookWeight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 317 10 314 11 217 12 379 And so on...

25. 250300350200400 BookWeight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 317 10 314 11 217 12 379 “95% confidence interval” That is, all the means that would give a p-value of over 0.05. It shows all the means that could be true based on hypothesis tests at the 5% level of significance. This is the (95%) confidence interval You could say it’s the means I am “happy to believe” based on my data.

26. 250300350200400 BookWeight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 317 10 314 11 217 12 379 “95% confidence interval” These ends would have given exactly 0.05 in the hypothesis test. BUT you don’t have to go through all this to find a confidence interval! You just have to figure out where the ends are. So you find them by doing the hypothesis test calculation backwards...

27. CI ends = mean ± t* × std. Error = 269 & 347 t = mean – 300 std. Error = 0.46 t* = ± 2.20 More than 0.05, so the mean could be 300. p = 0.65 HYPOTHESIS TEST Assume mean is 300 Use assumed mean and data to calculate a test statistic Use test statistic to get p-value. CONFIDENCE INTERVAL Assume P-value is 0.05 Use p-value to get “critical” test statistics. Use critical values to get two means. Compare p-value to 0.05 to decide. 95% confidence interval is between these. -2.00.01.02.0 95% CI is from 269 to 347 250300350200400

28. BookWeight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 317 10 314 11 217 12 379 Let’s go over that again… What is the mean weight in a novel? Randomly choose 12 books Choose a hypothesis test, and a significance level. T-Test, 0.05 Use critical values and your data to calculate the two ends. CI ends = mean ± t* × std. Error = 269 & 347 State the answer. The 95% CI is from 269g to 347g. Find the critical values of the test statistic. t* = ± 2.20 -2.00.01.02.0

29. So this is how to find a confidence interval: Have a “what’s the number?” question. Collect data. Choose a matching hypothesis test. Work backwards to calculate two ends. The confidence interval is between these two values.

30. And this is what a confidence interval means: The values in the CI would be retained with a matching hypothesis test. The values in the CI are those you are “happy to believe” based on your data. The values in the CI have a high chance of producing data like yours.