How confidence intervals work The Right Questions about Statistics: How confidence intervals work Maths Learning Centre The University of Adelaide A confidence interval is designed to give a RANGE of possible answers for a “WHAT’S THE NUMBER?” question, using DATA from a sample. You calculate which numbers could have been likely to produce your data.

What is the median number of chapters in a novel? Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 What is the median number of chapters in a novel? Randomly choose 12 books

Which possible medians might be likely to produce this data? Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Which possible medians might be likely to produce this data? For each particular median, we could do a hypothesis test to answer the question (in this case it’s called the sign test) … 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 10? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 10. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 11? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 11. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 12? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 12. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 13? SIGN TEST P-value = 0.001 Under 0.05, so NO, the median is not 13. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 14? SIGN TEST P-value = 0.02 Under 0.05, so NO, the median is not 14. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 15? SIGN TEST P-value = 0.23 Over 0.05, so YES, the median could be 15. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 16? SIGN TEST P-value = 0.75 Over 0.05, so YES, the median could be 16. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 17? SIGN TEST P-value = 1.0 Over 0.05, so YES, the median could be 17. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 18? SIGN TEST P-value = 0.56 Over 0.05, so YES, the median could be 18. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 19? SIGN TEST P-value = 0.11 Over 0.05, so YES, the median could be 19. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 20? SIGN TEST P-value = 0.01 Under 0.05, so NO, the median is not 20. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 21? SIGN TEST P-value = 0.001 Under 0.05, so NO, the median is not 21. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 22? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 22. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 Is the median for all books 23? SIGN TEST P-value = 0.0005 Under 0.05, so NO, the median is not 23. ? 12 14 16 20 18 10 11 13 15 17 19 21 22 23

This is the (95%) confidence interval Book # Chapters 1 21 2 16 3 14 4 18 5 6 7 19 8 15 9 17 10 13 11 12 20 This is the (95%) confidence interval It shows all the medians that could be true based on hypothesis tests at the 5% level of significance. That is, all the medians that would give a p-value of over 0.05. You could say it’s the medians I am “happy to believe” based on my data. “95% confidence interval” 12 14 16 20 18 10 11 13 15 17 19 21 22 23

What is the mean weight in a novel? Another example... Book Weight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 10 314 11 217 12 379 What is the mean weight in a novel? Randomly choose 12 books

Which possible means might be likely to produce this data? Another example... Book Weight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 10 314 11 217 12 379 Which possible means might be likely to produce this data? For each particular mean, we could do a hypothesis test to answer the question (in this case it’s called the t-test) … 200 250 300 350 400

Book Weight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 10 314 11 217 12 379 Is the mean for all books 200? T-TEST P-value = 0.00007 Under 0.05, so NO, the mean is not 200. ? 200 250 300 350 400

Book Weight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 10 314 11 217 12 379 Is the mean for all books 201? T-TEST P-value = 0.00008 Under 0.05, so NO, the mean is not 201. ? 200 250 300 350 400

Book Weight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 10 314 11 217 12 379 Is the mean for all books 202? T-TEST P-value = 0.00009 Under 0.05, so NO, the mean is not 202. ? 200 250 300 350 400

Book Weight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 10 314 11 217 12 379 And so on... 200 250 300 350 400

This is the (95%) confidence interval Book Weight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 10 314 11 217 12 379 This is the (95%) confidence interval It shows all the means that could be true based on hypothesis tests at the 5% level of significance. That is, all the means that would give a p-value of over 0.05. You could say it’s the means I am “happy to believe” based on my data. “95% confidence interval” 200 250 300 350 400

You just have to figure out where the ends are. Book Weight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 10 314 11 217 12 379 BUT you don’t have to go through all this to find a confidence interval! You just have to figure out where the ends are. These ends would have given exactly 0.05 in the hypothesis test. So you find them by doing the hypothesis test calculation backwards... “95% confidence interval” 250 300 350 200 400

HYPOTHESIS TEST CONFIDENCE INTERVAL 95% CI is from 269 to 347 95% confidence interval is between these. Assume mean is 300 Use assumed mean and data to calculate a test statistic t = mean – 300 std. Error = 0.46 CI ends = mean ± t* × std. Error = 269 & 347 Use critical values to get two means. Use test statistic to get p-value. t* = ± 2.20 p = 0.65 Use p-value to get “critical” test statistics. Compare p-value to 0.05 to decide. More than 0.05, so the mean could be 300. Assume P-value is 0.05 CONFIDENCE INTERVAL 250 300 350 200 400 -2.0 -1.0 0.0 1.0 2.0

Let’s go over that again… Book Weight (g) 1 242 2 366 3 424 4 312 5 307 6 238 7 317 8 265 9 10 314 11 217 12 379 What is the mean weight in a novel? Randomly choose 12 books Choose a hypothesis test, and a significance level. T-Test, 0.05 Find the critical values of the test statistic. t* = ± 2.20 Use critical values and your data to calculate the two ends. CI ends = mean ± t* × std. Error = 269 & 347 State the answer. The 95% CI is from 269g to 347g. -2.0 -1.0 0.0 1.0 2.0

So this is how to find a confidence interval: Have a “what’s the number?” question. Collect data. Choose a matching hypothesis test. Work backwards to calculate two ends. The confidence interval is between these two values.

And this is what a confidence interval means: The numbers in the CI would have been retained with a matching hypothesis test asking, “Is this number equal to this value?” The values in the CI have a high chance of producing data like yours. The values in the CI appear to be consistent with your data. OUTSIDE NOT CONSISTENT WITH DATA   INSIDE APPEARS CONSISTENT WITH DATA