1. Confidence intervals – the what, why and when
Stats Club 9
Marnie Brennan

2. References
Petrie and Sabin - Medical Statistics at a Glance: Chapter 11 Good
Petrie and Watson - Statistics for Veterinary and Animal Science: Chapter 6 Good
Dohoo, Martin and Stryhn – Veterinary Epidemiologic Research: Chapter 6
Other references – see the end of the presentation

3. What do you know about confidence intervals?

4. Follows on from hypothesis testing and p-values
Determining if comparisons between groups are statistically significantly different
Use a hypothesis test, and resulting p-values, to determine if sample groups are ‘related’ or not
But what about the ‘certainty’ around the results we get?
How certain are we that our sample results relate to our population of interest?
There are other things you can measure to help decide as well as p-values

5. Confidence intervals (CIs)
‘A range of values for a variable of interest constructed so that this range has a specified probability of including the true value of the variable’
‘ Shows the range within which the true treatment effect is likely to lie’
Davies and Crombie (2009)
Specified probability = confidence level
Range of values = confidence interval
End points of the confidence interval = confidence limits

6. Calculate them for any estimate
The way you calculate a confidence interval depends on what you are calculating:
For the mean
Normally distributed variables, or non-normally distributed variables
For the proportion
E.g. Binomial distribution
Kappa measurement of agreement
Correlations, regressions
Odds ratios, relative risks etc.

7. So a bit like a p-value?
Similar in that you can calculate a p-value or a CI for any estimate you are interested in
But p-values just give you information as to whether the estimate is different
Not the certainty behind this
Some may say they are preferable to p-values
Identify the range of possible effect sizes compatible with the data

8. An example of how to calculate them
Generally it is an equation incorporating the standard error (but exact calculation will be estimate specific)
E.g. For data that is normally distributed (and you want to work out the CI for the mean)
Lower limit = Sample mean x SEM
Upper limit = Sample mean x SEM
E.g. if mean = 416 and SEM = 5.5, CI would be:
416 – 1.96 (5.5); (5.5)
405.2; 426.8
Would write this as ‘The mean was 416 (CI – 426.8)’
Conventional to use the 95% level
If we were to repeatedly re-run the same study and create a CI each time, 95% of the CIs would contain the true parameter value
Can use others (90%, 99%) but this depends on what confidence you are looking for in your results

9. Petrie and Watson 2006

10. Why we use them
Indicates the precision of the estimate we are interested in
Helps us to decide whether something is biological importance versus statistical significance
Can be used to test a null hypothesis about a parameter of interest
Extrapolating from our sample to our population
See previous Stats Club session

11. Hypothesis testing

12. How to interpret CIs Two main things to look at: Final important point
The width of the CI
If the width of the CI is really wide, indicates great uncertainty about where the point estimate sits
Less confident about the results you’ve got
If the CI crosses the point of ‘no difference’
E.g. If we are doing a case-control study and we are using odds ratios, the point of no difference would be 1
If the CI includes 1, it means we cannot tell what affect the variable is having
Final important point
Is the result relevant?
‘Biologically important’ versus ‘Statistically important’

13. Factors affecting CIs Sample size
Smaller the sample, the wider the intervals
Standard deviation (how variable the values are from the mean)
More variability, the wider the intervals
Selected confidence level (e.g. 95%, 90%)
99% interval is wider than a 95% interval or a 90% interval
Must look at the sample biases when interpreting results
Even with large sample sizes and narrow confidence intervals, the results can be misleading
Usually if the P value is not significant, there will be an issue with the CIs as well (and vice versa)

14. du Prel et al. 2009

15. Example 1 What can we determine from these results?
Buchholz et al. (2011) German outbreak of E.coli 0104:H4 associated with sprouts. The New England Journal of Medicine

16. Example 2 What can we determine from these results?
Westgarth et al. (2015) Factors associated with daily walking of dogs. BMC Veterinary Research

17. Example 3 What can we determine from these results?
Bucher et al. (2012) Do any spray or dip treatments, applied on broiler chicken carcasses or carcass parts, reduce Salmonella spp. Prevalence and/or concentration during primary processing? A systematic review –meta-analysis. Food Control

18. Calculating them Analysis specific – check your programme for details!
How to do them in Microsoft Excel:

19. Calculating in SPSS

20. Calculating in SPSS

21. Summary
Confidence intervals can be useful to determine the uncertainty around a point estimate that you have calculated
Gives you more information than just the p-value
You should use/report both p-values and confidence intervals – and an understanding of where your sample has come from (bias)
Statistical versus biological significance

22. Next time Power calculations and sample size…… References:
Bucher, O., Rajić, A., Waddell, L.A., Greig, J. and McEwen, S.A., Do any spray or dip treatments, applied on broiler chicken carcasses or carcass parts, reduce Salmonella spp. prevalence and/or concentration during primary processing? A systematic review–meta-analysis. Food control, 27(2), pp
Davies, H.T. and Crombie, I.K., What are confidence intervals and p-values. London: Hayward Medical Communications.
du Prel, J.B., Hommel, G., Röhrig, B. and Blettner, M., Confidence interval or p-value?: part 4 of a series on evaluation of scientific publications. Deutsches Ärzteblatt International, 106(19), p.335.
Scottish Government (2017) Confidence intervals. Accessed at: Last accessed