1. Evidence-Based Medicine Appendix 1: Confidence Intervals
Warren R. Dunn, MD, MPH
Assistant Professor, Orthopaedics & Rehabilitation,
Vanderbilt Sports Medicine
Assistant Professor, Medicine & Public Health,
Center for Health Services Research

2. Alternate title slide: p values suck, Confidence Intervals rule
Warren R. Dunn, MD, MPH
Assistant Professor, Orthopaedics & Rehabilitation,
Vanderbilt Sports Medicine
Assistant Professor, Medicine & Public Health,
Center for Health Services Research

3. Confidence Intervals
The only way you can really get a population value is to measure everyone in the population
Typically impossible, so we use our sample to calculate a range within which the population value is likely to fall
"Likely" is usually taken to be "95% of the time," and the range is called the 95% confidence interval
The confidence interval is the likely range of the true value
Note that there is only one true value, and the CI defines the range where it's most likely to be; the CI is NOT the variability of the true value or of any other value between subjects

4. The CI also gives you a measure of the precision (or uncertainty) of study results for making inferences about the population of all such patients
A strictly correct definition of a 95% CI is that 95% of such intervals will contain the true population value
Little is lost by the less pure interpretation of the CI as “a range of values within which we can be 95% sure that the true value lies”

5. What you want to know is the 95% CI
95% CI = the likely range of the true value

6. p value
The p value is not intuitive, and relays nothing to you to help you with your conclusions
p is short for probability
the probability of getting something more extreme than your result, when there is no effect in the population
First you assume there is no effect in the population
Then you see if the value you get for the effect in your sample is the sort of value you would expect for no effect in the population
If the value you get is unlikely for no effect, you conclude there is an effect, and you say the result is "statistically significant"

7. Why you don’t care about the p value
The p value is not an estimate of any quantity but rather a measure of the strength of evidence against the null hypothesis of “no effect”
The p value by itself tells us nothing about the size of a difference, nor the direction of that difference
p values on their own are thus not informative in papers or abstracts

8. Take home message p values suck and tell you flipin nothing
CI tell something useful:
the likely range of the true value

9. Multiple Estimates of Treatment Effect
CI should relate to the contrast of clinical interest
When comparing 2 groups the important CI is that for the difference between the groups

10. Multiple Estimates of Treatment Effect

11. Confidence Intervals
Unlike p values, CIs indicate both quantities of direct interest, such as treatment benefit, and also the strength of the evidence
They are thus of particular relevance to practitioners of evidence-based medicine (EBM)

12. Things to Remember
The word “interval” means a range of values and is thus singular. The two values that define the interval are called “confidence limits”
The CI estimates the “sampling variation”
The CI does not reflect additional uncertainty due to other causes; in particular CIs do not incorporate the impact of selective loss to follow-up, poor compliance with treatment, imprecise outcome measurements, lack of blinding, and so on
CIs thus always underestimate the total amount of uncertainty

13. Clinical Significance
It is also important not to equate statistical significance with clinical importance
I think this has been emphasized enough

14. Take home message p values suck and tell you flipin nothing
CI tell something useful:
the likely range of the true value
which is what we so desperately want to know