Evidence-Based Medicine Appendix 1: Confidence Intervals Warren R. Dunn, MD, MPH Assistant Professor, Orthopaedics & Rehabilitation, Vanderbilt Sports Medicine Assistant Professor, Medicine & Public Health, Center for Health Services Research

Alternate title slide: p values suck, Confidence Intervals rule Warren R. Dunn, MD, MPH Assistant Professor, Orthopaedics & Rehabilitation, Vanderbilt Sports Medicine Assistant Professor, Medicine & Public Health, Center for Health Services Research

Confidence Intervals The only way you can really get a population value is to measure everyone in the population Typically impossible, so we use our sample to calculate a range within which the population value is likely to fall "Likely" is usually taken to be "95% of the time," and the range is called the 95% confidence interval The confidence interval is the likely range of the true value Note that there is only one true value, and the CI defines the range where it's most likely to be; the CI is NOT the variability of the true value or of any other value between subjects

The CI also gives you a measure of the precision (or uncertainty) of study results for making inferences about the population of all such patients A strictly correct definition of a 95% CI is that 95% of such intervals will contain the true population value Little is lost by the less pure interpretation of the CI as “a range of values within which we can be 95% sure that the true value lies”

What you want to know is the 95% CI 95% CI = the likely range of the true value

p value The p value is not intuitive, and relays nothing to you to help you with your conclusions p is short for probability the probability of getting something more extreme than your result, when there is no effect in the population First you assume there is no effect in the population Then you see if the value you get for the effect in your sample is the sort of value you would expect for no effect in the population If the value you get is unlikely for no effect, you conclude there is an effect, and you say the result is "statistically significant"

Why you don’t care about the p value The p value is not an estimate of any quantity but rather a measure of the strength of evidence against the null hypothesis of “no effect” The p value by itself tells us nothing about the size of a difference, nor the direction of that difference p values on their own are thus not informative in papers or abstracts

Take home message p values suck and tell you flipin nothing CI tell something useful: the likely range of the true value

Multiple Estimates of Treatment Effect CI should relate to the contrast of clinical interest When comparing 2 groups the important CI is that for the difference between the groups

Multiple Estimates of Treatment Effect

Confidence Intervals Unlike p values, CIs indicate both quantities of direct interest, such as treatment benefit, and also the strength of the evidence They are thus of particular relevance to practitioners of evidence-based medicine (EBM)

Things to Remember The word “interval” means a range of values and is thus singular. The two values that define the interval are called “confidence limits” The CI estimates the “sampling variation” The CI does not reflect additional uncertainty due to other causes; in particular CIs do not incorporate the impact of selective loss to follow-up, poor compliance with treatment, imprecise outcome measurements, lack of blinding, and so on CIs thus always underestimate the total amount of uncertainty

Clinical Significance It is also important not to equate statistical significance with clinical importance I think this has been emphasized enough

Take home message p values suck and tell you flipin nothing CI tell something useful: the likely range of the true value which is what we so desperately want to know