Lack of Confidence Interval Reporting in Dermatology: A Call to Action Z.H. Hopkins, MD1; C. Moreno, BS2; A.M. Secrest, MD, PhD3 1. Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States 2. College of Medicine, Texas A&M University, Temple, Texas, United States 3. Departments of Dermatology and Population Health Sciences, University of Utah, Salt Lake City, Utah, United States British Journal of Dermatology. DOI: 10.111/bjd.17126

Lead researcher: Zachary Hopkins, MD A current Preliminary Medicine Intern at University of Utah Hospital

Introduction What’s already known? Confidence intervals (CIs) have been long lauded for their ability to clearly and intuitively present hypothesis-testing outcomes. The prevalence of CI reporting in the dermatology literature is not known but should be high. One study of the orthopedic literature found CI reporting to be only 21%.

Objective Evaluated CI reporting prevalence in the dermatology literature, factors predicting reporting, and compare CI reporting in the dermatology literature with dermatology research in the New England Journal of Medicine (NEJM).

Methods MEDLINE® was queried for published studies over the past decade (2007- 2017) from the following journals: British Journal of Dermatology (BJD), Journal of Investigative Dermatology (JID), JAMA Dermatology, Journal of the American Academy of Dermatology (JAAD), European Journal of Dermatology (EJD), and Journal of Cosmetic Dermatology (JCD). These were chosen to represent a broad range of subjects and impact factors. 1284 articles were identified and a random number generator used to select articles for evaluation, with an a priori sample size calculation of 100 To compare the dermatology literature with dermatology studies published in the NEJM, the NEJM’s website (www.nejm.org) was queried for articles with a topic of “dermatology” in the same time period.

Methods Inclusion criteria Exclusion criteria Studies published between 2007-2017 in one of the aforementioned journals A study format of either clinical trial, randomized trial, controlled trial, or observational study. Exclusion criteria Descriptive studies and those for which no hypothesis testing was performed, unless said testing should have been performed and was not Tests for which CI reporting would not contribute any extra information Each journal article was reviewed by at least two reviewers

Methods Primary endpoint: CI reporting. Secondary end points: impact factor, journal name, year of publication, funding source, participation of a methodologist, study type, cosmetic subject, reporting of statistical methods, reporting of variance (e.g., standard deviation or standard error), and significance of study’s primary endpoint Prevalence of CI and variance reporting was calculated. Univariable and multivariable logistic regression analyses were used to identify predictors of CI reporting. CI reporting prevalence was compared between the dermatology literature and dermatologic articles published in NEJM over the same period using a two-sample test of proportions. All calculations were performed using Intercooled STATA v14.2 (StataCorp LP, College Station, TX).

Results 97 studies meeting the inclusion and exclusion criteria were randomly selected: 39 from BJD, 26 from JAAD, 8 from JAMA Dermatology, 6 from JID, 7 from EJD, and 11 from JCD 44 studies were randomly selected from the NEJM, of which 28 met the inclusion and exclusion criteria Of 97 articles in the dermatology literature, 22% (95%CI 14–31%) reported CIs 78% (95%CI 68–85%) had a significant primary outcome 15% (95%CI 9.5–24%) were cosmetic in nature 88% (95%CI 80–94%) described the statistical methods used 30% (95%CI 22–40%) mentioned using a statistical methodologist Of the 76 articles that did not report CIs, 57% (95%CI 45–67%) included a measure of variance for the primary outcome

Results In univariable analysis, government funding (OR 12.4, 95%CI 1.35–113, p=0.03), using a statistical methodologist (OR 3.54, 95%CI 1.30-9.70, p=0.01) and publication year (OR 1.25, 95% CI 1.04-1.51, p=0.02) predicted increased CI reporting. In multivariable analysis, only year of publication remained significant (OR 1.44, 95%CI 1.10–1.89, p=0.01).

Results CI reporting was significantly higher in the NEJM dermatology studies (64%, 95%CI 47–82%) compared to the dermatology literature (22%), with a between-group difference of 42% (95%CI 23–62%, p<0.001) Multivariable logistic regression was performed to control for study type, year, use of a methodologist, etc. Even after controlling for these factors CI reporting remained significantly more likely in the NEJM (OR=3.80, 95%CI 1.27–11.3, p=0.01).

Discussion p-values alone lack information necessary for making informed clinical decisions. The interpretation of CI’s is arguably more intuitive and important information regarding statistical significance, supported range of effect size estimates, and study power. Thus, CI’s are a simple, easy to report, outcome which can aid readers in interpreting the statistical and clinical significance of new studies.

Discussion Disappointingly, CI reporting in the dermatology literature is low (22%). It does appear that reporting is improving over time. A limited analysis of articles in our database from 2015-2017 suggest that reporting is as high as 26%, but power with this sample size was limited. A measure of variance was included in many studies without CI’s. While this allows the reader to calculate CI’s should they desire we argue that this is the duty of the author, not the reader

Discussion CI reporting was significantly higher in the NEJM than among articles sampled from the dermatology specific journals Greater emphasis by reviewers and authors to include CI’s in reports, as well as a more robust biostatistical training during residency may improve outcomes reporting in the dermatology literature. Limitations Limited scope of dermatology literature and journals. Study was not powered to assess secondary endpoints.

Conclusions What does this study add? CI reporting in dermatology journals is very low, and significantly lower than reported in dermatology research published in the New England Journal of Medicine. We call on dermatology researchers and journals to improve efforts to report CIs more frequently and consistently, as CIs provide additional information for data interpretation and application than do p-values alone.

Dr. Hopkins, Mr. Moreno, and Dr. Secrest

Call for correspondence Why not join the debate on this article through our correspondence section? Rapid responses should not exceed 350 words, four references and one figure Further details can be found here