Publication Bias : How to Avoid it An Editor's View Steven R Bailey MD FSCAI, FACC, FAHA Janey Briscoe Distinguished Professor of Medicine and Radiology Chairman, Division of Cardiology UTHSCSA Editor in Chief Catheterization and Cardiovascular Interventions baileys@uthscsa.edu

Steven R. Bailey, MD Consulting: Boston Scientific Corporation, Biostar Venture Grant Support: Abbott Vascular, SCAI, St. Jude Medical, Inc.

Bias Epidemiologic deviation of results or inferences from the truth, or processes leading to such systematic deviation any trend in the collection, analysis, interpretation, publication, or review of data that can lead to conclusions that are systematically incorrect McGraw-Hill Concise Dictionary of Modern Medicine. © 2002 by The McGraw-Hill Companies, Inc.

Bias, We All Have It Add Lincoln to this montage

Bias in Research Literature Review Study Design Study Execution Data Collection Analysis Interpretation of Results Publication Sackett DL. Bias in analytic research. J Chronic Dis 1979;32:51-63

Publication Bias Inappropriate bias can enter the literature when factors other than quality and scientific content influence a researcher’s selection of what to submit for publication, or a journal’s decision whether to publish a paper. Editorial New methods to deal with publication bias BMJ 2009

Types of Bias Related to Publication Publication bias Publication or Non-publication of research findings depending upon content and direction (+/-) results Time Lag Bias Rapid or delayed submission and publication depending upon content and direction (+/-) results Multiple Publication (duplicate) bias Location Bias Publication according to ease of access or levels of indexing Modified from Table 10.1 Cochrane Handbook for Systematic Reviews of Interventions

Types of Bias Related to Publication Citation bias Citation or Non-Citation of research findings depending upon content and direction (+/-) results Language Bias Submission and publication in a particular language depending upon content and direction (+/-) results Outcome Reporting Bias Selective reporting of some outcomes but not others depending upon content and direction (+/-) results Modified from Table 10.1 Cochrane Handbook for Systematic Reviews of Interventions

Why does it matter? Distorts the scientific record Cloaks the “truth” Encourages unnecessary repetition of research Influences decision making Physicians care of patients May cause harm to patients Costly for the health care systems Impacts policy makers A form of scientific and research misconduct

Publication Bias Who is to blame? Wicked researchers? Very wicked sponsors? Editors: the wickedest of all? Reviewers? Champion Cochrane Lecture 2010

Publication Bias: Study Design Emily Flake New Yorker Magazine

Publication Bias: Impact of Editorial Rejection Investigators reported that failure to submit for publication (on account of either null results, limitations in methodology, loss of interest, or unimportant results) accounted for 39 % of the reasons given for non-publication. Rejection of a manuscript by an editor was an infrequent reason (9%) for a study remaining unpublished, regardless of the results. The OR was 2.94 for a study with statistically significant versus null results being submitted . Easterbrook Lancet 1991 337: 867-72

Publication Bias Trials with positive findings (P < 0.05), or those findings perceived to be” important or striking” had nearly four times the odds of being submitted and published compared to findings that were not statistically significant (P ≥ 0.05) risk ratio of 1.78 (95% CI 1.58 to 1.95), Two studies found that trials with positive findings also tended to be published more quickly than trials with negative findings (4.7 vs. 8.0 yrs.). No influence was seen with size of the trial source of funding academic rank sex of the principal investigator Hopewell Cochrane Summaries 2009

Publication Bias: Impact of Positive Results on Full Publication after Abstract Presentation 'Positive' results resulted in more publications any 'significant' result showed an association with full publication (RR = 1.30; CI 1.14 to 1.47) 'positive' results defined as a result favoring the experimental treatment (RR =1.17; CI 1.02 to 1.35) ‘positive' results emanating from randomized or controlled clinical trials (RR = 1.18, CI 1.07 to 1.30). Scherrer DOI: 10.1002/14651858.MR000005.pub3 2011

Publication Bias Impact of Industry Sponsorship Published research from drug companies is more likely to be favourable to the product Companies may selectively fund trials Industry sponsored trials are of similar quality Sometimes inappropriate comparators are chosen Publication bias does occur Lexchin and Bero BMJ 2003;326:1167-70

Publication Bias Role of Conflicts of Interest Conditions of Project Support Study design Data collection Analysis Interpretation of data Writing the report N

Publication Bias Quality of Life Assessments Studies sponsored by industry were more than twice as likely as studies sponsored by non-industry sources to report ratios below $20, 000/QALY and over three times more likely to report ratios below $50 000/QALY or $100 000/QALY. Studies sponsored by industry were also more likely to be of lower methodological quality and to be published in journals with lower impact factors

Publication Bias Publication bias and other related biases can be summarized as statistically significant, 'positive' results being: more likely to be published (publication bias) more likely to be published rapidly (time lag bias) more likely to be published in English (language bias) more likely to be published more than once (multiple publication bias) more likely to be cited by others (citation bias) All of these biases make positive studies easier to find than those with non-significant or negative results

Editorial Publication Bias Editors must make decisions on what is suitable for their journal, and what will be of interest to their readers, limiting complete objectivity Editors may reject many submitted papers without independent review, representing bias in selecting which papers to send out to reviewers.

Editorial Publication Bias Unacceptable bias occurs when the decision to send a paper for review, or the decision to accept it, is influenced by factors other than the scientific content of the paper or its fit with the scope of the journal. papers from a particular country institution authors’ affiliations language gender

Publication Bias Impact of Reviewers Reviewers can exercise bias not only in the decision whether to recommend acceptance or rejection of a paper, but they can also delay its publication. They might reject or delay acceptance of papers that do not accord with their own beliefs they might be more likely to accept, or not critically judge, a paper that supports one of their previous findings, or one that cites them extensively.

Assessing Publication Bias

Publication Bias Improving the System Publication bias is an important problem that impacts patient care Strong evidence supports its existence It is a complex problem with many contributors There are several ways to reduce its effect Ultimately responsibility falls to all involved sponsors of trials authors reviewers editors

Publication Bias Improving the System Responsible authors Better conduct and reporting of RCTs Better conduct and reporting of systematic reviews Strong industry support for methods and models Clear definition of editorial policies Vigilant editors and reviewers Mechanism for “publication” of unpublished trials