1. Publication bias in clinical trials Kamran Abbasi Deputy editor, BMJ

2. Merhaba

3. I want to talk about...  What is publication bias?  Why does it matter?  What is the evidence for it?  What can be done about it?  How has the BMJ responded?

4. There are many types of bias  Selection bias: biased allocation to comparison groups  Performance bias: unequal provision of care except treatment being evaluated  Detection bias: biased assessment of outcome  Attrition bias: biased occurrence and handling of deviations from protocol and loss to follow up ... and on and on (From Egger et al BMJ 2001;323:42-46 (7 July)

5. What is publication bias (1)?  A definition: “Publication bias refers to the greater likelihood that studies with positive results will be published” JAMA 2002;287:2825-2828

6. What is publication bias (2)?  An alternative definition: Publication bias is the selective or multiple publication or suppression of trial results so that the scientific record is distorted

7. Why does it matter?  Distorts the scientific record  Hides the “truth”  Influences doctors’ decision making  Misleads policy makers  Causes harm to patients  Costly for the health service  A form of scientific and research misconduct

8. Who is to blame?  Wicked researchers?  Very wicked sponsors?  Editors: the wickedest of all?  (and let’s not forget reviewers)

9. What is the evidence for it (1)?  Stern and Simes BMJ 1997;315:640-645  Question: To what extent is publication influenced by study outcome?  Studies submitted to an Australian ethics committee over 10 years  Examined protocols  Questionnaire to authors (70% response)

10. Stern and Simes: results 4.7 vs 8.0 yrs 4.8 vs 8.0 yrs Time to publication 3.13 (1.76 to 5.58) 2.32 (1.47 to 3.66) Positive> negative Clinical trials (n=130) All studies (n=520)

11. Stern and Simes: conclusions  Positive trials are more likely to be submitted for publication  Positive trials are more likely to be published  Positive trials are more likely to be published quickly  Implications for systematic reviews  Important to register all trials

12. What is the evidence for it (2)?  Lexchin and Bero BMJ 2003;326:1167-70  Question: Does drug industry sponsorship influence research quality and outcome?  Meta-meta-analysis  Industry research less likely to be published (more likely in symposium proceedings)  No difference in methodological quality  More likely to have a positive finding (OR 4.05 95% CI 2.98 to 5.51)

13. Lexchin and Bero  A wide range of diseases eg osteoarthiritis of the knee, multiple myeloma, psychiatric problems, Alzheimer’s disease, venous thromboembolism  A wide range of drugs eg tacrine, clozapine, 3 rd generation OCP, erythropoietin, antidepressants, topical glucocorticoids, treatment for HIV

14. Lexchin and Bero: conclusions  Published research from drug companies is more likely to be favourable to the product  Do companies selectively fund trials? Unlikely  Is it of poorer quality? No  Are inappropriate comparators chosen? Sometimes/often/a lot  Is it publication bias? Yes

15. What is the evidence for it (3)?  Melander et al BMJ 2003;326:1171-3  Question: Is there selective reporting of sponsored studies by drug companies?  Trials submitted to the Swedish drug regulatory authority (5 SSRIs, 42 trials)  Multiple publication  Selective publication  Selective reporting

16. Melander et al: conclusion  “Any attempt to recommend a specific selective serotonin reuptake inhibitor from the publicly available data ONLY is likely to be based on biased evidence.”

17. What is the evidence for it (4)?  Olson et al JAMA 2002;287:2825-2828  Question: Is there publication bias in editorial decision making?  3 years, 745 manuscripts  Positive vs negative OR 1.30 (0.87 to 1.86)  Small effect of editorial decision making, much less than researchers not submitting negative studies  Will this be true for journals less grand than JAMA?

18. What can be done about it (1)?  Better conduct and reporting of RCTs (CONSORT)  Better conduct and reporting of systematic reviews (QUORUM)  “Publication” of unpublished trials  Enlightened sponsors (a code of good practice Wager et al 2003 http://www.gpp-guidelines.org )  Better editorial policies  Vigilant editors and reviewers  Responsible authors

19. What can be done about it (2)?  Publication of original protocols and deviations from protocol  Declaration of competing (financial) interests by authors, reviewers, and editors  Declaration of sponsorship/funding  Registering all clinical trials

20. How has the BMJ responded?  A change in editorial thinking: Is it the question that matters? It is  Amnesty on unreported clinical trials  More transparency (CONSORT, QUORUM, sponsorship, funding, competing interests)  Theme issue on doctors and the drug industry  ?Protocols  ?Registering clinical trials

21. Conclusions  Publication bias is an important problem that impacts on patient care  There is much evidence to support its existence  There are many players  There are many ways to reduce its effect, examples of good practice  Ultimately there is a big responsibility on sponsors of trials, authors, and editors