1. Reporting in health research Why it matters How to improve Presentation for the Center for Open Science July 10, 2015 April Clyburne-Sherin

2. Reporting in health research Why does reporting matter? What is a good report? – Signs you may have written a good report How can you improve reporting? – Reporting guidelines

3. Why does reporting matter? HTTP :// XKCD. COM /882/

4. Why does reporting matter? HTTP :// XKCD. COM /882/ “High consumption of green jelly beans was significantly associated with Acne vulgaris diagnoses (p < 0.05).”

5. HTTP :// XKCD. COM /882/

6. “High consumption of green jelly beans was significantly associated with Acne vulgaris diagnoses (p < 0.05).”

7. “Good reports are all alike; every poor report is poor in its own way.” Non-reporting Selective reporting Incomplete reporting Misleading reporting Ambiguous reporting

8. “Good reports are all alike; every poor report is poor in its own way.” Non-reporting “What jelly bean study?”

9. Why does non-reporting matter? Non-reporting  Publication bias Positive trials are 2 times more likely to be published than negative trials 1.Parekh, S., et al. Dissemination and publication of research findings: an updated review of related biases. Prepress Projects Limited, 2010. 2.Macleod, Malcolm R., et al. "Biomedical research: increasing value, reducing waste." The Lancet 383.9912 (2014): 101-104. 1/3 to 1/2 of trials are not published.

10. Why does non-reporting matter? Non-reporting  Publication bias Whittington, Craig J., et al. "Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data." The Lancet 363.9418 (2004): 1341-1345.

11. “Good reports are all alike; every poor report is poor in its own way.” Selective reporting “We studied green jelly bean consumption and acne. Just the green ones. Really.”

12. Why does selective reporting matter? Selective reporting  Outcome reporting bias 1.Chan, An-Wen, et al. "Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles." Jama 291.20 (2004): 2457-2465. 2.Macleod, Malcolm R., et al. "Biomedical research: increasing value, reducing waste." The Lancet 383.9912 (2014): 101-104. 62% of trials had at least one primary outcome changed, introduced or omitted 50%+ of pre-specified outcomes not reported

13. Why does selective reporting matter? Selective reporting  Outcome reporting bias Response from a trialist who had analysed data on a prespecified outcome but not reported them “When we looked at that data, it actually showed an increase in harm amongst those who got the active treatment, and we ditched it because we weren’t expecting it and we were concerned that the presentation of these data would have an impact on people’s understanding of the study findings. … The argument was, look, this intervention appears to help people, but if the paper says it may increase harm, that will, it will, be understood differently by, you know, service providers. So we buried it.” Smyth, R. M. D., et al. "Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists." Bmj 342 (2011): c7153.

14. “Good reports are all alike; every poor report is poor in its own way.” Incomplete reporting “Fifteen colours of jelly beans were consumed in this study.”

15. Why does incomplete reporting matter? Incomplete reporting  Research waste Glasziou, Paul, et al. "Reducing waste from incomplete or unusable reports of biomedical research." The Lancet 383.9913 (2014): 267-276. 40 – 89% of studies not replicable due to incomplete reporting of intervention

16. Why does incomplete reporting matter? Incomplete reporting  Research waste Macleod, Malcolm R., et al. "Biomedical research: increasing value, reducing waste." The Lancet 383.9912 (2014): 101-104.

17. “Good reports are all alike; every poor report is poor in its own way.” Misleading reporting “Salmon, red, turquoise, magenta, yellow, grey, tan, mauve, beige, lilac, black, peach, and orange jelly beans may prevent acne.” may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne may prevent acne

18. Why does misleading reporting matter? Misleading reporting  Spin Vera-Badillo, F. E., et al. "Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer." Annals of oncology (2013): mds636. 59% of breast cancer trials that had no significant difference in their primary outcome claim efficacy in their report

19. “Good reports are all alike; every poor report is poor in its own way.” Ambiguous reporting “We studied candy consumption and acne.”

20. Why does ambiguous reporting matter? Ambiguous reporting  Ambiguity “double-blinded” “randomized” “standard care”

21. Why does reporting matter? Non-reporting  Publication bias Selective reporting  Outcome reporting bias Incomplete reporting  Research waste Misleading reporting  Spin Ambiguous reporting  Ambiguity

22. Why does reporting matter? Non-reporting  Publication bias Selective reporting  Outcome reporting bias Incomplete reporting  Research waste Misleading reporting  Spin Ambiguous reporting  Ambiguity Not replicable + reproducible

23. What is a good report? Transparent + accurate reporting Leads to an understanding of – What was done – What was found

24. Signs you may have a good report Readers have enough information to reach their own conclusions use the findings Allows replication + reproduction Does not mislead Allows inclusion in systematic reviews + meta- analyses

25. How can you improve reporting? All studies registered. All results reported. Transparent reporting. Accurate reporting.

26. How can you improve reporting? All studies registered. All results reported. Transparent reporting. Accurate reporting.

27. How can you improve reporting? All studies registered. All results reported. Transparent reporting. Accurate reporting. ?

28. Reporting guidelines Enhancing the Quality and Transparency Of Health Research Minimum set of items to address in protocol + report Study design specific Developed through consensus by stakeholders Evidence-based to Contain bias Maximize transparency Maximize utility Moher, David, et al. "Guidance for developers of health research reporting guidelines." PLoS Med 7.2 (2010): e1000217.

29. Authors’ reasons for using reporting guidelines Fuller T, Pearson M, Peters J, Anderson R (2015) What Affects Authors’ and Editors’ Use of Reporting Guidelines? Findings from an Online Survey and Qualitative Interviews. PLoS ONE 10(4): e0121585. doi:10.1371/journal.pone.0121585 http://127.0.0.1:8081/plosone/article?id=info:doi/10.1371/journal.pone.0121585 “It improves the quality of the paper.” “It helped to avoid omissions.” “It helped validate the study design.”

30. Reporting guidelines

31. SPIRIT Standard Protocol Items: Recommendations for Interventional Trials CONSORT Consolidated Standards of Reporting Trials SAMPL Statistical analyses and methods in the published literature TIDIER Better reporting of interventions: template for intervention description and replication Enhancing the Quality and Transparency Of Health Research

32. Conclusions Poor reports Bias Research waste Impair clinical and policy decisions Prevent replicability and reproducibility Improved reports Improve quality of paper Prevent omissions Combat bias in the published literature Ensure your hard work does not go to waste