1. Software Sustainability Institute www.software.ac.uk Open science is impossible without software http://dx.doi.org/10.5281/zenodo.48749 5 th April 2016, Open Science Conference, Amsterdam Neil Chue Hong (@npch), Software Sustainability Institute ORCID: 0000-0002-8876-7606 | N.ChueHong@software.ac.uk Slides licensed under CC-BY where indicated: Supported by Project funding from

2. The research community relies on software Do you use research software? What would happen to your research without software Survey of researchers from 15 Russell Group universities conducted by SSI between August - October 2014. 406 respondents covering representative range of funders, discipline and seniority. 56% Develop their own software 71% Have no formal software training

3. £840m Investment in 2013-2014 financial year, an amount that has risen by 3% on average over last four years The cost of research that relies on software 30% Of total research investment has been spent on research which relies on software over the last four financial years Analysis of data from 49,650 grant titles and abstracts published on Gateway to Research covering 2010-2014.

4. The UK research community relies on software Survey of researchers from 15 Russell Group universities conducted by SSI between August - October 2014. 406 respondents covering representative range of funders, discipline and seniority. 56% Of UK researchers develop their own research software 71% Of UK researchers have had no formal software development training 140,000 UK researchers are relying on their own coding skills

5. Software Sustainability Institute www.software.ac.uk Raise standards for preclinical cancer research 47 out of 53 “landmark” publications could not be replicated Begley, Ellis. Nature, 483, 2012 doi:10.1038/483531a

6. Software Sustainability Institute www.software.ac.uk Repeatability of published microarray gene expression analyses 56% of analyses could not be repeated, of which 30% were because of software issues. 50% did not state software version, 39% did not provide raw data. Only 11% could be reproduced satisfactorily. Ioannidis et al. Nature Genetics, 41, 2010 doi:10.1038/ng.295

7. Software Sustainability Institute www.software.ac.uk Repeatability in Computer Science Of 401 papers in ACM Computer Science journals and proceedings, only 85 provided a link to software. For 176 the software could not be obtained. Collberg, Proebsting, Warren, University of Arizona TR 14-04, 2015 http://reproducibility.cs.arizona.edu/v2/RepeatabilityTR.pdf

8. Software Sustainability Institute www.software.ac.uk Errors due to bioinformatics pipeline The results presented in the Report “Ancient Ethiopian genome reveals extensive Eurasian admixture throughout the African continent“ were affected by a bioinformatics error – identified because of open science Llorente et al. Science, 350, 6262 doi:10.1126/science.aad2879

9. Software Sustainability Institute www.software.ac.uk T T There’s no such thing as irreproducible research Without access to software, research funded by public money is not transparent, not reproducible, difficult to reuse and exploit and certainly not open science

10. Software Sustainability Institute www.software.ac.uk The Research Cycle Create Test Interpret Publish Revise Paper Data Software Research Outputs Research is a continuous cycle. When we publish we are contributing to the body of knowledge.

11. Software Sustainability Institute www.software.ac.uk Research/Reuse/Reward Cycle Index Identify Cite Reward Create Test Interpret Publish Revise Research Reuse Reuse is also a cycle. We build our research on the work of others. Reward mechanisms should encourage reuse.

12. Software Sustainability Institute www.software.ac.uk Research/Reuse/Reward Cycle Index Identify Cite Reward Create Test Interpret Publish Revise Research Reuse Reuse is also a cycle. We build our research on the work of others. Reward mechanisms should encourage reuse.

13. Software Sustainability Institute www.software.ac.uk T T

14. Software Sustainability Institute www.software.ac.uk T T

15. Software Sustainability Institute www.software.ac.uk, it’ Victoria Stodden, AMP 2011 http://www.stodden.net/AMP2011/, Special Issue Reproducible Research Computing in Science and Engineering July/August 2012, 14(4) Howison and Herbsleb (2013) "Incentives and Integration In Scientific Software Production" CSCW 2013.

16. Software Sustainability Institute www.software.ac.uk T T Research Culture Needs Changing Our research culture presents barriers but few incentives to sharing code There is a fear of being “found out” for poor code, but no encouragement or resources to improve software engineering skills There is no reward for publishing code in the current system of metrics. Researchers fear being “scooped” or losing ability to publish. Many organisations do not understand how to exploit open source licenses

17. Software Sustainability Institute www.software.ac.uk Learning to be a carpenter Teach basic lab skills for scientific computing so that researchers can do more in less time and with less pain. Teach basic concepts, skills and tools for working more effectively with data. Workshops are designed for people with little to no prior computational experience. admin@datacarpentry.org admin@software-carpentry.org Open source learning, that can be tailored to disciplines. “Train the trainers”: building a capable base of instructors.

18. Software Sustainability Institute www.software.ac.uk T T Vandewalle (2012) DOI: 10.1109/MCSE.2012.63

19. Software Sustainability Institute www.software.ac.uk Community and contributions Open source is more than just a license  It’s about making it easier for people to collaborate  And reducing replication of effort Encouraging and expanding at the right pace  Efficient tools for research, not “generic” software  Increased robustness, reuse Being open from the start rather than waiting until it’s “ready”  Code is never perfect, always evolving

20. Software Sustainability Institute www.software.ac.uk Cultivate community Basics: Website, mailing list, code repository, issue resolution Remove barriers to participation, increase efficiency 1993: First public release; 2 devs 1995: Code open sourced; 3 devs 1996: r-testers list set up 1997: lists split: r-announce, r-help, r-devel; public CVS; 11 devs 2000: CRAN split and mirror 2001: BioConductor 2003: Namespaces 2005: I8n, L8n 2007: R-Forge Today: BioConductor (33 core devs), R-Forge (532 projects, 1562 devs), CRAN (1400+ packages) 20 http://cran.r-project.org/doc/html/interface98-paper/paper_2.html

21. Software Sustainability Institute www.software.ac.uk Open Source lets others benefit Slide courtesy of Nancy Wilkins-Diehr BEAST software licensed under LGPL

22. Software Sustainability Institute www.software.ac.uk T T Without data it’s difficult to validate results. But without code, we waste the opportunity to advance science. Slides: http://dx.doi.org/10.5281/zenodo.48749 “The only way to publish software in a scientifically robust manner is to share source code, and that means publishing via the internet in an open-access/open- source fashion. —Warren Lyford DeLano, Creator of PyMOL, 2005

23. Software Sustainability Institute www.software.ac.uk The Software Sustainability Institute A national facility for cultivating better, more sustainable, research software to enable world- class research Software reaches boundaries in its development cycle that prevent improvement, growth and adoption Providing the expertise and services needed to negotiate to the next stage Developing the policy and tools to support the community developing and using research software Supported by EPSRC Grant EP/H043160/1 + EPSRC/ESRC/BBSRC grant EP/N006410/1

24. Software Policy Training Community Outreach Delivering essential software skills to researchers via CDTs, institutions & doctoral schools Helping the community to develop software that meets the needs of reliable, reproducible, and reusable research Collecting evidence on the community’s software use & sharing with stakeholders Bringing together the right people to understand and address topical issues Exploiting our platform to enable engagement, delivery & uptake

25. Website & blog Campaigns Advice Guides Courses Workshops Fellowship Research Software Policy Training Community Consultancy 50+ projects 130+ evaluations 4 surgeries 35+ UK SWC workshops 1000+ learners 80+ guides 50,000 readers 61 domain ambassadors 20+ workshops organised 740 researchers 50,000 grants analysed 150+ contributed articles 20,000 unique visitors per month 3,000 Twitter followers 300+ RSEs engaged2100 signatures 13 issues highlighted Outreach

26. Software Sustainability Institute www.software.ac.uk Find out more about the SSI Community Engagement (Lead: Shoaib Sufi)  Fellowship Programme Fellowship Programme  Events and Workshops Events and Workshops Consultancy (Lead: Steve Crouch)  Open Call for Projects / Collaborations Open Call for ProjectsCollaborations  Software Evaluation Software Evaluation Policy and Publicity (Lead: Simon Hettrick)  Case Studies / Policy Campaigns Case StudiesPolicy Campaigns  Software and Research Blog Software and Research Blog Training (Lead: Aleksandra Pawlik)  Software Carpentry (300+ students/year) Software Carpentry  Guides and Top Tips GuidesTop Tips Journal of Open Research Software (Editor: Neil Chue Hong) Journal of Open Research Software Collaboration between universities of Edinburgh, Manchester, Oxford and Southampton Supported by EPSRC Grant EP/H043160/1 + EPSRC/ESRC/BBSRC grant EP/N006410/1

27. Software Sustainability Institute www.software.ac.uk