1. Software Sustainability Institute www.software.ac.uk Putting the user back into software sustainability 16 December 2013, Scientific Software Days, Austin Neil Chue Hong (@npch), Software Sustainability Institute ORCID: 0000-0002-8876-7606 | N.ChueHong@software.ac.uk Where indicated slides licensed under Supported by Project funding from Slides: http://dx.doi.org/10.6084/m9.figshare.878120

2. Software Sustainability Institute www.software.ac.uk Putting the user back into software sustainability If I was a crossword compiler… But I’m not, so this talk is going to draw on my experiences with scientific software

3. Software Sustainability Institute www.software.ac.uk The problem with mushrooms…

4. Software Sustainability Institute www.software.ac.uk OMII: Sowing the first seeds 11 initial projects funded by Managed Programme Many projects flourished But some wilted and decayed

5. Software Sustainability Institute www.software.ac.uk OMII-UK: Cultivating and Nurturing Emphasis on helping existing software grow  Extra gardeners brought in (Edinburgh and Manchester) with their own plant stock  Making the garden public through initiatives like Google Summer of Code and ENGAGE  Inviting specialists through the PALs scheme Cultivate and sustain community software important to research

6. Software Sustainability Institute www.software.ac.uk The Software Sustainability Institute A national facility for cultivating world- class research through software Better software enables better research Software reaches boundaries in its development cycle that prevent improvement, growth and adoption Providing the expertise and services needed to negotiate to the next stage Developing the policy and tools to support the community developing and using research software Supported by EPSRC Grant EP/H043160/1

7. Software Sustainability Institute www.software.ac.uk No one starts writing software intending to abandon it completely

8. Software Sustainability Institute www.software.ac.uk Can we learn? Picture by Heisenberg MediaPicture by PNNLPicture by nengard

9. Software Sustainability Institute www.software.ac.uk Startup companies are all about reaching sustainability

10. Software Sustainability Institute www.software.ac.uk Four stages of startups DiscoveryValidationEfficiencyScale Are you solving a problem that others are interested in? Have you implemented core features that users want? Can you support new users by refining your processes? Ready to drive growth. Back-end scalability refactoring http://blog.startupcompass.co/pages/marmer-stages

11. Software Sustainability Institute www.software.ac.uk Stage 1: Discovery Are you solving a problem that others are interested in?  Many pieces of software created by researchers have small user bases – this is particularly true of scripts  Not a problem if you are writing the software for yourself only – but it affects how large the project to support the software can be

12. Software Sustainability Institute www.software.ac.uk Water Swap Reaction Coordinate A water-swap reaction coordinate for the calculation of absolute protein-ligand binding free energies Woods CJ, Malaisree M, Hannongbua S, Mulholland AJ J. Chem. Phys. (2011) vol. 134, pp. 054114 http://dx.doi.org/10.1063/1.3519057

13. Software Sustainability Institute www.software.ac.uk Case Study: Ligand Binding Centre for Computational Chemistry, Bristol  New methods for rapid MC sampling of biomolecular systems modelled using QM/MM  Developed two codes ProtoMS (F77) + Sire (C++)  Water-Swap Reaction Coordinate method to calculate absolute protein-ligand binding free energies SSI’s work helped assess users + scale devs  Ran user observations with 4 different users  ASPIRE/ACQUIRE framework has multiple devs Split architecture between ASPIRE (adaptive multiresolution hybrid MD simulation) and ACQUIRE (WorkPacket scheduling system with optimisation for time to result vs “green-ness” http://www.software.ac.uk/resources/case-studies/getting-grips- molecules http://www.siremol.org/adaptive_dynamics

14. Software Sustainability Institute www.software.ac.uk Stage 2: Validation Have you implemented core features that users want?  Do you know who’s using your software? Why are they using it?  If you asked them “how would you feel if you can no longer use this software”, how many would be disappointed?  These are your core users: what do they want and how can you give it to them?

15. Software Sustainability Institute www.software.ac.uk Case Study: Climate Policy Modelling CIAS team at Tyndall Centre for Climate Change Research, University of East Anglia  Develop linked climate and economic models for detailed analysis  Their software was not ready to be used by other groups One researcher/developer at UEA, several users SSI’s work means the software is robust enough that it can be installed and used by others  Enabled use of the software by the WWFN’s Climascope project and James Cook University Documented software to allow extensions by contributors Made it easier to maintain and backup Added job scheduling to improve modeling throughput New modelling framework enables new models i.e. new science http://www.tyndall.ac.uk/research/cias

16. Software Sustainability Institute www.software.ac.uk Stage 3: Efficiency Can you support new users by refining your processes?  If you had conflicting requirements from users, how would you deal with them?  What infrastructure changes do you need to make to support new/additional users?  At this point, scientific software projects often start designating specific community/product managers, user support staff

17. Software Sustainability Institute www.software.ac.uk Case Study: ICAT Science and Technology Facilities Council  Metadata catalogue, used by RAL UK (ISIS, DIAMOND, CLF), SNS US, ELLETRA Italy  ICAT operationally critical at sites  Huge projects looking to use ICAT (PaNdataODI, EuDAT)  Scalability issues and lack of proper processes SSI’s work provided 33 recommendations  15 interviews with different stakeholders  92 observations set out in report  “…we must focus on doing the right things, and this report will help us” Alistair Mills, STFC  Governance and outreach changes to support additional users http://www.software.ac.uk/preparing-icat-thousands-new-users http://www.icatproject.org/

18. Software Sustainability Institute www.software.ac.uk Stage 4: Scale Are you ready to drive growth of users, to reengineer and refactor on an ongoing basis?  This is when software quality considerations become very important, as you have increased reputational risk  This is also the point where traditionally a PI would step aside to become Chief Technology Officer / Chief Scientist and enlist new management After this, the next stage is sustain  Though this might be at a small scale if appropriate

19. Software Sustainability Institute www.software.ac.uk Software development comes in stages Bridging criteria: strength of team; strength of market; proximity of software to market IdeaPrototypeResearch IdeaPrototype Idea PrototypeResearchSupportedProduct IdeaPrototypeResearchSupported An idea to solve a problem Scaling to work for others Understand the functionality Allow others to participate

20. Software Sustainability Institute www.software.ac.uk Open source software projects are all about reaching sustainability

21. Software Sustainability Institute www.software.ac.uk Open Source Software is Free...

22. Software Sustainability Institute www.software.ac.uk Free as in Puppy... Long term costs Needs love and attention May lose charm after growing up Occasional clean- ups required Many left abandoned by their owners

23. Software Sustainability Institute www.software.ac.uk Open Source Software Projects “Every good work of software starts by scratching a developer's personal itch.”  Eric Raymond, The Cathedral and the Bazaar Producers start by having a direct interest in the success of the software  Just like in science OSS projects need to satisfy two aims:  Acquire users (a.k.a. researchers)  Acquire contributors (a.k.a. collaborating researchers) Producing Open Source Software: How to Run a Successful Free Software Project by Karl Fogel

24. Software Sustainability Institute www.software.ac.uk Using infrastructure to become user aware Promotion: - website Communication: - mailing list, wiki Collaboration: - code repository Management: - issue tracker

25. Software Sustainability Institute www.software.ac.uk Scientific research is all about reaching other researchers

26. Software Sustainability Institute www.software.ac.uk Software is pervasive in research

27. Software Sustainability Institute www.software.ac.uk No common baseline Bhargen Basepair Bioinformatics RA Novice coding in Java and Perl to test groups pattern algorithms Helen Helmet Mech Eng intern Exploratory coding based on tinkering with legacy code Fan Fullerene Chemistry PhD Running analyses on behalf of his supervisor Mehrdad Mapping Forestry student Field data collection currently correlating using Matlab Slide courtesy of Greg Wilson

28. Software Sustainability Institute www.software.ac.uk Traditional teaching Traditional software teaching is aimed at “caged” students  Full-time  Able to allocate study- time  Able to space learning over time  Teach in groups Picture by Farm Sanctuary

29. Software Sustainability Institute www.software.ac.uk The free-range researcher The free range researcher has different requirements  Already working at 200%  No common “baseline” knowledge  Often just one or two researchers Picture by Brookford Farm

30. Software Sustainability Institute www.software.ac.uk The modern researcher… … worries about:  Data management and analysis  Reproducible research  Scalable simulations  Integration of models and workflows  Collaboration Picture of Otto Stern courtesy of Emilio Segre Visual Archives

31. Software Sustainability Institute www.software.ac.uk Why do you need users? Funding  Direct: fees, subscriptions, …  Indirect funding: letters of support, citations, collaborations  Advertising: recommendations and referrals Direction (indirection?)  Requirements, bug reports, change requests Community  Users supporting other users  Users becoming contributors  Sustainability and success

32. Software Sustainability Institute www.software.ac.uk 4 Golden Rules - Act Amiably - Breakdown Barriers - Cultivate Contributors - Diaspora + Dispersion

33. Software Sustainability Institute www.software.ac.uk A: Act Amiably Bugzilla project did analysis of their current and previous contributors  Once someone started contributing, what kept them around? Don’t freeze the trunk for long periods Turnover is inevitable Respond to contributions immediately Be extremely kind and visibly appreciative Encourage a total absence of negativity  Software sustainability relies on many people working happily together! http://www.codesimplicity.com/post/open-source-community-simplified/

34. Software Sustainability Institute www.software.ac.uk B: Breakdown Barriers Basics: Website, mailing list, code repository, issue resolution Remove barriers to participation, increase efficiency 1993: First public release; 2 devs 1995: Code open sourced; 3 devs 1996: r-testers list set up 1997: lists split: r-announce, r-help, r-devel; public CVS; 11 devs 2000: CRAN split and mirror 2001: BioConductor 2003: Namespaces 2005: I8n, L8n 2007: R-Forge Today: BioConductor (33 core devs), R-Forge (532 projects, 1562 devs), CRAN (1400+ packages) 34 http://cran.r-project.org/doc/html/interface98-paper/paper_2.html

35. Software Sustainability Institute www.software.ac.uk C: Cultivate Contributors Building intellectual access ramps to support incremental engagement – building capacity and capability Individual Group Consortium W/ industry Community Active Teams change as project matures 35 http://www.castep.org/

36. Software Sustainability Institute www.software.ac.uk D: Diaspora + Dispersion Sometimes, serendipity plays a part E.g. GEOS-CHEM Users spread across world means they have to scale But they also have great support and contributor mechanisms http://acmg.seas.harvard.edu/geos/

37. Software Sustainability Institute www.software.ac.uk Sustainability is also about choosing not to continue Software proliferation Time Customisation InnovationConsolidation Not all software should make it to the next stage Consolidation is as important as innovation in achieving sustainable scientific software

38. Software Sustainability Institute www.software.ac.uk Take home message Understand and embrace your users and contributors if you want your scientific software to be sustainable and successful Slides: http://dx.doi.org/10.6084/m9.figshare.878120

39. Software Sustainability Institute www.software.ac.uk Find out more about the SSI Community Engagement (Lead: Shoaib Sufi)  Fellowship Programme Fellowship Programme  Events and Workshops Events and Workshops Consultancy (Lead: Steve Crouch)  Open Call for Projects / Collaborations Open Call for ProjectsCollaborations  Software Evaluation Software Evaluation Policy and Publicity (Lead: Simon Hettrick)  Case Studies / Policy Campaigns Case StudiesPolicy Campaigns  Software and Research Blog Software and Research Blog Training (Lead: Aleksandra Pawlik)  Software Carpentry (300+ students/year) Software Carpentry  Guides and Top Tips GuidesTop Tips Journal of Open Research Software (Editor: Neil Chue Hong) Journal of Open Research Software Collaboration between universities of Edinburgh, Manchester, Oxford and Southampton Slides: http://dx.doi.org/10.6084/m9.figshare.878120

40. Software Sustainability Institute www.software.ac.uk A national facility for cultivating world-class research through software Become our next collaborators! Website:www.software.ac.uk Email: info@software.ac.uk Twitter: twitter.com/SoftwareSaved Some of our collaborations

41. Software Sustainability Institute www.software.ac.uk The Foundations of Digital Research Re- search Careers Recognition / Reward Skills and Capability Software Re-usable Re-producible www.software.ac.uk/ software-evaluation-guide resources/guides software-carpentry training www.software.ac.uk/blog/ 2012-11-09-craftsperson-and-scholar software.ac.uk/blog/2012-08-16-what-research- software-community-and-why-should-you-care www.software.ac.uk/blog/2011-05-02- publish-or-be-damned-alternative- impact-manifesto-research-software Prlić A, Procter JB (2012) Ten Simple Rules for the Open Development of Scientific Software PLoS Comput Biol 8(12): e1002802. doi:10.1371/journal.pcbi.1002802 Wilson G, et al. (2013) Best Practices for Scientific Computing Submitted to PNAS. http://arxiv.org/abs/1210.0530

42. Software Sustainability Institute www.software.ac.uk Academic Software Should Not Suck Courtesy of C. Titus Brown http://ivory.idyll.org/blog/ladder-of-academic-software-notsuck.html

43. Software Sustainability Institute www.software.ac.uk 5 Stars of Research Software Community  There is a community infrastructure Open  Software has permissive license Defined  Accurate metadata for the software Extensible  Usable, modifiable for my purpose Runnable  I can access and run software C O DE R c.f. 5 Stars of Linked Data (Berners-Lee) 5 Stars of Online Journals (Shotton) “Golden Star” Originally by Ssolbergj CC-BY

44. Software Sustainability Institute www.software.ac.uk Journal of Open Research Software http://openresearchsoftware.metajnl.com

45. Software Sustainability Institute www.software.ac.uk People UK Research Computing Ecosystem Computing Communities … Communities … Network/Collaboration Instruments Software Data Centres