Software Sustainability Institute Putting the user back into software sustainability 16 December 2013, Scientific Software Days, Austin Neil Chue Hong Software Sustainability Institute ORCID: | Where indicated slides licensed under Supported by Project funding from Slides:

Software Sustainability Institute Putting the user back into software sustainability If I was a crossword compiler… But I’m not, so this talk is going to draw on my experiences with scientific software

Software Sustainability Institute The problem with mushrooms…

Software Sustainability Institute OMII: Sowing the first seeds 11 initial projects funded by Managed Programme Many projects flourished But some wilted and decayed

Software Sustainability Institute OMII-UK: Cultivating and Nurturing Emphasis on helping existing software grow  Extra gardeners brought in (Edinburgh and Manchester) with their own plant stock  Making the garden public through initiatives like Google Summer of Code and ENGAGE  Inviting specialists through the PALs scheme Cultivate and sustain community software important to research

Software Sustainability Institute The Software Sustainability Institute A national facility for cultivating world- class research through software Better software enables better research Software reaches boundaries in its development cycle that prevent improvement, growth and adoption Providing the expertise and services needed to negotiate to the next stage Developing the policy and tools to support the community developing and using research software Supported by EPSRC Grant EP/H043160/1

Software Sustainability Institute No one starts writing software intending to abandon it completely

Software Sustainability Institute Can we learn? Picture by Heisenberg MediaPicture by PNNLPicture by nengard

Software Sustainability Institute Startup companies are all about reaching sustainability

Software Sustainability Institute Four stages of startups DiscoveryValidationEfficiencyScale Are you solving a problem that others are interested in? Have you implemented core features that users want? Can you support new users by refining your processes? Ready to drive growth. Back-end scalability refactoring

Software Sustainability Institute Stage 1: Discovery Are you solving a problem that others are interested in?  Many pieces of software created by researchers have small user bases – this is particularly true of scripts  Not a problem if you are writing the software for yourself only – but it affects how large the project to support the software can be

Software Sustainability Institute Water Swap Reaction Coordinate A water-swap reaction coordinate for the calculation of absolute protein-ligand binding free energies Woods CJ, Malaisree M, Hannongbua S, Mulholland AJ J. Chem. Phys. (2011) vol. 134, pp

Software Sustainability Institute Case Study: Ligand Binding Centre for Computational Chemistry, Bristol  New methods for rapid MC sampling of biomolecular systems modelled using QM/MM  Developed two codes ProtoMS (F77) + Sire (C++)  Water-Swap Reaction Coordinate method to calculate absolute protein-ligand binding free energies SSI’s work helped assess users + scale devs  Ran user observations with 4 different users  ASPIRE/ACQUIRE framework has multiple devs Split architecture between ASPIRE (adaptive multiresolution hybrid MD simulation) and ACQUIRE (WorkPacket scheduling system with optimisation for time to result vs “green-ness” molecules

Software Sustainability Institute Stage 2: Validation Have you implemented core features that users want?  Do you know who’s using your software? Why are they using it?  If you asked them “how would you feel if you can no longer use this software”, how many would be disappointed?  These are your core users: what do they want and how can you give it to them?

Software Sustainability Institute Case Study: Climate Policy Modelling CIAS team at Tyndall Centre for Climate Change Research, University of East Anglia  Develop linked climate and economic models for detailed analysis  Their software was not ready to be used by other groups One researcher/developer at UEA, several users SSI’s work means the software is robust enough that it can be installed and used by others  Enabled use of the software by the WWFN’s Climascope project and James Cook University Documented software to allow extensions by contributors Made it easier to maintain and backup Added job scheduling to improve modeling throughput New modelling framework enables new models i.e. new science

Software Sustainability Institute Stage 3: Efficiency Can you support new users by refining your processes?  If you had conflicting requirements from users, how would you deal with them?  What infrastructure changes do you need to make to support new/additional users?  At this point, scientific software projects often start designating specific community/product managers, user support staff

Software Sustainability Institute Case Study: ICAT Science and Technology Facilities Council  Metadata catalogue, used by RAL UK (ISIS, DIAMOND, CLF), SNS US, ELLETRA Italy  ICAT operationally critical at sites  Huge projects looking to use ICAT (PaNdataODI, EuDAT)  Scalability issues and lack of proper processes SSI’s work provided 33 recommendations  15 interviews with different stakeholders  92 observations set out in report  “…we must focus on doing the right things, and this report will help us” Alistair Mills, STFC  Governance and outreach changes to support additional users

Software Sustainability Institute Stage 4: Scale Are you ready to drive growth of users, to reengineer and refactor on an ongoing basis?  This is when software quality considerations become very important, as you have increased reputational risk  This is also the point where traditionally a PI would step aside to become Chief Technology Officer / Chief Scientist and enlist new management After this, the next stage is sustain  Though this might be at a small scale if appropriate

Software Sustainability Institute Software development comes in stages Bridging criteria: strength of team; strength of market; proximity of software to market IdeaPrototypeResearch IdeaPrototype Idea PrototypeResearchSupportedProduct IdeaPrototypeResearchSupported An idea to solve a problem Scaling to work for others Understand the functionality Allow others to participate

Software Sustainability Institute Open source software projects are all about reaching sustainability

Software Sustainability Institute Open Source Software is Free...

Software Sustainability Institute Free as in Puppy... Long term costs Needs love and attention May lose charm after growing up Occasional clean- ups required Many left abandoned by their owners

Software Sustainability Institute Open Source Software Projects “Every good work of software starts by scratching a developer's personal itch.”  Eric Raymond, The Cathedral and the Bazaar Producers start by having a direct interest in the success of the software  Just like in science OSS projects need to satisfy two aims:  Acquire users (a.k.a. researchers)  Acquire contributors (a.k.a. collaborating researchers) Producing Open Source Software: How to Run a Successful Free Software Project by Karl Fogel

Software Sustainability Institute Using infrastructure to become user aware Promotion: - website Communication: - mailing list, wiki Collaboration: - code repository Management: - issue tracker

Software Sustainability Institute Scientific research is all about reaching other researchers

Software Sustainability Institute Software is pervasive in research

Software Sustainability Institute No common baseline Bhargen Basepair Bioinformatics RA Novice coding in Java and Perl to test groups pattern algorithms Helen Helmet Mech Eng intern Exploratory coding based on tinkering with legacy code Fan Fullerene Chemistry PhD Running analyses on behalf of his supervisor Mehrdad Mapping Forestry student Field data collection currently correlating using Matlab Slide courtesy of Greg Wilson

Software Sustainability Institute Traditional teaching Traditional software teaching is aimed at “caged” students  Full-time  Able to allocate study- time  Able to space learning over time  Teach in groups Picture by Farm Sanctuary

Software Sustainability Institute The free-range researcher The free range researcher has different requirements  Already working at 200%  No common “baseline” knowledge  Often just one or two researchers Picture by Brookford Farm

Software Sustainability Institute The modern researcher… … worries about:  Data management and analysis  Reproducible research  Scalable simulations  Integration of models and workflows  Collaboration Picture of Otto Stern courtesy of Emilio Segre Visual Archives

Software Sustainability Institute Why do you need users? Funding  Direct: fees, subscriptions, …  Indirect funding: letters of support, citations, collaborations  Advertising: recommendations and referrals Direction (indirection?)  Requirements, bug reports, change requests Community  Users supporting other users  Users becoming contributors  Sustainability and success

Software Sustainability Institute 4 Golden Rules - Act Amiably - Breakdown Barriers - Cultivate Contributors - Diaspora + Dispersion

Software Sustainability Institute A: Act Amiably Bugzilla project did analysis of their current and previous contributors  Once someone started contributing, what kept them around? Don’t freeze the trunk for long periods Turnover is inevitable Respond to contributions immediately Be extremely kind and visibly appreciative Encourage a total absence of negativity  Software sustainability relies on many people working happily together!

Software Sustainability Institute B: Breakdown Barriers Basics: Website, mailing list, code repository, issue resolution Remove barriers to participation, increase efficiency 1993: First public release; 2 devs 1995: Code open sourced; 3 devs 1996: r-testers list set up 1997: lists split: r-announce, r-help, r-devel; public CVS; 11 devs 2000: CRAN split and mirror 2001: BioConductor 2003: Namespaces 2005: I8n, L8n 2007: R-Forge Today: BioConductor (33 core devs), R-Forge (532 projects, 1562 devs), CRAN (1400+ packages) 34

Software Sustainability Institute C: Cultivate Contributors Building intellectual access ramps to support incremental engagement – building capacity and capability Individual Group Consortium W/ industry Community Active Teams change as project matures 35

Software Sustainability Institute D: Diaspora + Dispersion Sometimes, serendipity plays a part E.g. GEOS-CHEM Users spread across world means they have to scale But they also have great support and contributor mechanisms

Software Sustainability Institute Sustainability is also about choosing not to continue Software proliferation Time Customisation InnovationConsolidation Not all software should make it to the next stage Consolidation is as important as innovation in achieving sustainable scientific software

Software Sustainability Institute Take home message Understand and embrace your users and contributors if you want your scientific software to be sustainable and successful Slides:

Software Sustainability Institute Find out more about the SSI Community Engagement (Lead: Shoaib Sufi)  Fellowship Programme Fellowship Programme  Events and Workshops Events and Workshops Consultancy (Lead: Steve Crouch)  Open Call for Projects / Collaborations Open Call for ProjectsCollaborations  Software Evaluation Software Evaluation Policy and Publicity (Lead: Simon Hettrick)  Case Studies / Policy Campaigns Case StudiesPolicy Campaigns  Software and Research Blog Software and Research Blog Training (Lead: Aleksandra Pawlik)  Software Carpentry (300+ students/year) Software Carpentry  Guides and Top Tips GuidesTop Tips Journal of Open Research Software (Editor: Neil Chue Hong) Journal of Open Research Software Collaboration between universities of Edinburgh, Manchester, Oxford and Southampton Slides:

Software Sustainability Institute A national facility for cultivating world-class research through software Become our next collaborators! Website: Twitter: twitter.com/SoftwareSaved Some of our collaborations

Software Sustainability Institute The Foundations of Digital Research Re- search Careers Recognition / Reward Skills and Capability Software Re-usable Re-producible software-evaluation-guide resources/guides software-carpentry training craftsperson-and-scholar software.ac.uk/blog/ what-research- software-community-and-why-should-you-care publish-or-be-damned-alternative- impact-manifesto-research-software Prlić A, Procter JB (2012) Ten Simple Rules for the Open Development of Scientific Software PLoS Comput Biol 8(12): e doi: /journal.pcbi Wilson G, et al. (2013) Best Practices for Scientific Computing Submitted to PNAS.

Software Sustainability Institute Academic Software Should Not Suck Courtesy of C. Titus Brown

Software Sustainability Institute 5 Stars of Research Software Community  There is a community infrastructure Open  Software has permissive license Defined  Accurate metadata for the software Extensible  Usable, modifiable for my purpose Runnable  I can access and run software C O DE R c.f. 5 Stars of Linked Data (Berners-Lee) 5 Stars of Online Journals (Shotton) “Golden Star” Originally by Ssolbergj CC-BY

Software Sustainability Institute Journal of Open Research Software

Software Sustainability Institute People UK Research Computing Ecosystem Computing Communities … Communities … Network/Collaboration Instruments Software Data Centres