1. Software Sustainability Institute www.software.ac.uk Referencing software to make it sustainable: what, why and how? SciencePAD Persistent Identifiers workshop 30 January 2013, CERN Neil Chue Hong (@npch) N.ChueHong@software.ac.uk Unless otherwise indicated slides licensed under

2. Why is this important? An animated explanation Featuring a frustrated panda http://tinyurl.com/datasharingpanda

3. Software Sustainability Institute www.software.ac.uk Software is no longer easy to define, let alone sustain

4. Software Sustainability Institute www.software.ac.uk Novel reuse of public sector data http://www.mysociety.org What do we sustain: - Map? - Software that creates map? - Software that uses map?

5. Software Sustainability Institute www.software.ac.uk What do we choose to identify: - Workflow? - Software that runs workflow? - Software referenced by workflow? - Software dependencies? What’s the minimum citable part? Boundary

6. Software Sustainability Institute www.software.ac.uk Algorithm Function Program Library / Suite / Package … Granularity

7. Software Sustainability Institute www.software.ac.uk Versioning Personal v1 Personal v1 Personal v2 Personal v2 Personal v3 Personal v3 Personal v2a Personal v2a Public v1 Public v1 Personal v3a Personal v3a Personal v2a Personal v2a Public v2 Public v2 Public v3 Public v3 Why do we version? - To indicate a change - To allow sharing - To confer special status

8. Software Sustainability Institute www.software.ac.uk Authorship Authorship Which authors have had what impact on each version of the software? Which authors have had what impact on each version of the software? Who had the largest contribution to the scientific results in a paper? Who had the largest contribution to the scientific results in a paper?http://beyond-impact.org/?p=175 OGSA-DAI projects statistics from Ohloh

9. Authorship http://beyond-impact.org/?p=175 http://beyond-impact.org/?p=175 Which authors have had what impact on each version of the software? Which authors have had what impact on each version of the software? Who is responsible for each component when authors leave? Who is responsible for each component when authors leave?

10. Software Sustainability Institute www.software.ac.uk Software sustainability is the ability to continue to use, support, maintain, and evolve software

11. Software Sustainability Institute www.software.ac.uk From xkcd.com

12. Software Sustainability Institute www.software.ac.uk 12 RCUK Gateway to Research

13. Software Sustainability Institute www.software.ac.uk The Software Sustainability Institute A national facility for cultivating world- class research through software Better software enables better research Software reaches boundaries in its development cycle that prevent improvement, growth and adoption Providing the expertise and services needed to negotiate to the next stage Developing the policy and tools to support the community developing and using research software Supported by EPSRC Grant EP/H043160/1

14. Software Sustainability Institute www.software.ac.uk SSI: Long Term Goals Provision of useful, effective services for research software community Development and sharing of research community intelligence and interactions Promotion of research software best practice Mantra:  Keep the software in its respective community  Work with the community, to increase ability  Don’t introduce dependency on SSI as the developer  Expand and exploit networks and opportunities

15. Software Sustainability Institute www.software.ac.uk SSI Drivers and Themes Two key drivers which cause people to seek the SSI’s advice:  They want to be more productive in their research  They don’t want to be embarrassed by appearing worse than their peers Broadly, our work falls into a few key themes:  Developing the scientific computing / software development skill base  The role and reward of software in research  Recognition of software career paths  Reproducible research

16. Software Sustainability Institute www.software.ac.uk The Foundations of Digital Research Re- search Careers Recognition / Reward Skills and Capability Software Re-usable Re-producible www.software.ac.uk/ software-evaluation-guide resources/guides software-carpentry training www.software.ac.uk/blog/ 2012-11-09-craftsperson-and-scholar software.ac.uk/blog/2012-08-16-what-research- software-community-and-why-should-you-care www.software.ac.uk/blog/2011-05-02- publish-or-be-damned-alternative- impact-manifesto-research-software Prlić A, Procter JB (2012) Ten Simple Rules for the Open Development of Scientific Software PLoS Comput Biol 8(12): e1002802. doi:10.1371/journal.pcbi.1002802 Wilson G, et al. (2013) Best Practices for Scientific Computing Submitted to PNAS. http://arxiv.org/abs/1210.0530

17. Software Sustainability Institute www.software.ac.uk SSI Organisation Community Engagement (Shoaib Sufi)  Fellowship Programme Fellowship Programme  Events and Roadshows Consultancy (Steve Crouch)  Open Call for Projects Open Call for Projects  Software Evaluation Software Evaluation Policy and Publicity (Simon Hettrick)  Guides and Case Studies Guides and Case Studies  Best Practice and Policy Training (Mike Jackson)  Software Carpentry Software Carpentry  Software Surgeries Collaboration between universities of Edinburgh, Manchester, Oxford and Southampton. 9.5 FTEs for 5 yrs supplemented by additional project funding.

18. Software Sustainability Institute www.software.ac.uk Case Study: Ligand Binding Centre for Computational Chemistry, Bristol  New methods for rapid MC sampling of biomolecular systems modelled using QM/MM  Developed two codes ProtoMS (F77) + Sire (C++)  Water-Swap Reaction Coordinate method to calculate absolute protein-ligand binding free energies SSI’s work is helping to scale development  ProtoMS and Sire both single developer codes  ASPIRE/ACQUIRE framework has multiple devs Split architecture between ASPIRE (adaptive multiresolution hybrid MD simulation) and ACQUIRE (WorkPacket scheduling system with optimisation for time to result vs “green-ness” http://www.siremol.org/adaptive_dynamics

19. Software Sustainability Institute www.software.ac.uk Case Study: Fusion Plasma Culham Centre for Fusion Energy  GS-2 used to study low-frequency turbulence in magnetized plasma  No common visualisation across different groups  Deliver mutually agreeable framework that can be extended easily and can be maintained by the small fusion community SSI’s work means the software can be used between groups  Simplified & enhanced plasma visualisation tool Based on ParaView o/s tool For simulations using GS-2 o/s package  Aim to allow CCFE to contribute back to GS-2 community  “I am very confident the tool will be invaluable” Colin Roach, CCFE http://www.software.ac.uk/who-do-we-work/culham-centre-fusion-energy http://www.ccfe.ac.uk/

20. Software Sustainability Institute www.software.ac.uk Case Study: Brain Imaging Brain Research Imaging Centre, Edinburgh  Develop PrivacyGuard software, a DICOM image deidentification toolkit  Created software to support new multispectral colouring modulation and variance identification technique (“MCMxxxVI”) to identify white matter lesions that are indicative of declining cognitive ability  BRIC are not principally software developers, but do provide software to other researchers SSI’s work means the software has been reviewed and refactored  Looked at exploitation Usability review, Naming/trademark review  Made it easier for BRIC staff to maintain and develop Move to standard repositories, testing and documentation processes Examination of licencing for MCMxxxVI Extraction and refactoring to create standalone tools http://www.software.ac.uk/who-do-we-work/brain-research-imaging-centre-edinburgh http://www.bric.ed.ac.uk/

21. Software Sustainability Institute www.software.ac.uk Case Study: SARoNGS UK NGS  Previous SARoNGS project aimed to provide a simple mechanism to obtain the security tokens required to access the National Grid Service.  This service uses the UK Access Management Federation to identify people using Shibboleth.  First in ongoing series of “NGS recommends…” SSI’s work means the software has been reviewed and issues corrected  Identified, fixed deep bugs in SARoNGS WMS It is now possible to run SARoNGS jobs successfully on NGS systems Though still needs NGS to roll out more widely…  “Happy with the report, using it to beat people now” David Wallom, OERC & NGS https://cts.ngs.ac.uk/

22. Software Sustainability Institute www.software.ac.uk Case Study: Climate Policy Modelling CIAS team at Tyndall Centre for Climate Change Research, University of East Anglia  Develop linked climate and economic models for detailed analysis  Their software was not ready to be used by other groups One researcher/developer at UEA, several users SSI’s work means the software is robust enough that it can be installed and used by others  Enabled use of the software by the WWFN’s Climascope project and James Cook University Documented software to allow extensions by contributors Made it easier to maintain and backup Added job scheduling to improve modeling throughput New modelling framework enables new models i.e. new science http://www.tyndall.ac.uk/research/cias

23. Software Sustainability Institute www.software.ac.uk Case Study: textual studies TextVRE team at CeRCH, Kings College London  Developed an environment which is used to integrate various tools used in the e-Humanities textual studies lifecycle  Builds on the German TextGrid project, and many other existing tools SSI’s work means the software is can be run “out of the box” – an important requirement for the researchers  Developed a VM image containing the TextVRE installation Improve installation instructions Develop tests to check each installed component Improve modularisation to allow others to contribute and maintain  Feeding back work to TextGrid http://textvre.cerch.kcl.ac.uk

24. Software Sustainability Institute www.software.ac.uk Case Study: NeISS Evaluate impact of traffic control measures over next 5/10/15 years Access baseline demographic data about the city Execute simulation of traffic system and population Visualise simulation outputs Augment with new forms of data Run dynamic models to assess future patterns (congestion, health, social inequality)

25. Software Sustainability Institute www.software.ac.uk Case Study: NeISS 25

26. Software Sustainability Institute www.software.ac.uk Case Study: NeISS 26

27. Software Sustainability Institute www.software.ac.uk Software sustainability requires thriving communities of users and developers: how do we do this?

28. Software Sustainability Institute www.software.ac.uk 5 Stars of Scientific Software We need a 5 stars for software!  Existence – there is accurate metadata that defines the software  Availability – you can access and run the software  Openness – the software has an open permissible license  Linked – related data, dependencies and papers are referenced  Assured – the software provides ways of demonstrating “correctness” c.f. 5 Stars of Linked Data (Berners-Lee) 5 Stars of Online Journals (Shotton)

29. Software Sustainability Institute www.software.ac.uk Discoverable Software? To grow a community around software, first it must be discoverable  For users, wanting to find a solution  For developers, wanting to reuse or extend  For funders, wanting to promote or feature For sustainability  Provide useful information  Make it easier to attract and add contributors  Enable dormant projects to re-activate?

30. Software Sustainability Institute www.software.ac.uk Software Hub Prototyping What information is useful? Do both provider and user benefit? What can be imported from other sites? What metadata must be collected to produce this information? Is it possible or easy to collect? How do people search for software?

31. Software Sustainability Institute www.software.ac.uk Levels of Showcasing Level 1: internal  Has had support from Jisc  Has produced a software output  Metadata is incomplete Level 2: awaiting approval  enough metadata to publish it externally  perhaps not all quality criteria met Level 3: published  meets quality criteria  enough information to allow comparison Level 4: featured  seen as particularly useful, exciting, best of breed etc.  associated screencasts, tutorials to show off Offer incentives to move up the levels

32. Software Sustainability Institute www.software.ac.uk Collecting Software Metadata Can we make the software metadata collection process work?  What are the benefits to provider and user?  Distinction between Project information and Product Information  Difference between information that enables discovery and choice, and the metadata that allows this information to be displayed E.g. “vitality” of project different for developer vs user

33. Software Sustainability Institute www.software.ac.uk Citable Software?

34. Software Sustainability Institute www.software.ac.uk Software Metapapers Create a complete scholarly record including “standard” publication, method, dataset and models, and software  e.g. modelling and simulation, statistical analysis  Enable replay, reproduction and reuse Pragmatic approach is to create a metadata record for the software, and link it to a copy of the software in some storage infrastructure  This is a software metapaper  Peer-review the metadata, not the software Journal of Open Research Software:  http://openresearchsoftware.metajnl.com/ http://openresearchsoftware.metajnl.com/ See: http://openresearchsoftware.metajnl.com/faq/http://openresearchsoftware.metajnl.com/faq/ and the work by B. Matthews et al: The Significant Properties of Software: A Study

35. Software Sustainability Institute www.software.ac.uk SoftwareCite Does the DataCite approach work with software?  What is the cost of minting a DOI?  What level do you mint DOIs for software?  What is the cost of storing the metadata associated with a software asset?  What is the cost of a software asset associated with a DOI disappearing?

36. Software Sustainability Institute www.software.ac.uk Alternative Impact Stories GitHub Repository Starring as a means of recommendation Forking analogous to citing Direct measure of software citation? Requires user IDs, repository IDs, APIs

37. Software Sustainability Institute www.software.ac.uk Software sustainability requires identifiers (and open APIs) for discovery, growth, attribution + reward

38. Why is this important? An animated explanation Featuring a frustrated panda http://tinyurl.com/datasharingpanda

39. Software Sustainability Institute www.software.ac.uk ISSUE TRACKER ISSUE TRACKER AUTOMATED BUILD&TEST AUTOMATED BUILD&TEST CODING STANDARDS CODING STANDARDS CODE REPOSITORY CODE REPOSITORY REAL TIME CHAT DEVELOPER EMAIL LIST DEVELOPER EMAIL LIST GENERAL EMAIL LIST GENERAL EMAIL LIST ANNOUNCE EMAIL LIST ANNOUNCE EMAIL LIST CRM USAGE TRACKING USAGE TRACKING CONTACT EMAIL CONTACT EMAIL REFERENCE MANAGER REFERENCE MANAGER DOMAIN NAME DOMAIN NAME PRIVATE WIKI/SITE SOCIAL NETWORKING SOCIAL NETWORKING PUBLIC WEBSITE PUBLIC WEBSITE BLOG FAQS FORUMS BROADCAST SERVICES BROADCAST SERVICES GOVERNANCE CONTINUOUS INTEGRATION CONTINUOUS INTEGRATION STYLE CHECKER STYLE CHECKER DEPENDENCY REPOSITORY DEPENDENCY REPOSITORY SHARED CALENDAR SHARED CALENDARPROTOTYPERESEARCHPRODUCTSUPPORTEDCOLLABORATION COMMUNICATION TRAINING