The 10 Best Practices for Workflow Design BioVeL M6 Workshop Göteborg, May 10-11, 2012 Kristina Hettne, Marco Roos (LUMC), Katy Wolstencroft, Carole Goble (myGrid) Thanks: BioSemantics Group (LUMC), myGrid team (UoM), Yassene Mohamed, Harish Dharuri (LUMC)

2 Our specialty: Knowledge Discovery Substrates for Knowledge Discovery Disambiguation * Text Mining Applications Predict protein-protein, protein-disease associations, gene prioritization Genotype-phenotype studies, e.g. Huntington’s Disease, Metabolic Syndrome Yours? Applications Predict protein-protein, protein-disease associations, gene prioritization Genotype-phenotype studies, e.g. Huntington’s Disease, Metabolic Syndrome Yours? * Global disambiguation initiative: Methods for Knowledge Discovery

3 Why build good workflows? Introduction Good workflow design = good science!

4 Best Practices for workflow design = Best Practices experimental science + Best Practices software engineering Introduction Best practices for workflow design

5 1 Make a sketch workflow

6 Powerpoint courtersy of Eleni Mina Sketch an Abstract Workflow Best practice 1

7 2 Use modules

8

9 3 Think about the output (and the data in your workflow in general)

10 Think about the output Best practice 3 ?

11 4 Provide example inputs and outputs

12 Taverna 2.3 Recipe Select input/output Select tab ‘Details’ Click ‘Annotation’ Add Example Taverna 2.3 Recipe Select input/output Select tab ‘Details’ Click ‘Annotation’ Add Example Taverna 2.4 Right-click input/output Select ‘Annotation’ Add Example Taverna 2.4 Right-click input/output Select ‘Annotation’ Add Example

13 5 Annotate

14 Annotate Best practice 5 Each component in Taverna can be annotated

15 Annotate and help your users Best practice 5

16 6 Make workflow executable from outside the local environment

17 Make workflow executable by others Best practice 6 »Try it! ›Ask a colleague ›Use an external t2web runner »Tips ›Use Web Services ›If you use local command line tools Install tools on a publicly accessible server (e.g. applies to Rserve) Use system that your users can set up (e.g. BioLinux) How to check that others can execute your workflow? Proof of executability

18 7 Choose services carefully

19 Choose services carefully Best practice 7

20 Choose services carefully Best practice 7

21 8 Reuse existing workflows

22 Invent a new wheel Search the internet The reuse workflow Best practice 8 Check workflows on myExperiment Contact authors Retry Contact authors Retry Use scripts from colleagues Not a best practice, but a tip: know-how is important for reuse Neg. Reuse, Attribute Respect licences Check services on BioCatalogue Pos.

23 9 Advertise

24 Advertise Unique reference for in your papers and for others to cite

25 10 Maintain

26 Maintain Best Practice 10 »Regularly check your workflow ›Ask colleagues »Enable support for maintenance ›Register your workflow on myExperiment ›Register Web Services on »Enable peers to repair: annotate! »Note about versioning ›No need to register all edits on myExperiment: use subversion ›Register important updates on myExperiment Best practices to support maintenance

27 Bonus tip Use common sense as scientist

28 Workflow 74 “Protein Discovery” 2005 Workflow 2876 “Match gene lists by literature” 2012 Preservation of good workflows for future applications Workflow Forever Workflow 2805 “Get Pathway genes” 2012

29 myExperiment 2.0 BioCatalogue Taverna Research Objects Linked Data Methods Protocols for Preservation and Conservation Wf4Ever Outcomes for BioVeL

30 1.Make a sketch workflow 2.Use modules 3.Think about the output 4.Provide example inputs and outputs 5.Annotate 6.Make it executable from outside the local environment 7.Choose services carefully 8.Reuse existing workflows 9.Advertise 10.Maintain Thank you The 10 Best Practices of Workflow Design Thank you for your attention More information:

31 Sneak preview Wf4Ever tooling

32 Workflow jargon Supporting information ›Scientific workflow Paradigm to describe, manage, and share complex scientific analyses ›Workflow system Software to design, execute, and monitor scientific workflows ›Module = nested workflow = workflow in a workflow = workflow component ›Beanshell script A Java-based scripting language. Typically used for data type conversions in Taverna. ›Provenance History or trace of a workflow run. Allows you to look at intermediate data, which workflows and services were run, with what data.