The Taverna Software Suite Prof Carole Goble FREng FBCS CITP The University of Manchester, UK

The Taverna Suite of Tools Client User Interfaces User Interfaces Workflow Repository Service Catalogue Third Party Tools Web Portals / Gateways Activity and Service Plug-in Manager Workflow ProvenanceWorkflow Server Secure Service Access OAuth1 & 2, username/password, certificates. Workflow Engine Virtual Machine Prog APIs Command Line Player Workflow Components WorkbenchTaverna Lite Interaction Server

VPH-Share Project Models of Human Physiology Eagle Genomics & NHS Next Generation Sequencing based Patient Diagnostics Astronomy & HelioPhysics Library Doc Preservation Systems Biology of Micro-Organisms OpenTox Project Chemistry Development Kit Drug Toxicity BioDiversity Invasive Species Modelling Metagenomics

5820 members, 304 groups, 2415 workflows, 604 files and 229 packs (research objects)

biovel.myexperiment.org

Taverna Engine Execution Scufl2 language Functional dataflow, simple control flows, implicit iteration Linking services and tools Data movement, monitoring, staging, reference “In Workflow Programming” Beanshell scripting Provenance collection: W3C PROV(+) format Plug-in Framework –Infrastructures: Grid, HPC, Web Services (SOAP, REST) –Domain: CDK, BioMart, VOTable, SADI –Common Tools: Excel Spreadsheets, Google Refine, R OAuth security plug-in

Taverna Pro-Workbench Desktop application GUI Intermediate results views Gateway to BioCatalogue and myExperiment Plug-in Framework

Workflow Blocks made of a workflow Well described Well behaved Well looked after Agreed fail Agreed formats in and out Agreed provenance Deposited in myExperiment Grouped into families Components

Workflow Blocks made of a workflow Well described Well behaved Well looked after Agreed fail Agreed formats in and out Agreed provenance Deposited in myExperiment Grouped into families Components

Workflow Blocks made of a workflow Well described Well behaved Well looked after Agreed fail Agreed formats in and out Agreed provenance Deposited in myExperiment Grouped into families Components

Desktop Client Data Centric Interface BIFI (Beautiful Interfaces for Inputs) Taverna Workbench Plug-in, GUI definition language

Data services Vanilla Taverna –Domain data type neutral AstroTaverna plug-in –IVOA data services –VOTables PyWPS plug-in –Exposes OGC-compliant Web Processing Services that can handle large data

Taverna Server Multiple clients, Multi-user SOAP and REST API Server Host Taverna Server “Client” Taverna Server “Client” Taverna Server Front End TavServ Back End Service

Taverna Server Family Taverna Server –Multiple clients, Multi-user –SOAP and REST API Taverna Server Amazon Machine Image –Bundled R server, Atom feed server –Multiple instances in Amazon Cloud and as required, for multiple users/uses and different security scenarios Taverna Virtual Machine Taverna Command Line Bundled Servers

Calling Grid/Cloud Services Expose services/tools as WSDL/REST services –HELIO: Fixed host name –VPH-Share: Services running on dynamically started instances –SZTAKI Desktop Grid – BOINC/Debian Package Specific service/extension to Taverna –UNICORE plugin: Ask grid what services are available, Include services in a workflow, Invoke services on the grid see talk by Shahbaz Memon Library to control job submission to grid –PBS plugin: beanshells in a workflow include invocations of jobs –KnowARC plugin: Advanced Resource Connector to submit jobs to NorduGrid

Web interface Input SNPs Results Storage (S3) Ensembl (mySQL) Cache (S3) Taverna Server Workflow engine orchestrator e-Hive other Taverna Common API Application specific tools and Web Services WS Tool WS All user interaction via web interface User data stored in the Cloud Data for all tools and Web Services stored in the Cloud Unified access to different workflow engines with our common REST API Tools and Web Services for each workflow are installed together for easy replication Cloud Analytics for Life Sciences

Tavoop—Taverna & Hadoop Compiles Taverna Workflow to collection of Hadoop jobs Designed for handling very large amounts of data –Overhead to using Hadoop, but wins if enough data –Data ingest (expensive step) must have already been done Supports Taverna Platform Execution interface Parallelisable service types labs.org/display/SP/PPLhttp://wiki.opf- labs.org/display/SP/PPL Hadoop Cluster Taverna Execution Interface Tavoop Compiler Portal (Taverna Player) GUI Application (Workbench)

Interacting with a workflow Many workflows need user interaction A workflow on a server does not need to be “press a button and wait” –VPH-Share opens a VNC connection to the spawned instance. Taverna Interaction Service –Users interact with a workflow (wherever it is running) in a web browser. –Interaction Service Plug-in in workbench

URLs and Frames

Taverna Tool Spectrum Technical Computational Scientist Domain Scientist Workbench Components Lite Domain-Specific Website / Tool / Portal Workflow Visibility Concept Knowledge TavernaDomain HighLow PlayerCommand Line

Taverna Client Family Java library / Ruby GEM Run a Taverna workflow in another workflow system e.g. Galaxy tools Command line Simple Taverna “player” –Fixed workflow Upload & run workflows and choose data –Universitat Pompeu Fabra’s “Soaplab MajorDomo” –Taverna Lite

Tavaxy Taverna-Galaxy

Taverna-Lite Generic Web-based Client Hide complexity Access to datasets Upload and interact with workflows Build Portal Homepage User-Sessions Workflow Management Run Management Server Credentials Uses Components for simpler assembly and workflow edits

Web apps to create and run workflows Service Chaining Editor Pete Walker et al Plymouth Marine Laboratory For chaining OGC Web Processing Service geospatial Web services

Web apps to create and run workflows Online Taverna Dr Vadim Surpin and Vitaly Sharanutsa Institute for Information Transmission Problems of Russian Academy of Sciences (IITP RAS) An online, in- browser application for assembling and running Taverna Workflows over a HPC platform Software Sustainability Institute Booth Dr Vadim Surpin

Upload workflow by URL Online Taverna

Taverna 3 Beta July 2013

Summary Taverna Suite for interactive and batch workflows Flexible Plug-ins and Flexibly Plugged-in Themed Taverna Establishing Taverna Foundation We welcome collaboration/contribution

Learn more…. myGrid – Taverna – myExperiment – BioCatalogue – Wf4ever – SCAPE – Software Sustainability Institute – BioVeL –