1. Scientific Workflow Support in the PL-Grid Infrastructure with HyperFlow Bartosz Baliś, Tomasz Bartyński, Kamil Figiela, Maciej Malawski, Piotr Nowakowski, Maciej Pawlik, Paweł Suder, Bartosz Wilk, Marian Bubak AGH University of Science and Technology Department of Computer Science & ACC CYFRONET AGH KU KDM 2015, 12-13.03.2015

2. Motivation & objectives Scientific workflow = scientific application as a graph of activities (multi-step ‘pipeline’) Arguably the most widely adopted programming & execution model for scientific applications Most workflow systems are oriented on domain scientists  Simplified programming approach (graphical graph composition)  Limited programming capabilities In many cases workflow developers are also skilled programmers  They want workflow development experience similar to ‘regular’ programming  They want capabilities of a mainstream programming language / ecosystem HyperFlow: a lightweight workflow programming and execution environment deployed as a service in PL-Grid Simple high-level workflow description + low-level programming capabilities for advanced developers Skilled programmers can be as productive as in any mainstream programming language

3. HyperFlow: overview function getPathWayByGene(ins, outs, config, cb) { var geneId = ins.geneId.data[0], url =... http({"timeout": 10000, "url": url }, function(error, response, body) {... cb(null, outs); }); }

4. HyperFlow scientific workflow service: PL-Grid deployment Running a workflow – simple command line client hflowc run HyperFlow runtime components and application VMs are started on-demand in the cloud Input and output files are automatically transferred from user directory to VMs and back contains: File workflow.json (workflow graph File workflow.cfg (workflow config) Optionally: file functions.js (advanced workflow activities) Input files