Scientific workflow in Kepler – hands on tutorial Marcin Plociennik (PSNC) Tomasz Zok (PSNC) A. Gomez (CIEMAT) 5/15/2018 EGI User Forum 2011

http://serpens.psnc.pl/uf2011 Agenda Kepler overview Main Kepler's concepts (actors, directors, etc.) Serpens for Kepler Hands on tutorials http://serpens.psnc.pl/uf2011 5/15/2018 EGI User Forum 2011

What you should know after this tutorial the general idea of Kepler how to build simple workflows how to build simple controls like loops how to use grid actors 5/15/2018 EGI User Forum 2011

About Kepler www.kepler-project.org Scientific Workflow System Builds upon the open-source Ptolemy II framework Current release: 2.1 (past 2.0, 1.0 – May 2008), initiated 2003 Allows scientists to visually design and execute scientific workflows Actor-oriented model with directors acting as the main workflow engine Enables different models of computation Workflows are saved as XML files - can easily be shared/published Kepler is supported by the NSF-funded Kepler/CORE team, which spans several of the key institutions that originated the Kepler project: UC Davis, UC Santa Barbara, and UC San Diego useful across disciplines: Ecology, Engineering, Geology, Physics, ... 5/15/2018 EGI User Forum 2011

About Kepler 5/15/2018 EGI User Forum 2011

Kepler is a Java based tool for building and executing workflows About Kepler Kepler is a Java based tool for building and executing workflows Kepler allows you to use existing elements Kepler allows you to extend it by creating new elements Kepler is an Open Source project (BSD License) Kepler can support you with: building and executing workflows executing tasks locally executing tasks within distributed environments http://kepler-project.org 5/15/2018 EGI User Forum 2011

How can I build workflow Kepler allows you to build workflows using standard elements: Workflow Director Actor Port Relation 5/15/2018 EGI User Forum 2011

Workflow allows to compose data flow between elements Accessing workflow Workflow allows to compose data flow between elements Elements can be Dragged and Dropped into Workflow's area After workflow is ready it can be started two ways: directly from the GUI from the command line 5/15/2018 EGI User Forum 2011

Accessing workflow 5/15/2018 EGI User Forum 2011

Accessing Actors and Directors Available elements can be accessed through the component's browser Elements can be Dragged and Dropped into Workflow's area You can search for elements if you don't know the exact name of the component You can add your own components into library 5/15/2018 EGI User Forum 2011

Accessing Actors and Directors 5/15/2018 EGI User Forum 2011

Accessing Supplementary Components Components that bind all elements together are accessible from toolbar Ports, relations, can be Dragged and Dropped into Workflow's area You can easily Start/Stop workflow directly from the GUI using toolbar icons 5/15/2018 EGI User Forum 2011

Accessing Supplementary Components 5/15/2018 EGI User Forum 2011

Main components 5/15/2018 EGI User Forum 2011

Directors are responsible for the orchestration of the workflow. Directors can provide various means of execution: CT Director (Continuous Time) DDF Director (Dynamic Data flow) SDF Director (Sequential Data flow) PN Director (Process Network) - parallel executions … 5/15/2018 EGI User Forum 2011

Actors perform actions basing on input ports and parameters 5/15/2018 EGI User Forum 2011

You can use already existing actors that are available within Kepler You can import actors from the other sources into Kepler You can develop your own actors and add them to the Kepler 5/15/2018 EGI User Forum 2011

Concept of composite actor 5/15/2018 EGI User Forum 2011

Composite actors allow to encapsulate workflows Composite actors can be copy-pasted between workflows Composite actors can be exported as components 5/15/2018 EGI User Forum 2011

Developed under EU FP7 Euforia project Used by Fusion community Serpens for Kepler Serpens: Actors, workflows, templates for using grid middleware: gLite Unicore QCG Accessing cloud: OpenNebula Amazon EC2 Developed under EU FP7 Euforia project Used by Fusion community Support for new workflow scenarios as a part of SA3 http://serpens.psnc.pl 5/15/2018 EGI User Forum 2011

Your feedback would be appreciated - please visit us on line at: User Forum 2011 Tutorial Survey Your feedback would be appreciated - please visit us on line at: https://www.egi.eu/online_surveys/index.html 11 - 14 April 2011

Thank you for your attention And now Let’s move to Hands on part http://serpens.psnc.pl/uf2011 5/15/2018 EGI User Forum 2011