Lightweight introduction WS-PGRADE Tutorial Zoltán Farkas zfarkas@sztaki.hu MTA SZTAKI LPDS Lightweight introduction 03/27/2012

Outline 03/27/2012 WS-PGRADE Tutorial Graph creation Concrete workflow creation Concrete workflow configuration Job types and corresponding properties Port properties Certificate handling Submission Log examination Submitted instance management Result evaluation Repository handling (export/import) Advanced features: Parameter study Embedded workflow support Hands-on part 03/27/2012 WS-PGRADE Tutorial

Important high-level graph structures in WS- PGRADE Template Constraints, Comments, Form Generators Graph Jobs, Edges, Ports Repository Item Application OR Project OR, Workflow part (G,T,CW) Concrete Workflow Algorithms, executable Resource references, Inputs Legend: a b a must reference b a b a may reference b Workflow Instance Running state, Outputs 03/27/2012 WS-PGRADE Tutorial

Workflow Graphs – The Graph Editor I. Create WF structure Add new jobs Add ports to jobs Define connections between jobs 03/27/2012 WS-PGRADE Tutorial

Workflow Graphs – The Graph Editor II. Define very basic job and port properties 03/27/2012 WS-PGRADE Tutorial

Concrete workflow creation Based on a graph, template or another WF Set the name Add an optional note (Type is always zen) 03/27/2012 WS-PGRADE Tutorial

Concrete WF configuration I. 03/27/2012 WS-PGRADE Tutorial

Concrete WF configuration II. Set job properties: Type: workflow, service, binary, cloud Type-dependent properties Set port properties: Internal filename (as opened by the job) Data source Port type 03/27/2012 WS-PGRADE Tutorial

Concrete WF configuration III. Select job type (execution model) Set type-dependent properties 03/27/2012 WS-PGRADE Tutorial

Concrete WF configuration IV. Set internal filename Set source (for non-channel ports) 03/27/2012 WS-PGRADE Tutorial

Certificate handling I. Manage X.509 certificates: Upload to MyProxy server Get new proxy from a MyProxy server Manage proxies on MyProxy servers 03/27/2012 WS-PGRADE Tutorial

Certificate handling - Upload Set information Click upload 03/27/2012 WS-PGRADE Tutorial

Certificate handling - Download Set MyProxy access information Download Assign to grid 03/27/2012 WS-PGRADE Tutorial

Really simple: click Submit Submission Really simple: click Submit 03/27/2012 WS-PGRADE Tutorial

Check submitted instance 03/27/2012 WS-PGRADE Tutorial

Check finished jobs I. 03/27/2012 WS-PGRADE Tutorial

Workflow instance management 03/27/2012 WS-PGRADE Tutorial

Examine the resulting zip file Result evaluation Click “get Outputs” Examine the resulting zip file 03/27/2012 WS-PGRADE Tutorial

Share your workflow! - Export Set type (application is OK) Add some fancy notes Click “Export” 03/27/2012 WS-PGRADE Tutorial

Use others' workflows! - Import I. Select type Click refresh And... 03/27/2012 WS-PGRADE Tutorial

Use others' workflows! - Import II. Override properties you like Click “Import” 03/27/2012 WS-PGRADE Tutorial

Advanced part – Parameter Study features Parameter study helpers: Generator jobs Parametric input ports Collector jobs Input assignment options: Dot product Cross product 03/27/2012 WS-PGRADE Tutorial

Generator-job-collector type PS Generator type job has at least one multiple output port. Output Number/Max Size: variable, number attribute, port specific. If the number of files produced by a single run is less than Output Number: the generated files will be encountered cyclically in further jobs. If the number of files exceed the Output Number the exceeding files will be not used Generator Generator Generator *K=3 *K=2 *K=4 1 run 1 run 1 run 1 1 1 1 1 1 2 2 2 2 2 3 3 1. run 1. run 1. run 2. run 2. run 2. run 3. run 3. run 4. run 03/27/2012 WS-PGRADE Tutorial

Set N during WF configuration: Parametric input port Simply upload a ZIP file called paramInputs.zip containing files: 0, 1, …, N Set N during WF configuration: 03/27/2012 WS-PGRADE Tutorial

Optional Conditions to Submit a Job: Collector Port-example Step 1 To set this option the input port must be “channel” Step 2 To access to the setting of this option the features of the Parametric Input must be in “View” state Step 3 To make the Port to be a Collector one the setting of “Waiting” must be “All”

Typical Parameter Sweep workflow 03/27/2012 WS-PGRADE Tutorial

Example parameter study workflow Generator Parametric inputs Cross product Dot product Collector port 03/27/2012 WS-PGRADE Tutorial

Advanced part – Embedded workflows A job can be a workflow! Rule: a concrete workflow created from a template may be embedded Port assignments have to be set: 03/27/2012 WS-PGRADE Tutorial

Embedded workflow support To ensure the compatibility of interfaces the embedded workflow must be defined by a Template The dummy job whose execution will be substituted by the call of the embedded one Original Workflow 03/27/2012 WS-PGRADE Tutorial

Questions? Let's move to the hands-on part! 03/27/2012 WS-PGRADE Tutorial