1. WS-PGRADE/gUSE in use Advance use of WS- PGRADE/gUSE gateway framework Zoltán Farkas and Peter Kacsuk zfarkas@sztaki.hu MTA SZTAKI LPDS

2. 07/02/2012 WS-PGRADE/gUSE in use 2 Advanced part – Parameter Study features Parameter study concepts: Generator jobs Parametric input ports Collector jobs Types of workflow nodes: Ordinary Generator Collector Input assignment options: Dot product Cross product

3. PS_1_2 PS_1_1 PS_2_2 PS_2_1 PS_1_1PS_2_1PS_1_2PS_2_1PS_1_1PS_2_2PS_1_2PS_2_2 WF or job One or more Parameter Sets are defined. Each has different parameters. (Sets are represented by remote file directories, parameters by files) PS DATA FLOW Content of Parameter Sets are combined (Descartes product) The workflow/job is executed as many times as many elements are in the Descartes product of input sets Result_1_1 Result_2_1 Result_1_2 Result_2_2 Store the results of the calculations (Results are represented by remote files of one or more common directories)

4. Applying several Generators and Collectors Gen1 Gen2 Input Col l Output Col l Output Col l Output

5. PS application with generator and collector nodes GEN SEQ COLL SEQ PS jobs: To be executed in Grids, Desktop Grids or Clouds Generates input parameter space Evaluates the results of the simulation

6. Generator Next Job 07/02/2012 WS-PGRADE/gUSE in use 6 The Generator-job has at least one generator output port. The executable produces more than one files at this port during a single job step. 0 12 0 12 1. run 2. run 3. run single run S Generator node and generator output port

7. Dataflow programming concept for appl. developers Cross & dot product data- pairing Concept similar to Taverna All-to-all vs. one-to-one pairing of data items Any component can be generator, PS node or collector, no ordering restriction Conditional execution based on equality of data embedding, recursion 5 5 1000 50 20 5000 1 1 11007 tasks

8. 07/02/2012 WS-PGRADE/gUSE in use 8 Parametric input port Simply upload a ZIP file called paramInputs.zip containing files: 0, 1, …, N Set N during WF configuration:

9. Optional Conditions to Submit a Job: Collector Port-example Step 1 To set this option the input port must be “channel” Step 2 To access to the setting of this option the features of the Parametric Input must be in “View” state Step 3 To make the Port to be a Collector one the setting of “Waiting” must be “All”

10. 07/02/2012 WS-PGRADE/gUSE in use 10 Typical Parameter Sweep workflow RandGen Call Collector single run 1. run 2. run 3. run 0 12 0 12

11. 07/02/2012 WS-PGRADE/gUSE in use 11 Concrete WF configuration IV. Set internal filename Set source (for non-channel ports)

12. 07/02/2012 WS-PGRADE/gUSE in use 12 Example parameter study workflow Generator Parametric inputs Cross product Dot product Collector port

13. 07/02/2012 WS-PGRADE/gUSE in use 13 Advanced part – Embedded workflows A job can be a workflow! Rule: a concrete workflow created from a template may be embedded: Port assignments have to be set:

14. 07/02/2012 WS-PGRADE/gUSE in use 14 Embedded workflow support Original Workflow Embedded Workflow To ensure the compatibility of interfaces the embedded workflow must be defined by a Template The dummy job whose execution will be substituted by the call of the embedded one

15. 07/02/2012 WS-PGRADE/gUSE in use 15 Questions?