P-GRADE overview and introduction: workflows & parameter sweeps (Advanced features) Gergely Sipos MTA SZTAKI www.lpds.sztaki.hu/pgportal pgportal@lpds.sztaki.hu.

Slides:

Advertisements

Similar presentations

1 CEOS WGISS Meeting, May 8-12, 2006, Budapest MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences

Advertisements

1 P-GRADE Portal and GEMLCA Legacy Code Architecture Peter Kacsuk MTA SZTAKI

LPDS MTA SZTAKI Equation Solver A Grid solution Important hint: A follow up example demonstrating the Parameter Study solution for the mass calculations.

P. Kacsuk, G. Sipos, A. Toth, Z. Farkas, G. Kecskemeti and G. Hermann P. Kacsuk, G. Sipos, A. Toth, Z. Farkas, G. Kecskemeti and G. Hermann MTA SZTAKI.

P-GRADE and WS-PGRADE portals supporting desktop grids and clouds Peter Kacsuk MTA SZTAKI

06/08/10 PBS, LSF and ARC integration Zoltán Farkas MTA SZTAKI LPDS.

1 MTA SZTAKI Application development on EGEE with P-GRADE Portal Gergely Sipos

MTA SZTAKI Hungarian Academy of Sciences Grid Computing Course Porto, January Introduction to Grid portals Gergely Sipos

WS-PGRADE: Supporting parameter sweep applications in workflows Péter Kacsuk, Krisztián Karóczkai, Gábor Hermann, Gergely Sipos, and József Kovács MTA.

Globus Toolkit 4 hands-on Gergely Sipos, Gábor Kecskeméti MTA SZTAKI

1 MTA SZTAKI Hungarian Academy of Sciences Grid application support by the P-GRADE Portal Peter Kacsuk.

1 Application Specific Module for P-GRADE Portal 2.7 Application Specific Module overview Akos Balasko MTA-SZTAKI LPDS

1 portal.p-grade.hu További lehetőségek a P-GRADE Portállal Gergely Sipos MTA SZTAKI Hungarian Academy of Sciences.

1 P-GRADE Portal: Towards a User-friendly Grid Environment Tamas Kiss Centre for Parallel Computing.

An Introduction to Designing and Executing Workflows with Taverna Aleksandra Pawlik University of Manchester materials by Dr Katy Wolstencroft and Dr Aleksandra.

Christopher Jeffers August 2012

1 portal.p-grade.hu Further information on P-GRADE Gergely Sipos MTA SZTAKI Hungarian Academy of Sciences.

07/06/11 New Features of WS-PGRADE (and gUSE) 2010 Q Q2 Miklós Kozlovszky MTA SZTAKI LPDS.

SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI Creating the Autodock gateway from WS-PGRADE/gUSE and making it cloud-enabled.

Enabling Grids for E-sciencE EGEE-II INFSO-RI E-Marketplace Model Integrated with Logistics MTA SZTAKI Hungarian Academy of Sciences International.

1 Next steps with EGEE and P-GRADE Portal Gergely Sipos MTA SZTAKI EGEE Training and Induction.

INFSO-RI Enabling Grids for E-sciencE Supporting legacy code applications on EGEE VOs by GEMLCA and the P-GRADE portal P. Kacsuk*,

Parameter Study (PS) Principles & Practices Note This document uses as starting example the non PS case discussed in document EquationSolver.ppt If you.

Supported by EU projects 12/12/2013 Athens, Greece Open Data in Agriculture Hands-on with data infrastructures that can power your agricultural data products.

An Introduction to Designing and Executing Workflows with Taverna Aleksandra Pawlik materials by: Katy Wolstencroft University of Manchester.

1 Advanced features of the P-GRADE portal Peter Kacsuk, Gergely Sipos Peter Kacsuk, Gergely Sipos MTA.

AgINFRA science gateway for workflows and integrated services 07/02/2012 Robert Lovas MTA SZTAKI.

EGEE-III INFSO-RI Enabling Grids for E-sciencE Application Porting Support in EGEE Gergely Sipos MTA SZTAKI EGEE’08.

MTA SZTAKI Hungarian Academy of Sciences Introduction to Grid portals Gergely Sipos

1 P-GRADE Portal: a workflow-oriented generic application development portal Peter Kacsuk MTA SZTAKI, Hungary Univ. of Westminster, UK.

The EDGeS project receives Community research funding 1 Porting Applications to the EDGeS Infrastructure A comparison of the available methods, APIs, and.

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Status report on Application porting at SZTAKI.

Parameter Study Principles & Practices. What is Parameter Study? Parameter study is the application of a single algorithm over a set of independent inputs:

EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Services for advanced workflow programming.

1 P-GRADE Portal: An easy to use graphical interface for Globus and EGEE Grids.

Parameter Study Principles & Practices. Outline Data Model of the PS Part I Simple PS –Generating simple PS Workflow by introducing PS Input port – using.

1 P-GRADE Portal tutorial at EGEE’09 Introduction to hands-on Gergely Sipos MTA SZTAKI EGEE.

1 Practical information for the GEMLCA / P-GRADE hands-on Gergely Sipos On behalf of: MTA.

1 P-GRADE Portal tutorial at EGEE'09 Gergely Sipos MTA SZTAKI EGEE Training and Induction.

1 Practical information for the GEMLCA / P-GRADE hands-on Tamas Kiss University of Westminster.

11 Introduction to EDGI Peter Kacsuk, MTA SZTAKI Start date: Duration: 27 months EDGI.

1 portal.p-grade.hu Workflow and parameter study management by P-GRADE Portal Gergely Sipos MTA SZTAKI Hungarian Academy of Sciences.

EGEE-II INFSO-RI Enabling Grids for E-sciencE P-GRADE overview and introduction: workflows & parameter sweeps (Advanced features)

1 Other features and next steps Gergely Sipos MTA SZTAKI Hungarian Academy of Sciences.

1 Support for Parameter Study applications in the P-GRADE Portal Cevat Şener Dept. Of Computer Engineering, METU.

Application Specific Module Tutorial Zoltán Farkas, Ákos Balaskó 03/27/

1 P-GRADE Portal hands-on Gergely Sipos MTA SZTAKI Hungarian Academy of Sciences.

1 Further information and next steps Further information and next steps Gergely Sipos MTA SZTAKI

07/02/2012 WS-PGRADE/gUSE in use Lightweight introduction Zoltán Farkas MTA SZTAKI LPDS.

WS-PGRADE/gUSE in use Advance use of WS- PGRADE/gUSE gateway framework Zoltán Farkas and Peter Kacsuk MTA SZTAKI LPDS.

1 DIRAC Data Management Components A.Tsaregorodtsev, CPPM, Marseille DIRAC review panel meeting, 15 November 2005, CERN.

1 Support for parameter study applications in the P-GRADE Portal Gergely Sipos MTA SZTAKI (Hungarian Academy of Sciences)

InSilicoLab – Grid Environment for Supporting Numerical Experiments in Chemistry Joanna Kocot, Daniel Harężlak, Klemens Noga, Mariusz Sterzel, Tomasz Szepieniec.

EGEE-II INFSO-RI Enabling Grids for E-sciencE Application specific portlet other portal features and next steps Miklos Kozlovszky.

Tamas Kiss University Of Westminster

Flowbster: Dynamic creation of data pipelines in clouds

How to connect your DG to EDGeS? Zoltán Farkas, MTA SZTAKI

Introduction to gUSE and WS-PGRADE portal

P-GRADE Portal tutorial

Grid Application Support Group Case study Schrodinger equations on the Grid Status report 16. January, Created by Akos Balasko

Introduction to P-GRADE Portal hands-on

P-GRADE and GEMLCA.

Recap: introduction to e-science

Remote Api Tutorial How to call WS-PGRADE workflows from remote clients through the http protocol?

Lightweight introduction

Lightweight introduction

MIK 2.1 DBNS - introduction to WS-PGRADE, 2013

Application development on EGEE with P-GRADE Portal

Introduction to the SHIWA Simulation Platform EGI User Forum,

Workflow level parametric study support by the P-GRADE portal

Presentation transcript:

P-GRADE overview and introduction: workflows & parameter sweeps (Advanced features) Gergely Sipos MTA SZTAKI www.lpds.sztaki.hu/pgportal pgportal@lpds.sztaki.hu

Parameter Sweep applications Typical usage of the Grid is that the same application is executed with many different parameters P-GRADE portal from release 2.5 supports this functionality The next part of the lecture shows how parameter sweep (PS) applications are supported by P-GRADE portal

Scaling up a workflow to a parameter study with P-GRADE Complete workflow Input file set in an LFC directory (e.g. lfc:/grid/gilda/sipos/myinputs) Output file set in an LFC directory (e.g. lfc:/grid/gilda/sipos/myoutputs)

Turning a WF into a parameter study By turning at least one of the open input ports into a “PS Input port” the WF is turned into a Parameter Study WF

PS Input Port of Simple PS Remote file Directory instead of FILE reference

Simple PS Activity 2: placement of result Menu item PS Properties is available in the Workflow menu Properties of the VO File Catalog The Output directory will contain the ZIP-ped result files of the Workflow executions One SE of the VO – Choose a reliable one!

Executing PS workflows PS Details for parameter sweep workflow applications

Workflow Manager List PS Details view showing eWF-s Status of complete application Statistics of workflows generated from the application Workflow list shows the state of the generated Workflows.

Advanced PS WFs in P-GRADE Portal

Advanced parameter studies Generator component(s) Initial input data Generate or cut input into smaller pieces Complete workflow Collector component(s) Aggregate result Files in an LFC directory (e.g. /grid/gilda/sipos/myinputs) Files in an LFC directory (e.g. /grid/gilda/sipos/myoutputs)

Concept of advanced parameter study workflows GEN Generator part generates the input parameter space SEQ SEQ SEQ SEQ Parameter study part COLL Collector part evaluates and integrates the results

The generic PS workflow application First phase: executing the generators Second phase: executing the “real” processor jobs for each input file combination Third phase: executing the collectors

Detailed view of a PS workflow Generator job(s) Overall statistics of workflow instances Workflow instances Collector job(s)

PS practical