Exposing WS-PGRADE/gUSE for large user communities Peter Kacsuk, Zoltan Farkas, Krisztian Karoczkai, Istvan Marton, Akos Hajnal, Tamas Pinter MTA SZTAKI SCI-BUS is supported by the FP7 Capacities Programme under contract nr RI

SCI-BUS Architecture 2

WS-PGRADE/gUSE Generic-purpose gateway framework 3 Based on Liferay General purpose Workflow-oriented gateway framework Supports the development and execution of workflow-based applications Supports the fast development of domain- specific gateways by a customization technology Most important design aspect is flexibility

Flexibility in exploiting parallelism Each job can be a parallel program – Parallel execution inside a workflow node – Parallel execution among workflow nodes Multiple jobs run parallel – Parameter study execution of the workflow Multiple instances of the same workflow with different data files

Flexibility of using various DCIs Flexible management of Security: –Individual users’ certificate –Robot certificates Flexible access to various types of DCIs: Clusters (PBS, LSF, MOAB, SGE) Cluster grids (ARC, gLite, GT2, GT4, GT5, UNICORE) Supercomputers (e.g. via UNICORE) Desktop grids (BOINC) Clouds (Via CloudBroker Platform) 5

Multi-cloud Integrated WS-PGRADE/CloudBroker Platform to access multi-clouds Cloud Broker Platform WS- PGRADE n IaaS Cloud 1 IaaS Cloud N SEQ WS- PGRADE 1 Supported clouds: Amazon, OpenStack, Eucalyptus, OpenNebula, IBM SaaS solution: Preregistered services/jobs can run IaaS solution: any services/jobs can run 6

WS-PGRADE UI to configure nodes for cloud services 7 Cloud selection During configuration WS-PGRADE queries the software list

Support for commercial clouds with costs Estimated job cost before submission Actual job and workflow cost after execution

Flexibility in data storage access Use Data Avenue Blacktop service –To access data storages in different DCIs –To transfer files among the storages of different DCIs –To upload/download files to/from the storages of different DCIs Data Avenue Liferay portlet to access the data transfer services of Data Avenue Blacktop See details: Currently supported protocols: –http, https, ftp, gsiftp, srm Soon coming protocols: –iRODS, LFC, cloud storage protocols 9

Data Avenue services 10 FS1FS2FS3FSn Data Avenue Blacktop service gLite GT5 Openstack Amazon Data SZTAKI Data XY Data Avenue Portlet WS-PGRADE gateway

Data SZTAKI 11

Data Avenue Liferay portlet 12

Data Avenue Liferay portlet 13

Data Avenue Liferay portlet 14

Data Avenue Liferay portlet 15

J1 J2 J3 J4 J5 DCI1 DCI2 FS2FS5 WS-PGRADE Workflow J: Job FS: File storage system, e.g. gsiftp, iRODS, SRM FS1FS3 Generic data transfer among WS-PGRADE workflow nodes The Data Avenue Blacktop services will be available not only by the Data Avenue portlet but also by the nodes of a WS-PGRADE workflow

SHIWA Repository WF uploadWF download gUSE Portal Cloud 1 OpenNebula Cloud 2 Amazon gUSE WF Repo 17 Cloud n OpenStack gUSE Portal gUSE WF Repo Flexibility for collaboration among community members

er Flexibility in using different workflow systems 18 EMI Grids Glob us Cloud Bio1 Bio2 BioN Cyberspace WS-PGRADE Gateways Infrastructures Combining SCI-BUS and SHIWA technologies (supported by ER- Flow) users can access and use many WFs and many infrastructures in an interoperable way no matter which is their home WF system Taverna Galaxy Kepler WF systems

Flexibility of gateway types and user views 1.Generic purpose gateways for clouds (workflow view) –Core WS-PGRADE/gUSE (e.g. Greek NGI) 2.Generic purpose gateway for specific technologies (workflow view) –SHIWA gateway for workflow sharing and interoperation 3.Domain-specific science gateway instance –Autodock gateway (end-user view) –Swiss proteomics portal (customized GUI using ASM API) –VisIVO Mobile (use of Remote API) 19

Some examples of SCI-BUS domain-specific gateways 20

VisIVO Astrophysics Science Gateway Architecture

VisIVO Mobile

23 SCI-BUS vision to build science gateways by communities SHIWA Workflow repository SHIWA Workflow repository fMRI MRI simulator Sequencing …. NonLinSyst simulator NonLinSyst simulator Application Domain specific Gateway Scientists Gateway Operators DCI 1 DCI 2 DCI n SCI-BUS Generic Framework based on WS- PGRADE and Liferay SCI-BUS Generic Framework based on WS- PGRADE and Liferay Gateway Developers SCI-BUS Generic Framework SCI-BUS Generic Framework Data transfer Profile Monitoring Workflow Portlet Developers Portlet SCI-BUS Portlet Repository SCI-BUS Portlet Repository Job submission …. Workflow Template A Data management Portlet Function Application Developers Application Download from sourceforge

gUSE based gateways 24 Close to 60 deployments world- wide More than downloads from sourceforge

Conclusions Why to select WS-PGRADE/gUSE and join the SCI-BUS community? 1.Robustness –Already large number of gateways used in production 2.Sustainability –The SCI-BUS project and its sustainability and commercialization plan guarantees it 3.Functionalities –Rich functionalities that are growing according to the SCI-BUS and sourceforge community needs 4.How easy to adapt for the needs of the new user community? –Already large number of gateways customized from gUSE/WS- PGRADE 5.You can influence the progress of WS-PGRADE/gUSE 25 Join SCI-BUS as associated member

26 Where to find further information? SCI-BUS web page: – gUSE/WS-PGRADE: – gUSE on sourceforge – – –