Converting P-GRADE Grid Portal into E-Science Gateways A. Balasko, M. Kozlovszky, K. Varga, A. Schnautigel, K. Karóckai, I. Márton, T. Strodl, P. Kacsuk MTA SZTAKI, Budapest, Hungary IWSG2010, , Catania, Italy

Outline Role definition of E-Science Gateways Common requirements of a generic e-science community Collaboration between community members P-GRADE Portal in a nutshell Application specific module – Goal – Basic idea – Concept – Detailed scenario Case studies – Application Specific Portals – E-science Gateway examples Conclusions

Who are the members of an e-science community? (from portal’s point of view) End-users (e-scientists) Execute the published applications with custom input parameters by creating application instances using the published applications as templates Grid Portal Developers Develop GUI for end users on the portal Connect the GUI to P-GRADE Portal using a programming API: Application Specific Module Grid Portal Administrators Install the portal Maintain and operate the portal Give feedback for the portal developers Grid Application Developers Develop grid applications by the portal (app. porting) Publish the completed applications for end-users

What does an e-science community need? App. Repository Portal Clouds Local clusters Supercomputers Desktop grids (DGs) (BOINC, Condor, etc.) Cluster based service grids (SGs) (EGEE, OSG, etc.) Supercomputer based SGs (DEISA, TeraGrid) Grid systems Application developers E-scientists

Requirements for an e-science portal from the e- scientists’ point of view It should be able to Support large number of e-scientists (~ 100) with good response time Enable the store and share of ready-to-run applications Enable to parameterize and run applications Enable to observe and control application execution Provide reliable application execution service even on top of unreliable infrastructures (like for example grids) Provide specific, user community views Enable the access of the various components of an e-science infrastructure (grids, databases, clouds, local clusters, etc.) Support user’s collaboration via sharing: – Applications (legacy, workflow, etc.) – Databases

Requirements for an e-science portal from the app. developers’ point of view Beyond the end-user requirements it should be able to Enable the store and share of half-made applications, application templates Provide graphical app. developing tools (e.g. workflow editor) to develop new applications Provide methods and API to customize the portal interface towards specific user community needs by creating user-specific portlets Enable the integration/call of other services

Collaboration between application developers App. Repository Portal E-science infrastructure Application developers Application developers use the portal to develop complex applications (e.g. parameter sweep workflow) for the e- science infrastructure Publish templates, legacy code apps. and half-made applications in the repository to be continued by other app. developers

Collaboration between e-scientists and application developers App. Repository Portal Application developers E-scientists End-users (e-scientists) Specify the problem/application needs Execute the published applications via the portal with custom input parameters by creating application instances Application Developers Develop e-science applications via the portal in collaboration with e-scientists Publish the completed applications for end-users via an application repository

P-GRADE portal family P-GRADE portal 2.4 NGS P-GRADE portal P-GRADE portal 2.5 Param. Sweep P-GRADE portal 2.8 P-GRADE portal Current release WS-PGRADE Portal Beta release 3.1 WS-PGRADE Portal Release 3.2 GEMLCA Grid Legacy Code Arch. GEMLCA, repository concept Basic concept Open source from Jan WS-PGRADE = Web Services Parallel Grid Runtime and Developer Environment gUSE = Grid User Support Environment

Main features of P-GRADE portal Supports generic, workflow-oriented applications parameter sweep (PS) applications with new super-workflow concept 3-level parallelism (MPI, WF-branch, PS) Simultaneous access of wide variety of resources – GT2, LCG-2, gLite, PBS, ARC, desktop grids (BOINC) and clouds Access to workflow repositories

P-GRADE Portal references P-GRADE Portal services: – SEE-GRID, BalticGrid – Central European VO of EGEE – GILDA: Training VO of EGEE – Many national Grids (UK, Ireland, Croatia, Turkey, Spain, Belgium, Malaysia, Kazakhstan, Switzerland, Australia, etc.) – US Open Science Grid, TeraGrid – Economy-Grid, Swiss BioGrid, Bio and Biomed EGEE VOs, MathGrid, etc. – EDGeS P-GRADE portal service

Creating application specific portals from a generic P- GRADE portal Creating an application spec. portal does not mean to develop it from scratch P-GRADE portal is a generic portal that can quickly and easily be customized to any application type Advantage: – Much shorter development time – Requires less efforts You do not have to develop the generic parts (WF editor, WF manager, job submission, monitoring, etc.) You can concentrate on the application spec. part

Basic idea A new module called “Application specific module” has been developed for P-GRADE Portal to facilitate the creation of Application Specified Portlets – The module provides programming API to control P-GRADE portal applications – The module provides programming API to access P-GRADE Portal services Any kind of Graphical interface can be developed on top of this module – GUI written in JSP, JSTL (Java Servlet Tag Library ) and Java – GUI can be customized for the grid application – GUI can be customized for end users – GUI can benefit from all the P-GRADE Portal services

Application Specific Module Concept Custom user interface (Written in Java, JSP, JSTL) Web browser EGEE and Globus Grid services (gLite WMS, LFC,…; Globus GRAM, …) Client P-GRADE Portal server Grid Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) Additional module for P-GRADE Core P-GRADE Portal Developer’s target Core ASM interfaces (Written in Java, JSP, JSTL)

P-GRADE PORTAL server Application Specific Module Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) Detailed Scenario (step 1.) EGEE and Globus Grid services (gLite WMS, LFC, …; Globus GRAM, GridFTP, …) Portal Administrator installs the P-GRADE portal 2.9.*. Then installs Application Specific Module Portlet to publish applications Portlet to set Roles Inner services

P-GRADE PORTAL server Application Specific Module Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) Detailed Scenario (step 2.) EGEE and Globus Grid services (gLite WMS, LFC, …; Globus GRAM, GridFTP, …) Grid Application Developer creates own Grid Application and publish it using the core ASM portlet. Portlet to publish applications Portlet to set Roles Inner services Grid Application

P-GRADE PORTAL server Application Specific Module Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) Detailed Scenario (step 3.) EGEE and Globus Grid services (gLite WMS, LFC, …; Globus GRAM, GridFTP, …) Grid Portal Developer creates Portlet for the application (mainly one JSP and one eventhandler java class) Then associates it with the role using the core portlet of ASM Portlet to publish applications Portlet to set Roles Inner services Grid Application Portlet for the Application

P-GRADE PORTAL server Application Specific Module Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) Detailed Scenario (step 4.) EGEE and Globus Grid services (gLite WMS, LFC, …; Globus GRAM, GridFTP, …) Finally the end-users parameterizes and executes the application using the developed portlet Portlet to publish applications Portlet to set Roles Inner services Grid Application Portlet for the Application Grid Application

The ASM API Provides: Listing published applications Listing available instances New instance creation Input data manipulation – FileUpload – Setting input text Managing instances' lifecycle – Create/Start/Check Status/Delete

Application Specific P-GRADE portals using ASM Rendering portal by Univ. of Westminster Traffic simulation portal by Univ. of Westminster EMMIL Procurement Portal by SZTAKI

eScience Gateway examples (based on P-GRADE Portal’s ASM)

OMNET++ Portal (Example 1.) OMNeT++ is a public-source, component-based, modular, discrete event simulation environment. Aim: Use of grid infrastructure for computing network simulations, queuing network simulations Gridification: –OpenSim Ltd. SZTAKI Important used/developed features – ASM + Application specific portlets – Parameter Study type solution – Parrot usage – Service like, easy-to-use login (non-gridsphere based authentication), captcha supported automatic user creation More detailed information

OMNET++ workflow (what is inside…)

Two types of OMNET portal usage scenario OMNeT++ user portal User accounts exist for 1 week Can handle NED files that use INET and Queuing module sets from OMNeT++ No binary come from end user –Portal does grid operations with one fixed certificate OMNeT++ developer portal Permanent user accounts Can handle custom OMNeT++ modules and NED files that handle such modules Binaries come from end users –Users are authenticated based on their personal certificates

OMNeT++ portal

Seismology E-Science Gateway (Example 2.) Operates within the seismology VO of the SEE-GRID-SCI project NMMC3D - Numerical Modeling of Mantle Convection SRA - Seismic Risk Assessment ELF – Earthquake Location finder "We shape our buildings, and afterwards our buildings shape us” -- Churchill

NMMC3D - Numerical Modeling of Mantle Convection Aim: better understand the influences of parameters on the behavior of upwellings. Function: The application solves the equations of thermal convection with a partly finite difference, partly spectral scheme. Gridification: GGRI /Geodetic and Geophysical Research Institute/ SZTAKI Bogazici University /Turkey/ Important used/developed features Application specific complex portlet structure Embedded picture rendering service Parameter Study type solution More detailed information

Example result file converted to an image Application output

SRA - Seismic Risk Assessment Aim: Application for public safety and hazards mitigation. Gridification: METU /Ankara, Turkey/ Important used/developed features Application specific portlet Embedded Google maps Embedded picture rendering service Parameter Study solution More detailed information

SRA: Seismic Risk Assessment developed as a portlet Google-powered map

Conclusions P-GRADE portal is a generic portal that can quickly and easily be customized to any application type Portal Developers can use Application Specific Module (ASM) to build eScience Gateways Advantages: – Much shorter development time – Requires less efforts – Portal Developers can customize /create portal interface faster using ASM - ASM provides high level API for P-GRADE's services - ASM for P-Grade Portal 2.7+ is available at Sourceforge Further information

Thank you for your attention! Questions?