CHAIN Worldwide Interoperability Test Giuseppe Andronico – INFN OGF 34 – Oxford, 12 Mar 2012 Research Infrastructures – Grant Agreement n. 260011

Interoperability (http://en.wikipedia.org/wiki/Interoperability) Interoperability is a property referring to the ability of diverse systems and organizations to work together (inter-operate). The term is often used in a technical systems engineering sense, or alternatively in a broad sense, taking into account social, political, and organizational factors that impact system to system performance According to ISO/IEC 2382-01 (Information Technology Vocabulary, Fundamental Terms), interoperability is defined as follows: "The capability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units"

Middleware in CHAIN Middleware Infrastructure UMD EGI Europe GOS CNGrid China GARUDA India OurGrid The OurGrid Community Brazil

Status of various middleware UMD Actually packaged by EGI, based on EMI and Globus Strong interaction with GIN@OGF and standards compliance

Status of various middleware GOS Actually at version 4.0, maintained from several institutions headed from Chinese Academy of Sciences This stage of the project is closing, they are considering to evolve in new directions Strongly based on GLOBUS, but actually completely rewritten in Java with a quite different layout

Status of various middleware GARUDA Based on GLOBUS 4.0.7 and the meta scheduler GridWay 5.7 the resources information system is based on GLOBUS MDS, not on BDII certificates of the euindia VO directly mapped on CEs’ gridmapfile without VOMS the data management is limited to gridftp in the head node of CE, without gridftp clients in the WNs.

Status of various middleware OurGrid Is a peer-to-peer computational grid targeted to run Bag-of- Tasks applications on the idle cycles of corporations' desktops. It is based on three main elements: OurGrid Broker: is the scheduler, usually installed on the machine of the user that is submitting jobs to the grid; OurGrid Peer: is in charge of managing the desktops on a site, i.e. an administrative domain; it also allows the discovery and allocation of desktops that are available in remote sites; OurGrid Worker: runs in the desktops that are made available to the grid; is in charge of identifying when the desktop can be used by the grid, and protect the desktop from any harm that could be caused by the grid jobs it executes.

Some considerations At least 2 interoperability levels: Services level Interoperability between web services of different middleware It could be so transparent to the users that they are not aware they are using resources outside their usual infrastructure Require consistent contribution from middleware developers More promising on long term, more difficult to implement in short term Applications level Solution to submit specific applications to several infrastructures; can be obtained, for example, using meta schedulers or scientific gateways Users should know details about middleware or some mechanism to make such a choice have to be implemented Require simple contribution from middlewares

CHAIN interoperability test We will relay on solution 2 (Application level) to demonstrate that: e-Infrastructures can be made interoperable to each other using standards (with the meaning of interoperability given above) VRC-specific applications can be submitted from anywhere and run everywhere

CHAIN interoperability test: requirements User interface is only web based, for simplicity Users must be transparently authenticated & authorised on all e-Infrastructures without any additional human/machine intervention There must be the smallest possible interaction with both site managers and e-Infrastructure operators No modification of the middleware should be requested to developers

Standards The framework for Science Gateways proposed and fostered by CHAIN is fully web-based and adopts official worldwide standards and protocols, through their most common implementations: Web interface: JSR 168 and JSR 286 standards (also known as "portlet 1.0" and "portlet 2.0" standards) Authentication: OASIS Security Assertion Markup Language (SAML) standard and its Shibboleth and SimpleSAMLphp implementations; Authorisation: Lightweight Direct Access Protocol, and its OpenLDAP implementation Digital certificate management: Cryptographic Token Interface Standard (PKCS#11) standard and its Cryptoki implementation Application interface: Open Grid Forum (OGF) Simple API for Grid Applications (SAGA) standard and its JSAGA implementation

Access: Science Gateway model Embedded Applications Administrator Power User Basic User ....... App. 1 App. 2 App. N Gateway Science Standard-based middleware-independent Grid Engine Users from different organisations having different roles and privileges

AuthN/AuthZ Infrastructure Portal Engine Liferay Community Edition 6.0.6 Authentication Shibboleth-2.4.3-2.1/log4shib-1.0.3-2.3/simpleSAMLphp Implementations of SAML for AuthN (Security data exchange among different Security Domains) OpenLDAP Takes care of user roles and privileges (AuthZ) Single Sign-On Users can only do what the SG allows them to do All transactions are tracked Authorization Authentication Science Gateway

Catania Grid Engine Liferay Portlets Grid Engine Data Engine Science GW 1 Science GW 2 Science GW 3 Grid Engine eToken Server Science GW Interface Data Engine Job Engine User Track. & Monit. User Tracking DB SAGA/JSAGA API Grid MW DONE DONE DONE By mid April By end of April 14

Job Engine The Job Engine is made of a set of libraries to develop applications able to submit and manage jobs on a grid infrastructure It is compliant with the OGF SAGA standard; It is optimized to be used in a web portal running an application server (e.g., Glassfish, Tomcat, etc.) based on J2EE It can be used also in stand-alone mode JSAGA is the SAGA implementation adopted 15

Job Engine -Architecture GRID INFRASTRUCTURE(S) Job Check Status/ Get Output Worker Threads for Status Checking MONITORING MODULE USER TRACKING DB WT WT WT WT WT WT WT Job Submission WT WT WT Job Queue Worker Threads for Job Submission

Job Engine - Features Feature Description Status The Job Engine has been designed with the following features in mind: Feature Description Status Middleware Independent Capacity to submit job to resources running different middleware DONE Easiness Create code to run applications on the grid in a very short time Scalability Manage a huge number of parallel job submissions fully exploiting the HW of the machine where the Job Engine is installed Performance Have a good response time Accounting & Auditing Register every grid operation performed by the users Fault Tolerance Hide middleware failure to end users ALMOST DONE Workflow Providing a way to easily create and run workflows IN PROGRESS 17

Job Engine – Scalability Job submission time (h) Submission time scales linearly with number of jobs; Actual time depends on HW capabilities and thread-pools configuration Time to submit 10000 jobs (h) 40000 Jobs submitted at the same time! 18

Job Engine – Accounting & Auditing A powerful accounting & auditing system is included in the Job Engine It is fully compliant with EGI VO Portal Policy and EGI Grid Security Traceability and Logging Policy The following values are stored in the DB for each job submitted: User ID Job Submission timestamp Job Done timestamp Application name Job ID Robot certificate ID VO name Execution site (name, lat, long) Overall usage (Dec. 2011 → today) 19

CHAIN interoperability test: e-Infrastructures involved so far gLite-based e-Infrastructures/Projects EUAsiaGrid EUChinaGRID EU-IndiaGrid EUMEDGRID GISELA IGI (Italy) SAGrid (South Africa)

Step 1 – Grid authorisation User VO(s) are authorized at all sites of the various e-Infrastructures This would break requirement 3 and could require quite some time to be done The certificates of the Science Gateway(s) to be used for the test are registered in the gLite VOMS servers and Unicore XUUDB

Step 2 – Resource discovery Create a (super)-topBDII that gathers all the topBDII’s of the various e-Infrastructures This would break requirement 3 and require a grid service to be managed by CHAIN Information can become outdated Insert the list of the topBDIIs/WMSs/TargetSystems of the various e-Infrastructures/middleware in the configuration of the portlets of the test applications and let the Grid Engine choose at job submission resource managers and services to be used

Multi-infra/multi-mw Job Engine EUMEDGRID e-Infrastructure gLite Infrastr. Info (BDII,VO, etc.) gLite Infrastr. Info Unicore Infrastr. Info GISELA e-Infrastructure Multi-infrastructure/ Multi-middleware Science Gateway Submit Juelich Supercomputing Centre User

MyJobsMap (1/4)

MyJobsMap (2/4)

MyJobsMap (3/4)

MyJobsMap (4/4) Both sequential and MPI-enabled jobs successfully executed

Full interoperability In the following, we will describe a longer term plan to achieve Service Level (i.e., full) interoperability

EGI standards Capability Standard SDO Phase Information.Messaging JMS 1.1 JCP USE AMQP 1.0 AMQP WG IMP Information.Discovery LDAPv3 IETF Information.Model GLUE 2.0 OGF DEP Compute.JobExecution BES, JSDL, HPC-BP, HPC File Staging Profile PGI DEV Compute.WorkflowExecution WS-BPEL 2.0 OASIS DCI-Federation Compute.ParallelJobExecution MPI MPI-Forum OpenMP OpenMP ARB HPC SPMD Compute.JobScheduling Storage.Management SRM v2.2 Storage.FileTransfer GridFTP Storage.FileTransferScheduling DMI Storage.FileAccess NSF 4.1 Storage.FileEncryption Data.Access WS-DAIR, WS-DAIX Data.MetadataCatalogue Instrumentation.Management Security.Authentication X.509 + RFC3820 SAML 2.0 OpenID OIDF Security.Authorisation SAML 2.0, XACML 2.0 Security.CredentialManagement Security.AttributeAuthority VirtualMachine.Management OCCI VirtualMachine.ImageFormat OVF DMTF VirtualMachine.ImageDistribution Operations.Monitoring Operations.Accounting UR RUS ClientAPI SAGA

EGI standards Capability Standard SDO Phase Information.Messaging JMS 1.1 JCP USE AMQP 1.0 AMQP WG IMP Information.Discovery LDAPv3 IETF Information.Model GLUE 2.0 OGF DEP Compute.JobExecution BES, JSDL, HPC-BP, HPC File Staging Profile PGI DEV Compute.WorkflowExecution WS-BPEL 2.0 OASIS DCI-Federation Compute.ParallelJobExecution MPI MPI-Forum OpenMP OpenMP ARB HPC SPMD Compute.JobScheduling Storage.Management SRM v2.2 Storage.FileTransfer GridFTP Storage.FileTransferScheduling DMI Storage.FileAccess NSF 4.1 Storage.FileEncryption Data.Access WS-DAIR, WS-DAIX Data.MetadataCatalogue Instrumentation.Management Security.Authentication X.509 + RFC3820 SAML 2.0 OpenID OIDF Security.Authorisation SAML 2.0, XACML 2.0 Security.CredentialManagement Security.AttributeAuthority VirtualMachine.Management OCCI VirtualMachine.ImageFormat OVF DMTF VirtualMachine.ImageDistribution Operations.Monitoring Operations.Accounting UR RUS ClientAPI SAGA

Referred standards AAI: SAML/Shibboleth for managing authorization and authentication Resources information index: BDII Job submission and management: SAGA + OGSA-BES to abstract from middleware implementation Data access and management: SRM + LFC

Full interoperability plan Every middleware should integrate such technologies in their stacks In this way jobs should be able to go from one infrastructure to another gLite GOS Standards Standards Job GARUDA Standards OurGrid Standards

AAI AAI: every middleware should add a plugin able to convert authorization and authentication information forth and back from Shibboleth token MW2 AAI  Shibboleth MW1 XML/SOAP

Job submission Abstraction from specific middleware implementations in: Sending job request Receiving and handle job request Returning job output Our trial solution is providing each middleware of software adapters based on SAGA and OGSA-BES

Job submission and management SOAP MW1 OGSA-BES interface SAGA + OGSA-BES adapter MW2 Every middleware should provide two elements: An OGSA-BES interface that will act as a standardization layer between the infrastructure and the rest of the world An element collecting request to other middleware(s) and handling them in SAGA, to be abstracted from the middleware, and the BES adapter to be issued to the other infrastructure without any knowledge of remote middleware This should be working for most of the job related features

Detailed view BDII: every middleware should provide a tool to publish its computing resources to a BDII query BDII to publish other infrastructure resources in its own system SRM: every middleware should provide a SRM interface to its storage elements LFC: every middleware should publish its files to LFC, making reference to the SRM interface query LFC to retrieve other infrastructures files and publish them in its internal system

Feedback from m/w developers CNGrid GARUDA OurGrid 1st phase Person months locally available for the SAGA adaptor development 3 Local plan or problems to sustain SAGA adaptor in the future 2nd phase Degree of expertise locally available on SAML/Shibboleth Nothing, willing to start Person months locally available to develop a converter between the local resource index system and GLUE schema to write on a BDII 6 Person months locally available to develop a SRM interface to local data system Person months locally available to develop both the OGSA-BES and the SAGA interface

Resources What Link AAI https://www.escholar.manchester.ac.uk/uk-ac-man-scw:99165 http://www.terena.org/activities/nrens-n-grids/workshop-06/slides/witzig-switch-slcs-vash.pdf BDII http://en.wikipedia.org/wiki/BDII SRM http://en.wikipedia.org/wiki/Storage_Resource_Manager LFC https://twiki.cern.ch/twiki/bin/view/LCG/LfcAdminGuide OGSA-BES http://www.ogf.org/documents/GFD.108.pdf SAGA http://saga.cct.lsu.edu/ DRMAA http://en.wikipedia.org/wiki/DRMAA

Summary and conclusions The adoption of standards (JSR 286, SAML, SAGA, etc.) in the Science Gateway proposed and fostered by CHAIN represents a first step towards a global interoperable architecture The model is used to demonstrate CHAIN global harmonisation of e-Infrastructures world-wide The model nicely implements HTC/HPC interoperability at the user/application level The service level interoperability is at proposal level It is currently used to develop in details a plan to implement interoperation between the various middleware

Thank you Acknowlodgments: Roberto Barbera, Marco Fargetta, Riccardo Rotondo, Diego Scardaci SAGA & JSAGA groups Depei Qian (CNGrid/GOS), Marco Verlato (GARUDA), Francisco Brasilero (OurGrid), Alberto Di Meglio (EMI)