1. CHAIN Worldwide Interoperability Test Giuseppe Andronico – INFN
OGF 34 – Oxford, 12 Mar 2012
Research Infrastructures – Grant Agreement n

2. Interoperability (http://en.wikipedia.org/wiki/Interoperability)
Interoperability is a property referring to the ability of diverse systems and organizations to work together (inter-operate). The term is often used in a technical systems engineering sense, or alternatively in a broad sense, taking into account social, political, and organizational factors that impact system to system performance
According to ISO/IEC (Information Technology Vocabulary, Fundamental Terms), interoperability is defined as follows: "The capability to communicate, execute programs, or transfer data among various functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of those units"

3. Middleware in CHAIN Middleware Infrastructure UMD EGI Europe GOS
CNGrid
China
GARUDA
India
OurGrid
The OurGrid Community
Brazil

4. Status of various middleware
UMD
Actually packaged by EGI, based on EMI and Globus
Strong interaction with and standards compliance

5. Status of various middleware
GOS
Actually at version 4.0, maintained from several institutions headed from Chinese Academy of Sciences
This stage of the project is closing, they are considering to evolve in new directions
Strongly based on GLOBUS, but actually completely rewritten in Java with a quite different layout

6. Status of various middleware
GARUDA
Based on GLOBUS and the meta scheduler GridWay 5.7
the resources information system is based on GLOBUS MDS, not on BDII
certificates of the euindia VO directly mapped on CEs’ gridmapfile without VOMS
the data management is limited to gridftp in the head node of CE, without gridftp clients in the WNs.

7. Status of various middleware
OurGrid
Is a peer-to-peer computational grid targeted to run Bag-of- Tasks applications on the idle cycles of corporations' desktops.
It is based on three main elements:
OurGrid Broker: is the scheduler, usually installed on the machine of the user that is submitting jobs to the grid;
OurGrid Peer: is in charge of managing the desktops on a site, i.e. an administrative domain; it also allows the discovery and allocation of desktops that are available in remote sites;
OurGrid Worker: runs in the desktops that are made available to the grid; is in charge of identifying when the desktop can be used by the grid, and protect the desktop from any harm that could be caused by the grid jobs it executes.

8. Some considerations At least 2 interoperability levels: Services level
Interoperability between web services of different middleware
It could be so transparent to the users that they are not aware they are using resources outside their usual infrastructure
Require consistent contribution from middleware developers
More promising on long term, more difficult to implement in short term
Applications level
Solution to submit specific applications to several infrastructures; can be obtained, for example, using meta schedulers or scientific gateways
Users should know details about middleware or some mechanism to make such a choice have to be implemented
Require simple contribution from middlewares

9. CHAIN interoperability test
We will relay on solution 2 (Application level) to demonstrate that:
e-Infrastructures can be made interoperable to each other using standards (with the meaning of interoperability given above)
VRC-specific applications can be submitted from anywhere and run everywhere

10. CHAIN interoperability test: requirements
User interface is only web based, for simplicity
Users must be transparently authenticated & authorised on all e-Infrastructures without any additional human/machine intervention
There must be the smallest possible interaction with both site managers and e-Infrastructure operators
No modification of the middleware should be requested to developers

11. Standards
The framework for Science Gateways proposed and fostered by CHAIN is fully web-based and adopts official worldwide standards and protocols, through their most common implementations:
Web interface: JSR 168 and JSR 286 standards (also known as "portlet 1.0" and "portlet 2.0" standards)
Authentication: OASIS Security Assertion Markup Language (SAML) standard and its Shibboleth and SimpleSAMLphp implementations;
Authorisation: Lightweight Direct Access Protocol, and its OpenLDAP implementation
Digital certificate management: Cryptographic Token Interface Standard (PKCS#11) standard and its Cryptoki implementation
Application interface: Open Grid Forum (OGF) Simple API for Grid Applications (SAGA) standard and its JSAGA implementation

12. Access: Science Gateway model
Embedded Applications
Administrator
Power User
Basic User
App. 1
App. 2
App. N
Gateway
Science
Standard-based middleware-independent
Grid Engine
Users from different organisations having different roles and privileges

13. AuthN/AuthZ Infrastructure
Portal Engine
Liferay Community Edition 6.0.6
Authentication
Shibboleth /log4shib /simpleSAMLphp
Implementations of SAML for AuthN (Security data exchange among different Security Domains)
OpenLDAP
Takes care of user roles and privileges (AuthZ)
Single Sign-On
Users can only do what the SG allows them to do
All transactions are tracked
Authorization
Authentication
Science Gateway

14. Catania Grid Engine Liferay Portlets Grid Engine Data Engine
Science
GW 1
Science
GW 2
Science
GW 3
Grid Engine
eToken
Server
Science GW Interface
Data Engine
Job Engine
User Track. &
Monit.
User
Tracking DB
SAGA/JSAGA API
Grid MW
DONE
DONE
DONE
By mid April
By end of April 14

15. Job Engine
The Job Engine is made of a set of libraries to develop applications able to submit and manage jobs on a grid infrastructure
It is compliant with the OGF SAGA standard;
It is optimized to be used in a web portal running an application server (e.g., Glassfish, Tomcat, etc.) based on J2EE
It can be used also in stand-alone mode
JSAGA is the SAGA implementation adopted 15

16. Job Engine -Architecture
GRID INFRASTRUCTURE(S)
Job
Check Status/
Get Output
Worker Threads for
Status Checking
MONITORING MODULE
USER
TRACKING DB WT WT WT WT WT WT WT
Job
Submission WT WT WT
Job Queue
Worker Threads for Job Submission

17. Job Engine - Features Feature Description Status
The Job Engine has been designed with the following features in mind:
Feature
Description
Status
Middleware Independent
Capacity to submit job to resources running different middleware
DONE
Easiness
Create code to run applications on the grid in a very short time
Scalability
Manage a huge number of parallel job submissions fully exploiting the HW of the machine where the Job Engine is installed
Performance
Have a good response time
Accounting & Auditing
Register every grid operation performed by the users
Fault Tolerance
Hide middleware failure to end users
ALMOST DONE
Workflow
Providing a way to easily create and run workflows
IN PROGRESS 17

18. Job Engine – Scalability
Job submission time (h)
Submission time scales linearly with number of jobs;
Actual time depends on HW capabilities and thread-pools configuration
Time to submit jobs (h)
40000 Jobs
submitted
at the same time! 18

19. Job Engine – Accounting & Auditing
A powerful accounting & auditing system is included in the Job Engine
It is fully compliant with EGI VO Portal Policy and EGI Grid Security Traceability and Logging Policy
The following values are stored in the DB for each job submitted:
User ID
Job Submission timestamp
Job Done timestamp
Application name
Job ID
Robot certificate ID
VO name
Execution site (name, lat, long)
Overall usage (Dec → today) 19

20. CHAIN interoperability test: e-Infrastructures involved so far
gLite-based e-Infrastructures/Projects
EUAsiaGrid
EUChinaGRID
EU-IndiaGrid
EUMEDGRID
GISELA
IGI (Italy)
SAGrid (South Africa)

21. Step 1 – Grid authorisation
User VO(s) are authorized at all sites of the various e-Infrastructures
This would break requirement 3 and could require quite some time to be done
The certificates of the Science Gateway(s) to be used for the test are registered in the gLite VOMS servers and Unicore XUUDB

22. Step 2 – Resource discovery
Create a (super)-topBDII that gathers all the topBDII’s of the various e-Infrastructures
This would break requirement 3 and require a grid service to be managed by CHAIN
Information can become outdated
Insert the list of the topBDIIs/WMSs/TargetSystems of the various e-Infrastructures/middleware in the configuration of the portlets of the test applications and let the Grid Engine choose at job submission resource managers and services to be used

23. Multi-infra/multi-mw Job Engine
EUMEDGRID
e-Infrastructure
gLite
Infrastr. Info
(BDII,VO, etc.)
gLite
Infrastr.
Info
Unicore
Infrastr.
Info
GISELA
e-Infrastructure
Multi-infrastructure/
Multi-middleware
Science Gateway
Submit
Juelich
Supercomputing Centre
User

24. MyJobsMap (1/4)

25. MyJobsMap (2/4)

26. MyJobsMap (3/4)

27. MyJobsMap (4/4) Both sequential and MPI-enabled
jobs successfully executed

28. Full interoperability
In the following, we will describe a longer term plan to achieve Service Level (i.e., full) interoperability

29. EGI standards Capability Standard SDO Phase Information.Messaging
JMS 1.1
JCP
USE
AMQP 1.0
AMQP WG
IMP
Information.Discovery
LDAPv3
IETF
Information.Model
GLUE 2.0
OGF
DEP
Compute.JobExecution
BES, JSDL, HPC-BP, HPC File Staging Profile
PGI
DEV
Compute.WorkflowExecution
WS-BPEL 2.0
OASIS
DCI-Federation
Compute.ParallelJobExecution
MPI
MPI-Forum
OpenMP
OpenMP ARB
HPC SPMD
Compute.JobScheduling
Storage.Management
SRM v2.2
Storage.FileTransfer
GridFTP
Storage.FileTransferScheduling
DMI
Storage.FileAccess
NSF 4.1
Storage.FileEncryption
Data.Access
WS-DAIR, WS-DAIX
Data.MetadataCatalogue
Instrumentation.Management
Security.Authentication
X RFC3820
SAML 2.0
OpenID
OIDF
Security.Authorisation
SAML 2.0, XACML 2.0
Security.CredentialManagement
Security.AttributeAuthority
VirtualMachine.Management
OCCI
VirtualMachine.ImageFormat
OVF
DMTF
VirtualMachine.ImageDistribution
Operations.Monitoring
Operations.Accounting UR
RUS
ClientAPI
SAGA

30. EGI standards Capability Standard SDO Phase Information.Messaging
JMS 1.1
JCP
USE
AMQP 1.0
AMQP WG
IMP
Information.Discovery
LDAPv3
IETF
Information.Model
GLUE 2.0
OGF
DEP
Compute.JobExecution
BES, JSDL, HPC-BP, HPC File Staging Profile
PGI
DEV
Compute.WorkflowExecution
WS-BPEL 2.0
OASIS
DCI-Federation
Compute.ParallelJobExecution
MPI
MPI-Forum
OpenMP
OpenMP ARB
HPC SPMD
Compute.JobScheduling
Storage.Management
SRM v2.2
Storage.FileTransfer
GridFTP
Storage.FileTransferScheduling
DMI
Storage.FileAccess
NSF 4.1
Storage.FileEncryption
Data.Access
WS-DAIR, WS-DAIX
Data.MetadataCatalogue
Instrumentation.Management
Security.Authentication
X RFC3820
SAML 2.0
OpenID
OIDF
Security.Authorisation
SAML 2.0, XACML 2.0
Security.CredentialManagement
Security.AttributeAuthority
VirtualMachine.Management
OCCI
VirtualMachine.ImageFormat
OVF
DMTF
VirtualMachine.ImageDistribution
Operations.Monitoring
Operations.Accounting UR
RUS
ClientAPI
SAGA

31. Referred standards
AAI: SAML/Shibboleth for managing authorization and authentication
Resources information index: BDII
Job submission and management: SAGA + OGSA-BES to abstract from middleware implementation
Data access and management: SRM + LFC

32. Full interoperability plan
Every middleware should integrate such technologies in their stacks
In this way jobs should be able to go from one infrastructure to another
gLite
GOS
Standards
Standards
Job
GARUDA
Standards
OurGrid
Standards

33. AAI
AAI: every middleware should add a plugin able to convert authorization and authentication information forth and back from Shibboleth token
MW2
AAI  Shibboleth
MW1
XML/SOAP

34. Job submission
Abstraction from specific middleware implementations in:
Sending job request
Receiving and handle job request
Returning job output
Our trial solution is providing each middleware of software adapters based on SAGA and OGSA-BES

35. Job submission and management
SOAP
MW1
OGSA-BES interface
SAGA + OGSA-BES adapter
MW2
Every middleware should provide two elements:
An OGSA-BES interface that will act as a standardization layer between the infrastructure and the rest of the world
An element collecting request to other middleware(s) and handling them in SAGA, to be abstracted from the middleware, and the BES adapter to be issued to the other infrastructure without any knowledge of remote middleware
This should be working for most of the job related features

36. Detailed view BDII: every middleware should provide a tool to
publish its computing resources to a BDII
query BDII to publish other infrastructure resources in its own system
SRM: every middleware should provide a SRM interface to its storage elements
LFC: every middleware should
publish its files to LFC, making reference to the SRM interface
query LFC to retrieve other infrastructures files and publish them in its internal system

37. Feedback from m/w developers
CNGrid
GARUDA
OurGrid
1st phase
Person months locally available for the SAGA adaptor development 3
Local plan or problems to sustain SAGA adaptor in the future
2nd phase
Degree of expertise locally available on SAML/Shibboleth
Nothing, willing to start
Person months locally available to develop a converter between the local resource index system and GLUE schema to write on a BDII 6
Person months locally available to develop a SRM interface to local data system
Person months locally available to develop both the OGSA-BES and the SAGA interface

38. Resources What Link AAI
BDII
SRM
LFC
OGSA-BES
SAGA
DRMAA

39. Summary and conclusions
The adoption of standards (JSR 286, SAML, SAGA, etc.) in the Science Gateway proposed and fostered by CHAIN represents a first step towards a global interoperable architecture
The model is used to demonstrate CHAIN global harmonisation of e-Infrastructures world-wide
The model nicely implements HTC/HPC interoperability at the user/application level
The service level interoperability is at proposal level
It is currently used to develop in details a plan to implement interoperation between the various middleware

40. Thank you Acknowlodgments:
Roberto Barbera, Marco Fargetta, Riccardo Rotondo, Diego Scardaci
SAGA & JSAGA groups
Depei Qian (CNGrid/GOS), Marco Verlato (GARUDA), Francisco Brasilero (OurGrid), Alberto Di Meglio (EMI)