Servizi di AAI e job management per Science Gateway basati su standard D. Scardaci INFN Catania

Reference model Science Gateway Science Gateway Appl 1 Appl 2 Appl N Grid Services Embedded Applications Administrator Power User Basic User Users from different Organisations Other Middleware

Motivations The distributed nature of Grid requires strong security mechanisms; No expert-users struggle to comply with complex security rules: – Create certificates, create proxy, update credentials and so on; Some institutions want to maintain the control of their users’ authentication and the service available: – Science Gateways have to be able to interact with other services.

Federated AAI In the web technology arena many approaches are available to federate the authentication among different entities; A standard provided by OASIS defines the Security Assertion Markup Language (SAML); Shibboleth is one of the most famous SAML-based tools: – Implement the SAML standard; – Allows different approaches to manage users: LDAP, CAS, Plain text, etc.; – Deployed in many universities and research institutes; – Free and Open Source; – Easy to integrate with Liferay; Shibboleth has been selected for the integration.

A&A Schema AuthorisationAuthorisation Science Gateway GrIDP (WAYF) (“catch-all”) GrIDP (WAYF) (“catch-all”) IDPCT IDP_y IDP_x LDAP CAS Access a Service 2. Login Authentication (“catch-all”)

6 The GARR-IDEM Identity Federation ( IDEM figures: 45 IDentity Providers: – 31 in production; – 14 in test; >2,700,000 end users (as of October 2010) ; ~50% of the Italian higher education & research community e-identified students in EU

Usage workflow Science Gateway 1. Portal Login 2. Actions request 4. Get proxy from robot 5. Perform actiion 6. Results Credentials exchange 3. Verify ACL eTokenServer

Role mapping Authorisation is centralised into the LDAP portal; Robot proxy may have VOMS attributes corresponding to the roles in LDAP: – For each application and user profile a LDAP role and a VOMS attribute is defined; Users have to explicitly request the authorisation for the roles they need: – A group of experts evaluates the requests; If users try to access Grid resources with other tools they do not gain more privileges; Roles coming from the federation are currently not accepted: – For other projects they could be granted.

Authenticated users without authorisation are not logged-in; Registration to the Science Gateway is mandatory: – Users can run only the applications they have explicitly requested and obtained authorisation for; – This is a requirement in several projects. User registration

Implementation Shibboleth-based authentication is not available in Liferay by default; A library for account operation has been developed and implemented the following operations: Login with provided by Shibboleth SAML attributes; Local logout from the service; Authorisation check against LDAP accounts.

12 Liferay-based Science Gateways are currently powered by Shibboleth at INFN Catania; 2 Federations supported: GrIDP (“catch-all”) and GARR-IDEM; 4 instances are in progress to be registered as IDEM Service Providers 3 Identity Providers are available in GrIDP: A “catch-all IdP” created at Catania; the MAAT-G (enterprise) IdP; INFN-AAI IdP. Current status

Grid access through robot certificates The REST-full “lightweight” crypto-library API: – The Architecture; – Software Requirements: Java™ PKCS#11, Bouncy Castle and Java CoG Kits; JAX-RS 1.2 Java APIs using Jersey implementation; VOMS-API v.3.0; Apache Tomcat as a Web Container; – Usage;

Background Robot certificates have been introduced to allow non-users to experience the Grid paradigm for research activity; – They are extremely useful, for instance, to automate Grid service monitoring, data processing production, distributed data collection systems; – Basically, these certificates can be used to identify a person responsible for an unattended service or process acting as client and/or server.

In order to strong reduce the risks to have the robot certificate compromised, the INFN CA decided to store this new certificate on board of the Aladdin eToken smart cards; The Aladdin eToken smart card can support many certificates; A token PIN is prompted every time the user needs to interact with the smart card; The adoption of robot certificates can reduce the gap to access Grid resources and help non-expert users to experience Grids technology in a easier way. See LIBI’s experience here.here Robot certificates & eTokens

Users Client Applications Grid Portals / Science Gateways The 3-tier architecture of the “lightweight” crypto-library

The Cryptographic Token Interface Standard (PKCS#11) is a standard introduced by RSA Data Security Inc;Cryptographic Token Interface Standard (PKCS#11)RSA Data Security Inc – It defines native programming interfaces to cryptographic tokens, (hardware cryptographic accelerators, smart cards, … ); PKCS#11 standard includes sixty function prototypes (also referred to as cryptoki library) that together can be used to perform a wide range of cryptographic operations. To make easier the integration of these PKCS#11 tokens, the PKCS#11 provider has been introduced. The PKCS#11 provider is supported on several platforms; The Cryptographic Token Interface Standard (PKCS#11)

The Bouncy Castle APIs provide support for creating two kinds of X.509 certificates: – version 1 They are used to create root certificates; org.bouncycastle.x509.X509V1CertificateGenerator; – version 3 They contain certificate extensions; org.bouncycastle.x509.X509V3CertificateGenerator; – PKCS10 certification requests org.bouncycastle.jce.PKCS10CertificationRequest. The BouncyCastle APIs

CoG Kits allow users to provide Globus Toolkit functionality within their code without calling scripts, or in some cases without having Globus installed. – CoGs are currently available for Java, Python, CORBA, Perl, and Matlab. The Java CoG Kits distributed under the Globus Toolkit Public License (GTPL) is an extension of the Java libraries and classes that provides Globus Toolkit functionality. – It provides Java classes for interfacing with the following Globus components/functions: Proxy: Credential creation and destruction; GRAM: Job submission and monitoring; MDS: Resource searching; RSL: Resource specification and job execution; GridFTP: Data Management; GASS: Data Management. The Java CoG Kits

VOMS-Admin library (ver. 3.0), developed in the context of the DILIGENT and D4Science projects, were used for interacting the VOMS server and retrieve the list of groups/roles per VO. DILIGENTD4Science VOMS-Admin APIs The library used by the eTokenServer can be downloaded from the ETICS Software RepositoryRepository

Deployed on Tomcat Application Server (ver ); Thread-safe access to the list of smart cards based on a Singleton; – Evaluated performance of the server using Apache Jmeter: ~ 6-8 sec. Waiting time for a new proxy; 20 msec. if the proxy is cached. SSL encryption using a trusted host certificate; Caching of proxy certificates for each valid requestID (serial+vo+fqan): – If lifetime(requestID)-3h>0 the proxy cached is sent to the Science Gateway Main Features

eTokenServer MyProxy Server ask for VOMS AC attributes VOMS Server store long proxy The working scenario (*) SSL encryption get results ask for a service list/create request execute service get results retrieve serials/proxy (*)

Listing the X.509 certificates installed on the eTokenServer: Some examples of usage (1/3) SSL mechanisms for encrypting the traffic with the eTokenServer Printing results in JSON format

Generating a VOMS proxy from a given robot certificate: Some examples of usage (2/3) Using VOMS-Admin library to update the list of groups/roles

Some examples of usage (3/3) Access the eTokenServer using a SSL Java client: Get the list of X.509 certificates Create a proxy certificate from a given serial

Standard-based job management A Simple API for Grid Applications (SAGA): – The OGF Standard; – A Java implementation of SAGA: JSAGA; A Generic Grid Engine for Science Gateways based on SAGA; – Grid Engine based on JSAGA: – EGI Portal Policy & Grid Security Traceability; References.

A Simple API for Grid Applications (SAGA) SAGA is an API that provides the basic functionality required to build distributed applications, tools and frameworks; It is independent of the details of the underlying infrastructure (e.g., the middleware); SAGA is an OGF specification: Several Implementations are available: A C++ and a Java implementation developed at the Louisiana State University / CCT and Vrije Universiteit Amsterdam ( A Java implementation developed at CCIN2P3 ( A Python implementation based on those above.

A Simple API for Grid Applications (SAGA) SAGA is composed by: SAGA Core Libraries: containing the SAGA base system, the runtime and the API packages (file management, job management, etc.); SAGA Adaptors: libraries providing access to the underlying grid infrastructure (adaptors are available for Globus, gLite, etc.); SAGA defines a standard We then need an implementation!

JSAGA JSAGA is a Java implementation of SAGA developed at CCIN2P3; JSAGA: Enables uniform data and job management across different grid infrastructures/middleware; Makes extensions easy: adaptor interfaces are designed to minimize coding effort for integrating support of new technologies/middleware; Is OS indenpendent: most of the provided adaptors are written in full Java and they are tested both on Windows and Linux.

JSAGA Adaptors JSAGA supports gLite, Globus, ARC, UNICORE, etc.

A Generic Grid Engine for Science Gateways based on JSAGA Grid Engine Users Tracking DB Science GW Interface JSAGA API Job Engine Data Engine Users Track & Monit. Science GW 1 Science GW 2 Science GW 3 Grid MW Liferay Portlets eToken Server

EGI Portal Policy - VO Portal Policy Portal Classes Portal ClassExecutableParametersInput Simple one-click provided by portal provided by portal Parameter provided by portal chosen from enumerable and limited set chosen from repository vetted by the portal Data processing provided by portal chosen from enumerable and limited set provided by user Job management provided by user provided by user Identified Web User

EGI Portal Policy - VO Portal Policy The Portal, the VO to which the Portal is associated, the Portal manager are all individually and collectively responsible and accountable for all interactions with the Grid; The Portal must be capable of limiting the job submission rate; The Portal must keep audit logs for all interactions with the Grid as defined in the Traceability and Logging Policy (minimun 90 days); The Portal manager and operators must assist in security incident investigations; Where relevant, private keys associated with (proxy) certificates must not be transferred across a network, not even in encrypted form.

Users’ Traceability in Science Gateways GRID USAGE TRACEABILITY Common NamePortal User Name as stored in LDAP IP + PortIP address and TCP port used by the requester TimestampIdentify the grid operation date/time Grid InteractionGrid Interaction Identification (Job “X” submission, file upload/download). The portal MUST classify all the grid operations allowed. This value will allow to identify both applications used and operation performed. Grid IDStore the actual GRID Interaction ID (Job ID for job submission and some other relevant information for data transfer) Robot CertificateIdentify the Robot Certificate used for the Grid Operation Two Tables, one for active Jobs and File Transfers and one for the finished ones. ID70 COMMON NAMEfpistagna IP + TCP Port :8162 Timestamp :16:29 Grid Interaction1 Grid ID[wms://infn-wms- 01.ct.pi2s2.it:7443/glite_wms_wmproxy_server]-[ lb-01.ct.pi2s2.it:9000/7rQ458xozactEEjoXMlxQg] Robot Certificate/C=IT/O=INFN/OU=Robot/L=COMETA/CN=Robot: ViralGrid Science Gateway - Roberto Barbera Virtual Organizationcometa An entry on the Users Tracking DB

Example of usage: Job Submission import it.infn.ct.JSagaJobSubmission; JSagaJobSubmission jobSubmission = new JSagaJobSubmission(); jobSubmission.setUserProxy(" "); //set the proxy jobSubmission.setExecutable("/bin/sh"); //set the executable jobSubmission.setTotalCPUCount("4"); //set the CPUNumber jobSubmission.setArguments("mpi-start-wrapper.sh,mpi-test,MPICH"); //set arguments separeted by ",“ jobSubmission.setOutputPath(" "); //set output path jobSubmission.setJobOutput("mpi-test.out"); //set std-output and std-error files jobSubmission.setJobError("mpi-test.err"); jobSubmission.setInputFiles("/opt/mpistart/mpi-start- wrapper.sh,/opt/mpistart/mpi-hooks.sh,/opt/mpistart/mpi-test.c"); //set input files separeted by "," jobSubmission.setJobQueue("grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda"); //set the queue String jdlRequirements[] = new String[2]; //JDL requirements jdlRequirements[0] = "JDLRequirements=(Member(\"MPI-START\", other.GlueHostApplicationSoftwareRunTimeEnvironment))"; jdlRequirements[1] = "JDLRequirements=(Member(\"MPICH\", other.GlueHostApplicationSoftwareRunTimeEnvironment))"; jobSubmission.setJDLRequirements(jdlRequirements); //submit your job to the Resource Manager (WMS in the sample) String newJobId = jobSubmission.submitJob(,,, );

Example of usage: Job Status import it.infn.ct.JSagaJobSubmission; JSagaJobSubmission jobSubmission = new JSagaJobSubmission(); jobSubmission.setUserProxy(" "); //set the proxy //jobId is the SAGA job id returned by jobSubmission.submitJob function. JobStatus cointains the job status. String JobStatus = jobSubmission.getJobStatus(jobId);

Example of usage: Job Output import it.infn.ct.JSagaJobSubmission; JSagaJobSubmission jobSubmission = new JSagaJobSubmission(); jobSubmission.setUserProxy(" "); //set the proxy //jobId is the SAGA job id returned by jobSubmission.submitJob function. JobStatus cointains the job status. String JobStatus = jobSubmission.getJobStatus(jobId); if (JobStatus.equals("DONE")) { //output file is a relative path pointing to a tar.gz file containing the job output files String outputFile = jobSubmission.getOutput(jobId); }

Valeria Ardizzone (COMETA); Roberto Barbera (UNICT & INFN) Riccardo Bruno (COMETA); Tony Calanducci (COMETA); Elisa Ingrà (GARR); Salvatore Monforte (INFN); Fabrizio Pistagna (INFN); Rita Ricceri (INFN); Riccardo Rotondo (INFN); Credits Vincenzo Ciaschini (INFN); Enrico Fasanelli (INFN); Maria Laura Mantovani (GARR); Barbara Monticini (GARR); Simona Venuti (GARR); Acknowledgments