FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America Alexandre Duarte CERN IT-GD-OPS UFCG LSD 1st EELA Grid School Itacuruçá, 04-15/12/2006 Overview of middleware

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, Job Workflow in gLite UI JDL Logging & Book-keeping ResourceBroker Job Submission ServiceStorageElementComputingElement InformationService Job Status LFCCatalog DataSets info Author. &Authen. Job Submit Event Job Query Job Status Input “sandbox” Input “sandbox” + Broker Info Globus RSL Output “sandbox” Job Status Publish voms-proxy-init Expanded JDL SE & CE info

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, Job Workflow in gLite UI JDL Logging & Book-keeping ResourceBroker Job Submission ServiceStorageElementComputingElement InformationService Job Status LFCCatalog DataSets info Author. &Authen. Job Submit Event Job Query Job Status Input “sandbox” Input “sandbox” + Broker Info Globus RSL Output “sandbox” Job Status Publish voms-proxy-init Expanded JDL SE & CE info

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, Middleware structure Applications have access both to Higher-level Grid Services and to Foundation Grid Middleware Higher-Level Grid Services are supposed to help the users building their computing infrastructure but should not be mandatory Foundation Grid Middleware will be deployed on the EGEE infrastructure –Must be complete and robust –Should allow interoperation with other major grid infrastructures –Should not assume the use of Higher-Level Grid Services

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, Grid Foundation: Security Authentication based on X.509 PKI infrastructure –Certificate Authorities (CA) issue (long lived) certificates identifying individuals (much like a passport)  Commonly used in web browsers to authenticate to sites –Trust between CAs and sites is established (offline) –In order to reduce vulnerability, on the Grid user identification is done by using (short lived) proxies of their certificates Proxies can –Be delegated to a service such that it can act on the user’s behalf –Include additional attributes (like VO information via the VO Membership Service VOMS) –Be stored in an external proxy store (MyProxy) –Be renewed (in case they are about to expire)

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, Virtual Organization Membership Service: –Extends the proxy with info on VO membership, group, roles –Fully compatible with GSI –Each VO has a database containing group membership, roles and capabilities informations for each user –User contacts VOMS server requesting his authorization info –Server sends authorization info to the client, which includes it in a proxy certificate sam]$ voms-proxy-init -voms gilda Your identity: /C=IT/O=GILDA/OU=Personal Enter GRID pass phrase: Creating temporary proxy Done Contacting egris197.eela.ufrj.br:15001 [/C=BR/O=ICPEDU/O=UFF BrGrid CA/O=UFRJ/OU=IF/CN=host/egris197.eela.ufrj.br] "gilda" Done Creating proxy Done Your proxy is valid until Tue Dec 5 10:55: Grid Foundation: VOMS Query Authentication Request Auth DB C=IT/O=INFN /L=CNAF /CN=Pinco Palla /CN=proxy VOMS AC

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, Grid foundation: Information Systems BDII: Information system in gLite 3.0 (by LCG) –LDAP database that is updated by a process –More than one DBs is used separate read and write –A port forwarder is used internally to select the correct DB R-GMA: provides a uniform method to access and publish distributed information and monitoring data –Used for job and infrastructure monitoring in gLite 3.0

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, Grid foundation: Computing Element gLite-CE: based on GSI enabled Condor-C –Coming from LCG-CE (based on GT2 GRAM) –Supported by Condor. More efficient. –Deployed for the first time in gLite 3.0 CREAM: new lightweight CE –Not yet in gLite 3 release. Will need exposure to users on dedicated system. –WSDL interface –Will support bulk submission of jobs from WMS and optimization of input/output file transfer. –Plans are to have a CE with both Condor-C and CREAM interfaces

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, Grid foundation: Storage Element Storage Element –Common interface: SRMv1,migrating to SRMv2 –Various implementation from LCG and other external projects  disk-based: DPM, dCache / tape-based: Castor, dCache –Support for ACLs in DPM (in future in Castor and dCache) –Common rfio library for Castor and DPM being added Posix-like file access: –Grid File Access Layer (GFAL) by LCG  Support for ACL in the SRM layer (currently in DPM only)  Support for SRMv2 being added

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, High Level Services: File transfer FTS: Reliable, scalable and customizable file transfer –Manages transfers through channels  mono-directional network pipes between two sites –Web service interface

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, High Level Services: Workload mgmt. WMS helps the user accessing computing resources –Resource brokering, management of job input/output,... gLite-RB: GT2 + Condor-G –To be replaced when the gLite WMS proves to be reliable gLite WMS: Web service (WMProxy) + Condor-G –Management of complex workflows (DAGs) and compound jobs  bulk submission and shared input sandboxes  support for input files on different servers (scattered sandboxes) –Job File Perusal: file peeking during job execution –Supports collection of information from CEMon, BDII, R-GMA and from DLI and StorageIndex data management interfaces –Support for parallel jobs (MPI) when the home dir is not shared –Deployed for the first time in gLite 3.0

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, High Level Services: Workflows Direct Acyclic Graph (DAG) is a set of jobs where the input, output, or execution of one or more jobs depends on one or more other jobs A Collection is a group of jobs with no dependencies –basically a collection of JDL’s A Parametric job is a job having one or more attributes in the JDL that vary their values according to parameters Using compound jobs it is possible to have one shot submission of a (possibly very large, up to thousands) group of jobs –Submission time reduction  Single call to WMProxy server  Single Authentication and Authorization process  Sharing of files between jobs –Availability of both a single Job ID to manage the group as a whole and an ID for each single job in the group nodeE nodeC nodeA nodeD nodeB

FP6−2004−Infrastructures−6-SSA E-infrastructure shared between Europe and Latin America 1 st EELA Grid School, Itacuruçá, Questions ?