Grid BIFI1 Grid computing at BIFI: Description of resources & examples of use seminar 8th June 2005 Guillermo Losilla Anadón

Grid BIFI2 BIFI: grid-related projects EGEE ("Enabling Grids for E-sciencE") –European grid for scientific use open 24 hours-a-day IrisGrid –National initiative I4Grid

Grid BIFI3 EGEE: predecessors DataGrid –First European initiative to create a pan-European Grid LCG (LHC Computing Grid) – Built to deal with the huge computing needs required to process the data that will be produced by the LHC taken from GridCafé ®

Grid BIFI4 EGEE: project structure Extremely hierarchized: –Three main areas organised into 11 activities: NA: Networking Activities (5 acts) SA: Specific Service Activities (2 acts) JRA: Joint Research Activities (4 acts) –11 partner federations BIFI SA1 SWE federation(Spain & Portugal)

Grid BIFI5 EGEE: current status Currently using LCG grid middleware EGEE grid midleware (gLite) still in development Some figures: taken from GSTAT ( 19:01h 7th June 2005 EGEE is currently the most powerful Grid infrastructure but......it is not enough for LHC!!!

Grid BIFI6 EGEE/LCG: architecture & services WMS (Workload Management System) –Manages the whole process of submitting a job to the Grid –Uses JDL (Job Description Language) for describing jobs DMS (Data Management System) –Manages the storing of data in the Grid –Identification(GUID, LFN, SURL) & copy of files –RLS(Replica Location Service) & LFC(LCG File Catalog)

Grid BIFI7 EGEE/LCG: architecture & services IS (Information System) –Provides information about the Grid resources and their status –Compliant with the GLUE schema –MDS (Monitoring & Discovery System) vs RGMA (Relational Grid Monitoring Architecture) –Source of information for Monitoring & Accounting services

Grid BIFI8 EGEE/LCG: node types UI (User Interface): point of entrance to the Grid CE (Computing Element): access point to a farm of computing nodes WN (Worker node): computing SE (Storage Element): access to storage element (disk, tape...) RB (Resource Broker): allocation and monitorization of jobs MON-BOX: Information System (RGMA-based) BDII (Berkeley DB Information Index): Information System cache PS (Proxy Server): proxy certificates renewals

Grid BIFI9 EGEE/LCG resources at BIFI 1 UI: ui-egee.bifi.unizar.es 1 CE: ce-egee.bifi.unizar.es 1 SE + 1 Mon-Box: se-egee.bifi.unizar.es 1 WN Using RB, PS & BDII from other sites(PIC & IFAE)

Grid BIFI10 EGEE: security & authorization Most services are built on top of GSI (Grid Security Infrastructure) Every user must own, not only an user account in a UI node, but a valid digital certificate (issued by a suported CA) to be able to use the Grid Authorization enabled by Virtual Organization Organizations Every user must join a virtual organization (CMS, Atlas, Alice, LHCb, SWETEST...)

Grid BIFI11 Examples of use: Acess to a UI ssh -X guillermo]$ grid-proxy-init Your identity: /C=ES/O=DATAGRID-ES/O=BIFI/CN=Guillermo Losilla Anadon Enter GRID pass phrase for this identity: Creating proxy Done Your proxy is valid until: Wed Jun 8 09:39:

Grid BIFI12 Examples of use: job submission guillermo]$ globus-job-run ce- egee.bifi.unizar.es /bin/pwd /home/swetest011 Simplest job submission: Complex job Submission tests]$ more testJob.jdl Executable = "testJob.sh"; StdOutput = "testJob.out"; StdError = "testJob.err"; InputSandbox = {"./testJob.sh"}; OutputSandbox = {"testJob.out","testJob.err"};

Grid BIFI13 Examples of use: job submission tests]$ edg-job-list-match --vo swetest testJob.jdl Selected Virtual Organisation name (from --vo option): swetest Connecting to host lcgrb02.ifae.es, port 7772 *************************************************************************** COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* ce00.inta.es:2119/jobmanager-lcgpbs-swetest ce01.lip.pt:2119/jobmanager-lcgpbs-swetest ce2.egee.cesga.es:2119/jobmanager-lcgpbs-swetest lcg-ce.usc.cesga.es:2119/jobmanager-lcgpbs-swetest lcg2ce.ific.uv.es:2119/jobmanager-lcgpbs-short ramses.dsic.upv.es:2119/jobmanager-pbs-swetest lcg2ce.ific.uv.es:2119/jobmanager-lcgpbs-swetest lcg2ce.ific.uv.es:2119/jobmanager-lcgpbs-swetestL ***************************************************************************

Grid BIFI14 Examples of use: job submission tests]$ edg-job-submit --vo swetest testJob.jdl Selected Virtual Organisation name (from --vo option): swetest Connecting to host lcgrb02.ifae.es, port 7772 Logging to host lcgrb02.ifae.es, port 9002 ********************************************************************************************* JOB SUBMIT OUTCOME The job has been successfully submitted to the Network Server. Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is: - ********************************************************************************************* tests]$ edg-job-status ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : Current Status: Scheduled Status Reason: Job successfully submitted to Globus Destination: lcg2ce.ific.uv.es:2119/jobmanager-lcgpbs-short reached on: Tue Jun 7 19:56: *************************************************************

Grid BIFI15 Examples of use: job submission tests]$ edg-job-get-output Retrieving files from host: lcgrb02.ifae.es ( for ) ********************************************************************************* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - have been successfully retrieved and stored in the directory: /tmp/jobOutput/guillermo_6scdvJ4Y1QU8yeMjH1zBhw ********************************************************************************* tests]$ ll /tmp/jobOutput/guillermo_6scdvJ4Y1QU8yeMjH1zBhw/ total 4 -rw-rw-r-- 1 guillermo guillermo 0 jun 7 22:02 testJob.err -rw-rw-r-- 1 guillermo guillermo 168 jun 7 22:02 testJob.out tests]$ more /tmp/jobOutput/guillermo_6scdvJ4Y1QU8yeMjH1zBhw/testJob.out Tue Jun 7 21:58:22 CEST 2005 wn066.ific.uv.es /home/swetest002/globustmp.wn /WMS_wn066_023996_https_3a_2f_2flcgrb02.ifae.es_3a9000_2f6scdvJ4 Y1QU8yeMjH1zBhw

Grid BIFI16 Examples of use: data management tests]$ edg-gridftp-ls gsiftp://se- egee.bifi.unizar.es/storage swetest alice atlas cms dteam lhcb sixt tests]$ edg-gridftp-mkdir gsiftp://se- egee.bifi.unizar.es/storage/swetest/prueba tests]$ edg-gridftp-ls gsiftp://se- egee.bifi.unizar.es/storage/swetest/ prueba

Grid BIFI17 Examples of use: data management $ lcg-cr -l mi_fichero_en_el_Grid -d sfn://se-egee.bifi.unizar.es/storage/swetest/test/mi_test --vo swetest file://`pwd`/mi_test Guid: guid:5def36aa-324e-4206-a631-a56093b8b02b $ lcg-lr --vo swetest guid:5def36aa-324e-4206-a631-a56093b8b02b Location: sfn://se-egee.bifi.unizar.es/storage/swetest/test/mi_test $ lcg-lr --vo swetest mi_fichero_en_el_Grid Location: sfn://se-egee.bifi.unizar.es/storage/swetest/test/mi_test $ lcg-cp --vo swetest guid:5def36aa-324e-4206-a631-a56093b8b02b file://`pwd`/mi_fichero_recuperado

Grid BIFI18 Examples of use: Information System tests]$ lcg-infosites --vo swetest se ************************************************************** These are the related data for swetest: (in terms of SE) ************************************************************** Avail Space(Kb) Used Space(Kb) Type SEs disk se01.lip.pt disk se00.inta.es disk aker.dsic.upv.es disk lcg-se.usc.cesga.es disk se2.egee.cesga.es mss castorgrid.ific.uv.es mss castorsrm.ifae.es mss castorsrm.ific.uv.es tests]$ ldapsearch -LLL -x -H ldap://ce-egee.bifi.unizar.es:2170 -b "mds-vo- name=bifi,o=grid"

Grid BIFI19 Useful links Grid computing at BIFI: omputing.html omputing.html EGEE project: LCG project: LCG-2 User Guide: UserGuide.html UserGuide.html