The Grid Production infrastructure Cristina Vistoli INFN CNAF

INFN-Grid – goals Promote computational grid technologies research & development: Middleware and grid tools Through european and national projects DataGrid, DataTAG, Firb-GRID-it, EGEE, LCG, CoreGRID etc Internal R&D activities Deploy – operate - support INFN grid production infrastructure: Grid as “coordinated resource sharing” on a large scale for a multi-institutional and dynamic virtual organisation Set up the national production grid Infrastructure open to the national research community FIRB: Grid.it – astrophysic, geophysic, biomedicine, computational chemistry etc Reserch community and industry

INFN-Grid – goals Provide operation and support of the EGEE/LCG production infrastructure Promote dissemination activity to ‘gridify’ scientific applications GILDA testbed Genius portal

INFN-GRID partecipation to EGEE EGEE SA1 – infrastructure operation and support ROC – Regional Operation Center CIC - Core Infrastructure Center EGEE JRA1 – IT-CZ cluster Workload management system Resource access - CE – accounting - policy VOMS EGEE NA4/NA3/NA2/NA5 HEP application Generic application Dissemination

INFN-GRID partecipation to LCG LHC Computing Grid main sites: T1 and n*T2 LHC Experiments and applications support Operation and deployment of the Grid infrastructure (national and international)

INFN-GRID participation to CoreGRID The CoreGRID Network of Excellence (NoE) aims at strengthening and advancing scientific and technological excellence in the area of Grid and Peer-to-Peer technologies Grid Information and Monitoring Services Knowledge & Data Management

INFN-GRID partecipation to GRID.IT/FIRB Set up the national production grid Infrastructure open to the national research community Grid management and support tools - system First tools in production R&D on Resource Utilization Policies Data Management Scientific Data Base grid integration Middleware porting

Italian – Grid (Site/resource map) INFN VO CMS Atlas Alice LHCb Babar VIRGO grid.it resources and VO TRENTO MILANO UDINE TORINO PADOVA LNL PAVIA FERRARA TRIESTE National Grid ) GENOVA PARMA CNAF BOLOGNA PISA FIRENZE S.Piero PERUGIA LNGS ROMA ROMA2 L’AQUILA LNF SASSARI NAPOLI BARI SALERNO LECCE CAGLIARI COSENZA PALERMO CATANIA LNS

Grid-it Status 22 Resource Centres 1 Tier1 : CNAF 4 Tier2: Roma1(2), Milano, Torino, LNL 14 siti INFN: Bologna(2), Bari, Catania, Ferrara, Firenze,Lecce, LNF, Napoli (3), Padova, Perugia, Pisa, Pavia, Roma2, Trieste, Cagliari 3 siti non INFN: INAF-TS, Uni-Na, Sns-Pisa Servizi : RBs, BDIIs, VOMS, VO-LDAP, Gridice servers, RLS……

INFN-GRID: Resources and supported VOs (**) Hyperthreaded

INFN-GRID Release INFN-GRID is a customized release of LCG All resources are fully managed via LCFGng; INFN-GRID does not support the middleware installation without LCFGng; Change with the next release based on SL3:YAIM and Quattor INFN-GRID 2.3.0 release is based upon the official LCG-2.3.0 and it is 100% compatible;

Grid.IT Production Grid: deployment portal User documentation site managers documentation Software repository Monitoring Trouble tickets system Knowledge base http://grid-it.cnaf.infn.it

INFN-GRID Release Main differences from LCG 2.3.0 to INFN-GRID 2.3.0: Added support for DAG jobs; Added support for AFS on the WorkerNodes; Added support for MPI jobs via home syncronisation with ssh; Documented installation of WNs on a private network; Added full function VOMS support: INFNGRID, CDF, COMPCHEM, PLANCK are completely managed via VOMS server.

grid-it … Cnaf/T1, LNL, To, Roma1,Milano, Padova, Napoli,…. Experiment Support EGEE/LCG CICs Controllo dei Servizi e dei Resource Centers, procedure di deployment, Produzione Release e certificazione Grid-it management Supporto Esperimenti, Virtual Organizations, Applicazioni Scientifiche CIC-On-Duty Cnaf/T1, LNL, To, Roma1,Milano, Padova, Napoli,…. Servizi GRID di Esperimento e/o di infrastruttura: RBs, VOMS, RLS, GIS, Monitoring…. grid-it Italian Roc Grid-it Operation-Support … CERN Spanish-Grid UK-Grid

Manage the Problem List Support workflow FAQ GOC Tools GridICE Gppmon Site CERT Gstat Etc… Problem 1001 Problem 1002 Problem 1003 Problem 1004 Problem 1005 Problem 1006 Problem 1007 Problem List Problem 1001 Problem 1002 Problem 1003 Problem 1004 Problem 1005 Problem 1006 Problem 1007 Problem List Problem 1001 Problem 1002 Problem 1003 Problem 1004 Problem 1005 Problem 1006 Problem 1007 Problem List Manage the Problem List DOC ROC ROC ROC ROC ROC RC (site) RC (site) RC (site) RC (site) RC (site) RC (site) RC (site) RC (site) RC (site)

CIC-On-Duty (P.Veronesi, A.Cavalli) Shift settimanale di controllo infrastruttura europea Interazione con Italian ROC e altri ROC europei

Riunioni periodiche di persona Iniziate a fine giugno phone conference periodiche (bisettimanali) di grid di produzione, EGEE-SA1 + site manager http://infnforge.cnaf.infn.it/cdsagenda/displayLevel.php?fid=4 Riunioni periodiche di persona Realizzazione release di middleware INFN-GRID – Release Team Gestione strumenti di installazione automatica Repository software e configurazioni Integrazione nuove funzionalità e certificazione Procedure di installazione, guide d’uso etc. sia automatiche che manuali, anche per SL

Supporto Realizzata la ‘checklist del turnista diligente’ con la collaborazione di tutti le sedi della Grid di produzione Istituiti turni 8.30 – 14.00 e 14.00 – 19.30, 5 giorni la settimanadi controllo Grid di produzione, risposta ai ticket, controllo dei problemi riscontrati a livello di CIC, stato dei servizi 2 persone per turno Report di fine turno per logging delle attivita’ pendenti, chat channel per colloquio durante il turno Siamo alla 2 settimana di turno, sistemato procedure e srtumenti siamo pronti per supportare VO CIC-on duty : turno settimanale, gli output verso l’italia sono gestiti dai turni nazionali

Ticketing system INFN-GRID ticketing system is used: from users to ask questions or to communicate troubles; from system manager to communicate about common grid tasks (ex: upgrading to a new grid release) from CMT to system manager to notify a problem Support Groups are “helper” groups and they exist to resolve the obvious problems arising with the grow of the grid: Support Grid Services (RB, RLS, VOMS, GridICE, etc) Group; Support VO Services Group (each for every VO); Support VOApplications Group (each for every VO); Support Site Group (each for every site) Operative Groups Operative Central Management Team (CMT); Operative Release & Deployment Team; Users -> Create a ticket Supporters/Operatives -> Open the ticket Users and/or Supporters/Operatives -> Update an open ticket Supporters/Operatives -> Close the ticket

EGEE/LCG: Production Grid services RB-BDII scope all european resources EGEE/LCG RB/UI with DAG Service Resources are open to all VOs supported by INFN-GRID and EGEE/LCG RB: egee-rb-01.cnaf.infn.it support BIOMED VO

Grid-it: Production Grid service Service Resources are open to all VOs supported RB-BDII scope Italian Grid NEW! Resource Broker/UI DAG prod-rb-01.pd.infn.it

Certification activity – TEST ZONE The Central Management Team is responsible of the resource centers certification: checking the functionalities of a site before joining the site to the production grid. Although all certification jobs are VO independent, the INFNGRID VO is used to perform these jobs; In particular are checked: GIIS' information consistence; Local jobs submission (LRMS); Grid submission with Globus (globus-job-run); Grid submission with the ResorceBroker; ReplicaManager functionalities; MPI functionalities In order to certificate a site the CMT uses dedicated grid services: RB & BDII: gridit-cert-rb.cnaf.infn.it In this way we avoid to have an uncertified site in the production grid services;

Attivita’ in corso Sistema di supporto: integrazione in EGEE e copertura supporto distribuito Evoluzione di Gridice per job monitoring, application monitoring, SLA monitoring, urgente configurazione notifiche Integrazione di DGAS in INFN-GRID  amministrazione sistema di accounting Porting di INFN-GRID a SL : nuovo sistema di installazione e configurazione Operation support infrastruttura EGEE/LCG a ‘rotazione’ tra IT/CERN/UK/FR Training: corso base e avanzato Allargamento infrastruttura a sedi non INFN: Spaci, Enea, etc Amministrazione Policy Pre-production service per definire il programma di migrazione a Glite Middleware certification testbed Operational requirements per il middleware

Useful links INFN Grid INFN production GRID infrastructure http://grid.infn.it INFN production GRID infrastructure http://grid-it.cnaf.infn.it/ http://grid-it.cnaf.infn.it/index.php?id=sa1italy INFN GRID development projects portal http://infnforge.cnaf.infn.it INFN GridICE http://grid-it.cnaf.infn.it/index.php?grisview&type=1 INFN Support http://grid-it.cnaf.infn.it/index.php?id=51&type=1 Contact eb@infn.it (management board) tb-grid@infn.it (technical board) grid-manager@infn.it (production grid management team)