Production Activities and Results by ALICE Patricia Méndez Lorenzo (on behalf of the ALICE Collaboration) Service Challenge Technical Meeting CERN, 15.

Slides:



Advertisements
Similar presentations
1 ALICE Grid Status David Evans The University of Birmingham GridPP 16 th Collaboration Meeting QMUL June 2006.
Advertisements

Exporting Raw/ESD data from Tier-0 Tier-1s Wrap-up.
Status GridKa & ALICE T2 in Germany Kilian Schwarz GSI Darmstadt.
ALICE Operations short summary and directions in 2012 Grid Deployment Board March 21, 2011.
ALICE Operations short summary LHCC Referees meeting June 12, 2012.
ALICE Operations short summary and directions in 2012 WLCG workshop May 19-20, 2012.
Ian M. Fisk Fermilab February 23, Global Schedule External Items ➨ gLite 3.0 is released for pre-production in mid-April ➨ gLite 3.0 is rolled onto.
Large scale data flow in local and GRID environment V.Kolosov, I.Korolko, S.Makarychev ITEP Moscow.
Production Activities and Requirements by ALICE Patricia Méndez Lorenzo (on behalf of the ALICE Collaboration) Service Challenge Technical Meeting CERN,
16 October 2005 Collaboration Meeting1 Computing Issues & Status L. Pinsky Computing Coordinator ALICE-USA.
CHEP – Mumbai, February 2006 The LCG Service Challenges Focus on SC3 Re-run; Outlook for 2006 Jamie Shiers, LCG Service Manager.
Computing Infrastructure Status. LHCb Computing Status LHCb LHCC mini-review, February The LHCb Computing Model: a reminder m Simulation is using.
ALICE Roadmap for 2009/2010 Patricia Méndez Lorenzo (IT/GS) Patricia Méndez Lorenzo (IT/GS) On behalf of the ALICE Offline team Slides prepared by Latchezar.
INFSO-RI Enabling Grids for E-sciencE Project Gridification: the UNOSAT experience Patricia Méndez Lorenzo CERN (IT-PSS/ED) CERN,
LCG Service Challenge Phase 4: Piano di attività e impatto sulla infrastruttura di rete 1 Service Challenge Phase 4: Piano di attività e impatto sulla.
IST E-infrastructure shared between Europe and Latin America High Energy Physics Applications in EELA Raquel Pezoa Universidad.
Status of the production and news about Nagios ALICE TF Meeting 22/07/2010.
Costin Grigoras ALICE Offline. In the period of steady LHC operation, The Grid usage is constant and high and, as foreseen, is used for massive RAW and.
CCRC’08 Weekly Update Jamie Shiers ~~~ LCG MB, 1 st April 2008.
1 LCG-France sites contribution to the LHC activities in 2007 A.Tsaregorodtsev, CPPM, Marseille 14 January 2008, LCG-France Direction.
The ALICE Distributed Computing Federico Carminati ALICE workshop, Sibiu, Romania, 20/08/2008.
Status of PDC’07 and user analysis issues (from admin point of view) L. Betev August 28, 2007.
The LHC Computing Grid – February 2008 The Challenges of LHC Computing Dr Ian Bird LCG Project Leader 6 th October 2009 Telecom 2009 Youth Forum.
Author: Andrew C. Smith Abstract: LHCb's participation in LCG's Service Challenge 3 involves testing the bulk data transfer infrastructure developed to.
1 LHCb on the Grid Raja Nandakumar (with contributions from Greig Cowan) ‏ GridPP21 3 rd September 2008.
1 Andrea Sciabà CERN Critical Services and Monitoring - CMS Andrea Sciabà WLCG Service Reliability Workshop 26 – 30 November, 2007.
Summary of Services for the MC Production Patricia Méndez Lorenzo WLCG T2 Workshop CERN, 12 th June 2006.
1 WLCG-GDB Meeting. CERN, 12 May 2010 Patricia Méndez Lorenzo (CERN, IT-ES)
Handling ALARMs for Critical Services Maria Girone, IT-ES Maite Barroso IT-PES, Maria Dimou, IT-ES WLCG MB, 19 February 2013.
SC4 Planning Planning for the Initial LCG Service September 2005.
Plans for Service Challenge 3 Ian Bird LHCC Referees Meeting 27 th June 2005.
Large scale data flow in local and GRID environment Viktor Kolosov (ITEP Moscow) Ivan Korolko (ITEP Moscow)
Materials for Report about Computing Jiří Chudoba x.y.2006 Institute of Physics, Prague.
LCG Service Challenges SC2 Goals Jamie Shiers, CERN-IT-GD 24 February 2005.
Distributed Physics Analysis Past, Present, and Future Kaushik De University of Texas at Arlington (ATLAS & D0 Collaborations) ICHEP’06, Moscow July 29,
ALICE experiences with CASTOR2 Latchezar Betev ALICE.
Status of AliEn2 Services ALICE offline week Latchezar Betev Geneva, June 01, 2005.
Service Challenge Report Federico Carminati GDB – January 11, 2006.
8 August 2006MB Report on Status and Progress of SC4 activities 1 MB (Snapshot) Report on Status and Progress of SC4 activities A weekly report is gathered.
CMS: T1 Disk/Tape separation Nicolò Magini, CERN IT/SDC Oliver Gutsche, FNAL November 11 th 2013.
PDC’06 - status of deployment and production Latchezar Betev TF meeting – April 27, 2006.
SAM Status Update Piotr Nyczyk LCG Management Board CERN, 5 June 2007.
GRID interoperability and operation challenges under real load for the ALICE experiment F. Carminati, L. Betev, P. Saiz, F. Furano, P. Méndez Lorenzo,
INFSO-RI Enabling Grids for E-sciencE File Transfer Software and Service SC3 Gavin McCance – JRA1 Data Management Cluster Service.
Status of GSDC, KISTI Sang-Un Ahn, for the GSDC Tier-1 Team
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES The Common Solutions Strategy of the Experiment Support group.
ALICE Physics Data Challenge ’05 and LCG Service Challenge 3 Latchezar Betev / ALICE Geneva, 6 April 2005 LCG Storage Management Workshop.
Gestion des jobs grille CMS and Alice Artem Trunov CMS and Alice support.
Dominique Boutigny December 12, 2006 CC-IN2P3 a Tier-1 for W-LCG 1 st Chinese – French Workshop on LHC Physics and associated Grid Computing IHEP - Beijing.
SAM architecture EGEE 07 Service Availability Monitor for the LHC experiments Simone Campana, Alessandro Di Girolamo, Nicolò Magini, Patricia Mendez Lorenzo,
VO Box discussion ATLAS NIKHEF January, 2006 Miguel Branco -
LCG GDB – Nov’05 1 Expt SC3 Status Nick Brook In chronological order: ALICE CMS LHCb ATLAS.
The ALICE Production Patricia Méndez Lorenzo (CERN, IT/PSS) On behalf of the ALICE Offline Project LCG-France Workshop Clermont, 14th March 2007.
ATLAS Computing Model Ghita Rahal CC-IN2P3 Tutorial Atlas CC, Lyon
LHCC meeting – Feb’06 1 SC3 - Experiments’ Experiences Nick Brook In chronological order: ALICE CMS LHCb ATLAS.
Pledged and delivered resources to ALICE Grid computing in Germany Kilian Schwarz GSI Darmstadt ALICE Offline Week.
CERN IT Department CH-1211 Genève 23 Switzerland t EGEE09 Barcelona ATLAS Distributed Data Management Fernando H. Barreiro Megino on behalf.
Availability of ALICE Grid resources in Germany Kilian Schwarz GSI Darmstadt ALICE Offline Week.
Federating Data in the ALICE Experiment
ARDA-ALICE activity in 2005 and tasks in 2006
Data Challenge with the Grid in ATLAS
INFN-GRID Workshop Bari, October, 26, 2004
CMS transferts massif Artem Trunov.
LHCb Computing Model and Data Handling Angelo Carbone 5° workshop italiano sulla fisica p-p ad LHC 31st January 2008.
Readiness of ATLAS Computing - A personal view
Installation and Commissioning of ALICE VO-BOXES and AliEn Services
Simulation use cases for T2 in ALICE
ALICE – FAIR Offline Meeting KVI (Groningen), 3-4 May 2010
LHC Data Analysis using a worldwide computing grid
The LHCb Computing Data Challenge DC06
Presentation transcript:

Production Activities and Results by ALICE Patricia Méndez Lorenzo (on behalf of the ALICE Collaboration) Service Challenge Technical Meeting CERN, 15 th September 2006

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September 2006 Outline  PDC’06/SC4 goals and tasks  1 st Phase: Principles of operation and results  2 nd Phase: Principles of operation and results  Conclusions and Plans

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Main tasks of the ALICE Physics Data Challenge 2006 (PDC’06)  Validation of the LCG/gLite workload management services  Stability of the services is fundamental for the entire duration of the exercise  Validation of the data transfer and storage services  2 nd phase of the PDC’06 (currently fundamental part)  The stability and support of the services have to be assured beyond the throughput tests  Validation of the ALICE distributed reconstruction and calibration model  Integration of all Grid resources within one single – interfaces to different Grids (LCG, OSG, NDGF)  End-user data analysis

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September PDC’06 Phases  First phase (ongoing):  Production of p+p and Pb+Pb MC events  Conditions and samples agreed with PWGs  Data migrated from all Tiers to  Second phase:  Scheduled data transfers T0-T1  Reconstruction of RAW data: 1 st pass reconstruction at CERN, 2 nd pass at T1  Scheduled data transfers T2- (supporting)T1  Third phase (end of the year)  End-user analysis on the GRID

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Principles of Operation: VO-box  VO-boxes deployed at all T0-T1-T2 sites providing resources for ALICE  Mandatory requirement to enter the production  Required in addition to all standard LCG Services  Entry door to the LCG Environment  Runs standard LCG components and ALICE specific ones  Uniform deployment  Same behaviour for T1 and T2 in terms of production  Differences between T1 and T2 a matter of QoS only.  Installation and maintenance entirely ALICE responsibility  Based on a regional principle  Set of ALICE experts matched to groups of sites  Site related problems handled by site administrators  LCG Service problems reported via GGUS  Not too much, ALICE has delivered experts in almost all sites

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September General Services Status  For production ALICE requires (beyond VO- BOXES), dedicated ALICE queues and if existing access to the local RB  Good respond of sites, good level of performance  Established a quite well define system of SW installation/running/monitoring/support  Important number of T2 sites joining (almost) continuously the production  However production unstable in some sites oCompetition with other VOs oLocal RB problems oProblems at the site (i.e space in local disk for each job, etc) oThe experts team is able to find and recover the problem in a relative short time

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September ALICE in the World

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Results for the PDC`06  Continuous running from April 2006

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Results of the PDC`06 (2)  Gradual inclusion of sites in the ALICE Grid - current status:  6 T1s: CCIN2P3, CERN, CNAF, GridKA, NIKHEF, RAL  30 T2s  Currently available CPU power – 2000 CPUs for ALICE (expected ~4000)  Competing for resources with the other LHC experiments  Computing centres are waiting for the last moment to buy hardware – will get more for the same price  Expect additional resources from Nordic countries and from US (LBL and LLNL)

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Resources Statistics  Resources contribution (normalized Si2K units): 50% from T1s, 50% from T2s  The role of the T2 remains very high!

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September nd Part of PDC`06  T0-T1 transfers: Methodology and Status  Next Part of the talk  Asynchronous to the Production (1st phase)  Combining the production with this exercise

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Principle of Operations: FTS and LFC  FTS Service deployed at all sites  Used for scheduled replication of data between computing centers  Lower level tool that underlies the data placement  Used as plug-in in the AliEn File Transfer Daemon (FTD) oFTS has been implemented through the FTS Perl APIs oFTD running in the VO-box as one of the ALICE services  LFC required at all sites  Used as a local catalogue for the site  Access to the SRM SE at all sites also required

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September FTS File Replication Job 1 Job 2 Job 3 Submits job User ALICETrnasfer queue ALICE central services VO-Box LCG User Job ALICE catalogues Transfer broker Origin SA SURL LFC GUID Destination SA LFC Update ALICE File Catalogue lfnguid{se’s} lfnguid{se’s} lfnguid{se’s} Optimizer Space reservation to SRM Update FC SRM FTD

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September FTS Tests: Strategy  The main goal is to test the stability of FTS as service and integration with FTD  T0-T1 (disk to tape): 7 days required of sustained transfer rates to all T1s oExercise still continuing  T1-T2 (disk to disk) and T1-T1 (disk to disk): 2 days required of sustained transfers to T2 oSTILL TO BE FULLY PERFORMED  Data types  T0-T1: Migration of raw and 1 st pass reconstructed data  T1-T2 and T2-T1: Transfers of ESDs, AODs (T1-T2) and T2 MC production for custodial storage (T2-T1)  T1-T1: Replication of ESDs and AODs

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September FTS: Transfer Rates  T0-T1: disk-tape transfers at an aggregate rate of 300MB/s from CERN  Distributed according the MSS resources pledged by the sites in the LCG MoU: oCNAF: 20% oCCIN2P3: 20% oGridKA: 20% oSARA: 10% oRAL: 10% oUS (one center): 20%  T1-T2: Following the T1-T2 relation matrix  Test of the services performance, no specific target for transfer rates

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September FTS Tests Remarks  The FTS transfers will not be synchronous with the data production  The sites should provide mechanism for garbage collector  Transfers based on LFN is not required  The automatic update of the LFC catalogue is not required  ALICE will take care of the catalogues update  Summary of requirements:  ALICE FTS Endpoints at the T0 and T1  SRM-enabled storage with automatic data deletion if needed  FTS service at all sites  Support during the whole tests (and beyond)

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Monitor Tools  We are controlling the status of the transfers with different tools  MonaLisa controls all FTD status  Dashboard follows the FTS errors  The status of the transfers are fully monitored also in the VO-BOXES through the FTD logs  All problems have been reported inmediatly using GGUS  Good support of the SC Experts

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Data Movement: MonaLisa

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Data Movement:DashBoard

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Status of FTS Transfers  These are the problems we are facing at this moment  Problems with Castor at the origin (CERN) and the destination (CNAF)  FTS server at CERN hanging  Problems with the access to the catalogue in all sites oSetcomment API may be having problems  Certain instabilities found in the VOBOXES oCERN (substituted by a new VOBOX) and SARA  Recovering now the transfers  From the ALICE site  Increase the size of the transferred files oDone  Increase the number of simultaneous transfers per site oDone, increased to 100 simultaneous transfers to all sites

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September After ALICE actions

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Proposed ALICE T1-T2 Connections  CCIN2P3  French T2s, Sejong (Korea), Lyon T2, Madrid (Spain)  CERN  Cape Town (South Africa), Kolkatta (India), T2 Federation (Romania), RMKI (Hungary), Athens (Greece), Slovakia, T2 Federation (Poland), Wuhan (China)  FZK  FZU (Czech Republic), RDIG (Russia), GSI and Muenster (Germany)  CNAF  ItalianT2sa) Still to be decided if this proposal  RAL is approved  Birmingham  SARA/NIKHEF b) Status of FTS services at T2  NDGF to be checked  PDSF  Houstonc) ALICE T2 sites aware of the requirements

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Conclusions  The ALICE PDC`06  Complete test of the ALICE computing model and Grid services readiness for data taking in 2007  Production of data ongoing, integration of LCG and ALICE specific services through the VO-box framework progressing extremely well  Building of support infrastructure and relations with ALICE sites is on track  The 2nd Phase of PDC’06 is a fundamental issue  Stability  Efficiency in the tickets response  Still to establish certain strategy points for the T1-T2 transfers  The 3rd phase (end-user analysis) at the end of the year

Patricia Mendez Lorenzo Service Challenge Technical Meeting 15th September Plans for Next Year  Continue with MC production for physics performance and detector studies  List of events/conditions from ALICE PWGs is growing  Continue the tests of the storage facilities at the sites and file replication of RAW data (CERN>>T1) and ESD/AOD (T1 >T2) through the FTS service  Continue and improve the user analysis of the Grid (starting October 2006) and the CAF  Include all detectors commisioning exercises and test data  Continue the build-up of ALICE related experts support at the computing centers providing ALICE resources