ATLAS CNAF in 2010  ATLAS data distribution: o import from T0, import/export from/to Atlas tier1s and It tiers2 o data disk crisis in May.

Slides:



Advertisements
Similar presentations
Grid and CDB Janusz Martyniak, Imperial College London MICE CM37 Analysis, Software and Reconstruction.
Advertisements

Sep Donatella Lucchesi 1 CDF Status of Computing Donatella Lucchesi INFN and University of Padova.
22/04/2005Donatella Lucchesi1 CDFII Computing Status OUTLINE:  New CDF-Italy computing group organization  Usage status at FNAL and CNAF  Towards GRID:
Ian M. Fisk Fermilab February 23, Global Schedule External Items ➨ gLite 3.0 is released for pre-production in mid-April ➨ gLite 3.0 is rolled onto.
WLCG/8 July 2010/MCSawley WAN area transfers and networking: a predictive model for CMS WLCG Workshop, July 7-9, 2010 Marie-Christine Sawley, ETH Zurich.
Ian Fisk and Maria Girone Improvements in the CMS Computing System from Run2 CHEP 2015 Ian Fisk and Maria Girone For CMS Collaboration.
December 17th 2008RAL PPD Computing Christmas Lectures 11 ATLAS Distributed Computing Stephen Burke RAL.
Zhiling Chen (IPP-ETHZ) Doktorandenseminar June, 4 th, 2009.
US ATLAS Western Tier 2 Status and Plan Wei Yang ATLAS Physics Analysis Retreat SLAC March 5, 2007.
Alexandre A. P. Suaide VI DOSAR workshop, São Paulo, 2005 STAR grid activities and São Paulo experience.
Claudio Grandi INFN Bologna CMS Operations Update Ian Fisk, Claudio Grandi 1.
Remote Production and Regional Analysis Centers Iain Bertram 24 May 2002 Draft 1 Lancaster University.
Jean-Yves Nief CC-IN2P3, Lyon HEPiX-HEPNT, Fermilab October 22nd – 25th, 2002.
Data Import Data Export Mass Storage & Disk Servers Database Servers Tapes Network from CERN Network from Tier 2 and simulation centers Physics Software.
LCG Service Challenge Phase 4: Piano di attività e impatto sulla infrastruttura di rete 1 Service Challenge Phase 4: Piano di attività e impatto sulla.
Interactive Job Monitor: CafMon kill CafMon tail CafMon dir CafMon log CafMon top CafMon ps LcgCAF: CDF submission portal to LCG resources Francesco Delli.
Status of the production and news about Nagios ALICE TF Meeting 22/07/2010.
F. Fassi, S. Cabrera, R. Vives, S. González de la Hoz, Á. Fernández, J. Sánchez, L. March, J. Salt, A. Lamas IFIC-CSIC-UV, Valencia, Spain Third EELA conference,
1. Maria Girone, CERN  Q WLCG Resource Utilization  Commissioning the HLT for data reprocessing and MC production  Preparing for Run II  Data.
1 LHCb on the Grid Raja Nandakumar (with contributions from Greig Cowan) ‏ GridPP21 3 rd September 2008.
CERN IT Department CH-1211 Genève 23 Switzerland t Frédéric Hemmer IT Department Head - CERN 23 rd August 2010 Status of LHC Computing from.
May Donatella Lucchesi 1 CDF Status of Computing Donatella Lucchesi INFN and University of Padova.
Service Availability Monitor tests for ATLAS Current Status Tests in development To Do Alessandro Di Girolamo CERN IT/PSS-ED.
Large scale data flow in local and GRID environment Viktor Kolosov (ITEP Moscow) Ivan Korolko (ITEP Moscow)
LHCb report to LHCC and C-RSG Philippe Charpentier CERN on behalf of LHCb.
Computing Issues for the ATLAS SWT2. What is SWT2? SWT2 is the U.S. ATLAS Southwestern Tier 2 Consortium UTA is lead institution, along with University.
ATLAS Distributed Computing perspectives for Run-2 Simone Campana CERN-IT/SDC on behalf of ADC.
Distributed Physics Analysis Past, Present, and Future Kaushik De University of Texas at Arlington (ATLAS & D0 Collaborations) ICHEP’06, Moscow July 29,
Jiri Chudoba for the Pierre Auger Collaboration Institute of Physics of the CAS and CESNET.
Belle II Computing Fabrizio Bianchi INFN and University of Torino Meeting Belle2 Italia 17/12/2014.
Status of GSDC, KISTI Sang-Un Ahn, for the GSDC Tier-1 Team
1 June 11/Ian Fisk CMS Model and the Network Ian Fisk.
Grid technologies for large-scale projects N. S. Astakhov, A. S. Baginyan, S. D. Belov, A. G. Dolbilov, A. O. Golunov, I. N. Gorbunov, N. I. Gromova, I.
Predrag Buncic CERN Plans for Run2 and the ALICE upgrade in Run3 ALICE Tier-1/Tier-2 Workshop February 2015.
BaBar & Grid Eleonora Luppi for the BaBarGrid Group TB GRID Bologna 15 febbraio 2005.
1 P. Murat, Mini-review of the CDF Computing Plan 2006, 2005/10/18 An Update to the CDF Offline Plan and FY2006 Budget ● Outline: – CDF computing model.
The CMS Beijing Tier 2: Status and Application Xiaomei Zhang CMS IHEP Group Meeting December 28, 2007.
LHCb Computing 2015 Q3 Report Stefan Roiser LHCC Referees Meeting 1 December 2015.
1-2 March 2006 P. Capiluppi INFN Tier1 for the LHC Experiments: ALICE, ATLAS, CMS, LHCb.
1 M. Paganoni, 17/1/08 Modello di calcolo di CMS M. Paganoni Workshop Storage T2 - 17/01/08.
Management of the Data in Auger Jean-Noël Albert LAL – Orsay IN2P3 - CNRS ASPERA – Oct Lyon.
ATLAS – statements of interest (1) A degree of hierarchy between the different computing facilities, with distinct roles at each level –Event filter Online.
Experiments and User Support
The Beijing Tier 2: status and plans
Xiaomei Zhang CMS IHEP Group Meeting December
L’analisi in LHCb Angelo Carbone INFN Bologna
LcgCAF:CDF submission portal to LCG
AWS Integration in Distributed Computing
LCG Service Challenge: Planning and Milestones
Overview of the Belle II computing
Added value of new features of the ATLAS computing model and a shared Tier-2 and Tier-3 facilities from the community point of view Gabriel Amorós on behalf.
Data Challenge with the Grid in ATLAS
ATLAS activities in the IT cloud in April 2008
Update on Plan for KISTI-GSDC
Status and Prospects of The LHC Experiments Computing
LHCb Computing Model and Data Handling Angelo Carbone 5° workshop italiano sulla fisica p-p ad LHC 31st January 2008.
Dagmar Adamova (NPI AS CR Prague/Rez) and Maarten Litmaath (CERN)
1 VO User Team Alarm Total ALICE ATLAS CMS
Evolution of the distributed computing model The case of CMS
Job workflow Pre production operations:
Artem Trunov and EKP team EPK – Uni Karlsruhe
Project Status Report Computing Resource Review Board Ian Bird
Universita’ di Torino and INFN – Torino
R. Graciani for LHCb Mumbay, Feb 2006
LHC Data Analysis using a worldwide computing grid
CC and LQCD dimanche 13 janvier 2019dimanche 13 janvier 2019
ATLAS DC2 & Continuous production
The ATLAS Computing Model
Dirk Duellmann ~~~ WLCG Management Board, 27th July 2010
The LHCb Computing Data Challenge DC06
Presentation transcript:

ATLAS CNAF in 2010  ATLAS data distribution: o import from T0, import/export from/to Atlas tier1s and It tiers2 o data disk crisis in May due to a delay of installation of pledged resources o 2011: data to be sent to cnaf according to pledged resources  MC production o no major issues o 2011: same load expected (new 8TeV)  DATA re-processing: May and November o In Nov reprocessed all 2010 p-p data o with available resources CNAF can take charge of more data to reprocess o HI data processing now also at cnaf o HI data to be reprocessed in Jan/Feb 2011  User ANALYSIS o started on July. new analysis schema: dynamical data distribution based on dataset popularity o observed high load of network resources o in 2011 expected similar or higher load

Comparazione tra i T1 di CMS: Metrica relativa alla stabilità del sito calcolata in base al successo di diversi test: SAM, JobRobot, Good links CNAF tra i migliori come stabilità Downtime critici: - problemi configurazione utenti (Marzo) - upgrade sistema di storage (Giugno/Luglio) - upgrade kernel (Settembre) - problemi GPFS (Ottobre) Problemi recenti: problemi di autorizzazione quando sono stati aggiunti due pool account nuovi per HI (l'aggiunta dei nuovi ruoli è stata un po' faticosa forse è necessaria una maggiore coordinazione) Richiesta sviluppo info provider per lo storage rimasta pending per mesi attualmente in fase di risoluzione

Trasferimenti Volume cumulativo di dati trasferiti dal CNAF nell'ultimo anno. Prevalenza dei T2 italiani Legnaro Pisa Job di CMS a CNAF Reprocessing Prompt skimming Da Agosto 2010 è iniziata l'attività di produzione montecarlo. (Prima solo ai T2)

Jobs succeded&failed vs time in all T1’s (last month) Quite stable running conditions for CNAF

Percentage of successes&failures CNAF had the lowest fraction of job failures (note: failures include human mistakes by production manager of analysis users): 85% & 15%

Share of succeded jobs amongst T1’s CNAF had the largest absolute number of succeded jobs (just after CERN) And one of the smallest for failed jobs (second only to smaller sites)

GPFS NSD  farm I/O last month Average of 266 MB/s with peaks up to 1.5 GB/s in output, negligible in input (the bulk of I/O is made by analysis user jobs which read a lot of data and produce small ntuples) Running jobs (T1 queue) last month Last week number of jobs increased  Start of 2010 data reprocessing  More analysis jobs to show results at the LHCb week

ALICE FARMING ALICE Activity at the Tier1: MC production RAW->ESD reconstruction User Analysis ALICE::CNAF activity last month: CNAF  14% of ALICE(last month) ALICE LSF queue last week In the last period there was an intensive activity of ALICE at CNAF Smaller sites are neglected in this plot

ALICE STORAGE 2 storage use cases: T1D0 (disk=385T), T0D1(disk=770T) -> 2 filesystems T1D0 Transfers T0->T1, T1->T1 raw data Target bandwidth 60MB/s T0D1 last week: 20 MB/s (MC) last week: 200 MB/s (analysis)

CDF Activities at CNAF in 2010 Moved CDF dedicated hardware to Virtual Machines Moved to SL5 on the farm  Validated experiment software  Tested official MC and Data catalog interface Tested succesfully data production for the first time outside FNAL, a relevant part of CDF reprocessing will be done at CNAF Continued with the usual data analysis activities Farm headnode - cdfhead.cnaf.infn.it (in progress) SAM station - cdfsamsrv.cr.cnaf.infn.it Parrot Code Server - parrot.cr.cnaf.infn.it Data Server - cdfdata01.cr.cnaf.infn.it Proxy Server - squid-cdf-2.cr.cnaf.infn.it test machines Running jobs 1k

BaBar activity 2010 at CNAF CNAF maintains a copy of all exp. data sample plus 1/3 of Charm events skim plus Y2S and Y3S data samples Skimming and simulation of new signal MC requested by Analysis Working Groups Analysis support (intense analysis period) at: SLAC, IN2P3, CNAF, Gridka 2010 Analysis performed at CNAF:  8 Charm analysis submitted/accepted to Journals  O(10) active users

SuperB activity 2010 at CNAF Design and development of distributed simulation production system: very fruitful interaction with CNAF experts CNAF is the center of distributed production system three successful official productions performed in 2010 core production services resides at CNAF  Bookkeeping DB (REST job interface), GANGA as submission system, Web production tools, Grid services: UI, SE(StoRM), WMS, VOMS, LFC Central official MC data repository (gpfs_superb) Most of analysis activity performed at CNAF farm

CSN2 Cosmic Ray Study ARGO-YBJ Surface Detector (in YBJ-Tibet m a.s.l). MAGIC Cherenkov Telescope (in La Palma Canary Islands, Spain) AUGER Surface + Fluorescence Detectors (in Malargue, Argentina) FERMI Gama-Ray Space Telescope PAMELA Space Observatory at 1AU AMS-02 Alpha Magnetic Spectrometer (on the main truss of the ISS) Gravitational Wave Study VIRGO interferometric detector in Cascina near Pisa

MAIN ACTIVITIES IN 2010 Data distribution ARGO-YBJ: data transfer from YBJ to CNAF on disk and backup on tape; data mirroring to Roma3 VIRGO: data transfer from Cascina to CNAF on disk and backup on tape AMS-02: data transfer from CERN to CNAF on tape PAMELA: data transfer from Mephi to CNAF on tape AUGER: Data mirroring from Lyon to CNAF

MAIN ACTIVITIES IN 2010 MC Production: local and grid usage of resources EAS Simulations, nucleons, electrons Detector response simulations Data processing ARGO-YBJ : raw data reconstruction; re-processing of 2008 raw data Data Reduction PAMELA: 2 data reduction per year User ANALYSIS ARGO-YBJ: EAS reconstructed analysis and user dedicated analysis

Atlas: Claudia Ciocca Lorenzo Rinaldi (Manoj Jha) CMS: Daniele Andreotti Chiara Genta LHCB: Paolo Franchini Alice: Francesco Noferini Babar: Armando Fella CDF: Gabriele Compostella CSN2: Anna Karen Calabrese Melcarne Giovanni Zizzi