2007 Workshop on INFN ComputingRimini, 7 May 2007 Conditions Database Service implications at Tier1 and Tier2 sites Andrea Valassi (CERN IT-PSS)

Slides:



Advertisements
Similar presentations
CERN - IT Department CH-1211 Genève 23 Switzerland t LCG Persistency Framework CORAL, POOL, COOL – Status and Outlook A. Valassi, R. Basset,
Advertisements

1 Databases in ALICE L.Betev LCG Database Deployment and Persistency Workshop Geneva, October 17, 2005.
RLS Production Services Maria Girone PPARC-LCG, CERN LCG-POOL and IT-DB Physics Services 10 th GridPP Meeting, CERN, 3 rd June What is the RLS -
CERN - IT Department CH-1211 Genève 23 Switzerland t Relational Databases for the LHC Computing Grid The LCG Distributed Database Deployment.
Ian M. Fisk Fermilab February 23, Global Schedule External Items ➨ gLite 3.0 is released for pre-production in mid-April ➨ gLite 3.0 is rolled onto.
CERN - IT Department CH-1211 Genève 23 Switzerland t Partitioning in COOL Andrea Valassi (CERN IT-DM) R. Basset (CERN IT-DM) Distributed.
Database monitoring and service validation Dirk Duellmann CERN IT/PSS and 3D
LCG 3D StatusDirk Duellmann1 LCG 3D Throughput Tests Scheduled for May - extended until end of June –Use the production database clusters at tier 1 and.
LCG Database Workshop Summary and Proposal for the First Distributed Production Phase Dirk Duellmann, CERN IT (For the LCG 3D Project :
SC4 Workshop Outline (Strong overlap with POW!) 1.Get data rates at all Tier1s up to MoU Values Recent re-run shows the way! (More on next slides…) 2.Re-deploy.
Computing Infrastructure Status. LHCb Computing Status LHCb LHCC mini-review, February The LHCb Computing Model: a reminder m Simulation is using.
ORACLE GOLDENGATE AT CERN
ATLAS Metrics for CCRC’08 Database Milestones WLCG CCRC'08 Post-Mortem Workshop CERN, Geneva, Switzerland June 12-13, 2008 Alexandre Vaniachine.
Update on Database Issues Peter Chochula DCS Workshop, June 21, 2004 Colmar.
LHC: ATLAS Experiment meeting “Conditions” data challenge Elizabeth Gallas - Oxford - August 29, 2009 XLDB3.
LCG Service Challenge Phase 4: Piano di attività e impatto sulla infrastruttura di rete 1 Service Challenge Phase 4: Piano di attività e impatto sulla.
ATLAS Scalability Tests of Tier-1 Database Replicas WLCG Collaboration Workshop (Tier0/Tier1/Tier2) Victoria, British Columbia, Canada September 1-2, 2007.
Daniela Anzellotti Alessandro De Salvo Barbara Martelli Lorenzo Rinaldi.
Workshop Summary (my impressions at least) Dirk Duellmann, CERN IT LCG Database Deployment & Persistency Workshop.
LCG 3D Project Status and Production Plans Dirk Duellmann, CERN IT On behalf of the LCG 3D project CHEP 2006, 15th February, Mumbai.
ATLAS Database Operations Invited talk at the XXI International Symposium on Nuclear Electronics & Computing Varna, Bulgaria, September 2007 Alexandre.
CHEP 2006, Mumbai13-Feb-2006 LCG Conditions Database Project COOL Development and Deployment: Status and Plans Andrea Valassi On behalf of the COOL.
Databases E. Leonardi, P. Valente. Conditions DB Conditions=Dynamic parameters non-event time-varying Conditions database (CondDB) General definition:
Web application for detailed real-time database transaction monitoring for CMS condition data ICCMSE 2009 The 7th International Conference of Computational.
CCRC’08 Weekly Update Jamie Shiers ~~~ LCG MB, 1 st April 2008.
Database Monitoring Requirements Salvatore Di Guida (CERN) On behalf of the CMS DB group.
The Persistency Patterns of Time Evolving Conditions for ATLAS and LCG António Amorim CFNUL- FCUL - Universidade de Lisboa A. António, Dinis.
LFC Replication Tests LCG 3D Workshop Barbara Martelli.
CERN - IT Department CH-1211 Genève 23 Switzerland t COOL Conditions Database for the LHC Experiments Development and Deployment Status Andrea.
Peter Chochula ALICE Offline Week, October 04,2005 External access to the ALICE DCS archives.
Database Readiness Workshop Summary Dirk Duellmann, CERN IT For the LCG 3D project SC4 / pilot WLCG Service Workshop.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Implementation and performance analysis of.
Integration of the ATLAS Tag Database with Data Management and Analysis Components Caitriana Nicholson University of Glasgow 3 rd September 2007 CHEP,
Michele de Gruttola 2008 Report: Online to Offline tool for non event data data transferring using database.
3D Testing and Monitoring Lee Lueking LCG 3D Meeting Sept. 15, 2005.
3D Project Status Dirk Duellmann, CERN IT For the LCG 3D project Meeting with LHCC Referees, March 21st 06.
Development, Deployment and Operations of ATLAS Databases XVI International Conference on Computing in High Energy and Nuclear Physics Victoria, British.
CERN IT Department CH-1211 Genève 23 Switzerland t Streams Service Review Distributed Database Workshop CERN, 27 th November 2009 Eva Dafonte.
Plans for Service Challenge 3 Ian Bird LHCC Referees Meeting 27 th June 2005.
LCG 3D Project Update (given to LCG MB this Monday) Dirk Duellmann CERN IT/PSS and 3D
Maria Girone CERN - IT Tier0 plans and security and backup policy proposals Maria Girone, CERN IT-PSS.
11th November Richard Hawkings Richard Hawkings (CERN) ATLAS reconstruction jobs & conditions DB access  Conditions database basic concepts  Types.
Victoria, Sept WLCG Collaboration Workshop1 ATLAS Dress Rehersals Kors Bos NIKHEF, Amsterdam.
1 A Scalable Distributed Data Management System for ATLAS David Cameron CERN CHEP 2006 Mumbai, India.
Status of tests in the LCG 3D database testbed Eva Dafonte Pérez LCG Database Deployment and Persistency Workshop.
ATLAS FroNTier cache consistency stress testing David Front Weizmann Institute 1September 2009 ATLASFroNTier chache consistency stress testing.
Database Project Milestones (+ few status slides) Dirk Duellmann, CERN IT-PSS (
ATLAS Distributed Analysis DISTRIBUTED ANALYSIS JOBS WITH THE ATLAS PRODUCTION SYSTEM S. González D. Liko
Maria del Carmen Barandela Pazos CERN CHEP 2-7 Sep 2007 Victoria LHCb Online Interface to the Conditions Database.
Replicazione e QoS nella gestione di database grid-oriented Barbara Martelli INFN - CNAF.
CERN - IT Department CH-1211 Genève 23 Switzerland t Persistency Framework CORAL, POOL, COOL status and plans Andrea Valassi (IT-PSS) On.
A quick summary and some ideas for the 2005 work plan Dirk Düllmann, CERN IT More details at
Database Requirements Updates from LHC Experiments WLCG Grid Deployment Board Meeting CERN, Geneva, Switzerland February 7, 2007 Alexandre Vaniachine (Argonne)
Dario Barberis: ATLAS DB S&C Week – 3 December Oracle/Frontier and CondDB Consolidation Dario Barberis Genoa University/INFN.
INFN Tier1/Tier2 Cloud WorkshopCNAF, 22 November 2006 Conditions Database Services How to implement the local replicas at Tier1 and Tier2 sites Andrea.
ATLAS Database Deployment and Operations Requirements and Plans
Dirk Duellmann CERN IT/PSS and 3D
Database Replication and Monitoring
LCG 3D Distributed Deployment of Databases
Evolution of Data(base) Replication Technologies for WLCG
Andrea Valassi (IT-ES)
3D Application Tests Application test proposals
Elizabeth Gallas - Oxford ADC Weekly September 13, 2011
Database Readiness Workshop Intro & Goals
Conditions DB Data Distribution
Conditions Data access using FroNTier Squid cache Server
Simulation use cases for T2 in ALICE
Workshop Summary Dirk Duellmann.
LHCb Conditions Database TEG Workshop 7 November 2011 Marco Clemencic
LHC Data Analysis using a worldwide computing grid
Presentation transcript:

2007 Workshop on INFN ComputingRimini, 7 May 2007 Conditions Database Service implications at Tier1 and Tier2 sites Andrea Valassi (CERN IT-PSS)

Andrea Valassi Conditions DatabasesRimini, 7 May 2007 What are Conditions Data? Describe the detector at the time of data taking –Event data (beam collisions) vs. conditions data (detector environment) –Conditions data vary with time and may exist in several versions Several sources of conditions data –Online: DCS (temperatures…), configuration (trigger…) and monitoring Generally single version (results of a measurement) –Offline: calibration and alignment Generally multi-version (results of a computation) This talk will focus on offline conditions databases –Online and offline conditions needed for offline event reconstruction –Main use case: retrieve the conditions data valid for one event/run

Andrea Valassi Conditions DatabasesRimini, 7 May 2007 Conditions DB in the 4 experiments ALICE –AliRoot (Alice-specific software) for time/version handling –ROOT files with AliEn file catalog ALICE-managed database infrastructure (AliEn MySQL at T0) CMS –CMSSW (CMS-specific software) for time/version handling –Oracle (via POOL-ORA C++ API) with Frontier web cache 3D + CMS database infrastructure (Oracle/Frontier at T0, Squid at T1/T2) ATLAS and LHCb –COOL (LCG AA common software) for time/version handling –Oracle, MySQL, SQLite, Frontier (via COOL C++ API) 3D + Alice/LHCb database infrastructure (Oracle at T0/T1 with Streams)

Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ALICE – use Alien/ROOT ALICE has no special needs for DB deployment/replication at T1 and T2 sites for managing its conditions data. AliEn Grid file catalog uses a 2-node MySQL database cluster at T0 (not replicated at T1/T2)

Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ALICE – OCDB Grid file catalog Offline Calibration DB: conditions data in AliEn file catalog –Metadata for ROOT files containing calibration and alignment data Files produced by many sources are registered into OCDB by a ‘shuttle’ –OCDB ROOT files are replicated to all T1’s (and on request to T2’s) Two types of conditions data –“Short-lived” (one update/run, ~2000/year) MB per data set –“Long-lived” (few updates/year) MB per data set –Total data size per year ~200 GB (short-lived) plus ~1 GB (long-lived) R/O data access from Grid jobs –ROOT files are downloaded to the local worker node

Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ALICE – Dress Rehearsal 2007

Andrea Valassi Conditions DatabasesRimini, 7 May D project (CMS, ATLAS, LHCb) Barbara Martelli, INFN T1/T2 Cloud Workshop, November 2006

Andrea Valassi Conditions DatabasesRimini, 7 May 2007 Oracle Streams (Capture, Propagation, Apply) Barbara Martelli, INFN T1/T2 Cloud Workshop, November 2006

Andrea Valassi Conditions DatabasesRimini, 7 May 2007 Streams downstream capture Objective –Source database (CERN) isolation against network or database problems at the replicas Status and plans –Four nodes allocated –Production setup with two servers (Atlas and LHCb) and two spares For LHCb will also be used for LFC replication Eva Dafonte Perez (IT-PSS)

10 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 Frontier Marc Paterno, CHEP 2004 T1/T2: Squid T0: Squid T0: Tomcat T0: Oracle

11 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 CMS – use Oracle/Frontier Vincenzo Innocente, CMS Software Week April 2007

12 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 CMS – conditions data at CERN Vincenzo Innocente, CMS Software Week, April 2007 ORCON and ORCOFF conditions data are in POOL_ORA format ORCON-ORCOFF Oracle Streams prototype set up (integration RAC). Production set up later in 2007.

13 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 CMS – conditions data distribution Vincenzo Innocente, CMS Software Week April 2007

14 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 CMS – deployment status Conditions data access via Frontier already successfully tested in CSA Many performance improvements since that time (improve cache reuse, fix many problems – D. Dykstra)

15 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 COOL: LCG Conditions database –Common development of IT-PSS, LHCb and ATLAS –Manage time variation and versioning of conditions data Four supported relational technologies (via CORAL) –Oracle and MySQL database servers –SQLite files –Frontier (read-only ): Squid + Tomcat + Oracle server COOL service deployment model –Based on generic 3D distributed db deployment model Oracle at T0 and T1 (Streams replication with downstream capture) Other technologies elsewhere if at all needed –Details depend on each experiment’s computing model ATLAS and LHCb – use COOL P. Mato

16 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 LHCb computing model COOL only stores the conditions data needed for offline event reconstruction –Low data volume ~ 1 GB/year COOL deployment model –Oracle at Tier0 –Oracle at Tier1’s (six sites) –Streams replication T0-T1 –COOL not needed at Tier2’s (only MC production there) SQLite files may in any case be used for any special needs (Marco Clemencic, 3D workshop 13 Sep 2006)

17 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 LHCb – COOL service model Two servers at CERN – essentially for online and offline –Replication to Tier1’s from the online database is a two-step replication –Online server at the pit (managed by LHCb); offline server in the CC GRIDKA RAL IN2P3 CNAF SARA PIC (Marco Clemencic, 3D workshop 13 Sep 2006) COOL (Oracle)

18 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 LHCb - status and plans Setup and tests of Oracle servers and Streams replication –CERN Online (prototype in CC), CERN Offline (integration RAC) and five Tier1 sites (GRIDKA, RAL, IN2P3, CNAF, SARA), with Streams downstream capture –Tested 2-step replication (Online-Offline-T1) at rates higher than expectations All OK with 100 IOVs/sec for one hour and 1 IOVs/sec for 24 hours –Tested simple read-back from T1 database Verified that latency in tag replication is negligible Future milestones –May 2007: complete T0/T1 deployment (CERN production offline RAC, add PIC) –June 2007: software release using LFC replica service (CORAL 1.8.0) DB lookup and secure user authentication using grid certificate (retrieve encrypted username and password – later on also secure SSH data transfer in LFC) –Summer 2007: test reconstruction at T1 sites using Oracle Streams replicas Conditions data for the ongoing data challenge are distributed as SQLite files –Later in 2007: online database server at the pit Online database prototype in the CC has already been switched off

19 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ATLAS – COOL service model COOL Oracle services at Tier0 and ten Tier1’s –Two COOL servers at CERN for online/offline (similar to LHCb) Online database within the Atlas pit network, but physically in the CC –In addition: Oracle at three ‘muon calibration center’ Tier2’s Online OracleDB Offline master CondDB Tier-0 Sqlite replica (1 file/run) Tier-1 replica Tier-1 replica Online / PVSS / HLT farm Tier-0 farm Dedicated 10Gbit link ATLAS pitComputer centreOutside world CERN public network Calibration updates Streams replication ATLAS pit network (ATCN) gateway (Sasha Vaniachine and Richard Hawkings, 3D workshop 14 Sep 2006) GRIDKA TAIWAN RAL IN2P3 CNAF SARA BNL TRIUMF PIC Nordugrid

20 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ATLAS – production setup Gancho Dimitrov, Florbela Viegas, Marco Clemencic 3D Amsterdam, March 2007 Online  Offline: both COOL and non-COOL

21 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ATLAS – current test setup Gancho Dimitrov, Florbela Viegas, Marco Clemencic 3D Amsterdam, March 2007

22 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ATLAS – COOL schemas Gancho Dimitrov, Florbela Viegas, Marco Clemencic 3D Amsterdam, March 2007 DCS ~300 GB/year (largest component) ONL ~few GB/year OFL ~60 GB/year

23 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ATLAS – status and plans Setup and tests of Oracle servers and Streams replication –Production Online and Offline RAC at CERN and six Tier1 sites (GRIDKA, RAL, IN2P3, CNAF, SARA, Taiwan), with Streams downstream capture –Ongoing “mini-CDC” production with 2-step replication (Online-Offline-T1) Continuous insertion of calibration-like CLOB data (30MB every 30min) Read back T1 data ~30 minutes after insertion at T0 –5% failure for most distant site (Streams latency – but the data gets there eventually) Future plans/milestones –May 2007: mini-CDC tests with DCS-like data (higher LCR/s for the same MB/s) –May 2007: software release 13 with replica sorting in dblookup.xml (CORAL 1.8.0) –June 2007: “closed loop” calibration/alignment challenge Realistic calibration setup (but lower data rates than current mini-CDC tests) The muon calibration centers will be tested later (~September) –Summer 2007: complete T1 deployment (add PIC, Nordugrid; fix BNL, Triumf)

24 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ATLAS – open issues Can the replication throughput meet the Atlas requirements? –Intrinsic limitation of Oracle Streams: single row updates in apply step Each LCR (logical change record) is replicated individually Current mini-CDC tests are very important to validate rates for COOL –Investigating transportable tablespaces for TAGS Higher data volume (6TB/year), much higher LCR/s (finer granularity) Replication to Tier2’s –Several use cases Simulation, calibration, end user analysis –Investigating several possibilities SQLite files are enough for some use cases (e.g. simulation) Evaluating COOL ‘dynamic replication’ to MySQL at remote sites Evaluating COOL Frontier backend (performance, cache consistency…)

25 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ATLAS – Streams tests Barbara Martelli, May 2007 Current Atlas mini-CDC tests (CNAF replica) Steady-state operation at ~80 LCR/s Gancho Dimitrov et al, SARA, March 2007 Oracle Streams stress tests (RAL replica) Maximum achieved ~1400 LCR/s

26 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 ATLAS – muon calibration centers (Sasha Vaniachine and Joe Rothberg, 3D workshop 14 Sep 2006) Muon calibration centers: Roma, Michigan, Munich Streams set up between Michigan (source) and CERN (target)

27 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 Summary *Not present in the table: online conditions other than DCS/PVSS (configuration for DAQ/trigger, monitoring…) Data volume for these data is generally smaller than for calibration/alignment or DCS CondDB. CondDB technologyCalibration and alignment (~Offline conditions) DCS from PVSS/Oracle (~Online conditions *) ALICEROOT files + Alien file catalog Alien uses MySQL cluster at T0 (DB is not replicated at T1/T2) CondDB: 200 GB/year ~100 MB/run (short-lived) ~150 MB/month (long-lived) (CondDB: included in 200 GB/y) PVSS archive to ROOT files Full PVSS archive: 7 TB/year CMS POOL-ORA (Oracle at T0) FroNTier launchpad at T0 Squid web cache at T0/T1/T2 CondDB: <500 GB/year (DB disk space requirements) 100 GB/Squid, 2-3 Squid/site (CondDB: included in <500 GB/y) P5 online DB disk space 500 GB ATLASCOOL Oracle at T0/T1 (Streams/3D) Sqlite/MySQL/Frontier at T2? CondDB: 60 GB/year ~20 GB/year (first-pass rec.) ~40 GB/year (re-rec.) CondDB: 300 GB/year One PVSS2COOL job/15 min Full PVSS archive ~1-2 TB/year LHCbCOOL Oracle at T0/T1 (Streams/3D) Sqlite at T2 (only if needed) CondDB: 1 GB/year? (CondDB: included in 1 GB/y) DCS CondDB filled by PVSS job Full PVSS archive < 100 GB/year?

28 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 Conclusions ALICE: use ROOT and Alien (no db services outside CERN) CMS: Oracle and Frontier at T0, Squid web caches at T1/2 –Frontier already successfully tested during CSA06 and 2007 productions –Oracle Streams between online and offline servers at T0 –No database service is required outside CERN for CMS LHCb (COOL): Oracle at T0 and T1 with Streams replication –Two servers at CERN: one at the pit (online) and one in the CC (offline) –No database service is required at T2’s for LHCb –Test Grid reconstruction (reading COOL from 6 T1 replicas) in summer 2007 Atlas (COOL): Oracle at T0 and T1 with Streams replication –Two servers at CERN, both in the CC (online server in the online network) –“Mini-CDC” high-rate tests ongoing with 6 T1 sites –Test “closed loop calibration” in summer 2007 –No single solution for Tier2 deployment – evaluating MySQL and Frontier

29 Andrea Valassi Conditions DatabasesRimini, 7 May 2007 Acknowledgements ALICE: Latchezar Betev CMS: Lee Lueking, Nicola De Filippis, Vincenzo Innocente Atlas: Richard Hawkings, Gancho Dimitrov, Jim Cook LHCb: Marco Clemencic, Philippe Charpentier IT-PSS, 3D: Eva Dafonte, Barbara Martelli, Dirk Duellmann Thanks to all for your help!