Athena/POOL integration

Slides:



Advertisements
Similar presentations
25 th March 2003 V.Fine, H.Ma CHEP 2003, San Diego 1 Root Based Persistency in Athena (ATLAS) by Valeri Fine and Hong Ma.
Advertisements

D. Düllmann - IT/DB LCG - POOL Project1 POOL Release Plan for 2003 Dirk Düllmann LCG Application Area Meeting, 5 th March 2003.
David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL March 25, 2003 CHEP 2003 Data Analysis Environment and Visualization.
David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL June 23, 2003 GAE workshop Caltech.
Automated Tests in NICOS Nightly Control System Alexander Undrus Brookhaven National Laboratory, Upton, NY Software testing is a difficult, time-consuming.
Simplify your Job – Automatic Storage Management Angelo Session id:
Module 8: Server Management. Overview Server-level and instance-level resources such as memory and processes Database-level resources such as logical.
SEAL V1 Status 12 February 2003 P. Mato / CERN Shared Environment for Applications at LHC.
STAR C OMPUTING Maker and I/O Model in STAR Victor Perevoztchikov.
M.Frank LHCb/CERN - In behalf of the LHCb GAUDI team Data Persistency Solution for LHCb ã Motivation ã Data access ã Generic model ã Experience & Conclusions.
David Adams ATLAS ATLAS Distributed Analysis David Adams BNL March 18, 2004 ATLAS Software Workshop Grid session.
DAT305 Boost Your Data-Driven Application Development Using SQL Server Centric.NET Code Generator Pascal Belaud Microsoft France.
Developer workshop on I/O and persistence evolution LAL,Orsay, Feb 2012 Marcin Nowak (PAS BNL) Extended T/P Converters.
Prelude Many questions/points have been raised in recent weeks, and forwarded by David Stickland to me: n Views on storage technology choices n Views on.
N ATIONAL E NERGY R ESEARCH S CIENTIFIC C OMPUTING C ENTER Charles Leggett The Athena Control Framework in Production, New Developments and Lessons Learned.
Event Data History David Adams BNL Atlas Software Week December 2001.
GEM Portal and SERVOGrid for Earthquake Science PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce Computer Science, Informatics, Physics.
Datasets on the GRID David Adams PPDG All Hands Meeting Catalogs and Datasets session June 11, 2003 BNL.
GLAST Gaudi Code Review, 10 Sept. 2002, H. Kelly, 2-1 GLAST Event Data Model and Persistency.
CHEP 2003 March 22-28, 2003 POOL Data Storage, Cache and Conversion Mechanism Motivation Data access Generic model Experience & Conclusions D.Düllmann,
1 What is OO Design? OO Design is a process of invention, where developers create the abstractions necessary to meet the system’s requirements OO Design.
24/06/03 ATLAS WeekAlexandre Solodkov1 Status of TileCal software.
Use of Gaudi in Reconstruction Weidong Li 23/06/2004.
Feedback from the POOL Project User Feedback from the POOL Project Dirk Düllmann, LCG-POOL LCG Application Area Internal Review October 2003.
ATLAS Offline Database Architecture for Time-varying Data, with Requirements for the Common Project David M. Malon LCG Conditions Database Workshop CERN,
David Adams ATLAS DIAL/ADA JDL and catalogs David Adams BNL December 4, 2003 ATLAS software workshop Production session CERN.
The POOL Persistency Framework POOL Project Review Introduction & Overview Dirk Düllmann, IT-DB & LCG-POOL LCG Application Area Internal Review October.
STAR Event data storage and management in STAR V. Perevoztchikov Brookhaven National Laboratory,USA.
GDB Meeting - 10 June 2003 ATLAS Offline Software David R. Quarrie Lawrence Berkeley National Laboratory
David Adams ATLAS Virtual Data in ATLAS David Adams BNL May 5, 2002 US ATLAS core/grid software meeting.
D. Adams, D. Liko, K...Harrison, C. L. Tan ATLAS ATLAS Distributed Analysis: Current roadmap David Adams – DIAL/PPDG/BNL Dietrich Liko – ARDA/EGEE/CERN.
STAR Schema Evolution Implementation in ROOT I/O V. Perevoztchikov Brookhaven National Laboratory,USA.
STAR C OMPUTING Plans for Production Use of Grand Challenge Software in STAR Torre Wenaus BNL Grand Challenge Meeting LBNL 10/23/98.
LCG Applications Area Meeting - 5 Feb 2003 ATLAS Strategy for the Data Dictionary & Persistency David R. Quarrie Lawrence Berkeley National Laboratory.
CHEP 2004, Core Software Integration of POOL into three Experiment Software Frameworks Giacomo Govi CERN IT-DB & LCG-POOL K. Karr, D. Malon, A. Vaniachine.
23/2/2000Status of GAUDI 1 P. Mato / CERN Computing meeting, LHCb Week 23 February 2000.
D. Duellmann - IT/DB LCG - POOL Project1 The LCG Pool Project and ROOT I/O Dirk Duellmann What is Pool? Component Breakdown Status and Plans.
K. Harrison CERN, 3rd March 2004 GANGA CONTRIBUTIONS TO ADA RELEASE IN MAY - Outline of Ganga project - Python support for AJDL - LCG analysis service.
Valeri Fine Athena/POOL integration.
- GMA Athena (24mar03 - CHEP La Jolla, CA) GMA Instrumentation of the Athena Framework using NetLogger Dan Gunter, Wim Lavrijsen,
D. Duellmann - IT/DB LCG - POOL Project1 The LCG Dictionary and POOL Dirk Duellmann.
Overview of C/C++ DB APIs Dirk Düllmann, IT-ADC Database Workshop for LHC developers 27 January, 2005.
Gaudi Framework Tutorial, April Creating Objects and Writing Data.
Introduction S. Rajagopalan August 28, 2003 US ATLAS Computing Meeting.
TAGS in the Analysis Model Jack Cranshaw, Argonne National Lab September 10, 2009.
INFSO-RI Enabling Grids for E-sciencE Using of GANGA interface for Athena applications A. Zalite / PNPI.
Summary of persistence discussions with LHCb and LCG/IT POOL team David Malon Argonne National Laboratory Joint ATLAS, LHCb, LCG/IT meeting.
ATLAS Data Dictionary A. Bazan, T. Bouedo, P. Ghez, T. Le Flour, S. Lieunard M. Marino, C. Tull.
STAR Persistent Pointers in the STAR Micro-DST V. Perevoztchikov Brookhaven National Laboratory,USA.
David Adams ATLAS ATLAS Distributed Analysis (ADA) David Adams BNL December 5, 2003 ATLAS software workshop CERN.
27/02/04 ATLAS weekTileCal Athena tutorial, part 21 TileCal Athena tutorial Part 2: Reading GEANT hits from ZEBRA and POOL Reading HLT and testbeam ByteStream.
N ATIONAL E NERGY R ESEARCH S CIENTIFIC C OMPUTING C ENTER Charles Leggett Prototype GEANT4 Service for ATHENA framework ATLAS Software Workshop Dec 3.
Next-Generation Navigational Infrastructure and the ATLAS Event Store Abstract: The ATLAS event store employs a persistence framework with extensive navigational.
D. Duellmann, IT-DB POOL Status1 POOL Persistency Framework - Status after a first year of development Dirk Düllmann, IT-DB.
POOL Based CMS Framework Bill Tanenbaum US-CMS/Fermilab 04/June/2003.
POOL Historical Notes POOL has been the most advanced and the most used AA project. Currently, excellent teamwork with experiments on new features and.
Mini-Workshop on multi-core joint project Peter van Gemmeren (ANL) I/O challenges for HEP applications on multi-core processors An ATLAS Perspective.
ATLAS Distributed Computing Tutorial Tags: What, Why, When, Where and How? Mike Kenyon University of Glasgow.
David Adams ATLAS DIAL Distributed Interactive Analysis of Large datasets David Adams BNL May 19, 2003 BNL Technology Meeting.
David Adams ATLAS Hybrid Event Store Integration with Athena/StoreGate David Adams BNL March 5, 2002 ATLAS Software Week Event Data Model and Detector.
Athena I/O Component Refactorization - Overview Random notes about todays agenda. Peter Van Gemmeren (Argonne National Laboratory (US))
Chapter 21 SGA Architecture and Wait Event Summarized & Presented by Yeon JongHeum IDS Lab., Seoul National University.
Root I/O and the Gaudi Framework
(on behalf of the POOL team)
POOL persistency framework for LHC
Dirk Düllmann CERN Openlab storage workshop 17th March 2003
Agile testing for web API with Postman
Event Storage GAUDI - Data access/storage Framework related issues
Lecture 13 Teamwork Bryan Burlingame 1 May 2019.
Planning next release of GAUDI
Presentation transcript:

Athena/POOL integration Valeri Fine fine@bnl.gov

AthenaPOOL components are components of the Athena framework From the Atlas “Athena” framework point of view the POOL provides just a new kind of I/O technology. This means the “integration” within Athena means a concrete implementation of Athena Conversion service for POOL-technology first of all. The implementation project was present collaboration http://www.usatlas.bnl.gov/~dladams/poolint/design.html Valeri Fine fine@bnl.gov

Conversion Process See: Athena. Chapter 15 Converters AthenaPoolConverter AthenaPoolCnv<T> AthenaPoolCnv<T> AthenaPoolCnvSvc AthenaPoolCnv<T> AthenaPoolAddress Valeri Fine fine@bnl.gov

POOL I/O for Athena algorithm File Catalog (XML) POOL Collection EventHeader (MySQL) Athena Algorithm jobOptions StoreGate C++ class headers AthenaPoolCnvSvc PoolSvc POOL SEAL POOL Db files (ROOT) IService Valeri Fine fine@bnl.gov

AtlasPOOL CVS repository Valeri Fine fine@bnl.gov

AtlasPOOL example package Valeri Fine fine@bnl.gov

AthenaPOOL MySQL “collecton” - EventHeader Valeri Fine fine@bnl.gov

Example of the AtlasPOOL collection Valeri Fine fine@bnl.gov

Job option to read /write via POOL One needs to remember very few things: Define the Athena Conversion service for POOL I/O technology. To do that it is sufficient to include in the jobOption file: #include "$ATHENAPOOLCNVSVCROOT/share/WriteAthenaPool_jobOptions.txt" This is to load and define two POOL-related Athena services: ApplicationMgr.DLLs += { "PoolSvc", "AthenaPoolCnvSvc" }; ApplicationMgr.ExtSvc += { "PoolSvc", "AthenaPoolCnvSvc" }; Define the Athena Converter for each package involved ApplicationMgr.DLLs += { "AthenaPoolExampleLcgDictionaryX" }; where AthenaPoolExampleLcgDictionaryX name of the package user-provided converter share library Provide the StoreGate Stream and the list of the class StoreGate ID’s for those instances of the classes one wants to be written out Stream1.ItemList += {"9876#*" }; Define the POOL Db collection to keep the track about the stored events PoolSvc.Collection = "NewPoolTry"; where "NewPoolTry" is a unique name of the current session in the MySQL Db. and the name of the ROOT file to write these events out. PoolSvc.Output = "SimplePoolFile.root"; POOL collection factory should be provided with the several parameters to describe the various type collections Valeri Fine fine@bnl.gov

Read the objects from the POOL To read the events back one has to load a special implementation of the Athena EventSelector class and the name of the input Db collection. To define the POOL event Selector implementation one can add the include statement #include $ATHENAPOOLCNVSVCROOT/share/ReadAthenaPool_jobOptions.txt" to his/her jobOptions. To define the POOL input collection the statement: EventSelector.InputCollection = "NewPoolTry"; where "NewPoolTry" is a name of the collection one wants to read the events from. The file ID is stored in the Db. By this reason one needs to provide NO file name to read the events back. Valeri Fine fine@bnl.gov

Quick Db check The named collection can be seen via from MySql Db. To see the browse the Atlas default Db http://atlassw1.phy.bnl.gov/phpMyAdmin/index.php select "pool_collection" from the drop-down "(databases)" menu and the select the name of your collection. Valeri Fine fine@bnl.gov

Conclusion At the moment the fundamental problem of providing a tool to build the class dictionary and object Athena converters automatically solved. The Atlas user can try it and the developer group can assist them to collect the valuable feedback, and provide the feedback to the POOL developers. Even though we may focus on other directions some problems should have been anticipated. Valeri Fine fine@bnl.gov

C++ class serialization problem There are two anticipated problems to be resolved: Not enough SEAL/POOL capability to serialize some particular Athena C++ class: Can be resolved by class redesign Fix SEAL/POOL Internal Athena class design problem caused its transient nature. The original I/O Gaudi paradigm came from the “Objectivity” that implied two hand-written classes, namely persistent class and class converter per each user data-class. Using ROOT-approach, generating automatically the class dictionary directly from its C++ definition eliminates those hand–written classes . However the classes originally were not design to be persistent that may have caused the problem. Valeri Fine fine@bnl.gov

How to find the “broken” class One has to try. It is a simplest way to find out the classes to be adjusted. Conclusion: The current level of implementation allows us to seek the collaboration-wide trial. Valeri Fine fine@bnl.gov

Three ways to evolve C++ class serialization problem (I/O and dictionary) – needs the collaboration participation. It is not clear why one is required to provide the list of the class data-members. The LCG OO dictionary is in possession of this infomrstion. Required the user to change “selection.xml” as soon as his /her class evolves. This would have tremendously simplified the user life. POOL Db related problem – just next Athena/POOL integration step to be done. Athena/Pool Optimization Removing the duplicated functionality Selection the proper combination of the Db options POOL provides. These cause no immediate problem. We have time to work out Atlas own proper solution or wait POOL Data Service evolves. Valeri Fine fine@bnl.gov

POOL is an OO Db User doesn’t play with any file. The file name is an internal POOL unit and available through the POOL file catalog. The user defines the object he wants to be written out (in term of StoreGate) and the type Db to keep the “event track” (for example MySQL collection name) Questions to be addressed yet: Who allocates the POOL file space and where? How to choose the Db collection name to avoid clashes? Such kind of questions are not new for the Athena. It the same problem we faced to select and assign a unique class id for each class kept in StoreGate. Valeri Fine fine@bnl.gov

Thank you. ATLAS team: POOL/LCG team David Adams (BNL) Dirk Dulleman Christian Arnault Markus Frank Hong Ma (BNL) Ioannis David Melon Torre Wenaus David Quarrie RD Schaffer Alex Undrus Valeri Fine fine@bnl.gov