“Primary Numbers” Database for ATLAS Detector Description Parameters March 24, 2003 S. Eckmann, D. Malon, A. Vaniachine (ANL) P. Nevski, T. Wenaus (BNL)

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Scale of the Problem A state-of-the-art detector is comprised of many elements: 29 M distinct volume copies 23 K different volume objects 4,673 volume types

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Detailed description of ATLAS detector geometry in simulations is unprecedented

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Primary Numbers The Detector Description task in ATLAS is accomplished through the use of “Primary Numbers" – parameters defining In simulations detector geometry digitization In reconstruction same detector geometry parameters of the reconstruction algorithms alignment parameters

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Virtual Data Aspect Virtual Data concepts from the GriPhyN project ( introduced a generic aspect of the problem across the science domainswww.griphyn.org The Detector Description parameters – “Primary Numbers” – can be considered as Virtual Data provided to control the applications performing the data transformations

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Parametrized simulation of the detector response Playing Central Role Raw Data … Simulated Raw Data … Reconstructed Event Objects Data Simulated Particle Event Data Event reconstruction data transformation Geant3 simulation of the detector response “Primary Numbers” for Detector Description Geant4 simulation of the detector response

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) XML Name-Value Pairs Indeed the Detector Description parameters – “Primary Numbers” providing control of applications simulating the detector response or performing the event reconstruction transformation in the first approximation fit well into the XML name- value pairs that can be arbitrarily structured to support content management

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Primary Numbers Parameters for Geant geometry e.g. ATLAS Mother Volume NameValueComment Version22001 VERSION WITH ENDCAP SHIFTED B Rmin.0 Inner Radius Rmax Outer Radius Zmax2350.0Maximum Z

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) More than XML Pairs A more detailed view of the Detector Description parameters provides Name of the parameter Value of the parameter Primitive type of the parameter (int, float,…) Textual comment describing the parameter Scope of the parameter (representing the structural container that this parameter belongs to) Additional information parameter unit of measurement parameter version structural container information

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Valuable Knowledge Verification of the “Primary Numbers“ requires considerable efforts Checking the engineering drawings Consulting with experts Our collaborator, Claire Bourdarios, reported that detailed analysis of LAr calorimeter barrel accordeon geometry parameters took about one month In total the barrel accordeon “Primary Numbers“ count was below one hundred Since the detailed ATLAS detector description needs more than 10K of such parameters, a preferred solution is to have a single verified source for all these data

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Knowledge Management Data Warehousing

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Binary Data Some of Detector Description parameters are better handled in the binary format Example of binary data stored in the “Primary Numbers” database are For simulations Magnetic field maps For reconstruction Identifier maps of the detector elements Typically these data are “secondary” - resulting from some computer calculations

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Heterogeneity: Organizing the “Chaos” The same “Primary Numbers" are served to many different clients accessing the database: ATLAS software framework Athena Geant4 developers' framework FADS/Goofy Geant3 legacy framework atlsim XML output generator for the detector description Interactive end-user browsing and navigation: Netscape IExplorer ROOT

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) S chema Evolution Evolution of data structures is challenging: Relational Databases is the technology of choice to address this problem Relational approach solution: Internal data representation in the database is not a one-to-one copy of the data user wants to use (transient representation) This is an opposite to the baseline OO DB approach When the structure of user data changes – the internal structure of the data in DB is not supposed to be changed If achieved, this is considered as a good design Transient data structures are, in essence, the “views” of the persistent data representation Effectively, database stores the persistent data dictionary for each parameter collection object, providing the schema evolution support for the object-based retrieval of parameters

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Layered Interface Database files: ISAM OS Level: disk storage Logical Data Model Structured Data Views Conversion Services Transient Detector Store Persistent Dictionary is stored together with the data Structured Data Persistent Dictionary Transient Dictionary Transient Objects Application Framework Persistent Media structuresparametersrelations

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Database Technology The relational database technology for storing the “Primary Numbers" was introduced earlier NOVA Data Model

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Primary Numbers Access Database Server Conversion Service Client Algorithms Transient DataObject Persistent DataObject Persistent Dictionary Transient Dictionary Parameters Collection Object Request Broker On-the-fly conversion G4 model by Adele Rimoldi

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Architecture Evolution An earlier approach when all the work was done in the object request broker base class is now evolved to support the new grid services architecture where the services discovery requests and the persistent-transient data conversions are now functionally separate This new approach fit best within the Gaudi/Athena architecture separating the transient and persistent data representations

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Nova Conversion Service To deliver this functionality we developed and deployed the Nova Conversion Service providing access to “Primary Numbers” within the mainstream ATLAS software framework Athena A novaClassGen utility automatically generates converters and the C++ headers Service registers “Primary Number” objects with the transient detector store Objects can be accessed in the usual StoreGate manner (retrieve) For more information see follow the link for “NOVA MySQL access ”

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) In MyAlg.h include NovaConverters header file #include "NovaConverters/NovaConverters.h” declare data member of needed types private: MintgeoBari* m_bari; In MyAlg.cxx declare DataHandle in MyAlg :: initialize() const DataHandle bari; sc = myStore->retrieve(bari); m_bari = bari; iterate over data in MyAlg :: execute() MintgeoBari::BARI* bari = m_bari->m_d; for (i=0; i size(); i++, bari++) { log ok = " ok << endreq; for (j = 0; j < 10; j++) { log type[" type[j] <<endreq; }

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Together with our collaborators we demonstrated that IOV database can provide validity- interval-based access to data that has been stored using an existing NOVA Conversion Service IOV/ConditionsDB Interface

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) NOVA Integration IOV database associates a folder name (e.g., “Tile/Pedestals”), an interval of validity [ti, tj), and a tag to a string String will contain an externalized IOpaqueAddress IOV integration with NOVA requires that NOVA: Stores multiple object instances of the same type Assign unique IOpaqueAddresses (instance names) Externalize IOpaqueAddresses as strings, “internalize” strings to create IOpaqueAddresses This capability would ideally be added to base classes for all conversion services

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Happy Users View Clients reading data do not need to know which conversion service (e.g. NOVA) is delivering the data StoreGate retrieve will typically use a key that is the folder name (e.g., “Tile/Pedestals”); timestamp information will come from the current event without user intervention (transient IOV service) Internally, this will be enough (along with a tag set in jobOptions) to retrieve the correct string; string will be changed to IOpaqueAddress; usual Athena/Gaudi conversion service mechanisms will be triggered to build an object in the transient store—all of this is hidden from the user Intent is to do nothing that will break or change current NOVA use that does not require intervals of validity: IOV database provides additional capabilities, but does not replace current mode of access to data in NOVA

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) MySQL Implementation Benefits Choice of the MySQL database for initial implementation added extra benefits: “Primary Numbers" database can be used on the developers' laptop when disconnected (using the MySQL embedded server technology), data can be updated when laptop is connected (using the MySQL database replication) MySQL support binary data transfers Cost-free scalability for content delivery: few writers thousands of readers Grid certificate authorization technologies

CHEP 2003, March 24-28, La Jolla Alexandre Vaniachine (ANL) Roadmap to Success Future directions: LCG Detector Description Project coming soon XML input implementation though the Athena framework dictionary services incorporating SEAL project deliverables Grid integration – “Primary Numbers” Service based on OGSA – DAI database services implementation including the needed XML binary extensions support from DAI