The AMI Database Project AMI Databases The AMI Database Project Atlas Data Challenge Bookkeeping, and the Tag Collector, a new tool for Release Management.   Solveig Albrand, Johann Collot, Jerôme Fulachier 30/11/2018 S.A. CHEP03

AMI Databases First Steps The AMI project started in the spring of 2000. The first realization was an electronic notebook for Atlas LAr testbeam runs. Soon, we had other requests for other database projects:  Needed to reuse software, and to make databases “self describing” so they can be used in a generic way. The AMI web site is at http://larbookkeeping.in2p3.fr/AMI (has a PDF version of this presentation) 30/11/2018 S.A. CHEP03

Some Design Principles AMI Databases Some Design Principles Use an RDBMS. Use many RDBMS. Be generic. Distribute geographically Manage long term projects (schema evolution) Place emphasis on user interfaces.  SQL and Java were chosen 30/11/2018 S.A. CHEP03

AMI Databases AMI Architecture 30/11/2018 S.A. CHEP03

The Deployment of AMI on Several Servers. AMI Databases The Deployment of AMI on Several Servers. Connections pass through a router database. This enables selection of the correct server, jdbc driver, and database for access to the data. The databases contain their own description.(Entities, relations, and meta – information and behaviour. 30/11/2018 S.A. CHEP03

The Indirect Database Connection Mechanism AMI Databases The Indirect Database Connection Mechanism Bookkeeping Database AMI DB Loader AMI AMI Router Loader * 1 AMI Router Database AMI tables AMI router tables BKK tables references project, process… BKK JDBC 30/11/2018 S.A. CHEP03

Current Status Used by ~6 projects. AMI Databases Current Status Used by ~6 projects. In particular for the Atlas Data Challenge 1, for the “Application Meta Data” alias Production Bookkeeping. (> 50 000 files) (Physics  logical file name) Does not deal with physical locations or replication. Atlas uses Magda for this function. Available interfaces: Java, Command line, Web, (C++ prototype) 30/11/2018 S.A. CHEP03

Database Schema Overview AMI Databases Instances of the “element” object Database Specific Objects Generic Objects Database Schema Overview 30/11/2018 S.A. CHEP03

AMI Databases 30/11/2018 S.A. CHEP03

AMI Databases 30/11/2018 S.A. CHEP03

Future Plans for AMI/ Atlas Bookkeeping AMI Databases Future Plans for AMI/ Atlas Bookkeeping Replication at several sites. (Atlas DB group) Build tools to manage AMI databases. Introduce WEB SERVICES GRID Integration, certificate authorisation (Spitfire) Communication with MAGDA (talk tomorrow Wensheng Deng) Build application specific tools especially for Atlas Bookkeeping.(Production Managers, more features for physicists) 30/11/2018 S.A. CHEP03

Tag Collector A database based application for release management. AMI Databases Tag Collector A database based application for release management. AMI compliant (but uses PHP). Sits on top of CVS, and CMT (our configuration management tool) In use since September 2001. Some History… 30/11/2018 S.A. CHEP03

Atlas Release Approaches Anarchy! AMI Databases Atlas Release Approaches Anarchy! I goofed with the tag for the External/GaudiInterface package for release 2.0.1. It won't affect the SRT release, but it will affect attempts to build the CMT release. I had forgotten (and I suspect I won't be the last one to do this) that CMT has a very formal response when the major version ID increases (I changed from 01-02-07 to 02-00-00). It takes it to mean that the new version is incompatible with the earlier one, and that packages that depend upon this package will essentially break. At the very least they should change their dependency on this package from something like: use GaudiInterface-01-* External to use GaudiInterface-02-* External I mindlessly chose 02-00-00 without thinking about this implication. In order to be backwards compatible I have created a new tag for External/GaudiInterface GaudiInterface-01-03-00 The code is identical to that of 02-00-00. But this version of the package should be used for any CMT release build, be it 2.0.1, 2.0.2 or 2.1.0. Sorry about that. Sigh... David OK, I've moved Reconstruction-01-03-01 on egammaRec-00-00-03 and on Reconstruction /ChangeLog. Sorry about that David On Thu, 18 Jan 2001 09:13:02 -0500 (EST) "S. Rajagopalan" <srinir@bnl.gov> wrote: > > > Hi - The tag for egammaRec should be egammaRec-00-00-03, not "02". > Please pick up the correct tags for this release. Thanks. srini > > On Thu, 18 Jan 2001, David Rousseau wrote: > > Hello Maya > > Please include Reconstruction-01-03-01 in the release (details below) > > Cheers > > David > > * egammaRec-00-00-02 > > - add _load and _entries files > > - update to CBNT ntuple Dear All, Many thanks to those of you who responded to the mail about missing tags. There are still some problems - mainly incomplete tags. Such accidents are inevitable if the same 2-3 people tag most of the packages for practically every release and if one person (myself) has to collect and sort out a large number of (sometimes incomplete or conflicting) tags. This has to stop. 30/11/2018 S.A. CHEP03

AMI Databases 30/11/2018 S.A. CHEP03

AMI Databases Enforce some control! Forbid anarchic tagging. (which packages are tagged, and when, respect the hierarchy) Reduce developer error. Tag selection and CMT requirements files. Make release building more efficient. Centralize information on packages, especially documentation. 30/11/2018 S.A. CHEP03

Tag Collector Architecture CVS Repository TagCollector Database ViewCVS TagCollector PHP Services Web Interface WWW Mail AFS CLIENT 30/11/2018 S.A. CHEP03

AMI Databases 30/11/2018 S.A. CHEP03

Conclusions Tag Collector is a victim of its own success. AMI Databases Conclusions Tag Collector is a victim of its own success. Will be rewritten using AMI base classes. Made more modular. More features. To look at the Tag Collector go to http://larbookkeeping.in2p3.fr/athena Username = “demo”, no password 30/11/2018 S.A. CHEP03