Report from the Working Group "Detector Database" W.Peryt (rapporteur), P.Buncic, D.Vicinanza, Y.Schutz. This WG has an urgent task to achieve that is the design and implementation of detector construction databases for the different detector group in ALICE. (from the Memo of F.C.) March 20 th, 2001 Detector Construction Database Working Group ALICE week, off-line software meeting, CERN, May 28th, 2001W.S. Peryt & Warsaw DB Group, WUT

Report from the Working Group "Detector Database" Konrad Bohuszewicz undergraduate student Maciej Czyzowicz undergraduate student Michal Janik Ph.D. student Rafal Holubowicz undergraduate student Dawid Jarosz undergraduate student Piotr Mazan undergraduate student Marcin Mierzejewski undergraduate student Mikolaj Olszewski undergraduate student Wiktor S. Peryt Sylwester Radomski undergraduate student Piotr Szarwas Ph.D. student Tomasz Traczyk Dominik Tukendorf undergraduate student Jacek Wojcieszuk undergraduate student Faculty of Electronics and Information Technology Faculty of Mathematics and Information Sciences Faculty of Physics

Alice detector databases – architecture Central database  Placed at CERN  Contains –central inventory of components –copies of data from laboratories Satellite databases  Placed in laboratories-participants  Contain source data –produced at laboratories –delivered by manufacturers Communication  Passing messages in XML  Off-line (batch processing)  No satellite-satellite communication! Central database Satellite databases

Central database versus satellite databases Central database  Usage characteristics –very large data volume –many concurrent users –twenty-four-hour utilisation –critical data (breakdown may be very costly)  Necessary features –7 × 24 availability –high reliability –continuous administration by professional DBA team –on-line backup –data partitioning  Proper solution –high-end commercial DBMS software is necessary Satellite databases  Usage characteristics –average or small data volume –only few concurrent users –used only few hours per day –not critical data (can be restored from backup or from central database)  Necessary features –should not need complex administration (no trained personnel at labs-participants) –should be monitored remotely by developers –off-line backup procedures are sufficient –data may not need to be partitioned  Proper solution –non-commercial database can be used

Monitoring of components Central database  Central inventory of components contains –actual location and status of each components –history of each component Flow of components  Creation of new component must be signalled to central inventory –a unique identifier for each component is generated by lab and registered in central inventory  Arrival and departure of each component to/from any laboratory must be recorded in the inventory  Destruction of the component must also be signalled to the inventory Solution  Components must be “checked-out” and “checked- in” from/to central inventory before/after they are processed by satellite databases  “Create”, “Check-in”,”Check-out”, and “Destroy” messages are passed as XML documents Central database Satellite databases 2. Check-in 3. Check-out 1. Creation 4. Destruction

Problems Compound components  Components may be composed of other components  Compound components –form hierarchy –must be treated in a special way  Changes of the components’ properties must be “cascaded” up and down the hierarchy, e.g. –“checked-in”, “checked-out” and “destroyed” statuses –location Communication problems  Breakdowns of telecommunications links cannot lock normal work procedures of satellite laboratories  There must be a possibility to ‘force’ check-in, check-out and creation of new component’s identifiers during telecommunication breakdowns  Changes made in this ‘force’ mode must be synchronised in central database as soon as the communication is recovered  Unique identifier (primary key) must be created for each new component to store its data in satellite database – even if during communication breakdown, so –globally unique id should be created by satellite database as compound id: database id + locally unique identifier –this identifier must be registered in central inventory as soon as possible

Development of the system Phase 1  Central database –contains only inventory –located in WUT –technology  Oracle DBMS  Java + Oracle XDK  Satellite databases –prototypes prepared concurrently on  MySQL  PostgreSQL –applications written in PHP –XML applications based on Oracle XDK  Central-satellite communication –in XML – prototype versions of messages –only inventory (Check-in/out, etc.) data Phase 2  Central database –contains copies of components’ data collected from satellite databases –all the data in one integrated structure –technology  Oracle DBMS (with use of object-relational technology)  Java technology (servlets, JSP)  Oracle XDK  dedicated Oracle technology (PL/SQL Web Agent, XSU, XSQL)  Satellite databases –all on the same selected DBMS –applications in PHP –XML applications: Java + Oracle XDK  Central-satellite communication –in XML – final versions of messages –inventory data and detailed components’ data

Design Repository What is needed?  Formalised representation of –user requirements –software requirements  Formalised description of –databases’ structures –applications  Extended data dictionary –prompts and descriptions –help texts –standard values of application parameters (fields’ sequence, dimensions, etc.)  Automated generation of –SQL scripts –parts of applications –help and documentation Possible solutions  CASE software –very powerful –with graphical interface –difficult and costly  “Poor man’s CASE” –proprietary formalisms and scripting –may be implemented in XML

Taken up tasks  Prototype model of interchange of measurement data between labs-participants and central database  “Envelope” (control) protocol for data interchange between central and satellite databases  Prototype system for control of components’ flow  Remote monitoring of satellite databases

Generic structures Object type C # Id * Property C1 * Property C2 * Property C3... Object type B # Id * Property B1 * Property B2 * Property B3... Object type A # Id * Property A1 * Property A2 * Property A3... “Typical” structure  Separate column for each property  Separate set of tables for each type of objects (having different set of properties) Generic structure  Dictionary-based –dictionaries of object types, properties, etc.  One generic set of tables for all objects Object # Id * Name... Type # Id * Name... Property definition # Id * Name * Optional? * Data type... Property value * Value

Human interface IMAGE Integrated MAnagement with Graphical InterfacE What it will be Tool for remote db control & administration Graphical user-database interface General for all local databases IMAGE – system features OS independence WWW user-interface Possibility of remote-control & administration Multi-access Divers levels of accessibility On-line documentation Integration with ROOT IMAGE – implementation Object-oriented technologies Open source development Modular structure of the application

Class structure for IMAGE Package IHObject IHContainer IHLink IHLaballed IHList IHDoc IHTCell IHTable IHForm IHApplet IHFrame IHImage IHString IDDatabase IDField IDDatabaseCtl IDMetaTableIDCollection IDQuerry IDTable ISAuthorizationISPermsISSessionISUser

IMAGE – documentation

Communication server (LabSERVER) for LabVIEW Originally developed by: Michał Janik & Piotr Szarwas Currently maintained by Marek Szuba Works at IReS, Strasbourg Available for all interested ALICE members immediately LabVIEWLabSERVER DBMS (MySQL or PostgreSQL) TCP/IP

Environment for test measurements LabVIEW NETWORK DBMS LabSERVER

Features of LabSERVER 100% C, POSIX-compliant Works as daemon Support for multiple clients connected simultaneously Access control Reliable data handling Highly configurable: TCP/IP port, connection timeout, maximum number of clients, allowed domains etc. Works with MySQL and PostgreSQL. On the other side: LabVIEW, HP VEE…

Milestones till September Installation of Oracle DMBS at WUT (Warsaw) Development of the prototype of the Central Database Beta-versions of satellite databases – implementation for SDD (Torino and Trieste) – implementation for SSD (partial – mainly for modules, microcables and object flow; Strasbourg, Utrecht, etc.) – maintenance of existing db concerning ladders for SSD (Utrecht) – Summer training for ~8 undergraduate students at SUBATECH (Nantes) July 15 th.. August 20 th