9 December 2003D. Menasce. S. Magni: Database requirements for the Silicon Tracker 1 Database requirements for the Inner Silicon Tracker in BTeV First thoughts D. Menasce, S Magni - I.N.F.N. Milano The Inner Silicon Tracker will presumably rely on the following categories of Databases to keep track of it ’ s inner status:  Configuration/Initialization constants  Calibration constants  Monitor values and reference plots In this memo we will try to give a first look at what we envisage/expect to be relevant for each of these categories, with particular emphasis to:  Purpose of the database  Source of the data  Expected size  Anticipated use and modes of access  Detector construction tracking  Relationship between databases

9 December 2003D. Menasce. S. Magni: Database requirements for the Silicon Tracker 2 Purpose of the databases  Detector construction tracking  Keeps track of parts of the detector:  elements already assembled, quality test results  stocks ordered, components tested, general inventory  spare parts,...  Configuration/Initialization constants  Holds the current status of all quantities needed to properly initialize the system and make it ready for run or for calibration  Calibration constants  Will contain historical archive of calibration data (raw histograms and/or summary)  Monitor values and reference plots  Data collected at periodic intervals to check the status of the detector  threshold curves  setup values used to collect current calibration data (data or pulser),...  logical addresses of strips, suppression masks, trigger masks  threshold values, high voltage settings and current limits...  profile histograms, occupancy plots, residuals  crude track segments reconstruction statistics, physics signals ( ?)

9 December 2003D. Menasce. S. Magni: Database requirements for the Silicon Tracker 3 Relationship between databases  May have a relationship with all databases to define which components are active.  Detector construction tracking  Configuration/Initialization constants  Calibration constants  Monitor values and reference plots  Gets updated once a new calibration indicates there is a need to change threshold values.  Definite relationship with the Calibration Constants database, source of data needed to compute new set of initialization constants  Possible relationship with the Monitor Values database: the latter gets here reference values to check against monitored ones  Possible relationship with the Detector Construction database: initialization might be done only on components declared installed by the latter database.  Definite relationship with the Configuration Constants database. Values of calibration are used to compute threshold, masks to kill noisy/dead channels used by the latter db.  Possible relationship with the Detector Construction database: calibration might be done only on components declared installed by the latter database.  Possible relationship with the Configuration Constants database. Values of calibration might be used as reference values for quantities needed to operate monitor.  Possible relationship with the Detector Construction database: monitor might be done only on components declared installed by the latter database.

9 December 2003D. Menasce. S. Magni: Database requirements for the Silicon Tracker 4 Source of the data  Detector construction tracking  Main source of the data will presumably be a WEB interface, to allow collaborators in charge of assembly of the detector to enter values remotely.  Should large numbers of data be available as files (e.g. preamplifier chips shipped by a foundry, scripts or programs will transfer them into the database)  Configuration/Initialization constants  Most part of the data (thresholds, kill/trigger masks) will be entered by specialized programs. These will get data from Calibration database and compute relevant quantities.  Small part will be entered from WEB interface.  Calibration constants  Source will be from programs gathering data from DAQ.  Monitor values and reference plots  Source will be from programs gathering data from DAQ. These databases will be populated at low I/O rate, no direct feed from DAQ in real time is envisaged (no time-critical I/O). At any rate, databases are known to be slow in write mode, but quite fast in read; the most frequent anticipated mode of access will be read (particularly true for WEB access)

9 December 2003D. Menasce. S. Magni: Database requirements for the Silicon Tracker 5 Expected size  Detector construction tracking  Configuration/Initialization constants  Calibration constants  Monitor values and reference plots  Rough estimate is a few hundred Mb.  The number of strips in the detector is of the order of 3x10 5 (36 X 36 cm 2 option)  For each strip logical address etc.. are recorded (static information, no historical archive)  There are ~2300 chips (128 channels each) each holding a threshold value; this information is time dependent and will be updated presumably (pessimistic) once a month. Biases, current limits etc… (for each chip or for each detector wafer) will be updated much less frequently.  Rough estimate is a few Mb per month.  Precise size of this database is difficult to predict.  Threshold and noise values are the main content of this database. It has not yet been decided whether these data will be stored as raw (threshold curves, histograms) or as simple threshold/noise pairs.  Opting for the most conservative choice (complete information as histograms) 3x10 5 strips x 100 bin/hist x 4 bytes/word = 120 Mb.  Considering a weekly update frequency estimate is ~500 Mb/month.  Difficult to predict, depending on type of quantity stored (simple numerical values or plots/distributions). Assuming profile/occupancy histograms, 3x10 5 strips x 4 bytes/word we get ~1.2 Mb. Assuming 10 possible quantities monitored per strip, we get an assumed size of about 20 Mb per shift (8 hours?).

9 December 2003D. Menasce. S. Magni: Database requirements for the Silicon Tracker 6 Anticipated use and access patterns  Detector construction tracking  Configuration/Initialization constants  Calibration constants  Monitor values and reference plots  Use will be through WEB forms and reports to allow people to track progress in commissioning, installation and testing of the complete detector. Programs will read these data as well to gather information of existing/available components (calibration and monitor are meaningful only for installed parts).  Of no lesser usefulness will be the disposal of an historical archive of each sub-component origin and handling. Spare parts location is also of consideration.  This database contains all the information needed to properly initialize the detector.  Data will be accessed both by detector initialization program and by WEB scripts to allow people on shift to check out what the current image of detector looks like.  Areas of use will therefore be: WEB, online systems  Monitor data will be used both to decide upon need of a calibration or an update of initialization constant values, and as source of timelines used by offline programs.  Montecarlo simulation needs to be knowledgeable about run periods shifts in data characteristics, information gathered in the form of timelines.  Area of use will therefore be: WEB, online systems, offline programs/data analysis.  Areas of use will therefore be: online systems

9 December 2003D. Menasce. S. Magni: Database requirements for the Silicon Tracker 7 Considerations about the DBMS (1)  Which DBMS meets the requirements specified for the Silicon tracker?  Given the anticipated sized and foreseen uses, most likely a open/source DBMS is appropriate. MySQL, miniSQL and POSTGRES are all good candidates: reliable, easy to use, full fledged relational databases.  Since anticipated use is, without exception, through specialized programs, which API is offered by the DBMS is of consideration. Those indicated above all have :  C and Perl APIs  What about backups?  Backups are of course essential. A general solution should be devised to encompass all databases in use or foreseen by the collaboration.  Possible solution could be mirroring of the database at off-site institutions (overnight update of the mirror sites such as is done with offline libraries)  Any connection with other databases?  Most likely: mechanical mounting of the Silicon tracker has parts in common with the Straws detector. In the design of each database this consideration should be taken into account.  miniSQL no longer developed  MySQL currently the most used  POSTGRES developed with inferential capabilities: update of tables can be triggered by pre-specified conditions. Algorithms moved from interface program to the database itself, with centralization of procedures.

9 December 2003D. Menasce. S. Magni: Database requirements for the Silicon Tracker 8 Considerations about the DBMS (2)  Standards are important  It would be safe to assume that several databases will constitute the repository of many BTeV data. Adoption of a specific DBMS and design of internals should be made keeping in mind that standards solve cross database communication issues.  as an example, many programmers take for granted that the DBSM keeps an internal ID for a table (useful to join tables together). This is NOT an SQL standard: porting data to another DBMS becomes suddenly a nightmare. This feature, available (and actually useful) in POSTGRES is not in MySQL, which adheres to standards more strictly  Data representation should be independent of particular DBMS choice: this insures maintainability of the data over long periods of time (should support for a particular DBMS dwindle in time)  Binary data?  It can be sometimes useful to store binary chunks of data (GIF file drawings, histogram files ecc…). Different DMBS ’ s provide specific features to this extent.  Operating systems  Some database could be available on Windows and some on Linux.  ODBC and JDBC mechanisms to allow inter-communication between platforms should be adopted. Particularly important for WEB served databases.

9 December 2003D. Menasce. S. Magni: Database requirements for the Silicon Tracker 9 Geometry description database  In the present context, this database has a special status  A Geometry Manager is deemed necessary to de-couple the various possible simulation codes from the actual data that represent shapes, volumes and their position in space. XML has been envisaged as a possible unifying description language.  The issues is whether we need a database description of the geometry from which an XML description can be build upon request, or if XML actually IS the database. A conventional database source for an XML description file is probably preferable since XML descriptions can be rebuilt from there selecting upon run periods or any other criteria.  This database should contain a geometrical description of the detector:  Purpose of this database  Positions in space (derived either from technical drawings, CAD files, or from technical surveys or even from offline alignment)  Possibly shapes description and hierarchical relationship between components  Architectural decisions upon this database should be postponed to a general discussion about Geometry Manager