Archiving System Deployed at Soleil Tango Meeting (Soleil, March 2007)

Archiving System Deployed at Soleil Archiving System General Description HDB System Architecture Assessment Improvements TDB Mecanism Evolution HDB / TDB Implementation Remarks Tango Meeting (Soleil, March 2007)

Archiving System General Description The tools : Historical DataBase (HDB) Data are kept permanently The minimal archiving period is 10 seconds Temporary DataBase (TDB) Data are kept 3 days The minimal archiving period is 100 milliseconds The two are managed via the graphical tool named Mambo Snap DataBase (SNAP) Allows to have a snapshot of a set of attributes values Allows to apply values on a set of attributes It is managed via the graphical tool named Bensikin Its Goal : the same system must be useful for the machine and the beam lines. The main distinction is Machine : Oracle DataBase BeamLine : MySql DataBase Tango Meeting (Soleil, March 2007)

Archiving System General Description On the Machine Control System HDB and TDB is largely used by machine operators To log the values of their equipments To analyze somes particulars phenomena To help them diagnose (beam interlocks …) Snap not used enough On the Beam Lines Control System Snap will be deployed as part of the Control System HDB and TDB will be installed on demand Tango Meeting (Soleil, March 2007)

HDB – System Architecture Jacquard SNAP HDB Devices (Watcher, Extractor, etc) Knuth HdbArchiver Currently : 2600 archived attributes 2574 scalars + 26 spectrums 380 at 10 s period 1350 at 30 s 700 at 60 s 150 at [ 100 – 1000] s The rest is upper 1000 s 25 DServers * 5 devices, so 125 devices Now each device supports about 20 attributes Number of devices has been estimated to support 6000 attributes so about 50 attributes / device Tango Meeting (Soleil, March 2007)

HDB - Assessment Now : Secured HDB archiving system. How to notice it Less call from the operators Via the Supervision Device : the Watcher Diagnosis tool : detecting abnormal archiving interruption Recovery : reactivate archiving on the failed attributes Tango Meeting (Soleil, March 2007)

HDB - Assessment How do we reached this goal ? The code is more reliable Some attributes were archived by more than one archiver Lots of abnormals behaviors has been resolved Recovery Oracle Database optimization The storage mode (data are splitted in n partitions) Oracle server memory management Tango Meeting (Soleil, March 2007)

HDB - Improvements Spectrum storage in the Oracle Database The Oracle System foundation must be reexamine The space storage is not enough to guarantee a correct DataBase backup copy system This is a very important point for the control : backup / restore quickly in case of crash In the initial machine specification the maximum archived attributes must be 6000 now users request near 16 000 attributes The Oracle configuration must be clearly reviewed Tango Meeting (Soleil, March 2007)

TDB – System Architecture AHO Devices TdbArchiver Blaise TDB Devices (Watcher, Extractor) Currently : 2950 archived attributes 2940 scalars + 10 spectrums 15 at 200 ms period 1030 at 500 ms 1840 at 1 s 50 at [ 10 – 60] s The rest is upper 60 s 40 DServers * 5 devices, so 200 devices Now each device supports about 15 attributes Number of devices has been estimated to support 6000 attributes so about 30 attributes / device Tango Meeting (Soleil, March 2007)

TDB – Mechanism Evolution 1 - At the beginning Considering the minimum required period (100 ms) it was not possible to directly insert the attributes values in the DataBase Solution : work with intermediates files 1 - File Creation 2 - Filling Thread File 4 – Read  Insertion line / line TDB 3 – Read order Observations : Too many simultaneous insertions in DB DataBase was saturated  the Archiver threads were blocked In this configuration the TDB Archiving System was hang with only 600 attributes (6000 must be stored !!! ) Remark : This solution is always true for MySql DataBase Tango Meeting (Soleil, March 2007)

TDB – Mechanism Evolution 2 – Read the files thanks to Oracle SqlLoader Massives insertions file by file called by an Oracle Job Thread File 1 - File Creation 2 - Filling TDB 3 – One New file is available Every 4 minutes Block Insertion Job Observations : Too many insertion in DB : the Job is blocked and so the DB itself On the other hand, the Archivers are no more blocked but we cannot support more than 1500 attributs Tango Meeting (Soleil, March 2007)

TDB – Mechanism Evolution 3 – Creation of 10 Oracle Jobs Call every 10 minutes Each of them managed the files of different attributes based on the last digit of the attribute identifier. Observations : Now near 3000 attributes are archived The system can support the 6000 attributes Tango Meeting (Soleil, March 2007)

TDB - Improvements Spectrum storage in the Oracle Database Some peace of code must be improved (still too many exceptions) New Specification : permanently store, on demand, a subset of logged attributes values Require to lay out a particular storage space in the DB (Oracle and MySql) to comply to this new requierement. Tango Meeting (Soleil, March 2007)

HDB / TDB – Implementation Remarks There are some divergences between the code for MySql and Oracle See the Tdb mechanism DataBaseApi design is being reviewed to well distinguish the MySql and Oracle code  refactoring ?! We don’t manage the milliseconds in the MySql Database : not supported by the TIME type Tango Meeting (Soleil, March 2007)