Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 1 Archiving ALBA Sergi Rubio Manrique

“Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 2 One Year Ago October 2007: Archiving meeting at Soleil Main actions: –Performance tests on MySQL (ALBA) –Deployment of Distributed Archiving (ALBA) –Events integration (Elettra) –Refactoring of Archiving API (Soleil) –Milliseconds resolution on MySQL tables (ALBA) –Widgets for ATK and Qt (Soleil/ALBA)

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 3 Performance Tests on MySQL Our Test Platform is a 64Bit CPU with 2 Xeon Dual processors, 16GB RAM and 6TB HardDisk. We used SuSE 10.2, MySQL 5.0, Archiving release 1.3 and the precious help of our Systems section Attributes were generated using PySignalSimulators Device Servers. HDB tested using 10 seconds period and daily backup. TDB tested using 1 seconds period, 10 minutes export period and 7 days round-buffer.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 4 Test Results 6000 Attributes were recorded using HDB Archiver without any problem of performance. A rate of 4K attributes/second was achieved using TDB Archiver. Tests were limited by Simulators cpu usage (6000 Tango devices running in a single machine!). It was not possible to run both Archivings on the same machine. Disk usage of MySQL tables were lower than Oracle (30% less!). All procedures, results and deployment issues has been documented.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 5 Missing Tests Live Backup-Restore of HDB database has been tested only with small databases (40GB). Bigger databases must be generated to perform this test. TDB limits must be evaluated generating attribute values on a separated machine. Due to similar reasons, Event-Based archiving cannot be tested in a single machine due to notifd cpu-usage. Effects of multiple client access during archiving has not been evaluated.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 6 Main Problems Found Archiving Servers Startup. –It’s a highly cpu-stressing process. –Archiver devices must be started sequentially. –Attribute values must be already available before archiver device servers are started. Configuration of big groups of attributes: –Thousands of attributes cannot be managed through Mambo

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 7 db_host archivers _host host1 host2 host3 Classic Archiving Structure MySQL DS Archiver DS Archiver DS Load distributed between archivers: 1 archiver server = 5 archiver devices 1 archiver device = 50 archived attributes Separated hosts for Hdb, Tdb and Database Data is polled from Archivers and filtered; only changes above range are archived.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 8 host1 host3 host2 archiving_host DS Archiver DS Archiver DS Distributed Archiving MySQL Archiver 1 Archiver server is running on each controls machine. Attributes assigned using isDedicated and reservedAttributes properties The limit of 50 attributes/archiver is respected. Not-dedicated archivers are still being used for general purpose. DS Archiver DS

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 9 Advantages Archivers are launched in the last runlevel of each host Starter. It guarantees that all reserved attributes are available. Cpu usage problems are highly reduced as Archivers are not started in the same machine. But there’s no tool, apart from Jive, to configure this ~6000 property values!

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 10 Main Problems Found Archiving Servers Startup. –It’s a highly cpu-stressing process. –Archiver devices must be started sequentially. –Attribute values must be already available before archiver device servers are started. Configuration of big groups of attributes: –Thousands of attributes are hard to manage through Mambo.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 11 Archiving Configuration Reasons for a new config tool: Tree-Like method of selection used by Mambo was not focused in configuring hundreds of attributes (e.g. 500 Ion Pumps). Device Properties used to configure Distributed Archiving are not managed by Mambo. Some additional information (date of creation, author) is stored only in local profiles that are hard to maintain. The Snapshot database and its Contexts allow to store this information; but it is a separated database with no link with Hdb or Tdb tables. Comments are also a feature implemented in Snap database that is missing in Hdb/Tdb.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 12 Configuration with CSV files To avoid some of these problems a solution based on.csv files has been adopted; providing all the information to create an Archiving Configuration and a Snapshot Context.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 13 Configuration with CSV files To avoid some of these problems a solution based on.csv files has been adopted; providing all the information to create an Archiving Configuration and a Snapshot Context. Host information is used to configure Starters and Dedicated is used to easily configure thousands of attributes Context allow to export config. files from DB and create a link between archived values and comments.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 14 Deployment: Starter/Astor A Python ServersDict object has been created to provide Astor functionalities to configuration and alarm scripts.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 15 PyTango_utils.archiving archiving archiving.snap archiving.utils archiving.xml archiving.api archiving.scripts This packages allowed to: Integrate Hdb and Snap archiving with other python/PyTango tools. Start/Stop Archiving devices in the appropiated order. Increase the capabilities of configuration and diagnostic. Import/Export.csv and.xml files between the archiving and the database.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 16 PyTango_utils A set of add-ons to PyTango (lot of them thanks to TAUcoutinho, AleHoms! and Gurú Taurel): PyTango device server logging Dynamic Attributes CSV files management Utils for Callbacks/Exceptions Starter DS configuration Multiple TangoClass Inheritance? Used by PyPLC, PyStateComposer, VacuumController, TAU and others …

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII New Chances Now PyPLC-like Device Servers are able to setup/modify its own attributes archiving, starter and events configuration. DeviceServer.Init() can check if Archiving events are enabled … and ask the ArchivingManager to create the appropiated tables. They even can access its archived data! It is a great feature … or a big danger? ArchivingManager will be always able to accept/reject any request.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII But life is not so easy … With PyPLC-like device servers events are working even worse than with other devices. Multiple dynamic attributes with read-time near to 0 cause the polling-thread to give not enough time to those attributes that access real hardware. Many attribute subscription caused massive cpu-usage by the notifd. Could we subscribe a single DevVarStringArray attribute that passes the list of Changed Attributes to the clients?!?!?!

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII New Chances

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 20 Integration with TAU, TauPlot This Python link allowed TauPlot (ATKTrend-like widget) to visualize archived values. The archiving values are inserted as an extension of the attribute polling buffer.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 21 Integration with TAU, TauPlot This example used the same widget for actual and archived values. Full integration between the Archiving, SNAP and existing TAU widgets is now the main target.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 22 Future Projects Continue with MySQL performance tests and deployment problems detection. Milliseconds resolution Events integration TdbArchiving API Integration of Snapshot Comments as Marks on HDB or TDB visualization.

Sergi Rubio Manrique “Archiving System at ALBA”. Tango Meeting. ALBA. October 16 th, MMVIII 23 Thanks for your attention