1. 25th Nov 2005CERN AB Controls Post Mortem data conversion G.Kruk

2. CERN – AB DepartmentG. Kruk – 25.11.2005 2 Agenda  Conversion overview Why do we want to convert the data? Conversion process BinX library Final data format  Current state  Issues & further steps

3. CERN – AB DepartmentG. Kruk – 25.11.2005 3 Why do we want to convert the data?  Minimize processing on the client side --> send the data as it is  Data sent by different clients may have different structure  To browse and analyze PM events we want to access (read) data coming from different systems in the same way (using the same API)  Let’s convert the different data into one, standardized format

4. CERN – AB DepartmentG. Kruk – 25.11.2005 4 Conversion process PM Server PMD file (CMW data object) Binary file with data sent by client Final data format Conversion program extracts converts Notifies through “LastDump” property 12 3 4

5. CERN – AB DepartmentG. Kruk – 25.11.2005 5 Conversion program It consists of two components:  PMManager Is notified when new PM data arrives (monitors “LastDump” property) Prepares directories structures Calls appropriate PMConverter to convert the data  PMConverter (interface) Performs the actual conversion from binary file to the final format Uses BinX library At the moment two implementations: FGCConverter, DefaultConverter (now used only for tests)

6. CERN – AB DepartmentG. Kruk – 25.11.2005 6 BinX library  Library (C++) facilitating reading/writing and conversion of binary files  Uses XML descriptor to decode the binary  Takes care of the endian-ness  Using it we can potentially avoid writing a separate PMConverter for each system/class  Only the conversion program depends on it

7. CERN – AB DepartmentG. Kruk – 25.11.2005 7 Final data format Information that need to be stored  For each signal: name type (int, float, string, …) values (array) units/precision? description?  Additional meta-information?

8. CERN – AB DepartmentG. Kruk – 25.11.2005 8 Final data format  After considering several options we decided to use SDDS as the final format  SDDS Fulfils requirements Is simple API in Java, C/C++, Python, … Supports both: ASCII and binary formats We have positive experiences with it

9. CERN – AB DepartmentG. Kruk – 25.11.2005 9 Current state  Converter program installed on the PM server machine (cs-ccr-pm1)  Supports conversion of FGC PM data - tested this week  Ready to perform tests with other systems

10. CERN – AB DepartmentG. Kruk – 25.11.2005 10 Current state (directories structure) SYSTEMS FGC 51 2005_11_24__14_44_45_240000000 RPMBB.UA83.RQSX3.L8.bin binx.xml RPMBB.UA83.RQSX3.L8-50Hz.sdds RPMBB.UA83.RQSX3.L8-100Hz.sdds RPMBB.UA83.RQSX3.L8-1kHz.sdds RPMBB.UA83.RQSX3.L8-events.sdds PROCESSED PMD FILES FGC 51 2005_11_24__14_44_45_240000000 RPMBB.UA83.RQSX3.L8.pmd

11. CERN – AB DepartmentG. Kruk – 25.11.2005 11 Issues & further steps  Performance ~600-700 ms to convert data from one PC There are places for improvements e.g. use binary SDDS rather than ASCII  No support for mapping from single bits into flag names e.g. 00000101  “FLAG_A FLAG_C” For FGCs at the moment it is hardcoded In the future  move the mapping to BinX descriptor or to some other descriptor  Directory structure modification (refinement)  Before extending functionality of the converter program – tests should be performed with other systems (not only with FGCs) and with analysis team: To validate current functionality, formats used, directory structure To understand better required refinements