1. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment Antonio Ceseracciu Gabriele Vedovato Antonello Ortolan

2. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 2- The AURIGA Gravitational Wave detector Location: Laboratori Nazionali di Legnaro – INFN First Acquisition Run: jun 1997 – nov 1999 Detector upgrades: transducer, suspension Next Acquisition Run: apr 2003 Detector: resonant bar Operating conditions: T=0.1K, attn 300dB Electronic readout: dc-SQUID, є=100 h Sensitivity: Δl/l ≈ 10 -20 Antenna readout: 5 kHz, 1.5 Gb/day Secondary readouts: seismometers, magnetometers, power monitor; 24 channels @200 Hz, 1 Gb/day International GW Observatory: UTC synchronization: GPS based system, accuracy 1 µs Data Exchange: common data format

3. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 3- DAQ hardware system HP1430 HP1413 RS232 MXI GPS Satellite AURIGA Detector 4882.8125 Hz – 23 bit Data Sampling 24 channels 200 Hz – 16 bit Data Sampling 10 MHz Synchronization Clock ADC Interrupts Data Synchronization GPS Date String DAQ Workstation GPS Environment

4. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 4- Requirements and Building Blocks Main Requirements for the DAQ system: ● Reliability: for long uninterrupted runs; ● Fault-Tolerance; ● Flexibility: modularity, real-time monitoring, components reuse, extensibility, network transparence. To fulfill at design level the above requirements, the envisioned architecture: ● is a multi-agent one, with multiple asynchronous processes doing specific tasks; ● Processes must be allowed to stop/reset/die at any time without forcing to reset the whole system. Multi-agent design needs a powerful framework to take care of all the common needs of processes control and intercommunication

5. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 5- Software Technology Fundamental platform choices (and specific implementation): ● Object Oriented design methodology ● C++ language ● CORBA interprocess communication (OmniORB, AT&T) ● O.S. GNU/Linux (RedHat); ● Compiler GCC ● Persistence format: IGWD Frame (Interferometric Gravitational Wave Detector) :the format adopted by GW experiments ● QT graphic toolkit (Trolltech) Development tools: ● CVS ● IDE: Kdevelop (KDE project), Designer, Trolltech

6. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 6- DAQ software architecture Log Antenna ANT Auxiliary AUX GPS RS232 Builder BLD Disk DSK MXI gps timeraw data Raw data 4882.8125 Hz – 23 bit Data Sampling 24 channels 200 Hz – 16 bit Data Sampling GPS Date String Logger Log messages Acquisition info commands Permanent Data storage Frequency (Hz) Monitor data

7. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 7- the PCL (Process Control Library) Working Thread Log msg queue Commands Spy Queue (frame) From Remote Queue CORBA Interface Device Interface From Device Data queue (frame) Data output Data Input Commands Status Info ● Defines a common architecture for processes, thus called PCL processes ● All the DAQ processes are PCL processes ● The PCL takes full care of: a) InterProcess Communication b) Synchronization c) Transient and Persistent storage d) Process Control ● The architecture provided by the PCL is inherently multithreaded ● Part of the PCL is provided as a shared library; part as template classes All of the custom process code is embedded in the Working Thread

8. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 8- The PCL intercommunication model ACTIVE LOADED CONFIGUREDINITIAL FAILURE Boot StopAbortShutdown Start Config Reset Process control is modeled by ● defining a small FSM ● associating control commands to state transitions ● All produced data is stored / transmitted by pushing it into a queue ● Consumers will poll asynchronously for new data. ● At every poll request, all the pending items in the data queue are fetched ● Benefits: asynchronous intercomm, simple design, but safe and efficient W T Data queue Working Thread Data queue Poll Request Producer Consumer Working Thread DATA

9. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 9 – PCL process model and CORBA PclInterface_impl +pclProducer: T* +Boot() +Configure() +Start() +Stop() +Abort() +Shutdown() T POA_PclInterface PortableServer::RefCountServantBase PclProducer #qpcl: FrameQueues* #qpclx: FrameXQueues* +Boot() +Config() +Start() +Stop() +Abort() +Shutdown() +Work() main «bind» ProcessProducer omni_thread Producer PCL process PCL PclOmniBind +startServer(prc:int,nQueues:int) T OmniORB ● Specialized code is in Producer ● Producer inherits the process model, and is bound as template parameter

10. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 10- performance The performance requirements for the AURIGA experiment are not very high. The requested throughput is about 30kB/sec, and the DAQ system easily proved more than one order of magnitude above that. Peak performance tests were conducted on two networked (100 Mb) Intel(R) Xeon(TM) CPU 2.40 Ghz machines For any block size, a limiting factor of 50Hz, probably due to the CORBA data transport, is present. The maximum throughput at 9600kB is limited by the network interface Antenna ANT Builder BLD Disk DSK 100Mb Block size (kB)HzThroughput (kB/sec) 1650 800 64503200 192509600

11. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment 11- Summary The first achievement for the Data Acquisition System: Providing all the desired functionality. Many improvements were brought in: Dataflow and persistent data: the frame format has been adopted to transport and store any information (GW and aux data, configuration...) Object Oriented design proved consistent and flexible; basic classes reused within the analysis environment Open source components and tools proved flexible and powerful, and ensure the longevity of the system The DAQ system has been in use for more than one year, requiring no maintenance nor code tweaking. It is ready for the upcoming new acquisition run of the AURIGA experiment.

12. CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment