6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 1 Virgo Data Acquisition D. Verkindt, LAPP DAQ Purpose DAQ Architecture Data Acquisition examples Connection to DAQ and monitoring tools Data Streams Online analysis tools

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 2 DAQ purpose DAQ requirements: collection of distributed data (timing system, optical links) flexibility in data flow (frame format) reliability (at least 1 month without crash) easyness of use and restart (DAQ graphical client) DAQ requirements: collection of distributed data (timing system, optical links) flexibility in data flow (frame format) reliability (at least 1 month without crash) easyness of use and restart (DAQ graphical client) Laser

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 3 DAQ purpose Control BuildingCentral Building Get data from various synchronized sources, sometimes 3 km away North Building

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 4 Data Acquisition DetectionEnvironmentControls DAQ purpose Collect distributed data from: ITF environment ITF controls ITF output detection Collect distributed data from: ITF environment ITF controls ITF output detection Env. monitoring Suspension control Output MC Bench Detection Bench

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 5 DAQ architecture Central data collection 9 MB/s (compressed = 4MB/s) local data collector 2.7 MB/s frames Input bench monitoring, Vacuum monitoring, Environment monitoring 3.3 MB/s local data collector Suspensions data Locking and alignment data frames Environment Monitoring 3.0 MB/s frames local data collector Photodiodes data det. Bench monitoring DetectionControls

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 6 DAQ architecture FbF 1 GxFbF PhotodiodesAlignement FbS Susp. CtrlGlobal Ctrl 6 Gx 4 Fbf Laser + env + towers + tubes + calib.itf FbF3 Gx local Main Frame Builder Central Main Frame Builder FbS Det. Bench Ctrl FbS 3.0 MB/s3.3 MB/s2.7 MB/s 9 MB/s (compressed = 4MB/s) Environment MonitoringControlsDetection frames local Main Frame Builder DOL > 30 VME crates

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 7 DAQ architecture More than 30 VME crates, but a reduced set of standard tools: Digital Optical Links (DOL) for controls Fast Ethernet and Gbit Ethernet for central data collection VME crates for front-end data acquisition Workstations for central data collection Standard format for data collection : frames encapsulated in Ethernet messages

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 8 Frame format Frame = elementary time slice of data GW signal channel 1 frame 1 frame 2... Time channel 2 channel 3 channel n Contains: GPS time stamp ITF informations raw data channels processed data events Contains: GPS time stamp ITF informations raw data channels processed data events Common format of several gravitational waves detectors

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 9 Timing system overview GPS Timing Laser Data Acquisition DetectionEnvironmentControls Timing Distributor Crate User’s Timing Crates ADC,DAC Camera,DOL Timing CPU Coax Cables ADC,DAC Camera,DOL Timing CPU ADC,DAC Camera,DOL Timing CPU ADC,DAC Camera,DOL Timing CPU OPT/TTL Run OPT/TTL Frame OPT/TTL Sampling OPT/TTL Fast Clock TTL/OPT Frame Sampling Timing Distributor Crate User’s Timing Crates ADC,DAC Camera,DOL Timing CPU Coax Cables ADC,DAC Camera,DOL Timing CPU ADC,DAC Camera,DOL Timing CPU ADC,DAC Camera,DOL Timing CPU OPT/TTL Run OPT/TTL Frame OPT/TTL Sampling OPT/TTL Fast Clock TTL/OPT Frame Sampling Timing Distributor Crate User’s Timing Crates ADC,DAC Camera,DOL Timing CPU Coax Cables ADC,DAC Camera,DOL Timing CPU ADC,DAC Camera,DOL Timing CPU ADC,DAC Camera,DOL Timing CPU OPT/TTL Run OPT/TTL Frame OPT/TTL Sampling OPT/TTL Fast Clock TTL/OPT Frame Sampling Timing Distributor Crate User’s Timing Crates ADC,DAC Camera,DOL Timing CPU Coax Cables ADC,DAC Camera,DOL Timing CPU ADC,DAC Camera,DOL Timing CPU ADC,DAC Camera,DOL Timing CPU OPT/TTL Run OPT/TTL Frame OPT/TTL Sampling OPT/TTL Fast Clock TTL/OPT Frame Sampling Optical fibers Timing Information (Cm) all VME crates synchronized by Master clock Fast Clock (2.5 Mhz) Sampling (20 kHz) Frame (1 Hz) Monitoring & Control Part Generator & Distributor Part GPS CPU TTL/OPT Run TTL/OPT Frame TTL/OPT Fast Clock Timing TTL/OPT Sampling OPT/TTL Sampling OPT/TTL Frame Build. Return Timing Return GPS GPS Thanks to A. Masserot Purpose Synchronization (of controls) Frame and sampling numbers GPS time stamp for data exchange

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 10 SMS data Main Frame Builder frames timing info Slow Frame Builder GPS Timing timing info Data acquisition examples Slow Monitoring Stations query Sensor (temp. pressure…)

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 11 accelerometers, microphones, … Fast Frame Builder BNC cables GPS Timing timing signals Main Frame Builder frames Eth. 100 Mbps timing info Data acquisition examples

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 12 Fast Frame Builder Optical line (DOL) Main Frame Builder frames Eth. 100 Mbps timing info Data acquisition examples Photodiode Pre-ampli, demodulation& filtering Photodiode Readout GPS Timing timing signals Optical line Interferometer controls (DOL)

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 13 Connection to DAQ Slow Frame Builder Main Frame Builder Consumer 2 Consumer 1 Fast Frame Builders Central Main Frame Builder Main data stream Producer DAQ world Data Storage Shared Memory Online Processing Main Frame Builder: Use shared memory and 2 processes Producer: merge input frames and put result in shared memory Consumer: read frames in shared memory and send them on network Main Frame Builder: Use shared memory and 2 processes Producer: merge input frames and put result in shared memory Consumer: read frames in shared memory and send them on network dataDisplay Monitoring world requested data request Dynamical connection connect: send request with list of channels disconnect: automatic minimal perturbation on main stream. Dynamical connection connect: send request with list of channels disconnect: automatic minimal perturbation on main stream.

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 14 Online Monitoring using dataDisplay

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 15 Offline use of dataDisplay

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 16 DAQ control and monitoring

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 17 Web DAQ Monitoring

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 18 DAQ Performances nADC nBytes Run almost continuously since Sept DAQ efficiency during last engineering runs > 99.8% Minimized latency --> DAQ can be used for online control

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 19 Current data streams Raw data frames: most of channels sampled at 20 kHz or 10 kHz frame = 1 sec of raw data = 4MB (day=345 GB year=120 TB) Raw data frames: most of channels sampled at 20 kHz or 10 kHz frame = 1 sec of raw data = 4MB (day=345 GB year=120 TB) 50Hz data frames: 3% of raw data storage provide fast access to raw data in low frequency band resampling at 50Hz (with filtering) all the fast data channels frame = 10 sec of resampled data = 1.1 MB (day=9 GB year=3300 GB) 50Hz data frames: 3% of raw data storage provide fast access to raw data in low frequency band resampling at 50Hz (with filtering) all the fast data channels frame = 10 sec of resampled data = 1.1 MB (day=9 GB year=3300 GB) Trend data frames: 0.1% of raw data storage provide fast access to long (weeks) stretch of data trend data = min, max, mean, rms computed for each fast sampled channel, over one frame frame = 30mn of trend data = 9.6 MB (day=460 MB year=170 GB) Trend data frames: 0.1% of raw data storage provide fast access to long (weeks) stretch of data trend data = min, max, mean, rms computed for each fast sampled channel, over one frame frame = 30mn of trend data = 9.6 MB (day=460 MB year=170 GB)

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 20 Trend data acquisition Trend Frames Disks Trend Frame Builder Full Frame Storage (disks) Main Frame Builder Controls Frame Builder Detection Frame Builder Env. Moni Frame Builder Vega DB (Root) Web

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 21 Online Monitoring using trend data Example 1 : output of ITF over 8 hours, during engineering run E4 (min, max, mean) Use of Vega tool and Web browser

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 22 Offline use of trend data Example 2 : max of output of ITF, building temp. and seismic motion near north tower over 3 days

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 23 50Hz data acquisition Trend Frames Disks Trend Frame Builder Full Frame Storage (disks) Main Frame Builder Controls Frame Builder Detection Frame Builder Env. Moni Frame Builder 50Hz Frames Disks 50 Hz Frame Builder 50Hz processing 50Hz processing 50Hz processing Vega DB (Root) Web

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 24 Online monitoring using 50 Hz data Example 1 : monitoring of seismic activity over 8 hours, in 3 frequency bands

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 25 Offline use of 50 Hz data Example 2 : spectral density of output of ITF over 3 hours of data (made in 30 sec)

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 26 Online analysis tools GAI (General Algorithm Interface): A software tool to interface algorithms to online processing stream of data. Used to run online algorithms during engineering runs Used also offline to analyse engineering runs data Improved thanks to requests and comments from users and algorithm developers Some of the algorithms developed up to now with GAI for online and offline analysis: Algorithm 1 : monitoring of spectral lines in ITF output channels Algorithm 2 : search of glitches in ITF output channels Algorithm 3 : monitoring of the stationarity and gaussianity of the ITF output.

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 27 Online analysis tools Gai library Disk Shared Memory Ethernet GAI process Algorithm Disk Shared Memory Ethernet frames

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 28 Online analysis tools Disk Shared Mem Ethernet Algorithm2 Disk Shared Mem EthernetAlgorithm5 Algorithm4 Algorithm3 Algorithm1 : data under frame format

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 29 Online Analysis: current scheme Full Frame Storage (disks) Main Frame Builder Online Processing Frame Distributor Algo1 data storage Algorithm 2Algorithm 3Algorithm 1 Algo2 data storage Algo3 data storage frames raw data frames

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 30 Online Analysis: futur scheme Full Frame Storage (disks) Main Frame Builder Trigger manager Algorithm 2 Algorithm 3 Algorithm 1 frames Processed data storage frames raw data frames

6-10 Oct 2002GREX 2002, Pisa D. Verkindt, LAPP 31 Conclusion Virgo DAQ and online monitoring tools like dataDisplay or Vega+Web have been extensively used since year DAQ has shown to be: modular (lego pieces with standard connections between them) reliable and quite easy to use (and to restart) flexible and evolutive latency minimized Beyond DAQ: Useful data streams (raw data, trend data, 50Hz data, processed data, …) are under definition Online analysis has started