LHC BLM Software audit June 2008

BLM Software components Handled by BI Software section Expert GUIs Not discussed as part of this audit Real-Time software Topic for this presentation Handled by OP / CO Fixed displays Settings management Operational GUIs

Real-Time software architecture Based on FESA (AB standard framework for real-time software), which basically provides… Real-time action scheduling In the case of the BLMs, most actions are triggered by events from the central timing system Plus some special triggers from the collimation system Communication mechanism (server actions) with clients GET, SET and SUBSCRIBE Basic configuration using NFS xml files Internal memory organisation All variables and constants shared between real-time actions and server actions are formally defined in the design. The BLM design has > 450 such variables!

Real-time action scheduling - Acquisition Triggered by the 1Hz tick Acquires the 12 running sum data from the 16 DABs (Digital Acquisition Boards). Also reads the currently active thresholds for each monitor Reads all the thresholds and monitor names from the non-volatile memory This may be removed as it is quite heavy at 1Hz! Gets status from the 16 DABs + status from the combiner card Copies some data / status info from the 16 DABs to the combiner card when the combiner sets a test flag Keeps a history of the last 512 running sums (used later in the post-mortem data) Presently 1 Acquisition action handles everything, but this may be changed to have individual actions to allow accurate diagnostics and benchmarking No change in functionality though!

Real-time action scheduling - Capture Started by operators sending the capture event Buffers are actually started using BI’s BST timing which is triggered by the capture event 2048 packets with 2 different capture modes – SLOW (2.54ms running sum) and FAST (40us running sum) The control room can SET this mode to SLOW or FAST before sending the capture event On the CPU, 2 local events are created based on the capture event SLOW – triggered on capture event + 6 seconds FAST – triggered on capture event + 90mS One real-time action woken up by these 2 local events (+90ms & +6s) Uses the ‘data-ready’ flag on the acquisition cards to determine if data capture is complete If the capture mode is set to FAST, this flag will be set at capture event + ~82ms Otherwise, the flag will be set at capture event + ~5.2 seconds

Real-time action scheduling - Beam Dump (XPOC) & Post Mortem Unlike Capture buffers which are started on demand, the Beam Dump & Post Mortem buffers are frozen on demand Beam Dump buffers contain 200*40us samples Using the BST, an additional 4ms delay is applied to the Beam Dump trigger Therefore we measure 4ms before the beam dump and 4ms after the beam dump Post Mortem buffers contain 2048*40us samples BST delay is 4ms Therefore we measure ~78ms before the beam dump and 4ms after the beam dump The 2 corresponding real-time actions are triggered at least 4ms after the respective event from the central timing The Beam Dump action notifies clients that data is ready The Post Mortem action pushes the data to a post-mortem server

Real-time action scheduling - Parallelization of the actions The 4 real-time actions are potentially fighting for CPU time The 1Hz acquisition is synchronous The Capture/Beam Dump and Post Mortem acquisitions are asynchronous The post mortem data transfer is quite time consuming (>1 second) so causes holes in the Acquisition (i.e. we miss an acquisition) Not acceptable! FESA allows us to prioritise actions The Capture/Beam Dump/Post Mortem data will stay there until we’ve read it so it takes least priority! So in our case, priority is: 1. Acquisition 2. Collimation data (not discussed today) 3. Capture data 4. Beam Dump data 5. Post Mortem data

Server (client-side) actions Implemented actions (a.k.a. Properties) include 4 GET actions to compliment the 4 main real-time actions Acquisition Capture Beam Dump Post Mortem 2 additional actions, to GET expert data (not intended for OP) BLETCExpertAcquisition BLECSExpertAcquisition An action to GET and SET the threshold tables and monitor names / details Special action that inhibits all real-time actions to avoid corruption of data An action to GET and SET the configuration of the combiner card May be integrated with the action for the thresholds in the future? As with the real-time actions, we also have the possibility to prioritise these server actions Basically, make sure that the post-mortem action has lower priority than all the others! And the Acquisition has the highest priority

Configuration Depending on its configuration, the front-end controls what is sent to the logging, fixed displays, operational GUIs, etc – it’s very important! FESA provides persistency using NFS files Deemed not reliable enough so configuration data stored on non-volatile memory on the acquisition cards Instead, a server action accepts the new threshold data and monitor names, and sends this to the non-volatile memory For a trace of what was sent, a copy of every new configuration is stored in a directory on NFS The same system is used for the combiner card configuration The source of this configuration is handled by the LSA settings management Plus MCS (critical settings) And some dedicated BLM tables for the data

Finally, some statistics (less is better) Real-time processes 25 instances of the BLM software accessing up to 16 acquisition cards and a combiner card + 2 extra instances deployed in CMS 25 (+2?) instances of the BOBR (managed by BI SW) and LTIM (managed by CO) software All instances for the LHC have been configured and deployed Lines of code FESA framework C++ 17’908 lines XML 18’553 lines Perl 4’842 lines Java 39’941 lines BLM specific code C++ 6’210 lines Java 8’860 lines XML (design) 3’566 lines We rely heavily on FESA!

More details On the LIDS page Link found on BI SW’s homepage Details of the BLMLHC design Details of the BLMLHC deployment … and much more Also watch the LHC Technical Board Wiki Link also found on our homepage