1. LHC BLM Software audit
June 2008

2. BLM Software components
Handled by BI Software section
Expert GUIs
Not discussed as part of this audit
Real-Time software
Topic for this presentation
Handled by OP / CO
Fixed displays
Settings management
Operational GUIs

3. Real-Time software architecture
Based on FESA (AB standard framework for real-time software), which basically provides…
Real-time action scheduling
In the case of the BLMs, most actions are triggered by events from the central timing system
Plus some special triggers from the collimation system
Communication mechanism (server actions) with clients
GET, SET and SUBSCRIBE
Basic configuration using NFS xml files
Internal memory organisation
All variables and constants shared between real-time actions and server actions are formally defined in the design. The BLM design has > 450 such variables!

4. Real-time action scheduling - Acquisition
Triggered by the 1Hz tick
Acquires the 12 running sum data from the 16 DABs (Digital Acquisition Boards). Also reads the currently active thresholds for each monitor
Reads all the thresholds and monitor names from the non-volatile memory
This may be removed as it is quite heavy at 1Hz!
Gets status from the 16 DABs + status from the combiner card
Copies some data / status info from the 16 DABs to the combiner card when the combiner sets a test flag
Keeps a history of the last 512 running sums (used later in the post-mortem data)
Presently 1 Acquisition action handles everything, but this may be changed to have individual actions to allow accurate diagnostics and benchmarking
No change in functionality though!

5. Real-time action scheduling - Capture
Started by operators sending the capture event
Buffers are actually started using BI’s BST timing which is triggered by the capture event
2048 packets with 2 different capture modes – SLOW (2.54ms running sum) and FAST (40us running sum)
The control room can SET this mode to SLOW or FAST before sending the capture event
On the CPU, 2 local events are created based on the capture event
SLOW – triggered on capture event + 6 seconds
FAST – triggered on capture event + 90mS
One real-time action woken up by these 2 local events (+90ms & +6s)
Uses the ‘data-ready’ flag on the acquisition cards to determine if data capture is complete
If the capture mode is set to FAST, this flag will be set at capture event + ~82ms
Otherwise, the flag will be set at capture event + ~5.2 seconds

6. Real-time action scheduling - Beam Dump (XPOC) & Post Mortem
Unlike Capture buffers which are started on demand, the Beam Dump & Post Mortem buffers are frozen on demand
Beam Dump buffers contain 200*40us samples
Using the BST, an additional 4ms delay is applied to the Beam Dump trigger
Therefore we measure 4ms before the beam dump and 4ms after the beam dump
Post Mortem buffers contain 2048*40us samples
BST delay is 4ms
Therefore we measure ~78ms before the beam dump and 4ms after the beam dump
The 2 corresponding real-time actions are triggered at least 4ms after the respective event from the central timing
The Beam Dump action notifies clients that data is ready
The Post Mortem action pushes the data to a post-mortem server

7. Real-time action scheduling - Parallelization of the actions
The 4 real-time actions are potentially fighting for CPU time
The 1Hz acquisition is synchronous
The Capture/Beam Dump and Post Mortem acquisitions are asynchronous
The post mortem data transfer is quite time consuming (>1 second) so causes holes in the Acquisition (i.e. we miss an acquisition)
Not acceptable!
FESA allows us to prioritise actions
The Capture/Beam Dump/Post Mortem data will stay there until we’ve read it so it takes least priority!
So in our case, priority is:
1. Acquisition
2. Collimation data (not discussed today)
3. Capture data
4. Beam Dump data
5. Post Mortem data

8. Server (client-side) actions
Implemented actions (a.k.a. Properties) include
4 GET actions to compliment the 4 main real-time actions
Acquisition
Capture
Beam Dump
Post Mortem
2 additional actions, to GET expert data (not intended for OP)
BLETCExpertAcquisition
BLECSExpertAcquisition
An action to GET and SET the threshold tables and monitor names / details
Special action that inhibits all real-time actions to avoid corruption of data
An action to GET and SET the configuration of the combiner card
May be integrated with the action for the thresholds in the future?
As with the real-time actions, we also have the possibility to prioritise these server actions
Basically, make sure that the post-mortem action has lower priority than all the others!
And the Acquisition has the highest priority

9. Configuration
Depending on its configuration, the front-end controls what is sent to the logging, fixed displays, operational GUIs, etc – it’s very important!
FESA provides persistency using NFS files
Deemed not reliable enough so configuration data stored on non-volatile memory on the acquisition cards
Instead, a server action accepts the new threshold data and monitor names, and sends this to the non-volatile memory
For a trace of what was sent, a copy of every new configuration is stored in a directory on NFS
The same system is used for the combiner card configuration
The source of this configuration is handled by the LSA settings management
Plus MCS (critical settings)
And some dedicated BLM tables for the data

10. Finally, some statistics (less is better)
Real-time processes
25 instances of the BLM software accessing up to 16 acquisition cards and a combiner card
+ 2 extra instances deployed in CMS
25 (+2?) instances of the BOBR (managed by BI SW) and LTIM (managed by CO) software
All instances for the LHC have been configured and deployed
Lines of code
FESA framework
C++ 17’908 lines
XML 18’553 lines
Perl 4’842 lines
Java 39’941 lines
BLM specific code
C++ 6’210 lines
Java 8’860 lines
XML (design) 3’566 lines
We rely heavily on FESA!

11. More details On the LIDS page Link found on BI SW’s homepage
Details of the BLMLHC design
Details of the BLMLHC deployment
… and much more
Also watch the LHC Technical Board Wiki
Link also found on our homepage