Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 1/22 ALICE High Level Trigger Interfaces and Data Organisation CHEP 2006 – Mumbai ( – ) Sebastian Robert Bablok, Matthias Richter, Dieter Roehrich, Kjetil Ullaland (Department of Physics and Technology, University of Bergen, Norway) Torsten Alt, Volker Lindenstruth, Timm M. Steinbeck, Heinz Tilsner (Kirchhoff Institute of Physics, Ruprecht-Karls-University Heidelberg, Germany) Harald Appelshaeuser (Institute for Nuclear Physics, University of Frankfurt, Germany) Haavard Helstrup (Faculty of Engineering, Bergen University College, Norway) Bernhard Skaali, Thomas Vik (Department of Physics, University of Oslo, Norway) Cvetan Cheshkov (CERN, Geneva, Switzerland) for the ALICE Collaboration

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 2/22 Outline HLT Overview HLT - ECS Interface HLT - DAQ Data Format HLT - GRID Interface Summary & Outlook

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 3/22 Outline HLT Overview HLT - ECS Interface HLT - DAQ Data Format HLT - GRID Interface Summary & Outlook

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 4/22 HLT Overview Purpose: online event reconstruction and analysis providing of trigger decisions to DAQ monitoring and performance analysis of the ALICE detectors usage of its spare computer power (HLT cluster) for GRID Architecture: ca 500 node cluster (similar to Tier 2) dual SMP board with dual core CPUs (  up to 2000 processing units in sum) Gigabit Ethernet includes up to 300 FEP (Front-End-Processing) nodes receiving data from detectors dynamic software data transport framework (PubSub- and TaskManager system)

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 5/22 HLT Overview (300 FEP) (DDL)

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 6/22 HLT Overview Interfaces: raw data input copied via D-RORC from DAQ output of processed data and trigger decisions to the DAQ LDCs ECS interface as integration into the ALICE controlling system DCS interface to receive configuration data AliRoot interface to analysis code and for online monitoring AliEn interface to use idle nodes for GRID computing internal node interfaces using the PubSub - and TaskManager system HLT – cluster Detector Event Building / DAQ Trigger GRID DCS Online / Offline Software ECS Monitoring, Logging L2 Accept HLT Accept

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 7/22 Outline HLT Overview HLT - ECS Interface HLT - DAQ Data Format HLT - GRID Interface Summary & Outlook

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 8/22 Control of HLT through Experiment Control System (ECS) for initialisation and runtime operations Interface: represented as well defined states and transitions managed by Finite State Machines (FSM) ECS controls the HLT via HLT proxy implemented in SMI++ (external communication via DIM) internal communication is using interface library of TaskManager control system HLT - ECS Interface

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 9/22 HLT - ECS Interface HLT proxy states and transitions: subdivided into two type of states “stable” states (OFF, INITIALIZED, CONFIGURED, READY, RUNNING, ERROR): transitions only via well defined commands intermediate states (RAMPING_UP_&_INITIALIZING, CONFIGURING, ENGAGING, DISENGAGING, RAMPING_DOWN): longer procedure of transition – implicit transition, when procedure is finished configuration is passed as parameters of configure command mode of the HLT: “A” (no HLT)  no output to DAQ “B” (data processing only)  processed event data to DAQ, HLT trigger decisions ignored by DAQ “C” (full HLT functionality)  HLT trigger and pre-processed event data provided to DAQ output data format: output format version number of data sent back to DAQ trigger classes: tag identifying the desired physics triggers list of DDLs: on which H-RORCs receive event data for the HLT

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 10/22 HLT proxy - States ERROR OFF RAMPING_UP_&_INITIALIZING > RAMPING_DOWN > INITIALIZED CONFIGURED READY RUNNING CONFIGURING > ENGAGING > DISENGAGING > initialize implicit transtion shutdown reset shutdown configure reset engage disengage start stop implicit transtion

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 11/22 HLT - ECS Interface Implementation: SMI++ (common among all ALICE systems) single instance not partitioned HLT proxy is only active during (de-) initialisation- and configuration phase, in RUNNING state the cluster processes event data autonomously internal communication: interface library of TaskManager (TM) control system HLT proxy contacts to multiple master TMs of the cluster (majority vote – fault tolerance, avoiding single-point-of-failure) master TMs are networked with all slave TMs slave TMs control the PubSub system (RORC publisher and others)

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 12/22

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 13/22 Outline HLT Overview HLT - ECS Interface HLT - DAQ Data Format HLT - GRID Interface Summary & Outlook

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 14/22 HLT - DAQ Data Format HLT – output to DAQ DAQ handles HLT output like data from any other detector (except for the trigger decision). HLT output is received by the DDLs on D-RORCs (DAQ - ReadOut Receiver Cards) located on the LDCs (Local Data Concentrators).

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 15/22 HLT - DAQ Data Format Output data format: trigger decision: list of DDLs to read (detector DDLs [e.g. RoI] and HLT DDLs) Event Summary Data (ESD) in case of an accept candidate ID (ID from used trigger class) preprocessed event data [optional]: (partially) reconstructed event compressed raw data preprocessed detector specific data (e.g. Photon Spectrometer pulse shape analysis) Start-Of-Data (SOD) and End-Of-Data (EOD)

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 16/22 Outline HLT Overview HLT - ECS Interface HLT - DAQ Data Format HLT - GRID Interface Summary & Outlook

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 17/22 HLT - GRID Interface HLT cluster is similar to Tier 2 center:  up to 2000 processing units Cluster usage: HLT processing (ALICE data taking periods) GRID job execution (non data taking periods and unused nodes during data taking)  priority lies on the HLT functionality GRID middleware in ALICE is AliEn (Alice Environment)  HLT cluster offers Computing Element (CE) functionality to AliEn  due to priority in HLT tasks no (permanent) Storage Element (SE)

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 18/22 HLT - GRID Interface Dedicated node acts as GRID portal node: has AliEn installed contacts AliEn master takes care of certificates and authentication administration of GRID nodes includes unused nodes into GRID informs AliEn master about estimated Time-To-Live of included nodes (TTL) excludes HLT required nodes from GRID uses Condor as batch system underneath  Condor also installed on each node sharing of GRID applications inside cluster via network

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 19/22 HLT - GRID Interface GRID usage Free nodes administration (1): unused nodes (for HLT) are collected in “buffered” pool pool consists of three regions divided by two thresholds: GRID usage: nodes above the soft reserve threshold are offered for GRID jobs. below soft reserve: no more nodes are offered for GRID, unused nodes from the HLT are only collected in reserve, nodes finishing GRID jobs are recollected to reserve (  “soft” stop), nodes needed for HLT are taken from the pool. below hard reserve: amount of reserve nodes is below the safety minimum, GRID nodes are immediately recollected (GRID jobs are canceled) to the reserve (  “hard” stop) until hard reserve threshold is exceeded again. pool soft reserve hard reserve reserve

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 20/22 HLT - GRID Interface Free nodes administration (2):  aim of the hard reserve: immediately available spare nodes for HLT  aim of the soft reserve: avoiding “hard” stops as far as possible a sophisticated algorithm will estimate TTL of free nodes (this will develop together with the experience gathered during the use of the HLT cluster)  non data taking periods in this context are just special cases of the normal pool administration

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 21/22 Outline HLT Overview HLT - ECS Interface HLT - DAQ Data Format HLT - GRID Interface Summary & Outlook

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 22/22 Summary & Outlook HLT: will consist of a large PC cluster (similar to Tier 2) internally managed by TaskManager control system overall system to be configured and controlled by ECS  via well defined states and transitions (commands)  interface implemented in SMI++  internal communication to multiple master TaskManagers via own interface library will provide (pre-) processed data and trigger decisions to DAQ planned to offer unused nodes for GRID computing  organised in “buffered” pool (three regions divided by two thresholds)  interface to AliEn, Condor as batch system  utilisation of significant computing power for multiple applications

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 23/22 HLT proxy – States (additional slide)

Normal text - click to edit CHEP Mumbai ALICE High Level Trigger Interfaces … 24/22 HLT - GRID Interface (additional slide)