ALFA - a common concurrency framework for ALICE and FAIR experiments Mohammad Al-Turany GSI-IT/CERN-PH

ALFA April 25, 2014M. Al-Turany, CWG13 meeting Common layer for parallel processing. Common algorithms for data processing. Common treatment of conditions database. Common deployment and monitoring infrastructure. 2

Current status of FairRoot April 25, 2014M. Al-Turany, CWG13 meeting 5/18/13M. Al-Turany, ACAT 2013, Beijing Florian Uhlig ROOT Users Workshop, Saas Fee Root TEve ROOT IO TGeo TVirtualMC Cint TTree … Proof Geant3 Geant4 Genat4_VMC Librarie s … VGM FairRoot … Run Manager IO Manager Runtime DB DB Interface Runtime DB DB Interface Event Display MC Application Module Detector Module Detector Task Magnetic Field … Event Generator Event Generator CbmRoot PandaRoot AsyEosRoot R3BRoot SofiaRoot MPDRoot FopiRoot EICRoot 3

What we went to do with FairRoot April 25, 2014M. Al-Turany, CWG13 meeting ROOT Geant3 Geant4 Genat4_VMC Libraries … VGM Runtime DB Module Detector Magnetic Field Event Generator CbmRoot PandaRoot AsyEosRoot R3BRoot SofiaRoot MPDRoot FopiRoot EICRoot MC Application Transport (FairMQ) Building configuraion Testing Fair DB CMake ZeroMQ 4

How is it with ALFA and FairRoot? April 25, 2014M. Al-Turany, CWG13 meeting ROOT Geant3 Geant4 Genat4_VMC Libraries and Tools … VGM Runtime DB Module Detector Magnetic Field Event Generator MC Application Fair MQ Building configuraion Testing Fair DB CMake ZeroMQ DDS BOOST Protocol Buffers FairRoot ALFA CbmRoot PandaRoot AsyEosRoot R3BRoot SofiaRoot MPDRoot FopiRoot EICRoot AliRoot6 (O 2 ) ???? 5

Current Status The Framework delivers some components which can be connected to each other in order to construct a processing pipeline(s). All components share a common base called Device Devices are grouped by three categories: – Source: Data Readers (Simulated, raw) – Message-based Processor: Sink, Splitter, Merger, Buffer, Proxy – Content-based Processor: Processor M. Al-Turany, CWG13 meeting April 25, 20146

The Dynamic Deployment System (DDS) Should: Deploy task or set of tasks Use (utilize) any RMS (Slurm, Grid Engine, … ), Secure execution of nodes (watchdog), Support different topologies and task dependencies Support a central log engine …. April 25, 2014M. Al-Turany, CWG13 meeting See Talk by Anar Manafov on Alice Offline week (March 2014) 7

Test Cluster 8 machines from DAQ lab with SLC6-64 – aidrefma01-08 FairRoot + FairSoft are installed – /local/home/cwg13/install/FairRoot Tutorial3 can be used as a demonstrator April 25, 2014M. Al-Turany, CWG13 meeting 8

Some remarks Mixture of SL6.5 and SL6.4 – Binary compatible but I had to switch off the test in config.sh /etc/issue is not available which also require a manual change for FairRoot Is there any way to export the display from these machines? April 25, 2014M. Al-Turany, CWG13 meeting 9

First test Two Samplers – aidrefma03 and aidrefma08 Six Processors – aidrefma04 and aidrefma05 One File writer (sink) – aidrefma02 April 25, 2014M. Al-Turany, CWG13 meeting 10

Processor1 startTestDectorProcessor1.sh ID=“301" processorTask="FairTestDetectorMQRecoTask" numIoThreads="1" inputSocketType="pull" inputRcvBufSize=$buffSize inputMethod="connect" inputAddress="tcp://aidrefma08:5565" outputSocketType="push" outputSndBufSize=$buffSize outputMethod="connect" outputAddress="tcp://aidrefma02:5568" /local/home/cwg13/install/FairRoot/build/bin/testDetectorProcessor$dataFormat $ID $processorTask $numIoThreads $inputSocketType $inputRcvBufSize $inputMethod $inputAddress $outputSocketType $outputSndBufSize $outputMethod $outputAddress April 25, 2014M. Al-Turany, CWG13 meeting 11

Processor2 startTestDectorProcessor2.sh ID="401" processorTask="FairTestDetectorMQRecoTask" numIoThreads="1” // How many IO threads ZeroMQ should use. //it is relevant only after 1 GB/s per output. inputSocketType="pull" inputRcvBufSize=$buffSize // Output Buffer size (in # of messages for ZeroMQ, in bytes for nanomsg). inputMethod="connect" inputAddress="tcp://aidrefma08:5565" outputSocketType="push" outputSndBufSize=$buffSize outputMethod="connect" outputAddress="tcp://aidrefma02:5568” /local/home/cwg13/install/FairRoot/build/bin/testDetectorProcessor$dataFormat $ID $processorTask $numIoThreads $inputSocketType $inputRcvBufSize $inputMethod $inputAddress $outputSocketType $outputSndBufSize $outputMethod $outputAddress April 25, 2014M. Al-Turany, CWG13 meeting 12

Sampler startTestDetectorSampler.sh ID="101" inputFile=”~/install/FairRoot/example/…/data/testdigi_TGeant3.root" parameterFile=“~/install/FairRoot/example/…/data/testparams_TGeant3.root" branch="FairTestDetectorDigi" eventRate="0" numIoThreads="1" outputSocketType="push" outputBufSize=$buffSize outputMethod="bind" outputAddress="tcp://*:5565" /local/home/cwg13/install/FairRoot/build/bin/testDetectorSampler$dataFormat $ID $inputFile $parameterFile $branch $eventRate $numIoThreads $outputSocketType $outputBufSize $outputMethod $outputAddress April 25, 2014M. Al-Turany, CWG13 meeting 13

FileSink startFileSink.sh ID="601" numIoThreads="1" inputSocketType="pull" inputRcvBufSize=$buffSize inputMethod=“bind" inputAddress="tcp://*:5568" /local/home/cwg13/install/FairRoot/build/bin/fileSin$dataFormat $ID $numIoThreads $inputSocketType $inputRcvBufSize $inputMethod $inputAddress April 25, 2014M. Al-Turany, CWG13 meeting 14

The Test April 25, 2014M. Al-Turany, CWG13 meeting 15 Sampler Processor1 FileSink

April 25, 2014M. Al-Turany, CWG13 meeting 16 TClonesArray Payload TClonesArray Payload TClonesArray Sampler Processor Sink From digits to hits with :

Example: Sampler April 25, 2014M. Al-Turany, CWG13 meeting 17

Example: Processor April 25, 2014M. Al-Turany, CWG13 meeting 18

Example: FileSink April 25, 2014M. Al-Turany, CWG13 meeting 19

The Test April 25, 2014M. Al-Turany, CWG13 meeting 20 Sampler Processor1 FileSink Processor1 The processors are “connecting” on both input and output, thus They can be added (removed) On the fly

April 25, 2014M. Al-Turany, CWG13 meeting 21

FileSink April 25, 2014M. Al-Turany, CWG13 meeting 22