1. The Data Flow System of the ATLAS DAQ/EF "-1" Prototype Project G. Ambrosini 3,9, E. Arik 2, H.P. Beck 1, S. Cetin 2, T. Conka 2, A. Fernandes 3, D. Francis 3, Y. Hasegawa 4, M. Joos 3, G. Lehmann 1,3, J. Lopez 3,10, A. Mailov 2, L. Mapelli 3, G. Mornacchi 3, Y. Nagasaka 7, M. Niculescu 3,5, K. Nurdan 2, J. Petersen 3, D. Prigent 3, J. Rochez 3, L. Tremblet 3, G. Unel 3, S. Veneziano 3,6, Y. Yasu 8 1. Laboratory for High Energy Physics, University of Bern, Switzerland 2. Department of Physics, Bogazici University, Istanbul, Turkey 3. CERN, Geneva, Switzerland 4. ICEPP, University of Tokio, Tokio, Japan 5. Institute of Atomic Physics,Bucharest, Romania 6. I.N.F.N. Sezione di Roma, Roma, Italy 7. Nagasaki Institute for Applied Science, Nagasaki, Japan 8. High Energy Accelerator Research Organization (KEK), Japan 9. Now at Lightning Instrumentation S.A., Lausanne, Switzerland 10. Now at EDF, Grenoble, France

2. Padova, CHEP 2000 2 G. Lehmann, LHEP Bern / CERN The DAQ/EF “-1” Project Study of a vertical slice of the ATLAS DAQ system to: –define requirements on different sub-systems, –design the elements of the DAQ with their boundaries and their interaction with other components, –implement a prototype to evaluate technological solutions with the requested performance. The project has been organized in 4 main activities: –Detector interface –Data Flow –Event Filter –Back-End (see talk by I. Soloviev on Thursday)

3. Padova, CHEP 2000 3 G. Lehmann, LHEP Bern / CERN View of the Data Flow System See talk by S. Veneziano

4. Padova, CHEP 2000 4 G. Lehmann, LHEP Bern / CERN Prototype Implementation of the Data Flow

5. Padova, CHEP 2000 5 G. Lehmann, LHEP Bern / CERN Global Performance Measurements 3000 Event Rate = f (ROB fragment size) 0 0.5 1 1.5 2 2.5 3 3.5 05001000150020002500 ROB fragment size (bytes) Event Builder rate (kHz) 2x2 system Event Building Rate = f (L2R) 0 0.5 1 1.5 2 2.5 8990919293949596979899100 L2 reject factor Event Building rate (kHz) ROB fragment size variable, with mean ~1.5 kBytes The EB dictates the performance for a L2 rejection ratio below 95% Measurements with no L2 rejection The performance is limited by the EB interface which collects fragments over VME and sends them out on the ATM network.

6. Padova, CHEP 2000 6 G. Lehmann, LHEP Bern / CERN The Event Builder (EB) The EB is responsible for merging data fragments to complete, formatted events The EB design is based on a two layer approach which separates the technology specific aspects from the functionality of the EB elements and their interaction protocol Bsy/NotBsy EoT Transfer EoE GetId Src DFM Dst The EB has been studied through prototyping and simulation

7. Padova, CHEP 2000 7 G. Lehmann, LHEP Bern / CERN EB Performance Measurements: Gigabit Ethernet Processor: Intel Pentium PC @ 450 MHz OS: Linux Protocol: TCP/IP

8. Padova, CHEP 2000 8 G. Lehmann, LHEP Bern / CERN EB Performance Measurements: ATM Processor: RIOII 8062 SBC @ 200 MHz OS: LynxOS 2.5.1 Protocol: AAL5 ATM bandwidth Data

9. Padova, CHEP 2000 9 G. Lehmann, LHEP Bern / CERN Modelling of the EB Main purpose: study of the scaling of the EB system performance to ATLAS sizes Model design: 2 layer approach as in the prototype in order to be capable of studying different technologies with the same model Simulation Program: implementation with the discrete event simulation domain of the PTOLEMY (http://ptolemy.eecs.berkeley.edu) simulation tool

10. Padova, CHEP 2000 10 G. Lehmann, LHEP Bern / CERN Network Node Scheme Check queue Check User Buffer Send buffer queue Receive buffer queue C P U r u n n i n g t h e E B a p p l i c a t o n NIC Flow of data Flow control: checking of queues and requesting data Every EB element is a network node. Only the application specific part distinguishes between DFM, EBIF and SFI. The Network is modelled as an ideal router introducing a constant delay between input and output.

11. Padova, CHEP 2000 11 G. Lehmann, LHEP Bern / CERN EB Scalability Studies The time to build an event increases fairly linearly with the number of EBIFs; nevertheless the data cannot exclude a weak quadratic dependency yet. ATM

12. Padova, CHEP 2000 12 G. Lehmann, LHEP Bern / CERN Results of the simulation calibrated with the processing times and the ATM technology parameters measured in the prototype. The performance is strongly dependent on the evolution of the processing time as a function of the number of nodes. Scaling of the EB Link speed EB performance ATLAS region

13. Padova, CHEP 2000 13 G. Lehmann, LHEP Bern / CERN Scaling: Possible Improvements Reduce the number of nodes in the system by introducing higher bandwidth links Increase the processing power of the nodes 400x400 ATM155 with 3 times faster CPU

14. Padova, CHEP 2000 14 G. Lehmann, LHEP Bern / CERN Conclusions An Event Builder prototype has been implemented for Gigabit Ethernet and ATM; the latter has been used to calibrate a computer model of the EB. The model shows that the EB design is scalable and that the required performance is in reach. A small but complete Data Flow prototype has been designed and implemented; a LVL2 accept rate of 2.3 kHz could be sustained for ROB fragment sizes of ~1.5 kBytes.