1. Online-Offsite Connectivity Experiments Catalin Meirosu *, Richard Hughes-Jones ** * CERN and Politehnica University of Bucuresti ** University of Manchester The ATLAS Computing Model Workshop, Freiburg-im-Breisgau, Oct. 3 rd, 2004

2. Catalin Meirosu Oct. 3rd, 2004 Outline What are we trying to prove and why Network setup Network measurements – tools and results Conclusion

3. Catalin Meirosu Oct. 3rd, 2004 Why Connectivity Experiments for ATLAS Bandwidth from the online not going via Tier-0 yet to be evaluated Clear requirement for direct outward flow for  Some calibration tasks  Many monitoring tasks Potential problem: ATLAS might be low on computing resources, especially at the beginning  Could require more use of offsite resources  Even occasional remote event filtering might be useful We also need to prove that the network can handle the traffic originating in the Tier-0  Tier-0/Tier-1 phase of DC2 will test part of this, but not high bandwidth At CERN, negotiations with bandwidth providers are happening now ! (carried on by the CERN IT, for all the LHC experiments)  Any substantial non-Tier-0 requirements need to be established quickly These experiments are a proof of concept:  How much bandwidth can be used on a connection with a given capacity  What can be achieved using ATLAS-related applications  Proof of capability of the existing TDAQ software interworking with remote sites

4. Catalin Meirosu Oct. 3rd, 2004

5. Catalin Meirosu Oct. 3rd, 2004 Bandwidth measurements The networking is layered, from the physical transmission media (layer 1) to the application (layer 7) Tests at layer 2,3  relevant when the remote and the local sites are logically in the same LAN  programmable Gigabit Ethernet network interfaces used for traffic generation  example: throughput between CERN – INP Krakow, August 2004:~1000 Mbit/s Layer 4 tests: TCP, UDP  Relevant for general-purpose, Internet-style connectivity  Performed tests between Geneva and Copenhagen, Edmonton, Krakow, Manchester  Test equipment: server PCs, running patched Linux kernels and open source software for network measurements Example: Geneva – Manchester The network can sustain 1Gbps of UDP traffic, but the average server has problems with smaller packets Degradation for packets smaller than ~1000bytes, caused by the PC receiving the traffic

6. Catalin Meirosu Oct. 3rd, 2004 Real application in an ATLAS context Simple request-response program  Emulation of the request-response communication between the SFI and EFD in the Event Filter  Runs over TCP/IP  The client sends a small request message  The server answers with an up to 2 MB message Results … to be understood EF SFI request event network ATLAS Event Filter scenario

7. Catalin Meirosu Oct. 3rd, 2004 Request – Response results, CERN – Uni. Manchester connection Good response of the standard Linux TCP stack if properly tuned, poor if not Need to understand TCP implementation issues, not only the generic protocol 800Mbit/s achievable with tuned stack, 120 Mbit/s without – the same end nodes were used in both cases ! Out-of-the-box TCP settings Tuned TCP stack 64 byte Request green 1 MByte Response blue

8. Catalin Meirosu Oct. 3rd, 2004 Network Monitoring Essential for End-to-End understanding  Reduced performance on data transfer applications is often due to packet loss … so where did the loss occur ? Need access to the statistics on the network providers’ routers along our paths Also need campus-level information, requires collaboration with people responsible for networking at each site

9. Catalin Meirosu Oct. 3rd, 2004 Conclusion We are investigating the technical feasibility of remote real-time computing for ATLAS Have exercised multiple 1 Gbit/s connections between CERN and Universities located in Canada, Denmark, Poland and the UK  Network providers very helpful and interested in our experiments Developed a set of tests for a throughout characterization of the network connections  Very good results obtained, due to the excess of bandwidth available in the backbone  Properly configured end nodes essential for getting good results with real applications Next steps  Complete the characterization of all the connections  Interested in replicating calibration / monitoring style traffic, when model available  Evaluate new data transfer protocols in ATLAS-specific scenarios  Explore some of the above topics over a 10 Gbit/s connection  Investigate the scalability of the TDAQ DataFlow in this scenario