Slide: 1 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 1 Update on Remote Real-Time Computing Farms For ATLAS Trigger DAQ. Richard Hughes-Jones.

Slides:

Advertisements

Similar presentations

DataTAG CERN Oct 2002 R. Hughes-Jones Manchester Initial Performance Measurements With DataTAG PCs Gigabit Ethernet NICs (Work in progress Oct 02)

Advertisements

G ö khan Ü nel / CHEP Interlaken ATLAS 1 Performance of the ATLAS DAQ DataFlow system Introduction/Generalities –Presentation of the ATLAS DAQ components.

Sander Klous on behalf of the ATLAS Collaboration Real-Time May /5/20101.

Project Partners Project Collaborators The Council for the Central Laboratory of the Research Councils Funded by EPSRC GR/T04465/01

CALICE, Mar 2007, R. Hughes-Jones Manchester 1 Protocols Working with 10 Gigabit Ethernet Richard Hughes-Jones The University of Manchester

The International Grid Testbed: a 10 Gigabit Ethernet success story in memoriam Bob Dobinson GNEW 2004, Geneva Catalin Meirosu on behalf of the IGT collaboration.

Copyright© 2000 OPNET Technologies, Inc. R.W. Dobinson, S. Haas, K. Korcyl, M.J. LeVine, J. Lokier, B. Martin, C. Meirosu, F. Saka, K. Vella Testing and.

JIVE VLBI Network Meeting 15 Jan 2003 R. Hughes-Jones Manchester The EVN-NREN Project Richard Hughes-Jones The University of Manchester.

TCP and ATLAS T/DAQ Dec 2002 R. Hughes-Jones Manchester TCP/IP and ATLAS T/DAQ With help from: Richard, HansPeter, Bob, & …

Xmas Meeting, Manchester, Dec 2006, R. Hughes-Jones Manchester 1 e-Science work ESLEA & EXPReS vlbi_udp Multiple Flow Tests DCCP Tests EXPReS-Dante Collaboration.

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Networking for ATLAS Remote Farms Richard Hughes-Jones The University.

GridPP meeting Feb 03 R. Hughes-Jones Manchester WP7 Networking Richard Hughes-Jones.

CdL was here DataTAG/WP7 Amsterdam June 2002 R. Hughes-Jones Manchester 1 EU DataGrid - Network Monitoring Richard Hughes-Jones, University of Manchester.

DataGrid WP7 Meeting CERN April 2002 R. Hughes-Jones Manchester Some Measurements on the SuperJANET 4 Production Network (UK Work in progress)

JIVE VLBI Network Meeting 28 Jan 2004 R. Hughes-Jones Manchester Brief Report on Tests Related to the e-VLBI Project Richard Hughes-Jones The University.

T2UK RAL 15 Mar 2006, R. Hughes-Jones Manchester 1 ATLAS Networking & T2UK Richard Hughes-Jones The University of Manchester then.

CALICE UCL, 20 Feb 2006, R. Hughes-Jones Manchester 1 10 Gigabit Ethernet Test Lab PCI-X Motherboards Related work & Initial tests Richard Hughes-Jones.

DataTAG Meeting CERN 7-8 May 03 R. Hughes-Jones Manchester 1 High Throughput: Progress and Current Results Lots of people helped: MB-NG team at UCL MB-NG.

PFLDNet Argonne Feb 2004 R. Hughes-Jones Manchester 1 UDP Performance and PCI-X Activity of the Intel 10 Gigabit Ethernet Adapter on: HP rx2600 Dual Itanium.

© 2006 Open Grid Forum Interactions Between Networks, Protocols & Applications HPCN-RG Richard Hughes-Jones OGF20, Manchester, May 2007,

Slide: 1 Richard Hughes-Jones CHEP2004 Interlaken Sep 04 R. Hughes-Jones Manchester 1 Bringing High-Performance Networking to HEP users Richard Hughes-Jones.

CdL was here DataTAG CERN Sep 2002 R. Hughes-Jones Manchester 1 European Topology: NRNs & Geant SuperJANET4 CERN UvA Manc SURFnet RAL.

ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester 1 VLBI & Protocols vlbi_udp Multiple Flow Tests Richard Hughes-Jones The University.

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 1 Discussion of Methodology for MPLS QoS & High Performance High throughput Investigations.

GGF4 Toronto Feb 2002 R. Hughes-Jones Manchester Initial Performance Measurements Gigabit Ethernet NICs 64 bit PCI Motherboards (Work in progress Mar 02)

Slide: 1 Richard Hughes-Jones Summer School, Brasov, Romania, July 2005, R. Hughes-Jones Manchester 1 TCP/IP and Other Transports for High Bandwidth Applications.

Virtual Organization Approach for Running HEP Applications in Grid Environment Łukasz Skitał 1, Łukasz Dutka 1, Renata Słota 2, Krzysztof Korcyl 3, Maciej.

Worldwide event filter processing for calibration Calorimeter Calibration Workshop Sander Klous September 2006.

EVN-NREN Meeting, Zaandan, 31 Oct 2006, R. Hughes-Jones Manchester 1 FABRIC 4 Gigabit Work & VLBI-UDP Performance and Stability. Richard Hughes-Jones The.

Copyright © 2000 OPNET Technologies, Inc. Title – 1 Distributed Trigger System for the LHC experiments Krzysztof Korcyl ATLAS experiment laboratory H.

Slide: 1 Richard Hughes-Jones e-VLBI Network Meeting 28 Jan 2005 R. Hughes-Jones Manchester 1 TCP/IP Overview & Performance Richard Hughes-Jones The University.

High Rate Internet Data Transfer for eVLBI Ralph Spencer, Richard Hughes-Jones and Simon Casey The University of Manchester All Hands Sept 2005.

Remote Online Farms Sander Klous

John D. McCoy Principal Investigator Tom McKenna Project Manager UltraScienceNet Research Testbed Enabling Computational Genomics Project Overview.

Greedy Applications Ethernet Everywhere Real Time Networks for the Atlas Experiment Brian Martin CERN In memory of Bob Dobinson And on behalf of a large.

Network Performance for ATLAS Real-Time Remote Computing Farm Study Alberta, CERN Cracow, Manchester, NBI MOTIVATION Several experiments, including ATLAS.

Connect. Communicate. Collaborate 4 Gigabit Onsala - Jodrell Lightpath for e-VLBI The iNetTest Unit Development of Real Time eVLBI at Jodrell Bank Observatory.

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester MB – NG SuperJANET4 Development Network SuperJANET4 Production Network Leeds RAL / UKERNA RAL.

ESLEA Bits&Bytes, Manchester, 7-8 Dec 2006, R. Hughes-Jones Manchester 1 Protocols DCCP and dccpmon. Richard Hughes-Jones The University of Manchester.

Slide: 1 Richard Hughes-Jones IEEE Real Time 2005 Stockholm, 4-10 June, R. Hughes-Jones Manchester 1 Investigating the Network Performance of Remote Real-Time.

ESLEA-FABRIC Technical Meeting, 1 Sep 2006, R. Hughes-Jones Manchester 1 Multi-Gigabit Trials on GEANT Collaboration with Dante. Richard Hughes-Jones The.

Connect. Communicate. Collaborate perfSONAR MDM Service for LHC OPN Loukik Kudarimoti DANTE.

Prospects for the use of remote real time computing over long distances in the ATLAS Trigger/DAQ system R. W. Dobinson (CERN), J. Hansen (NBI), K. Korcyl.

Online-Offsite Connectivity Experiments Catalin Meirosu *, Richard Hughes-Jones ** * CERN and Politehnica University of Bucuresti ** University of Manchester.

Geneva – Kraków network measurements for the ATLAS Real-Time Remote Computing Farm Studies R. Hughes-Jones (Univ. of Manchester), K. Korcyl (IFJ-PAN),

CAIDA Bandwidth Estimation Meeting San Diego June 2002 R. Hughes-Jones Manchester UDPmon and TCPstream Tools to understand Network Performance Richard.

PFLDNet Workshop February 2003 R. Hughes-Jones Manchester Some Performance Measurements Gigabit Ethernet NICs & Server Quality Motherboards Richard Hughes-Jones.

Collaboration Meeting, 4 Jul 2006, R. Hughes-Jones Manchester 1 Collaborations in Networking and Protocols HEP and Radio Astronomy Richard Hughes-Jones.

2003 Conference for Computing in High Energy and Nuclear Physics La Jolla, California Giovanna Lehmann - CERN EP/ATD The DataFlow of the ATLAS Trigger.

DataGrid WP7 Meeting Amsterdam Nov 01 R. Hughes-Jones Manchester 1 UDPmon Measuring Throughput with UDP  Send a burst of UDP frames spaced at regular.

Kraków4FutureDaQ Institute of Physics & Nowoczesna Elektronika P.Salabura,A.Misiak,S.Kistryn,R.Tębacz,K.Korcyl & M.Kajetanowicz Discrete event simulations.

Xmas Meeting, Manchester, Dec 2006, R. Hughes-Jones Manchester 1 ATLAS TDAQ Networking, Remote Compute Farms & Evaluating SFOs Richard Hughes-Jones The.

Experience with multi-threaded C++ applications in the ATLAS DataFlow Szymon Gadomski University of Bern, Switzerland and INP Cracow, Poland on behalf.

GridPP Meeting Jan 2003 R. Hughes-Jones Manchester ATLAS Trigger/DAQ Real-time use of the Grid Network Richard Hughes-Jones The University of Manchester.

Networks ∙ Services ∙ People Richard-Hughes Jones eduPERT Training Session, Porto A Hands-On Session udpmon for Network Troubleshooting 18/06/2015.

Final EU Review - 24/03/2004 DataTAG is a project funded by the European Commission under contract IST Richard Hughes-Jones The University of.

1 eVLBI Developments at Jodrell Bank Observatory Ralph Spencer, Richard Hughes- Jones, Simon Casey, Paul Burgess, The University of Manchester.

ESLEA VLBI Bits&Bytes Workshop, 31 Aug 2006, R. Hughes-Jones Manchester 1 vlbi_udp Throughput Performance and Stability. Richard Hughes-Jones The University.

ATLAS Computing Model & Service Challenges Roger Jones 12 th October 2004 CERN.

L1/HLT trigger farm Bologna setup 0 By Gianluca Peco INFN Bologna Genève,

The Evaluation Tool for the LHCb Event Builder Network Upgrade Guoming Liu, Niko Neufeld CERN, Switzerland 18 th Real-Time Conference June 13, 2012.

Connect. Communicate. Collaborate 4 Gigabit Onsala - Jodrell Lightpath for e-VLBI Richard Hughes-Jones.

DataGrid WP7 Meeting Jan 2002 R. Hughes-Jones Manchester Initial Performance Measurements Gigabit Ethernet NICs 64 bit PCI Motherboards (Work in progress)

CALICE TDAQ Application Network Protocols 10 Gigabit Lab

Bio-Informatics Workshop, Beijing , October 2009

Networking between China and Europe

Mar 2001 ATLAS T2UK Meeting R. Hughes-Jones

MB-NG Review High Performance Network Demonstration 21 April 2004

Remote Online Farms TDAQ Sander Klous ACAT April

MB – NG SuperJANET4 Development Network

Presentation transcript:

Slide: 1 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 1 Update on Remote Real-Time Computing Farms For ATLAS Trigger DAQ. Richard Hughes-Jones University of Manchester In Collaboration with: Bryan Caron University of Alberta Krzysztof Korcyl IFJ PAN Krakow Catalin Meirosu TERANA Jakob Langgard Nielsen Niels Bohr Institute

Slide: 2 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 2 Tests from Krakow u Configure/Test/Operate Krakow – CERN ANT GPS box Learn many details !!! Successful GPS and ANT operation using GUI at CERN Histograms still to be done u Synchronized Manchester – Krakow clocks with GPS u Configured ANT test Krakow-Manchester BUT … Manc Alteon card 66 MHz Manc Netgear card 0.5Mbyte Exchange at CERN next week u Online help from Catalin u udpmon tests Krakow – CERN CERN – Krakow Manc – CERN

Slide: 3 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 3 GÉANT2 Topology

Slide: 4 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 4 GÉANT2: The Convergence Solution NREN Access Existing IP Router Existing IP Router GÉANT2 POP B GÉANT2 POP A Managed Lambda’s 1626 LM L2 Matrix L2 TDM Matrix TDM 1678 MCC Dark Fiber EXPReS PC 10 GE EXPReS PC 10 GE

Slide: 5 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 5 Pionier Weather Map

Slide: 6 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 6 Test Traffic on Pionier

Slide: 7 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 7 CERN  Krakow: Throughput CERN  Krakow Krakow  CERN u800Mbit/s u20% Loss in the network u Mbit/s u40% Loss in the network u~3 * more re-ordering

Slide: 8 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 8 CERN  Krakow: Latency CERN  Krakow Krakow  CERN uFwhm ~10us uPeak separation us uFwhm 50 us uPeak separation ~135 us

Slide: 9 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 9 CERN  Krakow: 1-way delay CERN  Krakow Krakow  CERN u20 us packet spacing uInterrupt Coalescence

Slide: 10 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 10 Thanks to all who helped, including: uNational Research Networks Canarie, Dante, DARENET, Netera, PSNC and UKERNA u“ATLAS remote farms” J. Beck Hansen, R. Moore, R. Soluk, G. Fairey, T. Bold, A. Waananen, S. Wheeler, C. Bee u“ATLAS online and dataflow software” S. Kolos, S. Gadomski, A. Negri, A. Kazarov, M. Dobson, M. Caprini, P. Conde, C. Haeberli, M. Wiesmann, E. Pasqualucci, A. Radu

Slide: 11 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 11 Any Questions?

Slide: 12 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 12 Backup Slides

Slide: 13 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 13 Remote Computing Concepts ROB L2PU SFI PF Local Event Processing Farms ATLAS Detectors – Level 1 Trigger SFOs Mass storage Experimental Area CERN B513 Copenhagen Edmonton Krakow Manchester PF Remote Event Processing Farms PF lightpaths PF Data Collection Network Back End Network GÉANT Switch Level 2 Trigger Event Builders

Slide: 14 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 14 ATLAS Remote Farms – Network Connectivity

Slide: 15 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 15 ATLAS Application Protocol u Event Request EFD requests an event from SFI SFI replies with the event ~2Mbytes u Processing of event u Return of computation EF asks SFO for buffer space SFO sends OK EF transfers results of the computation u Tcpmon - instrumented tcp request-response program emulates the Event Filter EFD to SFI communication. Send OK Send event data Request event ●●● Request Buffer Send processed event Process event Time Request-Response time (Histogram) Event Filter Daemon EFD SFI and SFO

Slide: 16 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 16 End Hosts & NICs CERN-Manc. Request-response Latency Throughput Packet Loss Re-Order uUse UDP packets to characterise Host & NIC SuperMicro P4DP8 motherboard Dual Xenon 2.2GHz CPU 400 MHz System bus 66 MHz 64 bit PCI bus

Slide: 17 Richard Hughes-Jones T2UK, October 06 R. Hughes-Jones Manchester 17 TCP (Reno) – Details uTime for TCP to recover its throughput from 1 lost packet given by: u for rtt of ~200 ms: 2 min UK 6 ms Europe 20 ms USA 150 ms