Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Networking for ATLAS Remote Farms Richard Hughes-Jones The University of Manchester DataGrid WP7 – Dante Tests on the GÉANT Core End-2-End Measurements from the 4 th Year VLBI Project at Manchester New TCP stacks – the effect on throughput Some Simple Network Tests CERN-Manchester

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester DataGrid WP7 – Dante Tests on the GÉANT Core Set-up Supermicro PC in: London GEANT PoP Amsterdam GEANT PoP Smartbits in: London GEANT PoP Frankfurt GEANT PoP Long link UK-SE-DE2-IT-CH-FR-BE-NL Short Link UK-FR-BE-NL

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Tests GÉANT Core: UDP throughput UDP Throughput London-Amsterdam Available BW to packet on wire Then 1/t Wire rate 998 Mbit/s for packets > 1400 bytes Packet Loss None for large packets Dips in BW lined to packet loss SysKonnect NIC int. per packet CPU load important

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Tests GÉANT Core: Packet re-ordering Effect of Packet size London-Amsterdam Packets at 10 µs – line speed 10,000 sent Packet Loss ~ 0.1% Re-order Distribution

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Tests GÉANT Core: Packet re-ordering Effect of LBE background Amsterdam-London BE Test flow Packets at 10 µs – line speed 10,000 sent Packet Loss ~ 0.1% Re-order Distributions:

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Tests GÉANT Core: Packet Jitter Amsterdam-London BE Test flow Packet spacing 80 µs IPPremium Test flow BE Test flow + Background: 60% BE 1.4Gbit + 40% LBE 780Mbit IPPremium Test flow + Background

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Tests GÉANT Core: 1-way Delay Amsterdam-London IPPremium Test flow Packet spacing 80 µs BE Test flow + Background: 60% BE 1.4Gbit + 40% LBE 780Mbit

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester VLBI Project: Test Topology SuperJANET4 Jodrell Manchester SURFnet JIVE Dwingaloo Adam Mathews Steve O’Toole Univ of Manchester

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Manchester to Dwingeloo 2.0G Hz Xeon 1.2 GHz PIII Re-ordering vs Offered Load VLBI Project: Throughput

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester 1472 byte Packets man -> JIVE FWHM 22 µs (B2B 3 µs ) VLBI Project: Jitter & 1-way Delay 1-way Delay – note the packet loss (points with 0 –way delay)

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Aggregated Variance Method Divide time series length N into blocks of size m Calc mean of each section Xm(k) k= 1 … N/m Calc variance VXm of these Xm(k) Vary m size of the blocks Plot on log-log & fit slope β Hurst parameter H β = 2H -2 Measure: β = which gives H H =1 no long range dependence VLBI Project: Packet Loss – Long Range Effects?

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Traffic Flows uManchester – NetNorthWest - SuperJANET Access links Two 1 Gbit/s uAccess links: SJ4 to GÉANT GÉANT to SurfNet

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester High Performance TCP – DataTAG uDifferent TCP stacks tested on the DataTAG Network u128 ms round trip time uDrop 1 in 10 6 uHigh-Speed Rapid recovery uScalable Very fast recovery uStandard Recovery would take ~ 10 mins

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester uDrop 1 in 25,000 uRtt 6.2 ms uRecover in 1.6 s High Performance TCP – MB-NG StandardHighSpeed Scalable

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Some Network Tests uTCP Request – Response Zero stats OK done Send statistics: CPU load & no. int 1-way delay Send event data Request-Response time (Histogram) Request event Get remote statistics ●●●

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Lab Test: TCP Request-Response Histograms PC – router – PC BE Test flow Request spacing 0 µs Request spacing 10 ms

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Man-CERN: TCP Request-Response Latency DataTAG PC – backup link BE Tests Request spacing 0 µs Win size 2.5Mbytes Compare with UDP latency Large differences Rtt of 20 ms delay*bw = 2.5 Mbytes 1Mbyte data = 690 pkts  interesting bursts !

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Man-CERN: UDP Throughput & Packet Loss DataTAG PC – backup link BE Tests Throughput Packet loss

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Traffic Flows uManchester – NetNorthWest - SuperJANET Access links Link to PC in M/c uAccess links: 1 GE Man to NNW Total Man to NNW NNW to SuperJANET4

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Divide time series of packets into 1000 slices of 50 packets Total lost packets 1410 Average number / slice = 1.4 Calc Poisson Probability P(n, µ) = µ n e -µ n! Curves close but not exact Could be more than 1 process VLBI Project: Packet Loss – Is it Poisson?

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Traffic QoS Classes on GÉANT Backbone uNormal Traffic uNormal Traffic + uLess Than Best Effort 2.0 Gbit/s uNormal Traffic + uRadio Astronomy Data 500 Mbit/s uNormal Traffic + uRadio Astronomy Data + uLess Than Best Effort 2.0 Gbit/s Max Throughput on 2.5 G PoS

Meeting on ATLAS Remote Farms. Copenhagen 11 May 2004 R. Hughes-Jones Manchester Some Measurements made during ER2002