1. DataGrid WP7 Meeting CERN April 2002 R. Hughes-Jones Manchester Some Measurements on the SuperJANET 4 Production Network (UK Work in progress)

2. DataGrid WP7 Meeting CERN April 2002 R. Hughes-Jones Manchester The Measurements (1) uLatency Round trip times measured using Request-Response UDP frames Latency as a function of frame size Slope gives sum of individual data transfer rates end-to-end Mem copy + pci + Gig Ethernet + pci + mem copy Histograms of individual measurements uUDP Throughput Send a burst of UDP frames spaced at regular intervals Vary the frame size and the frame transmit spacing Record: The time to send and the time to receive the frames The number received, the number lost, number out of order CPU load, Number of interrupts uPacket Jitter Send a series of UDP frames spaced at regular intervals Record: The received inter-packet spacing

3. DataGrid WP7 Meeting CERN April 2002 R. Hughes-Jones Manchester MAN-RAL SuperJANET4 Topology Gigabit Ethernet 2.5 Gbit POS Access 2.5 Gbit POS core 622 Mbit Access WorldCom Super Janet 4 Core SJ4 C-PoP Warrington 12016 SJ4 C-PoP London 12016 WorldCom SJ4 C-PoP Reading 12016 WorldCom SJ4 C-PoP London 12016 WorldCom MAN 6500 (2) MAN BAR Test PC RAL ?? RAL BAR Test PC

4. DataGrid WP7 Meeting CERN April 2002 R. Hughes-Jones Manchester Latency: UDP Man - RAL Gigabit Interfaces Latency 5.2 mS Slope 0.0904 us/byte For 2 PCs expect: PCI0.00188 GigE0.008 PCI0.00188 Total 0.0118 us/byte 7 routers extra links 3 GigE0.024 2 2.5G PoS0.0064 1 622Mbit0.012 Total0.0424 Structure seen: Period 88 bytes Variation 150 – 190 us

5. DataGrid WP7 Meeting CERN April 2002 R. Hughes-Jones Manchester Latency: Histograms Peak separation ~120 us Multiple packets have fine structure ~10 us

6. DataGrid WP7 Meeting CERN April 2002 R. Hughes-Jones Manchester UDP Throughput: UDP Man - RAL Gigabit Interfaces Manc Motherboard: SuperMicro 370DLE Chipset: ServerWorks III LE Chipset CPU: PIII 800 MHz PCI:64 bit 66 MHz RedHat 7.1 Kernel 2.4.14 NIC: NetGear RAL RedHat 7.1 Kernel 2.2.16-3 NIC: Intel pro 1000 Max throughput 550Mbit/s Some throughput drop for packets < 20 us spacing Packet loss similar to PC-PC

7. DataGrid WP7 Meeting CERN April 2002 R. Hughes-Jones Manchester Throughput: Intel Pro/1000 on P4DP6 Max throughput 950Mbit/s Some throughput drop for packets >1000 bytes Packet loss small 800 – 1000 byte packets Motherboard: SuperMicro P4DP6 Chipset: Intel E7500 (Plumas) CPU: Dual Xeon Prestonia (2cpu/die) 2.2 GHz Slot 4: PCI, 64 bit, 66 MHz RedHat 7.2 Kernel 2.4.14