MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester MB – NG SuperJANET4 Development Network SuperJANET4 Production Network Leeds RAL / UKERNA RAL / UKERNA SJ4 Dev C-PoP Reading SJ4 Dev C-PoP Warrington Manc MCC ULCC UCL SJ4 Dev C-PoP London Gigabit Ethernet 2.5 Gbit POS 2.5 Gbit POS core WorldCom Dark Fibre (SSE)

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Some Edge (Bbone) Router requirements Connect to the test systems in the IP domain. Accept marked packets and be able to re-mark them Mark the packets on input according to at least: Source – dest IP address IP port Layer3 and Layer4 headers TCP/UDP Support Diffserve code points Perform policing and admission control Provide a range of Queue type options – Several queues with different priorities Real-time queue Provide flexible Queue Scheduling Weighted round robin Weighted fair queuing Provide Congestion control such as RED ECN Be able to assign MPLS labels to the packets in a flexible manner.

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Equipment for the External / Edge Domains Functionality Load PC Test PC Load PC Test PC Load PC Test PC Load PC Difserve router OR MPLS Edge Router 4 * 1 Gigabit Ethernet blade QoS Marking Gigabit Ethernet 2.5 Gbit POS 2.5 Gbit POS core SJDN Core MPLS Backbone Router * 1 Gigabit Ethernet blade 2.5 Gbit POS 10 Gbit POS 4 * 1 Gigabit Ethernet blade MPLS 2.5 Gbit POS MPLS MPLS Backbone Router OSR IP or MPLS Test PC Supervisor Load PC MPLS Classify packets QoS Difserv QoS – WRED MPLS Generate MPLS frames MPLS Multi-loop

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Ethernet Switch 3512 Local site Ethernet Ethernet Switch 3512 Local site Ethernet Equipment for the External / Edge Domains Connectivity Load PC Test PC Load PC Test PC Load PC Difserve router OR MPLS Edge Router 4 * 1 Gigabit Ethernet blade QoS Marking Gigabit Ethernet 2.5 Gbit POS 2.5 GbitPOS core 4 * 1 Gigabit Ethernet blade MPLS 2.5 Gbit POS MPLS MPLS Backbone Router OSR Spy Test PC Supervisor Load PC MPLS to core 3508 Monitor PC

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester IP Access to MB-NG Systems A suggestion only – we all must decide ! u End/Test systems routing: The default route from the PC is set to the Gigabit link i.e. into the development network. Static routes for researchers only are set for the 100 Mbit interface, e.g. to Manchester HEP, MCC, RAL, UCL, ULCC, UKERNA. The link between the Cisco 3512 and the Campus is set to 10 Mbit – this would minimise any unintentional traffic. uIP access to core routers for researchers only Similar approach? via the test PCs at the CPoP

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Some Questions ??? Can one Gigabit Ethernet blade perform the IP and MPLS functions we need or do we require two cards – one for IP input functions, the other for MPLS output? What is the performance of the 4-port Gigabit Ethernet interfaces? Confirm that an interface can support both IP and MPLS frames at same time. Can the GSRs use MPLS to loop the frames round n times? Can all our “requirements be” met?

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester uCisco OSR based on 6500 Are the details correct ? Focus on the routers - Hardware More memory

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Focus on the routers - Summary

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Hardware & Software Tools Optical splitter or opto-electrical tap opto-electrical tap constructed for logic analyser Software use of : tcpdump tcptrace Web100 4 * 1 Gigabit Ethernet blade Test PC 2.5 Gbit POS Monitor PC NIC High Bandwidth High Throughput Protocol Investigations (mainly) 4 * 1 Gigabit Ethernet blade Test PC 2.5 Gbit POS Monitor PC NIC EthernetSwitch 3508 Monitor port

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Latency Tests with 64-bit PCI uAlteon Gigabit NIC + RAW uSuperMicro 370DLE ServerWorks III LE Chipset 64 bit 33 MHz uRequest – Response uTransfer rates: observe expect us/B bit > 108Mbyte/s bit bit byte Request 1400 byte Response

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Throughput Tests with 64-bit PCI uRAW Stream uNot using IP stack uVary time between packets u1.6 factor improvement over bit transfers

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Throughput Tests UDP SysKonnect uUDP Stream uVary time between packets u us uNo interrupt coalescing u15 μs between packets 1400 byte sent 1400 byte received Pci-pci latency 28.3 μs

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Latency Tests UDP SysKonnect uUDP request - response uLatency and transfer rates very similar to raw frames observe us/B offset us bit bit (raw) 1400 byte sent μs pci time 1400 byte received μs pci time Pci-pci latency 28.3 μs

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Throughput Tests UDP Alteon uUDP Stream uVary time between packets u us uinterrupt coalescing u15 μs between packets

MB - NG MB-NG Meeting Dec 2001 R. Hughes-Jones Manchester Latency Tests UDP Alteon uUDP request – response uDue to interrupt coalescing