MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 1 Discussion of Methodology for MPLS QoS & High Performance High throughput Investigations

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 2 MB – NG SuperJANET4 Development Network Gigabit Ethernet 2.5 Gbit POS Access 2.5 Gbit POS core MPLS Admin. Domains Dark Fiber (SSE) RAL/ UKERNA OSM-4GE- WAN-GBIC WorldCom SuperJANET4 Production Network Leeds SJ4 Dev C-PoP Reading SJ4 Dev C-PoP Warrington SJ4 Dev C-PoP London WorldCom UCL OSM-4GE- WAN-GBIC OSM- 1OC48- POS-SS UCL OSM-4GE- WAN-GBIC MCC OSM-4GE- WAN-GBIC OSM- 1OC48- POS-SS MAN OSM-4GE- WAN-GBIC

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 3 High Performance High Throughput Show and Tell & Deliverables  Bringing Gigabit throughput to the Grid applications  Show and tell demos  TCP streams operating  GridFTP operation  100 Mbit long transfers  500 Mbit long transfers  1 Gbit long transfers  Multiple host-host – Host-striping  High Throughput operations between multi-MPLS domains  High Throughput & QoS – Realtime (EF - spq) / best-effort / Scavenger (qbss)  Deliverables - Reports  PC Lab tests  Protocol monitoring tools  Gigabit WAN tests  Enhanced TCP stacks  Non TCP  Best practice for TCP & GridFTP in Grid Middleware and Applications

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 4 High Performance High Throughput The Tools  Understanding the operation of the end system HW  Latency vs frame size  Throughput vs message size  PCI bus load  CPU load  Effect of “protocol stacks”  Using Smartbits as a traffic generator  Best way to monitor traffic and protocol packets  Optical splitter + Cambridge Card or NICs  Monitor ports on Gigabit Switch  Web100 instrumentation  Type and effect of background traffic generators  Uniform  Random – Poisson  Bursty – web mirror – wget  Self-similar

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 5 High Performance High Throughput Tests on Intermediate Systems  Understand the management & configuration of routers  IP operation  MPLS  QoS/Diffserv management & configuration  Understand the performance of routers and (LAN) interconnects  Latency vs frame size  Throughput vs message size  Link framing overheads  The effect of load on switch/router thoughput  Offered vs achieved thoughput

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 6 High Performance High Throughput Tests on the WAN 1  TCP/IP Throughput vs rtt  Throughput vs window-size  Dependency of Latency & Throughput on end system hardware  Throughput using multiple TCP streams, and effect on the Network- how does the background throughput change?  The effect of background traffic loads  Effect of different TCP stacks and TCP tuning  Reno  New Reno  Vegas  Enhanced TCP algorithms – Sally Floyd  Delayed-Ack, SACK, …  ECN

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 7 High Performance High Throughput Tests on the WAN 2  TCP Behaviour in QoS environment  Marking  Rate / Congestion Control CAR RED ECN  Configurable Scheduling WFQ weighted fair queuing Priority / realtime WRR weighted round robin / DRR …  …  Bringing Gigabit throughput to the Grid applications  GridFTP at Gigabit  BBftp and BBscp at Gigabit  Data Replication GDMP  Use of non-TCP protocols, and effect on the Network  UDP Blast  via

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 8 High Performance High Throughput Time-Lines (1) Month1 NOW  Investigate NICs & End System behaviour  Investigate Monitoring techniques – Code modifications ?  Purchase end systems & monitoring tools !  Define test traffic Month 2  Install / configure End systems at sites  Set up GridFTP between sites – minimum Globus RH7.1? K2.4.14?  Set up BBftp BBscp  Test on SJ4 at 100 Mbit  Investigate TCP/IP Stacks & Tuning – Agreement on Kernel / details of tests /..  Define MPLS needs – multi-loop to extend the rtt Month 3  Delivery & installation of routers and access links (TM1)  Understand configuration of 7600s – IP only and MPLS  Understand LAN and blade performance of routers – Lab tests  Configuration of Development GSRs at PoPs Discussion! Time

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 9 High Performance High Throughput Time-Lines (2) Month4  Gigabit WAN tests e.g. UDP and TCP mem – mem streams  End-to-end IP only 2 sites / no background / with background  1 MPLS domain / n MPLS domains  Test with QoS – Realtime (EF - pf) / best-effort / Scavenger (qbss)  Think about results! Month 5  Simple Test GridFTP / BBscp BBftp over the development network  Transfers with QoS Realtime (EF - spq) / best-effort / Scavenger (qbss) Discussion! Time

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 10 High Performance High Throughput Time-Lines (3) Month 6  Initial demo of sustained throughput for different traffic classes (TM1)  Establish connection between systems with real data e.g. BaBar HEP Month 9  Demonstration of GridFTP operation  100 Mbit long transfers, 500 Mbit long transfers (TP1) Month 12  Initial demo & report of end-to-end guaranteed BW & Qos (TM2) Month18  Advanced demos including use of Qos Techniques (TM2)  Demo of reliable transport at > 1Gbit/s - including with QoS (TP1)  Grid Data Replication GDMP Month 24  Final report on High throughput transport across the WAN (TP1)  Final report on end-to-end traffic management over multiple MPLS domains (TM2) Discussion! Time

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 11 Next Steps  Draft document on High Performance Throughput Investigations to be circulated.  Technical “bits&bytes” meetings to discuss workplans and doc Discussion! Time

MB - NG MB-NG Meeting UCL 17 Jan 02 R. Hughes-Jones Manchester 12 Hardware & Software Tools High Bandwidth High Throughput Protocol Investigations (mainly) Optical splitter or opto-electrical tap opto-electrical tap constructed for logic analyser Software use of : tcpdump tcptrace 4 * 1 Gigabit Ethernet blade Test PC 2.5 Gbit POS Monitor PC NIC