Networking between China and Europe Richard Hughes-Jones DANTE Delivery of Advanced Network Technology to Europe Data transmission for international genomics projects, October 2010

GÉANT & NREN Topology & Connectivity Services 38 European Countries Dark Fibre + DWDM Hybrid network: Packet routed IP IPv6, multicast, VPN Point-to-point Circuits typically 1Gbit/s Dedicated Lambdas Full 10Gbit/s 25 POPs (+4) serve >30 NRENs 11600 km of fibre + 140 ILA sites 50+ x (own) 10G lambdas 9 x (leased) 10G lambda 8 x 2.5G (leased) “lambda” + some lower speed links Juniper T640, M160, M40 routers NREN accesses at up to 10Gbps (+ backup) + P2P GÉANT topology – expected by end 2010

GÉANT Global Connectivity

TEIN3 Topology January 2010 Trans-Eurasia Information Network Federation of NRENs 45 million users 8,000 Institutes 2.5 Gbit Links to Europe NOC in HC managed by DANTE Promotes EU-Asian collaborative work Catalyst for developing regional NRENs

Network Traffic

Traffic on the ORIENT Link 22 Oct 09 100-350 Mbit/s 3 Oct 10 350Mbit/s – 1 Gbit/s Week 16 Aug - 25 Sep 09 Week 17 Sep - 22 Oct 09

Traffic on the ORIENT Link Plot of traffic over the year: Nov 09 – Oct 10 Peak values important for achieving throughput Risen to 1.2-1.5 Gbit/s out of Europe 1-1.4 Gbit into Europe

Traffic To ORENT By TEIN Region Data for just one day China Dominant Many European countries and subject areas Senders

Traffic From ORENT By Region Many destination countries involved but quite variable China Dominant sender Senders

Network Protocols

Transport Layer 4 : UDP Connection less service over IP Minimal overhead – high performance It is unreliable, provides best effort delivery: Packet may be lost Duplicated Out of order Application is responsible for: Data reliability Packet Spacing Flow control Error handling

Transport Layer 4 : TCP Connection oriented service over IP It delivers a reliable end-2-end byte stream. TCP takes care of: Lost packets Duplicated packets Out of order packets Needs Tuning to get performance. TCP provides : Data buffering Flow control Error detection & handling Limits network congestion CWND slow start: exponential increase congestion avoidance: linear increase packet loss time retransmit: slow start again timeout

TCP (Reno) – Recovery Time The Time for TCP to recover its throughput from 1 lost 1500 byte packet given by: For rtt of ~200 ms @ 1 Gbit/s: 2 min Packet loss is a killer UK 6 ms Europe 25 ms USA 150 ms China 200ms 1.6 s 26 s 16min 28min

Performance

Network Topology EBI, London, Beijing Multi: Vendor Technology Domain

UDP Throughput & loss London – Beijing London  BJ 100 Mbit interface No packet loss for ≤ 100 Mbit/s BJ London No packet loss, receiver 1 Gbit

TCP reno Throughput Buffer Scan ah04 – London Traceroute: PC-lon-cop-BJ-PC RTT 197 ms BDP TCP buffer 2.5 MByte BJLon as expected. LonBJ not clear yet why 70 Mbit/s plateau.

SC2004: Disk-Disk bbftp bbftp file transfer program uses TCP/IP UKLight: Path:- London-Chicago-London; PCs:- Supermicro +3Ware RAID0 MTU 1500 bytes; Socket size 22 Mbytes; rtt 177ms; SACK off Move a 2 Gbyte file Web100 plots: Standard TCP Average 825 Mbit/s (bbcp: 670 Mbit/s) Scalable TCP Average 875 Mbit/s (bbcp: 701 Mbit/s ~4.5s of overhead) Disk-TCP-Disk at 1Gbit/s is here!

SC|05 HEP: Moving data with bbcp What is the end-host doing with your disks & network protocol? Look at the PCI-X 3Ware 9000 controller RAID0 1 Gbit Ethernet link 2.4 GHz dual Xeon Data ~660 Mbit/s PCI-X bus with RAID Controller PCI-X bus with Ethernet NIC Read from disk for 44 ms every 100ms Write to Network for 72 ms Power needed in the end hosts Careful Application design

ORIENT Plus New joint Chinese-European proposal for infrastructure 4-way funding: EU, GEANT, CERNET, CSTNET 4 year project Start with 2.5 Gbit plan for increase to 10 Gbit Aim to improve connectivity into China: equal access for CSTNET & CERNET Appreciates the need for helping user communities achieve improved throughput. EU call Dec 2010

Any Questions ?

APAN Asia-Pacific Advanced Network International Consortium Coordinate & promote Networking for the Asia-Pacific R&E Community.

Some members of APAN Translight/ Pacific wave IRNC Funding AARNet TransPAC3 IRNC Funding