Efficient utilization of 40/100 Gbps long-distance network by a single　TCP stream -- Outline of Project and Invitation to collaboration -- SURFnet Amsterdam IEEAF Seattle Chicago JGN2 Tokyo Kei Hiraki†1, Akira Kato†2, Mary Inaba†1 †1　　The University of Tokyo †2　 Keio University

Short history of High-speed networks 100BASE-Tx 1995 1000BASE-X 1998 Popular use of GbE 2001～ 10GBASE-X 2002 Local Use of 10GbE 2002～ Global Use of 10GbE 2003～ Popular use of 10GbE 2005～ 40/100GBASE-X 2010 Global Use of 40/100GbE 2010～ Local Use of 40GbE 2011～ Popular Use of 40/100GbE　Not yet Today

Popular use of High-speed networks Step 1 bundle of small streams High-speed uplink 1G or 10G 1G or 10G End system End system Switch Switch 40G/100G End system End system End system End system

Popular use of High-speed networks Step 1 bundle of small streams High-speed uplink Step 2 UDP-streams of a single user High-speed NIC on end systems Low efficiency of TCP on LFN Switch End system 40G Network I/F UDP based application (Tiled display) 40G/100G Ethernet

Popular use of High-speed networks Step 1 bundle of small streams High-speed uplink Step 2 UDP-streams of a single user Step 3 Single TCP-stream apps Progress of TCP technology Switch Switch 40G 40G Long Fat Pipe 40G Network I/F 40G Network I/F TCP based application (Data Transfer)

Popular use of High-speed networks Step 1 bundle of small streams High-speed uplink Step 2 UDP-streams of a single user Step 3 Single TCP-stream apps Step 4 TCP-streams from general apps Web browsing, file transfer, cloud computing etc. 40G Switch Switch 40G Long Fat Pipe 40G Network I/F + 5GB/s File system 40G Network I/F

Popular use of High-speed networks Step 1 bundle of small streams High-speed uplink Step 2 UDP-streams of a single user High-speed NIC on end systems Low efficiency of TCP on LFN Step 3 Single TCP-stream apps Progress of TCP technology Step 4 TCP-streams from general apps Web browsing, file transfer, cloud computing etc. Our target in 2013-14

History of single-stream IPv4 Internet Land Speed Record Distance bandwidth product Pbit m / s Our records 1,000 10 Gbps * 30,000km Target Performance 1100 Pbit m/s 2004/12/24 216 Pbit m / s 100 2006/2/20 264 Pbit m / s 2005/11/10 240 Pbit m / s 10 2004/11/9 Data Reservoir project WIDE project 149 Pbit m / s 1 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Year

Difficulty in 40GbE Many others still uknown CPU speed PCIe speed 1 IvyBridge core memory BW ~= 100 Gbps PCIe speed Gen3 x8 = 64 Gbps TCP scalability – Influence of random packet losses Software compatibility E.g. CUBIC may overflow Many others still uknown linux/net/ipv4/tcp_cubic.c /* !!! The following code does not have overflow problems, * if the cwnd < 1 million packets !!! */ 40 Gbps * 400 ms / 1500 byte = 1.33 million packets!!!

TCP technology for 40G/100G All the optimization for 10G LFN NIC parameters PCIe Parameters Offloading(TSO) TCP/IP parameters Zerocopy TCP Packet Pacing We achieved 99% TCP efficiency on 35,000 Km Internet

New TCP technology for 40G/100G Network loss classification (Poster TNC2013) Distinguish TCP losses and random network losses Use of machine learning for TCP control Without loss classification Congestion window size With loss classification

Call for collaboration We need 40G/100G long distance network Japan is a small country ( < 1,000 Km) No oversea network on Pacific Oceaan We want to evaluate our 40G single stream TCP technology on LFN > 20,000 Km

Outline of experiments Q4 2013 (Oct to Dec 2013) Length of the experiment 1～2　Weeks 40G/100G network with > 10,000Km Single stream TCP data transfer between 2 end systems We will carry all the necessary end systems

Please contact Kei Hiraki University of Tokyo hiraki@is.s.u-tokyo.ac.jp Mary Inaba University of Tokyo mary@is.s.u-tokyo.ac.jp Akira Kato Keio University kato@wide.ad.jp