1. udt.sourceforge.net 1 :: 50 BREAKING THE DATA TRANSFER BOTTLENECK Yunhong GU gu@lac.uic.edu National Center for Data Mining University of Illinois at Chicago October 10, 2005 Updated on August 8, 2009 udt.sourceforge.net UDT: A High Performance Data Transport Protocol

2. udt.sourceforge.net 2 :: 50 Outline INTRODUCTION PROTOCOL DESIGN & IMPLEMENTATION CONGESTION CONTROL PERFORMANCE EVALUATION COMPOSABLE UDT CONCLUSIONS

3. udt.sourceforge.net 3 :: 50 >> INTRODUCTION PROTOCOL DESIGN & IMPLEMENTATION CONGESTION CONTROL PERFORMANCE EVALUATION COMPOSABLE UDT CONCLUSIONS

4. udt.sourceforge.net 4 :: 50 Motivations  The widespread use of high-speed networks (1Gb/s, 10Gb/s, etc.) has enabled many new distributed data intensive applications  Inexpensive fibers and advanced optical networking technologies (e.g., DWDM - Dense Wavelength Division Multiplexing)  10Gb/s is common in high speed network testbeds, 40 Gb/s is emerging  Large volumetric datasets  Satellite weather data  Astronomy observation  Network monitoring  The Internet transport protocol (TCP) does NOT scale well as network bandwidth-delay product (BDP) increases  New transport protocol is needed!

5. udt.sourceforge.net 5 :: 50 Data Transport Protocol  Functionalities  Streaming, messaging  Reliability  Timeliness  Unicast vs. multicast  Congestion control  Efficiency  Fairness  Convergence  Distributedness Physical Layer Applications Data link Layer Network Layer Transport Layer

6. udt.sourceforge.net 6 :: 50 TCP  Reliable, data streaming, unicast  Congestion control  Increase congestion window size (cwnd) one full sized packet per RTT  Halve the cwnd per loss event  Poor efficiency in high bandwidth-delay product networks  Bias on flows with larger RTT ½ Bandwidth * RTT

7. udt.sourceforge.net 7 :: 50 TCP 0.01% 0.05% 0.1% 0.5% 1000 800 600 400 200 110100200400 1000 800 600 400 200 Throughput (Mb/s) Packet Loss Round Trip Time (ms) LANUS-EUUS-ASIAUS

8. udt.sourceforge.net 8 :: 50 Related Work  TCP variants  HighSpeed, Scalable, BiC, FAST, H-TCP, L-TCP  Parallel TCP  PSockets, GridFTP  Rate-based reliable UDP  RBUDP, Tsunami, FOBS, FRTP (based on SABUL), Hurricane (based on UDT)  XCP  SABUL

9. udt.sourceforge.net 9 :: 50 Problems of Existing Work  Hard to deploy  TCP variants and XCP  Need modifications in OS kernel and/or routers  Cannot be used in shared networks  Most reliable UDP-based protocols  Poor fairness  Intra-protocol fairness  RTT fairness  Manual parameter tuning

10. udt.sourceforge.net 10 :: 50 A New Protocol 0.01% 0.05% 0.1% 0.5% 1000 800 600 400 200 110100200400 1000 800 600 400 200 Throughput (Mb/s) Packet Loss Round Trip Time (ms) LANUS-EUUS-ASIAUS

11. udt.sourceforge.net 11 :: 50 UDT (UDP-based Data Transfer Protocol)  Application level, UDP-based  Similar functionalities to TCP  Connection-oriented reliable duplex unicast data streaming  New protocol design and implementation  New congestion control algorithm  Configurable congestion control framework

12. udt.sourceforge.net 12 :: 50 Objective & Non-objective  Objective  For distributed data intensive applications in high speed networks  A small number of flows share the abundant bandwidth  Efficient, fair, and friendly  Configurable  Easily deployable and usable  Non-objective  Replace TCP on the Internet

13. udt.sourceforge.net 13 :: 50 UDT Project  Open source (udt.sourceforge.net)  Design and implement the UDT protocol  Design the UDT congestion control algorithm  Evaluate experimentally the performance of UDT  Design and implement a configurable protocol framework based on UDT (Composable UDT)

14. udt.sourceforge.net 14 :: 50 >> PROTOCOL DESIGN & IMPLEMENTATION INTRODUCTION CONGESTION CONTROL PERFORMANCE EVALUATION COMPOSABLE UDT CONCLUSIONS

15. udt.sourceforge.net 15 :: 50 UDT Overview  Two orthogonal elements  The UDT protocol  The UDT congestion control algorithm  Protocol design & implementation  Functionality  Efficiency  Congestion control algorithm  Efficiency, fairness, friendliness, and stability

16. udt.sourceforge.net 16 :: 50 UDP Socket API Applications UDT Overview TCP Socket API Applications UDT UDT Socket

17. udt.sourceforge.net 17 :: 50 Functionality  Reliability  Packet-based sequencing  Acknowledgment and loss report from receiver  ACK sub-sequencing  Retransmission (based on loss report and timeout)  Streaming and Messaging  Buffer/memory management  Connection maintenance  Handshake, keep-alive message, teardown message  Duplex  Each UDT instance contains both a sender and a receiver

18. udt.sourceforge.net 18 :: 50 Protocol Architecture Sender Receiver  Sender Receiver  UDP Channel UDP Seq. NoTSPayload ACKSeq. NoNAKLoss List Sender AB

19. udt.sourceforge.net 19 :: 50 Software Architecture API UDP Channel Sender Receiver Sender's Buffer Receiver's Buffer CC Sender's Loss List Receiver's Loss List Listener

20. udt.sourceforge.net 20 :: 50 Efficiency Consideration  Less packets  Timer-based acknowledging  Less CPU time  Reduce per packet processing time  Reduce memory copy  Reduce loss list processing time  Light ACK vs. regular ACK  Parallel processing  Threading architecture  Less burst in processing  Evenly distribute the processing time

21. udt.sourceforge.net 21 :: 50 Application Programming Interface (API)  Socket API  New API  sendfile/recvfile: efficient file transfer  sendmsg/recvmsg: messaging with partial reliability  selectEx: a more efficient version of “select”  Rendezvous Connect  Firewall traversing

22. udt.sourceforge.net 22 :: 50 >> CONGESTION CONTROL INTRODUCTION PROTOCOL DESIGN & IMPLEMENTATION PERFORMANCE EVALUATION COMPOSABLE UDT CONCLUSIONS

23. udt.sourceforge.net 23 :: 50 Overview  Congestion control vs. flow control  Congestion control: effectively utilize the network bandwidth  Flow control: prevent the receiver from being overwhelmed by incoming packets  Window-based vs. rate-based  Window-based: tune the maximum number of on-flight packets (TCP)  Rate-based: tune the inter-packet sending time (UDT)  AIMD: additive increases multiplicative decreases  Feedback  Packet loss (Most TCP variants, UDT)  Delay (Vegas, FAST)

24. udt.sourceforge.net 24 :: 50 AIMD with Decreasing Increases  AIMD  x = x +  (x), for every constant interval (e.g., RTT)  x = (1 -  ) x, when there is a packet loss event where x is the packet sending rate.  TCP   (x)  1, and the increase interval is RTT.   = 0.5  AIMD with Decreasing Increase   (x) is non-increasing, and lim x->+   (x) = 0.

25. udt.sourceforge.net 25 :: 50 AIMD with Decreasing Increases (x)(x) x AIMD (TCP NewReno) UDT HighSpeed TCP Scalable TCP

26. udt.sourceforge.net 26 :: 50  Increase   (x) = f( B - x ) * c where B is the link capacity (Bandwidth), c is a constant parameter  Constant rate control interval (SYN), irrelevant to RTT  SYN = 0.01 seconds  Decrease  Randomized decrease factor   = 1 – (8/9) n UDT Control Algorithm (x)(x) x

27. udt.sourceforge.net 27 :: 50 The Increase Formula: an Example Bandwidth (B) = 10 Gbps, Packet size = 1500 bytes x (Mbps)B - x (Mbps)Increment (pkts/SYN) [0, 9000)(1000, 10000]10 [9000, 9900)(100, 1000]1 [9900, 9990)(10, 100]0.1 [9990, 9999)(1, 10]0.01 [9999, 9999.9)(0.1, 1]0.001 9999.9+<0.10.00067

28. udt.sourceforge.net 28 :: 50 Dealing with Packet Loss  Loss synchronization  Randomization method  Non-congestion loss  Do not decrease sending rate for the first packet loss M=5, N=2M=8, N=3  Packet reordering

29. udt.sourceforge.net 29 :: 50 Bandwidth Estimation  Packet Pair  Filters  Cross traffic  Interrupt Coalescence  Robust to estimation errors  Randomized interval to send packet pair P2P1 P2P1 P2P1 Packet Size / Space  Bottleneck Bandwidth

30. udt.sourceforge.net 30 :: 50 >> PERFORMANCE EVALUATION INTRODUCTION PROTOCOL DESIGN & IMPLEMENTATION CONGESTION CONTROL COMPOSABLE UDT CONCLUSIONS

31. udt.sourceforge.net 31 :: 50 Performance Characteristics  Efficiency  Higher bandwidth utilization, less CPU usage  Intra-protocol fairness  Max-min fairness  Jain's fairness index  TCP friendliness  Bulk TCP flow vs Bulk UDT flow  Short-lived TCP flow (slow start phase) vs Bulk UDT flow  Stability (oscillations)  Stability index (standard deviation)

32. udt.sourceforge.net 32 :: 50 Evaluation Strategies  Simulations vs. experiments  NS2 network simulator, NCDM teraflow testbed  Setup  Network topology, bandwidth, distance, queuing, Link error rate, etc.  Concurrency (number of parallel flows)  Comparison (against TCP)  Real world applications  SDSS data transfer, high performance mining of streaming data, etc.  Independent evaluation  SLAC, JGN2, UvA, Unipmn (Italy), etc.

33. udt.sourceforge.net 33 :: 50 Efficiency, Fairness, & Stability Flow 1 Flow 2 Flow 3 Flow 4 0100200300400500600700 Time (sec) 206.220.241.16 206.220.241.15 206.220.241.14 206.220.241.13 145.146.98.81 145.146.98.80 145.146.98.79 145.146.98.78 1Gb/s bandwidth, 106 ms RTT, StarLight, ChicagoSARA, Amsterdam

34. udt.sourceforge.net 34 :: 50 Efficiency, Fairness, & Stability Flow 1902466313215301452885 Flow 2446308216310452 Flow 3302202307 Flow 4197 Efficiency902912923830918904885 Fairness10.999 0.9980.99911 Stability0.11 0.080.160.040.020.04

35. udt.sourceforge.net 35 :: 50 TCP Friendliness 012345678910 20 30 40 50 60 70 80 Number of UDT flows TCP Throughput (Mb/s)  500 1MB TCP flows vs. 0 – 10 bulk UDT flows  1Gb/s between Chicago and Amsterdam

36. udt.sourceforge.net 36 :: 50 >> COMPOSABLE UDT INTRODUCTION PROTOCOL DESIGN & IMPLEMENTATION CONGESTION CONTROL PERFORMANCE EVALUATION CONCLUSIONS

37. udt.sourceforge.net 37 :: 50 Composable UDT - Objectives  Easy implementation and deployment of new control algorithms  Easy evaluation of new control algorithms  Application awareness support and dynamic configuration

38. udt.sourceforge.net 38 :: 50 Composable UDT - Methodologies  Packet sending control  Window-based, rate-based, and hybrid  Control event handling  onACK, onLoss, onTimeout, onPktSent, onPktRecved, etc.  Protocol parameters access  RTT, loss rate, RTO, etc.  Packet extension  User-defined control packets

39. udt.sourceforge.net 39 :: 50 Composable UDT - Evaluation  Simplicity  Can it be easily used?  Expressiveness  Can it be used to implement most control protocols?  Similarity  Can Composable UDT based implementations reproduce the performance of their native implementations?  Overhead  Will the overhead added by Composable UDT be too large?

40. udt.sourceforge.net 40 :: 50 Simplicity & Expressiveness  Eight event handlers, four protocol control functions, and one performance monitoring function.  Support a large variety of protocols  Reliable UDT blast  TCP and its variants (both loss and delay based)  Group transport protocols

41. udt.sourceforge.net 41 :: 50 Simplicity & Expressiveness CCC Base Congestion Control Class CTCP TCP NewReno CGTP Group Transport Protocol CUDPBlast Reliable UDP Blast CFAST FAST TCP CVegas TCP Vegas CScalable Scalable TCP CHS HighSpeed TCP CBiC BiC TCP CWestwood TCP Westwood 28 73 / +132-611 / +192-298 / +27-111 / +192-2927 / +145-2 37 / +351-2

42. udt.sourceforge.net 42 :: 50 Similarity and Overhead  Similarity  How Composable UDT based implementations can simulate their native implementations  CTCP vs. Linux TCP Flow # ThroughputFairnessStability TCPCTCPTCPCTCPTCPCTCP 1112122110.5170.415 21912080.9970.9990.4760.426 43223230.9490.9990.4840.492 83784220.9710.9990.6330.550 166726420.9580.9850.5020.482 328777990.9880.9970.4910.470 649217160.9940.9960.5690.529  CPU usage  Sender: CTCP uses about 100% more times of CPU as Linux TCP  Receiver: CTCP uses about 20% more CPU than Linux TCP

43. udt.sourceforge.net 43 :: 50 >> CONCLUSIONS INTRODUCTION PROTOCOL DESIGN & IMPLEMENTATION CONGESTION CONTROL PERFORMANCE EVALUATION COMPOSABLE UDT

44. udt.sourceforge.net 44 :: 50 Contributions  A high performance data transport protocol and associated implementation  The UDT protocol  Open source UDT library (udt.sourceforge.net)  User includes research institutes and industries  An efficient and fair congestion control algorithm  DAIMD & the UDT control algorithm  Packet loss handling techniques  Using bandwidth estimation technique in congestion control  A configurable transport protocol framework  Composable UDT

45. udt.sourceforge.net 45 :: 50 Selected Publications  Papers on the UDT Protocol  UDT: UDP-based Data Transfer for High-Speed Wide Area Networks, Yunhong Gu and Robert L. Grossman, Computer Networks (Elsevier). Volume 51, Issue 7. May 2007.  Supporting Configurable Congestion Control in Data Transport Services, Yunhong Gu and Robert L. Grossman, SC 2005, Nov 12 - 18, Seattle, WA.  Experiences in Design and Implementation of a High Performance Transport Protocol, Yunhong Gu, Xinwei Hong, and Robert L. Grossman, SC 2004, Nov 6 - 12, Pittsburgh, PA.  An Analysis of AIMD Algorithms with Decreasing Increases, Yunhong Gu, Xinwei Hong and Robert L. Grossman, First Workshop on Networks for Grid Applications (Gridnets 2004), Oct. 29, San Jose, CA.  Internet Draft  UDT: A Transport Protocol for Data Intensive Applications, Yunhong Gu and Robert L. Grossman, draft-gg-udt-02.txt.

46. udt.sourceforge.net 46 :: 50 Commercialization  Baidu Hi Messenger  Maidsafe  Movie2Me by broadcasting center europe  NiFTy TV  Sterling File Accelerator (SFA) by Sterling Commerce  Tideworks  PowerFolder  GridFTP  etc.

47. udt.sourceforge.net 47 :: 50 Support and Services  Online Forum  https://sourceforge.net/forum/?group_id=115059  Support both English and Chinese languages  Consulting service  Dedicated service to customers’ projects  Provided by UDT developers

48. udt.sourceforge.net 48 :: 50 Achievements  SC 2002 Bandwidth Challenge “Best Use of Emerging Network Infrastructure” Award  SC 2003 Bandwidth Challenge “Application Foundation” Award  SC 2004 Bandwidth Challenge “Best Replacement for FedEx / UDP Fairness” Award  SC 2006 Bandwidth Challenge Winner  SC 2008 Bandwidth Challenge Winner

49. udt.sourceforge.net 49 :: 50 Vision  Short-term  A practical solution to the distributed data intensive applications in high BDP environments  Long-term  Evolve with new technologies (open source & open standard)  More functionalities and support for more use scenarios  Network research platform (e.g., fast prototyping and evaluation of new control algorithms)

50. udt.sourceforge.net 50 :: 50 The End Thank You! Yunhong Gu, October 10, 2005 Updated on August 8, 2009