1. TCP Incast in Data Center Networks
A study of the problem and proposed solutions

2. Outline TCP Incast - Problem Description Motivation and challenges
Proposed Solutions
Evaluation of proposed solutions
Conclusion
References

3. Outline TCP Incast - Problem Description Motivation and challenges
Proposed Solutions
Evaluation of proposed solutions
Conclusion
References

4. TCP Incast – Problem Description
Incast jargons:
Barrier Synchronized Workload
SRU (Server Request Unit)
Goodput, Throughput
MTU
BDP
and TCP acronyms like RTT, RTO, CA, AIMD, etc.

5. TCP Incast – Problem
A typical implementation scenario in the Data Centers

6. TCP Incast - Problem Many-to-one barrier synchronized workload:
Receiver requests k blocks of data from S storage servers.
Each block of data striped across S storage servers
Each server responses with a “fixed” amount of data. (fixed-fragment workload)
Client won’t request block k+1 until all the fragments of block k have been received.
Datacenter scenario:
k=100
S = 1-48
fragment size : 256KB

7. TCP Incast - Problem
Goodput Collapse

8. TCP Incast - Problem
Switch buffers are inherently small in magnitude i.e. 32KB-128KB per port
Bottleneck switch buffer gets overwhelmed by synchronous sending of data by servers and consequently switch drops the packets
RTT is typically 1-2ms in datacenters and RTOmin is 200ms. This gap results in packets not getting retransmitted soon
All the other senders who have already sent the data have to wait until the dropped packet gets retransmitted.
But large RTO implies that retransmission will be delayed resulting in decrease in goodput

9. Outline TCP Incast - Problem Description Motivation and challenges
Proposed Solutions
Evaluation of proposed solutions
Conclusion
References

10. Motivation
Internet datacenters support a myriad of service and applications.
Google, Microsoft, Yahoo, Amazon
Vast majority of datacenter use TCP for communication between nodes.
Companies like Facebook have adopted UDP as their transport layer protocol to avoid TCP incast and endowed the responsibility of flow control to the application layer protocols
The unique workload such as Mapreduce , Hadoop, scale and environment of internet datacenter violate the WAN assumption on which TCP was originally designed.
Ex: In a Web search application, many workers respond near simultaneously to search queries in which key-value pairs from many Mappers are transferred to appropriate Reducers during the shuffle stage

11. Incast in Bing (Microsoft)
Ref : Slide from Albert Greenberg(Microsoft) presentation at SIGCOMM’10

12. Challenges Minimum changes to TCP implementation needed
Cannot decrease the RTO min to less than 1ms as operating systems fail to work with high resolution timers for RTO
Have to address internal and external flows
Cannot afford large buffer at the switch because it is costly
Solution needs to be easily deployed and should be cost effective

13. Outline TCP Incast - Problem Description
Characteristics of the problem and challenges
Proposed Solutions
Evaluation of proposed solutions
Conclusion
References

14. Proposed Solutions Solutions can be divided into
Application level solutions
Transport layer solutions
Transport layer solutions aided by switch’s ECN and QCN capabilities.
Alternative way to categorize the solutions
Avoiding timeouts in TCP
Reducing RTOmin
Replace TCP
Call lower layer functionalities like Ethernet Flow control for help

15. Understanding the problem…
Collaborated study by EECS Berkeley and Intel labs[1]
Their study focused on
proving this problem is general,
deriving an analytical model
Studying the impact of various modifications to TCP on incast behavior.

16. Different RTO Timers Observations:
Initial goodput min occurs at the same number of servers.
Smaller RTO timer value has faster goodput “recovery” rate
The decrease rate after local max is the same between different min RTO settings.

17. Decreasing the RTO – proportional increase in the goodput
Surprisingly, 1ms RTO with delayed ack enabled was a better performer
Delayed ack disabled in 1ms forces overriding of TCP congestion window on the sender side due to high transmission of acks resulting in fluctuations in smoothed RTT

18. QUANTITATIVE MODEL:
D: total amount of data to be sent, 100 blocks of 256KB
L: total transfer time of the workload without and RTO events.
R: the number of RTO events during the transfer
S: number of server:
r: the value of the minimum RTO timer value
I : Interpacket wait time
Modeling of R and I was done based on empirically observed behavior
Net goodput:

19. Key Observations
A smaller minimum RTO timer value means larger goodput values for the initial minimum.
The initial goodput minimum occurs at the same number of senders, regardless the value of the minimum RTO times.
The second order goodput peak occurs at a higher number of senders for a larger RTO timer value
The smaller the RTO timer values, the faster the rate of recovery between the goodput minimum and the second order goodput maximum.
After the second order goodput maximum, the slope of goodput decrease is the same for different RTO timer values.

20. Application level solution[5]
No changes required to the TCP stack or network switches
Based on scheduling server responses to the same data block so that no data loss occurs
Caveats:
Retransmissions can be interesting
Scheduling at the application level cannot be easily synchronized
Limited control over transport layer

21. Application level solution

22. Application level solution

23. ICTCP-Incast Congestion Control for TCP in Data Center Networks[8]
Features
Solution based on modifying Congestion window dynamically
Can choose implementation on the receiver side only
focuses on avoiding packet losses before the incast congestion occurs
Test implementation on Windows NDIS
Novelties in the solution:
Using Available bandwidth to coordinate the receive window increase in all incoming connections
Per flow congestion control is performed independently in slotted time of RTT
Receive window adjustment is based on the ratio of difference of measured and expected throughput over expected one.

24. Design considerations
Receiver knows how much throughput is achieved and what is the available bandwidth
While overly controlled window mechanism may constrain TCP performance, less controlled does not prevent incast congestion
Only low latency flows less than 2ms are considered
Receive window increase is determined by the available bandwidth
Frequency of receive window based congestion control should be per- flow
Receive window based scheme should adjust the window according to link congestion and application requirement

25. ICTCP Algorithm Control trigger: Available bandwidth
Calculate available bandwidth
Estimate the potential throughput/flow increase before increasing receive window
Time divided into two slots
For each network interface, measure available bandwidth in first sub-slot and calculate quota for window increase in second sub-slot
Ensure the total increase in the receive window is less than the total available bandwidth calculated in the first sub-slot

26. ICTCP Algorithm per connection control interval: 2*RTT
to estimate the throughput of a TCP connection for receive window adjustment, the shortest time scale is an RTT for that connection
control interval for a TCP connection is 2*RTT in ICTCP
One RTT latency for adjusted window to take effect
One additional RTT for measuring throughput with the newly adjusted window
For any TCP connection, if now time is in the second global sub-slot and it observes that the past time is larger than 2*RTT since its last receive window adjustment, it may increase its window based on newly observed TCP throughput and current available bandwidth.

27. ICTCP Algorithm Window adjustment on single connection
Receive window is adjusted based on its incoming measured throughput
Measured throughput is current requirement of the application over that TCP connection
Expected throughput is the expectation of throughput on that TCP connection if throughput is constrained by receive window
Define ratio of throughput difference
Make receive adjustment based on the following conditions
MSS and i increase receive window if it’s now in global second sub-slot and there is enough quota of available bandwidth on the network interface. Decrease the quota correspondingly if the receive window is increased.
decrease receive window by one MSS if this condition holds for three continuous RTT. The minimal receive window is 2*MSS.
Otherwise, keep current receive window.

28. ICTCP Algorithm Fairness Controller for multiple connections
Fairness considered only for low latency flows
Decrease window for fairness only when BWA < 0.2C
For window decrease, we cut the receive window by one MSS3, for some selected TCP connections.
Select those connections that have receive window larger than the average window value of all connections.
For window increase, this is automatically achieved by our window adjustment

29. ICTCP Experimental Results
Testbed
47 servers
1 LB4G 48-port Gigabit Ethernet switch
Gigabit Ethernet Broadcom NIC at the hosts
Windows Server 2008 R2 Enterprise 64-bit version

31. Issues with ICTCP
ICTCP scalability to a large number of TCP connections is an issue because receive window can decrease below 1 MSS resulting in degraded TCP performance
Extending ICTCP to handle congestion in general cases where sender and receiver are not under the same switch and bottleneck link is not the last link to the receiver
ICTCP for future high bandwidth low latency networks

32. DCTCP Features TCP like protocol for data centers
It uses ECN (Explicit Congestion Notification) to provide multi-bit feedback to the end-hosts
Claim is that DCTCP provides better throughput than TCP using 90% less buffer space
Provides high burst tolerance and low latency for short flows
Also can handle 10X increase in foreground and background traffic without significant hit on the performance front.

33. DCTCP Overview Applications in data centers largely require
Low latency for short flows
High burst tolerance
High utilization for long flows
Low latency for short flows have real time deadlines of about approximately ms
To avoid continuously modifying internal data structures, high utilization for long flows is essentia
Study analyzed production traffic from app servers with app. 150 TB of traffic for a period of 1 month
Query traffic (of 2KB to 20KB) experience incast impairment

34. DCTCP Overview (Contd)
Proposed DCTCP uses ECN capability available in most modern switches
Uses multi-bit feedback on congestion from single bit stream of ECN marks
Essence of the proposal is to keep switch buffer occupancies persistently low, while maintaining high throughput
to control queue length at switches, use Active Queue management(AQM) approach that uses explicit feedback from congested switches
Claim is also that only as much as 30 LoC to TCP and setting of a single parameter on switches is needed
DCTCP focuses on 3 problems
Incast
Queue Buildup
Buffer pressure
Our area

35. DCTCP
Algorithm
Mainly concentrates on extent of congestion rather than just the presence of it.
Act of deriving multi-bit feedback from single bit sequence of marks
Three components of the algorithm
Simple marking at the switch
ECN-echo at the receiver
Controller at the sender

36. DCTCP Simple marking at the switch ECN-ECHO at the receiver:
An arriving packet is marked with CE (Congestion Experienced) codepoint if the queue occupancy is greater than K (marking threshold)
Marking is not based on average queue length , but instantaneous
ECN-ECHO at the receiver:
Normally, in TCP, an ECN-ECHO is set to all packets until receiver gets CWR from the sender
DCTCP receiver sends a ECN-ECHO only if a CE codepoint is seen on the packet

37. DCTCP Controller at the sender:
Sender maintains an estimate of the fraction of packets that are marked and updated every window
When 𝛼 is close to 0 , low congestion and 𝛼 close to 1 indicate high congestion
While TCP cuts its window by half, DCTCP uses 𝛼 to determine the sender’s window size
cwnd  cwnd x (1 - 𝛼/2)

38. DCTCP
Modeling of when window reaches W*(when K is at the critcal point)
Maximum size Q max of the queue depends on the number of synchronously sending servers N
Lower bound for K can be derived by

39. DCTCP How DCTCP solves Incast? TCP suffers from timeouts when N>10
DCTCP senders receive ECN marks, slow their rate
Suffers timeouts when N large enough to overwhelm static buffer size
Solution is Dynamic Buffering

40. Outline TCP Incast - Problem Description Motivation and challenges
Proposed Solutions
Evaluation of proposed solutions
Conclusion
References

41. Evaluation of proposed solutions
Application level solution
Genuine retransmissions  cascading timeouts  congestion
Scheduling at the application level cannot be easily synchronized
Limited control over transport layer
ICTCP- Solution that needs minimal change and is cost effective
ICTCP scalability to a large number of TCP connections is an issue
Extending ICTCP to handle congestion in general cases has a limited solution
ICTCP for future high bandwidth low latency networks will need extra support from Link layer technologies
DCTCP- Solution that needs minimal change but requires switch support
DCTCP requires dynamic buffering for larger number of senders

42. Conclusion
No solution completely solves the problem other than configuring less RTO
Solutions have less focus on foreground an background traffic together
Need solutions which are cost effective+ requiring minimal change to environment+ and of course solving incast!!!

43. References
Y.Chen, R.Griffith, J.Liu, R.H.Katz, and A.D.Joseph, “Understanding TCP Incast Throughput Collapse in Datacenter Networks” in Proc. of ACM WREN, 2009.
 Kulkarni.S., Agrawal. P, “A Probabilistic Approach to Address TCP Incast in Data Center Networks”,Distributed Computing Systems Workshops (ICDCSW), 2011
 Peng Zhang, Hongbo Wang, Shiduan Cheng “Shrinking MTU to Mitigate TCP Incast Throughput Collapse in Data Center Networks”, Communications and Mobile Computing (CMC), 2011
 Yan Zhang, Ansari, N.,”On mitigating TCP Incast in Data Center Networks”, INFOCOM Proceedings-IEEE,2011
Maxim Podlesny, Carey Williamson, “An application-level solution for the TCP-incast problem in data center networks”,IWQoS ’11: Proceedings of the 19th International Workshop on Quality of Service,IEEE, June,2011
Mohammad Alizadeh, Albert Greenberg, David A. Maltz, Jitendra Padhye, Parveen Patel, Balaji Prabhakar, Sudipta Sengupta, Murari Sridharan, “Data center TCP (DCTCP)”, SIGCOMM ’10: Proceedings of the ACM\SIGCOMM, August 2010
Hongyun Zheng, Changjia Chen, Chunming Qiao, “Understanding the Impact of Removing TCP Binary Exponential Backoff in Data Centers”, Communications and Mobile Computing (CMC), 2011
 Haitao Wu, Zhenqian Feng, Chuanxiong Guo, Yongguang Zhang, “ICTCP: Incast Congestion Control for TCP in data center networks”,Co-NEXT ’10: Proceedings of the 6th International COnference, ACM,November 2010