Low-Latency Software Rate Limiters for Cloud Networks Keqiang He, Weite Qin, Qiwei Zhang, Wenfei Wu, Junjie Yang, Tian Pan, Chengchen Hu, Jiao Zhang, Brent Stephens, Aditya Akella, Ying Zhang

Bandwidth Allocation in Clouds Provide performance isolation Allocate bandwidth for tenants Prioritize applications Implemented by Software Rate Limiters Configured on NICs E.g., Linux Hierarchical Token Buffer filter (HTB) 10G server VM1 VM2 VM3 RL1 RL2 RL3 4Gbps 2Gbps RL: Rate Limiter

Software Rate Limiters Maintain a queue to absorb packets Assigned tokens Dequeue packets according to tokens Refresh tokens periodically according to bandwidth allocation

However, queuing delay is introduced Motivating Experiment Rate limiting number of flows

Software Rate Limiters introduce queuing latency TCP RTT is increased by 10X or even more Goal: To build a low-latency software rate limiter

Contents Background & Problem Statement Design Evaluation Conclusion & Future Work

Strawman Solution: Extending ECN into Rate Limiters VM DCTCP VM DCTCP VM DCTCP Rate Limiter Queues Marking Threshold Extending ECN into Rate Limiters: set ECN bits in IP header based on queue length

ECN+DCTCP: Lead to Throughput Oscillation Network Latency is reduced, but throughput oscillates

Reasons of Oscillation Control loop latency One RTT Inflated by in-network latency Coarse-grained ECN marking Marking segments, not packets 64KB segment ≈ 43 packets (1500B) Aggressive CWND Computation

Direct ECE Marking (DEM) 1. Control loop latency is almost ZERO Outgoing data packets sender receiver 2. Fine-grained congestion feedback Incoming TCP ACK Rate limiter queue

Further Improvement: get rid of ECN? ECN may not be enabled Tenant VM, Internet end hosts Solution: SPRING Replay congestion control algorithm at rate limiter Enforce computed CWND to RWND field in ACK packets

Contents Background & Problem Statement Design Evaluation Conclusion & Future Work

Performance of DEM Both low latency and high throughput are achieved using DEM DEM works ECN-enabled flows only

Performance of SPRING Both low latency and high throughput are achieved using SPRING SPRING works for both ECN and non-ECN flows

Conclusions Software rate limiters introduce network latency DCTCP+ECN causes throughput oscillation One-RTT control loop latency Segment-level marking Direct ECE Marking high bandwidth saturation, low latency, low oscillation SPRING gets rid of ECN Future work Fairness, overhead, compatibility

End Thanks! Q&A wenfeiwu@tsinghua.edu.cn