1. Low-Latency Software Rate Limiters for Cloud Networks
Keqiang He, Weite Qin, Qiwei Zhang, Wenfei Wu, Junjie Yang, Tian Pan, Chengchen Hu, Jiao Zhang, Brent Stephens, Aditya Akella, Ying Zhang

2. Bandwidth Allocation in Clouds
Provide performance isolation
Allocate bandwidth for tenants
Prioritize applications
Implemented by Software Rate Limiters
Configured on NICs
E.g., Linux Hierarchical Token Buffer filter (HTB)
10G server
VM1
VM2
VM3
RL1
RL2
RL3
4Gbps
2Gbps
RL: Rate Limiter

3. Software Rate Limiters
Maintain a queue to absorb packets
Assigned tokens
Dequeue packets
according to tokens
Refresh tokens periodically
according to bandwidth allocation

4. However, queuing delay is introduced
Motivating Experiment
Rate limiting
number of flows

5. Software Rate Limiters introduce queuing latency
TCP RTT is increased by 10X or even more
Goal: To build a low-latency software rate limiter

6. Contents Background & Problem Statement Design Evaluation
Conclusion & Future Work

7. Strawman Solution: Extending ECN into Rate LimitersVM
DCTCP VM
DCTCP VM
DCTCP
Rate Limiter
Queues
Marking
Threshold
Extending ECN into Rate Limiters: set ECN bits in IP header based on queue length

8. ECN+DCTCP: Lead to Throughput Oscillation
Network Latency is reduced, but throughput oscillates

9. Reasons of Oscillation
Control loop latency
One RTT
Inflated by in-network latency
Coarse-grained ECN marking
Marking segments, not packets
64KB segment ≈ 43 packets (1500B)
Aggressive CWND Computation

10. Direct ECE Marking (DEM)
1. Control loop latency is almost ZERO
Outgoing data packets
sender
receiver
2. Fine-grained congestion feedback
Incoming TCP ACK
Rate limiter queue

11. Further Improvement: get rid of ECN?
ECN may not be enabled
Tenant VM, Internet end hosts
Solution: SPRING
Replay congestion control algorithm at rate limiter
Enforce computed CWND to RWND field in ACK packets

12. Contents Background & Problem Statement Design Evaluation
Conclusion & Future Work

13. Performance of DEM
Both low latency and high throughput are achieved using DEM
DEM works ECN-enabled flows only

14. Performance of SPRING
Both low latency and high throughput are achieved using SPRING
SPRING works for both ECN and non-ECN flows

15. Conclusions Software rate limiters introduce network latency
DCTCP+ECN causes throughput oscillation
One-RTT control loop latency
Segment-level marking
Direct ECE Marking
high bandwidth saturation, low latency, low oscillation
SPRING gets rid of ECN
Future work
Fairness, overhead, compatibility

16. End
Thanks!
Q&A