Presentation is loading. Please wait.

Presentation is loading. Please wait.

Low-Latency Software Rate Limiters for Cloud Networks

Similar presentations


Presentation on theme: "Low-Latency Software Rate Limiters for Cloud Networks"— Presentation transcript:

1 Low-Latency Software Rate Limiters for Cloud Networks
Keqiang He, Weite Qin, Qiwei Zhang, Wenfei Wu, Junjie Yang, Tian Pan, Chengchen Hu, Jiao Zhang, Brent Stephens, Aditya Akella, Ying Zhang

2 Bandwidth Allocation in Clouds
Provide performance isolation Allocate bandwidth for tenants Prioritize applications Implemented by Software Rate Limiters Configured on NICs E.g., Linux Hierarchical Token Buffer filter (HTB) 10G server VM1 VM2 VM3 RL1 RL2 RL3 4Gbps 2Gbps RL: Rate Limiter

3 Software Rate Limiters
Maintain a queue to absorb packets Assigned tokens Dequeue packets according to tokens Refresh tokens periodically according to bandwidth allocation

4 However, queuing delay is introduced
Motivating Experiment Rate limiting number of flows

5 Software Rate Limiters introduce queuing latency
TCP RTT is increased by 10X or even more Goal: To build a low-latency software rate limiter

6 Contents Background & Problem Statement Design Evaluation
Conclusion & Future Work

7 Strawman Solution: Extending ECN into Rate Limiters
VM DCTCP VM DCTCP VM DCTCP Rate Limiter Queues Marking Threshold Extending ECN into Rate Limiters: set ECN bits in IP header based on queue length

8 ECN+DCTCP: Lead to Throughput Oscillation
Network Latency is reduced, but throughput oscillates

9 Reasons of Oscillation
Control loop latency One RTT Inflated by in-network latency Coarse-grained ECN marking Marking segments, not packets 64KB segment ≈ 43 packets (1500B) Aggressive CWND Computation

10 Direct ECE Marking (DEM)
1. Control loop latency is almost ZERO Outgoing data packets sender receiver 2. Fine-grained congestion feedback Incoming TCP ACK Rate limiter queue

11 Further Improvement: get rid of ECN?
ECN may not be enabled Tenant VM, Internet end hosts Solution: SPRING Replay congestion control algorithm at rate limiter Enforce computed CWND to RWND field in ACK packets

12 Contents Background & Problem Statement Design Evaluation
Conclusion & Future Work

13 Performance of DEM Both low latency and high throughput are achieved using DEM DEM works ECN-enabled flows only

14 Performance of SPRING Both low latency and high throughput are achieved using SPRING SPRING works for both ECN and non-ECN flows

15 Conclusions Software rate limiters introduce network latency
DCTCP+ECN causes throughput oscillation One-RTT control loop latency Segment-level marking Direct ECE Marking high bandwidth saturation, low latency, low oscillation SPRING gets rid of ECN Future work Fairness, overhead, compatibility

16 End Thanks! Q&A


Download ppt "Low-Latency Software Rate Limiters for Cloud Networks"

Similar presentations


Ads by Google