Egress traffic shaping on Linux using Hierarchical Token Bucket (HTB) Brad Baker CS

Traffic Shaping Overview Manages limited network resources Manages limited network resources Has many applications where desired network use exceeds available resources Has many applications where desired network use exceeds available resources Can provide control, efficiency, and guaranteed quality of service, QoS Can provide control, efficiency, and guaranteed quality of service, QoS Limits contention for resources Limits contention for resources Most useful on outbound (Egress) streams, some options for Ingress. Most useful on outbound (Egress) streams, some options for Ingress.

Traffic Control Feature included in current Linux distributions Feature included in current Linux distributions Includes classless and classful queuing disciplines Includes classless and classful queuing disciplines Need TC function to be bottleneck, not external router or other device Need TC function to be bottleneck, not external router or other device Several classless techniques available Several classless techniques available FIFO, SFQ, TBF FIFO, SFQ, TBF Several classful techniques available Several classful techniques available HTB, CBQ, PRIO HTB, CBQ, PRIO

HTB – Hierarchical Token Bucket Classful queuing discipline Classful queuing discipline Similar to Class Based Queuing (CBQ) Similar to Class Based Queuing (CBQ) Hierarchy of classes combined with filters are used to control traffic Hierarchy of classes combined with filters are used to control traffic System periodically produces tokens, needed to send frames over the interface System periodically produces tokens, needed to send frames over the interface Classes send data if tokens are available Classes send data if tokens are available If no tokens, can borrow from parent If no tokens, can borrow from parent

HTB classes Root classes cannot borrow from one another Root classes cannot borrow from one another Traffic can only be classified to leaf classes Traffic can only be classified to leaf classes Inner classes allow borrowing to children Inner classes allow borrowing to children Egress packets are classified and placed in correct subclass Egress packets are classified and placed in correct subclass Dequeue request given to root, proceeds down tree until it can be fulfilled Dequeue request given to root, proceeds down tree until it can be fulfilled

HTB specifics Bucket based storage of tokens accommodates burst traffic, such as web Bucket based storage of tokens accommodates burst traffic, such as web Same concept in classless discipline TBF Same concept in classless discipline TBF Can use same feature to smooth bursts Can use same feature to smooth bursts Good for situations with fixed bandwidth to distribute among competitors Good for situations with fixed bandwidth to distribute among competitors Can provide guaranteed min service rate, and constrain to max bandwidth use to ceiling rate Can provide guaranteed min service rate, and constrain to max bandwidth use to ceiling rate Supports priority values associated with classes Supports priority values associated with classes More efficient and fewer parameters than CBQ More efficient and fewer parameters than CBQ

Some HTB results

HTB syntax tc qdisc del dev eth0 root tc qdisc add dev eth0 root handle 2: htb default 1 tc class add dev eth0 parent 2: classid 2:1 htb rate 100kbps ceil 100kbps prio 0 tc qdisc add dev eth0 parent 2:1 handle 1: htb default 1 tc class add dev eth0 parent 1: classid 1:1 htb rate 70kbps ceil 70kbps prio 0 tc class add dev eth0 parent 1:1 classid 1:10 htb rate 35kbps ceil 70kbps prio 1 burst 10k tc class add dev eth0 parent 1:1 classid 1:11 htb rate 35kbps ceil 70kbps prio 1 burst 10k tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10 tc qdisc add dev eth0 parent 1:11 handle 11: sfq perturb 10 tc class add dev eth0 parent 1: classid 1:2 htb rate 30kbps ceil 30kbps prio 0 tc class add dev eth0 parent 1:2 classid 1:20 htb rate 15kbps ceil 30kbps prio 1 burst 10k tc class add dev eth0 parent 1:2 classid 1:21 htb rate 15kbps ceil 30kbps prio 1 burst 10k tc class add dev eth0 parent 1: classid 1:3 htb rate 100kbps ceil 100kbps prio 1 burst 10k Creates four qdiscs and eight classes One non-root qdisc used to enforce 100K Two non-root qdiscs used for SFQ

HTB improvements Problem: HTB divides predetermined bandwidth values between classes Problem: HTB divides predetermined bandwidth values between classes Works well in situations with fixed bandwidth Works well in situations with fixed bandwidth Improvement: Process could be enhanced by giving classes relative bandwidth values Improvement: Process could be enhanced by giving classes relative bandwidth values Find potential channel capacity and use as root bandwidth Find potential channel capacity and use as root bandwidth

HTB improvements Problem: Discrepancy between ‘quantum’ value and actual packet size Problem: Discrepancy between ‘quantum’ value and actual packet size Can cause errors in cases of high bandwidth contention Can cause errors in cases of high bandwidth contention Improvement: Calculate bandwidth based on actual dequequed bytes Improvement: Calculate bandwidth based on actual dequequed bytes Instead of bytes specified by the model Instead of bytes specified by the model

Conclusion Traffic shaping is useful and important Traffic shaping is useful and important When resources exceed demand When resources exceed demand When an interface changes from fast connection to slow connection When an interface changes from fast connection to slow connection HTB provides a well supported TC solution HTB provides a well supported TC solution Efficient and accurate shaping traffic among predetermined bandwidth values Efficient and accurate shaping traffic among predetermined bandwidth values Allows resource control comparable to high-end dedicated bandwidth management systems. Allows resource control comparable to high-end dedicated bandwidth management systems. Included in recent ( or later) Linux kernels Included in recent ( or later) Linux kernels

References Linux Advanced Routing & Traffic Control HOWTO (LARTC) Linux Advanced Routing & Traffic Control HOWTO (LARTC) Author: Bert Hubert and others Author: Bert Hubert and others Traffic Shaping with Linux Traffic Shaping with Linux Author: Shane Chen Author: Shane Chen HTB Linux queuing discipline manual - user guide HTB Linux queuing discipline manual - user guide Author: Martin Devera Author: Martin Devera Traffic Control HOWTO Traffic Control HOWTO Author: Martin A. Brown Author: Martin A. Brown