1. ECE 544: Traffic engineering (supplement)
Abhigyan Sharma

2. Traffic engineering Problem: adapt network routing to traffic load
Goals:
Increase network capacity
Avoid congestion hotspots
Compute backup paths for failures

3. Traffic engineering problem
Link capacity
Traffic engineering algorithm
Network
routing 4 2 9 6 7 5 8 1 3
Traffic from node i to node j
The third decision affecting geo-distributed systems is the network routing, or as it is often called traffic engineering.
Traffic engineering is a problem is solved by ISPs today to configure routing in their networks.
There are two inputs to the traffic engineering problem.
First, is the link capacity of all network links.
Next, is a traffic matrix. The i,j-th entry in this matrix is the traffic from node I to node j in the network topology.
Based on these inputs, traffic engineering computes a network routing, that is, the physical paths between all pairs of nodes in the network.
[pause]
In computing the routing, traffic engineering seeks to minimize cost functions …
Traffic matrix

4. Network cost function
Convex functions of link utilization, e.g., maximum link utilization (MLU) A B
1 Mbps
3 Mbps
1.5 Mbps
Shortest path
MLU = 2/3 = 0.67
Flow split (Optimal)
2 Mbps
0.5 Mbps
MLU = 0.5/1 = 0.5
The cost function for traffic engineering are commonly defined as the convex functions of link utilization.
For example, a common cost function is the maximum link utilization, MLU.
Traffic engineering seeks to minimize these cost function to achieve a lower link utilization.
A low utilization of all links in the network means that the network is free from congestion hotspots.
Further, a low link utilization means that the network has more spare capacity to tolerate sudden or long-term increase in demand.
We take two examples of routing schemes to show a careful routing can reduce link utilization.
One the left, we use a shortest path routing and send traffic via the lower

5. Traffic engineering in wide-area ISP networks
Approach: configure routing protocols (e.g., OSPF, MPLS) based on previously estimated traffic matrices (e.g., previous day)
OSPF (Open Shortest Path First)
Configure link weights that result in optimal paths
Use local search algorithms to determine (integer) link weights
MPLS (Multi-Protocol Label Switching)
Compute tunnel paths between src/dst and fraction of traffic on each path
Use linear programming solvers to compute tunnel paths

6. Datacenter basics Common topology: 2-level leaf spine
Other topologies: hypercube, random graphs
Leaf-spine topology
Spine switches (S) A B
Total core BW = L*S*(cap of each link) D E F G
Leaf switches (L)
Total edge BW = H*L*(cap of each link)
Oversubscription ratio = (total edge BW)/(total core BW)
H hosts/switchs
Oversubscription > 1 is common since more traffic within racks

7. Equal cost multipath (ECMP) load balancing
Load balance traffic equally among all equal length paths
Use hash value based on packet header to select path
Single path for all packets in a flow prevents reordering of packets A B
50%
50% D E F G

8. ECMP Problems 1: Elephant flow collision
Elephant (large) flows collide reducing throughput A B D E F G

9. ECMP Problems 2: Path asymmetry
Asymmetry: One of the link BD is failed!
Same flow will have 2x throughput on DAE compared DBE A B D E F G

10. Datacenter traffic engineering
Centralized approaches
Have complete view of topology and traffic matrix
Can compute optimal solution
But, need to run frequently (100 ms) due to bursty datacenter traffic
Can work if traffic is predictable for a few seconds

11. Datacenter traffic engineering
Advanced distributed approaches
Packet-level ECMP: solves elephant flow problem but cause packet reordering
Multipath TCP: new form of TCP
adapts flow rate on each path based on congestion
requires host TCP modification and performs poorly in many-to-one traffic (called incast problem)
CONGA: distributed congestion control based on link-level feedback
Solves packet reordering problem by load balancing at flowlet-level (flowlet = burst of packets)

12. Assignment 2: traffic engineering algorithm
Implement and evaluate traffic engineering algorithm via simulation
Simulation parameters (symbols in red):
Leaf-spine topology
L leaf nodes, S spine (L >= S)
(L=4, S=4), (L=8, S=4), (L=8, S=8)
Uniform capacity link between each leaf and spine node
x fraction of links chosen randomly whose capacity is reduced by half
X = 0 (default), X = 0.25
Traffic load
Number of elephant flows = f * (L*S)
f = {1/8, ¼, ½, 1, 2, 4, 8}
Each flow across leaf nodes l1 and l2 chosen randomly (l1 != l2)
Each flow is of infinite length

13. Assignment 2: traffic engineering algorithm
Objective: Maximize sum of throughput of all flows
(FIG A) If a flow does not share any link with any other flow, its throughput = 1
(FIG B) If a flow shares a link with n other flows, all flows receive equal share of link
(Not shown) Flow throughput is determined by the most bottlenecked link A B A B
Tput = 1
Tput = (1/3, 1/3,1/3) D E F G D E F G
FIG A
FIG B

14. Assignment 2: traffic engineering algorithm
Evaluation criteria:
Compare you scheme/s against these baselines
Random: select one of the path randomly
Round-robin: if n outgoing flows and k paths, i-th flow takes (i%k)-th path
Vary simulation parameters and evaluate
Topology: (L, S) = (4, 4), (8, 4), (8, 8)
Reduced capacity link fraction X = {0, 0.25}
Number of flows: f = f = {1/8, ¼, ½, 1, 2, 4, 8}
For each scenario, repeat 10 times and show median throughput

15. Assignment 2: traffic engineering algorithm
Submit the following:
Description possibly including pseudo-code and diagrams of your algorithm
Graphs: (one for each combination of (L,S) and X) (3 * 2 graphs)
X-axis: increasing value of f (1/8 to 8)
Y-axis: total throughput
Keys: different schemes
Interpret your findings and provide explanation for the trends you observe
Your simulation code as a zip file with a brief README