ECE 544: Traffic engineering (supplement) Abhigyan Sharma

Traffic engineering Problem: adapt network routing to traffic load Goals: Increase network capacity Avoid congestion hotspots Compute backup paths for failures

Traffic engineering problem Link capacity Traffic engineering algorithm Network routing 4 2 9 6 7 5 8 1 3 Traffic from node i to node j The third decision affecting geo-distributed systems is the network routing, or as it is often called traffic engineering. Traffic engineering is a problem is solved by ISPs today to configure routing in their networks. There are two inputs to the traffic engineering problem. First, is the link capacity of all network links. Next, is a traffic matrix. The i,j-th entry in this matrix is the traffic from node I to node j in the network topology. Based on these inputs, traffic engineering computes a network routing, that is, the physical paths between all pairs of nodes in the network. [pause] In computing the routing, traffic engineering seeks to minimize cost functions … Traffic matrix

Network cost function Convex functions of link utilization, e.g., maximum link utilization (MLU) A B 1 Mbps 3 Mbps 1.5 Mbps Shortest path MLU = 2/3 = 0.67 Flow split (Optimal) 2 Mbps 0.5 Mbps MLU = 0.5/1 = 0.5 The cost function for traffic engineering are commonly defined as the convex functions of link utilization. For example, a common cost function is the maximum link utilization, MLU. Traffic engineering seeks to minimize these cost function to achieve a lower link utilization. A low utilization of all links in the network means that the network is free from congestion hotspots. Further, a low link utilization means that the network has more spare capacity to tolerate sudden or long-term increase in demand. We take two examples of routing schemes to show a careful routing can reduce link utilization. One the left, we use a shortest path routing and send traffic via the lower

Traffic engineering in wide-area ISP networks Approach: configure routing protocols (e.g., OSPF, MPLS) based on previously estimated traffic matrices (e.g., previous day) OSPF (Open Shortest Path First) Configure link weights that result in optimal paths Use local search algorithms to determine (integer) link weights MPLS (Multi-Protocol Label Switching) Compute tunnel paths between src/dst and fraction of traffic on each path Use linear programming solvers to compute tunnel paths

Datacenter basics Common topology: 2-level leaf spine Other topologies: hypercube, random graphs Leaf-spine topology Spine switches (S) A B Total core BW = L*S*(cap of each link) D E F G Leaf switches (L) Total edge BW = H*L*(cap of each link) Oversubscription ratio = (total edge BW)/(total core BW) H hosts/switchs Oversubscription > 1 is common since more traffic within racks

Equal cost multipath (ECMP) load balancing Load balance traffic equally among all equal length paths Use hash value based on packet header to select path Single path for all packets in a flow prevents reordering of packets A B 50% 50% D E F G

ECMP Problems 1: Elephant flow collision Elephant (large) flows collide reducing throughput A B D E F G

ECMP Problems 2: Path asymmetry Asymmetry: One of the link BD is failed! Same flow will have 2x throughput on DAE compared DBE A B D E F G

Datacenter traffic engineering Centralized approaches Have complete view of topology and traffic matrix Can compute optimal solution But, need to run frequently (100 ms) due to bursty datacenter traffic Can work if traffic is predictable for a few seconds

Datacenter traffic engineering Advanced distributed approaches Packet-level ECMP: solves elephant flow problem but cause packet reordering Multipath TCP: new form of TCP adapts flow rate on each path based on congestion requires host TCP modification and performs poorly in many-to-one traffic (called incast problem) CONGA: distributed congestion control based on link-level feedback Solves packet reordering problem by load balancing at flowlet-level (flowlet = burst of packets)

Assignment 2: traffic engineering algorithm Implement and evaluate traffic engineering algorithm via simulation Simulation parameters (symbols in red): Leaf-spine topology L leaf nodes, S spine (L >= S) (L=4, S=4), (L=8, S=4), (L=8, S=8) Uniform capacity link between each leaf and spine node x fraction of links chosen randomly whose capacity is reduced by half X = 0 (default), X = 0.25 Traffic load Number of elephant flows = f * (L*S) f = {1/8, ¼, ½, 1, 2, 4, 8} Each flow across leaf nodes l1 and l2 chosen randomly (l1 != l2) Each flow is of infinite length

Assignment 2: traffic engineering algorithm Objective: Maximize sum of throughput of all flows (FIG A) If a flow does not share any link with any other flow, its throughput = 1 (FIG B) If a flow shares a link with n other flows, all flows receive equal share of link (Not shown) Flow throughput is determined by the most bottlenecked link A B A B Tput = 1 Tput = (1/3, 1/3,1/3) D E F G D E F G FIG A FIG B

Assignment 2: traffic engineering algorithm Evaluation criteria: Compare you scheme/s against these baselines Random: select one of the path randomly Round-robin: if n outgoing flows and k paths, i-th flow takes (i%k)-th path Vary simulation parameters and evaluate Topology: (L, S) = (4, 4), (8, 4), (8, 8) Reduced capacity link fraction X = {0, 0.25} Number of flows: f = f = {1/8, ¼, ½, 1, 2, 4, 8} For each scenario, repeat 10 times and show median throughput

Assignment 2: traffic engineering algorithm Submit the following: Description possibly including pseudo-code and diagrams of your algorithm Graphs: (one for each combination of (L,S) and X) (3 * 2 graphs) X-axis: increasing value of f (1/8 to 8) Y-axis: total throughput Keys: different schemes Interpret your findings and provide explanation for the trends you observe Your simulation code as a zip file with a brief README