Infocom 2003 An Approach to Alleviate Link Overload as Observed on an IP Backbone Tuesday, April 1 st Infocom 2003 Sundar Iyer 1,2, Supratik Bhattacharrya 2, Nina Taft 2, Christophe Diot 2 1 Stanford University, 2 ATL SprintLabs

Infocom Contents 1. Introduction 2. Pathology of link overload 3. Alleviate overload - deflection routing 4. Performance analysis

Infocom There should be no link overload  IP backbones are  Overprovisioned  low average utilization  Have multiple paths  Routing algorithms  balance load across multiple shortest paths  should reduce the likelihood of overload  Overload: More than 50% utilization

Infocom But there is link overload  Shortest path routing  puts load on a small set of equal cost shortest paths  causes unequal use of link capacity  Unpredictable traffic  Short term load fluctuations e.g. hotspots  Failure  Link failures, fiber cuts, network maintenance  Hard to predict all factors apriori

Infocom Why bother about link overload?  Operators upgrade persistently overloaded links  Peaks in link utilization  cannot increase average utilization  Severe link overload causes packet drops  Interactive, real-time applications make it mandatory to overcome overload

Infocom Contents 1. Introduction 2. Pathology of link overload 3. Alleviate overload - deflection routing 4. Performance analysis

Infocom Methodology  Measurement of data from the Sprint backbone  Analyzed 138 backbone links for 9 months  SNMP link utilization data polled every 5 minutes  The link utilization is an exponentially weighted moving average (EWMA)  Measurements under-estimate overload  Short term fluctuations are missed

Infocom Maximum load  Observation 1: There is always some overloaded link Maximum Load

Infocom Contribution of links to overload  Observation 2: Most of the links are not overloaded Non-Overloaded links Overloaded links

Infocom Types of link overload  Observation 3: Two types — Persistent Periods of link overload and temporary overload  Observation 4: Often just 1-2 links are simultaneously overloaded

Infocom Causes of temporary link overload  Observation 5: Link failures cause temporary overload Link Utilizations  Observation 6: Fiber cuts cause severe overload

Infocom Contents 1. Introduction 2. Pathology of link overload 3. Alleviate overload - deflection routing 4. Performance analysis

Infocom The case for deflection routing  Previous techniques  useful for long term overload  change normal functioning of the network  useful when overload is common  We observe that link overload  is relatively rare (  0.1% of the time on any link)  are typically caused due to link failures/maintenance  lasts for minutes-hours on average  occurs on maximum of 1-2 links simultaneously  can be easily overcome by deflecting packets Allow normal network operation most of the time

Infocom Problem  Problem:  How can we design a simple, stateless, loop-free deflection algorithm to overcome link overload?  Theorem 1: (sufficiency)  Any deflection algorithm which deflects packets with “strictly decreasing cost” is loop-free

Infocom Explanation of Theorem 1  A packet is forwarded from node s to d according to the strictly decreasing cost criteria as follows 1. If shortest path not overloaded Forward the packet on the shortest path with cost C 2. If link to neighboring node n is not overloaded Forward the packet to n if n’s cost to d is  C 3. Else Forward the packet on the shortest path

Infocom Intuition for Theorem 1  Shortest path routing:  forward packet on the shortest path  the sequence of costs to a destination is strictly decreasing 30 Router: s Router: n3 Router: n2 Router: n1 Router: d 15  Loop-free deflection routing: Yes No  we do not consider the cost of reaching the deflection node

Infocom Problem  Problem:  Can we always find loop-free deflection paths according to the strictly decreasing cost criteria?  Theorem 2: (sufficiency)  A network with redundant equal length paths always has a loop-free deflection path if the link weights are in a ratio 1 + 1/(d-1), where d is the diameter of the network

Infocom Requirements  Intuition:  All link weights are in the range [ W min,W min x ]  the minimum cost of the shortest path is dW min  the maximum cost of the deflection path is (d-1)W min x  (d-1)W min x  dW min  x  1 + 1/(d-1)  Criteria for Theorem 2  Need equal length shortest paths between any two nodes  Weights need to be within a bounded ratio “ 1 + 1/(d-1) ”  The diameter d of the network should be small

Infocom Topology Considerations Inter-PoP Network Large inter-POP weights are within ratio Redundant equal length paths are guaranteed NYC-2 NYC-4 NYC-1 NYC-3 RTP-2 RTP-4 RTP-1 RTP-3 FW-2 FW-4 FW-1 FW-3 CHI-2 CHI-4 CHI-1 CHI-3 ANA-2 ANA-4 ANA-1 ANA-3 SJ-2 SJ-4 SJ-1 SJ-3 PoP San Jose PoP Anaheim PoP Chicago PoP Fort-Worth PoP New York PoP RTP Small diameter, d=3

Infocom Topology Considerations Complete Network Large Inter-POP Weights NYC-2 NYC-4 NYC-1 NYC-3 RTP-2 RTP-4 RTP-1 RTP-3 FW-2 FW-4 FW-1 FW-3 CHI-2 CHI-4 CHI-1 CHI-3 ANA-2 ANA-4 ANA-1 ANA-3 SJ-2 SJ-4 SJ-1 SJ-3 Perfect Mesh in PoPs Small ( w max ) Intra-POP Weights Diameter is larger Redundant equal length paths not guaranteed

Infocom Problem  Inter-PoP Network: PoPs as a single ‘logical node’ + All criteria for theorem 2 are satisfied  The complete network - Equal length redundant paths does not exist - Diameter of the network is not small - Maximum intra-PoP link weight w max is unrelated and very small compared to inter-PoP link weights  Problem - Cannot satisfy theorem 2 for the complete network

Infocom Practical deflection routing algorithm Solution: Clumping a PoP  A packet is forwarded from node s to d as follows, where w gain = w max 1. If shortest path not overloaded Forward the packet on the shortest path (with cost C ) 2. If link to neighboring node n is not overloaded Forward the packet to n if n’s cost to d is  C – w gain 3. Else if link to (intra-PoP) node n’ is not overloaded Forward the packet if its cost to d is  C + w ma x 4. Forward the packet on the shortest path Inter- PoP Intra- PoP

Infocom Theorem 3  Theorem 3:  The practical deflection routing algorithm has no inter-PoP loops  Comments  The sequence of costs strictly decreases across PoPs  This is in keeping with the idea of ‘PoPs’  Link failures  The algorithm is extended by setting w gain = (n-1)w max

Infocom Contents 1. Introduction 2. Pathology of link overload 3. Alleviate overload - deflection routing 4. Performance analysis

Infocom Simulations  Simulation parameters  14 node inter-PoP network and 4-5 node intra-PoP network  Estimated traffic matrix with gravity models & link measurements  Deflection threshold was set to 45%  Deflection based on fast EWMA  Simulations for link failures and fiber cuts

Infocom Link overload due to a fiber cut  Deflection routing decreases the maximum load amongst all links in the backbone

Infocom Conclusions  Deflection routing algorithm  Based on practical considerations and overload pathology  Exploits backbone architecture, meshed topology  Mandates a condition on weights which is not too restrictive  Is loop-free across PoPs  Note  Needs a redundant backbone network with equal-length paths  Useful when average utilization is low  Future Work  Stability needs to be investigated