Traffic Dynamics at a Commercial Backbone POP Nina Taft Sprint ATL Co-authors: Supratik Bhattacharyya, Jorjeta Jetcheva, Christophe Diot

Outline Part 1: what are the traffic demands between pairs of POPs? How stable is this demand? Part 2: what are the paths taken by those demands? Are link utilizations levels similar throughout the backbone? Part 3: is there a better way to spread the traffic across paths? Can we divert some traffic to lightly loaded paths?

The Sprint IPMon Project Passive monitoring Capture header (44 bytes) from every packet full TCP/IP headers, no http information Use GPS time stamping - allows accurate correlating of packets on different links Day long traces Simultaneously monitor multiple links and sites. Collect routing information along with packet traces. Traces archived for future use

IP Backbone : POP-to-POP view POP fanout: one row of POP-to-POP traffic matrix OC-48 OC-12 OC-192

POP-to-POP Traffic Matrix City A City B City C City A City B City C Measure traffic over different timescales Divide traffic per destination prefix, protocol, etc. For every ingress POP : Identify total traffic to each egress POP Further analyze this traffic

The Mapping Problem What is the egress POP for a packet entering a given ingress POP?

Monitored links at a single POP Core Public peer 2 Access web host ISP Public peer 1 Date : Aug 9, 2000

Traffic Fanout: POP level granularity

Fanout: web host links

Time-of-Day for POP level granularity

Day-Night Variation : Webhost #1 % reduction at night between 20-50% depending upon access link

Summary so far... Wide disparity in traffic demands among egress POPs POPs can be roughly categorized as : small, medium, large; and they maintain their rank during the day. Traffic is heterogeneous in space yet stable in time % reduction at night

Outline Part 1: what are the traffic demands between pairs of POPs? How stable is this demand? Part 2: what are the paths taken by those demands? Are link utilizations levels similar throughout the backbone? Part 3: is there a better way to spread the traffic across paths? Can we divert some traffic to lightly loaded paths?

Paths used by traffic demands Our Observations (summary) routing policies concentrate traffic on a few paths: between two POPs, all the traffic uses either the same route, or 1 or 2 routes the ISIS weights are changed very infrequently (once a month), so routing is fairly static there are many underutilized routes

Is backbone traffic balanced?

Part 3 : Can we divert some traffic to lightly loaded paths? Approach: to improve load balancing by rerouting only a few flows scalable Which flows? Heavy hitters. How identify heavy hitters: Consider: destination prefix-based flows at fixed prefix lengths: 8 and 16 BGP table entries (variable prefix length)

Streams based on destination prefix Stream : all packets in a group with same /8 destination address prefix Traffic grouped by egress POPs Ingress : Webhost Link Similar results for /16 and bgp table prefixes

Stability of prefix-based streams R i (n) = the rank of flow i at time slot n i,n,k = | R i (n) - R i (n+k) | each time slot Stability of prefix rank

Conclusions We have used our data to build components of traffic matrices for traffic engineering Heterogeneous traffic fanout from POP Current routing practices lead to many underutilized links and paths thus, there is a lot of room for improved load balancing techniques. Load-balancing using flows selected via destination-prefixes is a simple and promising criterion

Ongoing Work Intra-domain Routing : Choosing ISIS link weights Multi-path routing Flow Characterization at the network prefix level Inference techniques for building POP-to- POP traffic matices