April 4th, 2002George Wai Wong1 Deriving IP Traffic Demands for an ISP Backbone Network Prepared for EECE565 – Data Communications

April 4th, 2002George Wai Wong2 Difficulties in configuring a large IP backbone network  Engineering a large IP backbone network without an accurate view of the traffic demand is difficult  Significant and sudden fluctuation in load can be caused by –Shift in user behavior –Changes in routing polices –Failure of network elements

April 4th, 2002George Wai Wong3 Difficulties in configuring a large IP backbone network (cont.)  IP network engineers do not have end-to- end control of the path from source to destination  The majority of traffic in an ISP network travels across multiple administrative domain

April 4th, 2002George Wai Wong4 Point-to-multipoint IP traffic demand  A given destination network address is typically reachable from multiple edge routers

April 4th, 2002George Wai Wong5 Point-to-multipoint IP traffic demand (cont.)  IP traffic demands are naturally modeled as point-to-multipoint volumes  Connection-oriented networks, such as Frame Relay, are modeled as point-to-point volumes

April 4th, 2002George Wai Wong6 Traffic flows in an ISP backbone

April 4th, 2002George Wai Wong7 Peering links in BC Obtained from

April 4th, 2002George Wai Wong8 Tracing the route on Monday 12:32am March 18, 2002 ug15{georgew}104: traceroute traceroute to ( ), 30 hops max, 40 byte packets 1 ugrad-route ( ) ms ms ms 2 turing ( ) ms ms ms 3 ee-gw ( ) ms ms ms ( ) ms ms ms 5 anguhub9.net.ubc.ca ( ) ms ms ms 6 c7507-a9.BC.net ( ) ms ms ms 7 ATM PEERB-VANCBC.IP.GROUPTELECOM.NET ( ) ms ms ms 8 GE3-0.WANB-VANCBC.IP.GROUPTELECOM.NET ( ) ms ms ms 9 GE4-0.PEERA-VANCBC.IP.GROUPTELECOM.NET ( ) ms ms ms ATM3-0.GW3.VAN1.ALTER.NET ( ) ms ms ms ATM2-0.XR1.VAN1.ALTER.NET ( ) ms ms ms 12 0.so XL1.VAN1.ALTER.NET ( ) ms ms ms 13 0.so TL1.VAN1.ALTER.NET ( ) ms ms ms 14 0.so TL1.SAC1.ALTER.NET ( ) ms ms ms 15 0.so XL1.PAO1.ALTER.NET ( ) ms ms ms 16 POS1-0.XR1.PAO1.ALTER.NET ( ) ms ms ms

April 4th, 2002George Wai Wong9 Tracing the route on Monday 12:32am March 18, 2002 (cont.) ATM6-0.GW10.PAO1.ALTER.NET ( ) ms ms ms 18 opentransit2-gw.customer.ALTER.NET ( ) ms ms ms 19 P0-1.TKYBB2.Tokyo.opentransit.net ( ) ms ms ms 20 P2-0.TKYBB1.Tokyo.opentransit.net ( ) ms ms ms 21 P0-0.HKGBB1.Hong-kong.opentransit.net ( ) ms ms ms 22 EquantHongKong.GW.opentransit.net ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms ( ) ms ( ) ms * ( ) ms ms ms ( ) ms ms ms 30 ( ) ms ms ms

April 4th, 2002George Wai Wong10 Tracing the route on Monday 2:36am March 18, 2002 ug15{georgew}105: traceroute traceroute to ( ), 30 hops max, 40 byte packets 1 ugrad-route ( ) ms ms ms 2 turing ( ) ms ms ms 3 ee-gw ( ) ms ms ms ( ) ms ms ms 5 anguhub9.net.ubc.ca ( ) ms ms ms 6 c7507-a9.BC.net ( ) ms ms ms 7 ATM PEERB-VANCBC.IP.GROUPTELECOM.NET ( ) ms ms ms 8 GE3-0.WANB-VANCBC.IP.GROUPTELECOM.NET ( ) ms ms ms 9 GE4-0.PEERA-VANCBC.IP.GROUPTELECOM.NET ( ) ms ms ms ATM3-0.GW3.VAN1.ALTER.NET ( ) ms ms ms ATM2-0.XR1.VAN1.ALTER.NET ( ) ms ms ms 12 0.so XL1.VAN1.ALTER.NET ( ) ms ms ms 13 0.so TL1.VAN1.ALTER.NET ( ) ms ms ms 14 0.so TL1.SAC1.ALTER.NET ( ) ms ms ms 15 0.so XL1.PAO1.ALTER.NET ( ) ms ms ms 16 POS1-0.XR1.PAO1.ALTER.NET ( ) ms ms ms

April 4th, 2002George Wai Wong11 Tracing the route on Monday 2:36am March 18, 2002 (cont.) ATM6-0.GW10.PAO1.ALTER.NET ( ) ms ms ms 18 opentransit2-gw.customer.ALTER.NET ( ) ms ms ms 19 P0-1.TKYBB2.Tokyo.opentransit.net ( ) ms ms ms 20 P2-0.TKYBB1.Tokyo.opentransit.net ( ) ms ms ms 21 P0-0.HKGBB1.Hong-kong.opentransit.net ( ) ms ms ms 22 EquantHongKong.GW.opentransit.net ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms ( ) ms ms ms ( ) ms ( ) ms * ( ) ms ms ms ( ) ms ms ms 30 ( ) ms ms ms Conclusion: Internet is so unpredictable !!!

April 4th, 2002George Wai Wong12 Measurement Methodology  To compute the traffic demand –Fine-grain traffic measurements are collected at ALL ingress links (too expensive, impractical) –Flow-level statistics should be collected at each ingress link. The measurement can be collected directly by the incident router using Netflow (Cisco traffic measurement tool)

April 4th, 2002George Wai Wong13 Netflow Data Record Source IP Address Destination IP Address Source IP Address Destination IP Address Next Hop Address Source AS Number Dest. AS Number Source Prefix Mask Dest. Prefix Mask Next Hop Address Source AS Number Dest. AS Number Source Prefix Mask Dest. Prefix Mask Input Interface Port Output Interface Port Input Interface Port Output Interface Port Type of Service TCP Flags Protocol Type of Service TCP Flags Protocol Packet Count Byte Count Packet Count Byte Count Start Timestamp End Timestamp Start Timestamp End Timestamp Source TCP/UDP Port Destination TCP/UDP Port Source TCP/UDP Port Destination TCP/UDP Port Usage QoS Time of Day Application Routing and Peering Port Utilization From/To The who, what, where, when and how much IP traffic questions are answered

April 4th, 2002George Wai Wong14 Set of egress links  The flow record collected at the ingress link has sufficient information for computing a set of egress links: –IP destination address –routing table (next-hop link(s) for a particular prefix)

April 4th, 2002George Wai Wong15 Problems with this methodology  The routers that terminate these ingress links often vary in functionality and must perform computationally intensive access control functions such as filtering  Hence, collecting flow-level statistics at every ingress link is not always feasible in practice

April 4th, 2002George Wai Wong16 Measuring at peering links  We shall extend our methodology to measurements collected at a much smaller number of peering links.  A small number of high-end routers are used to connect to a neighboring provider

April 4th, 2002George Wai Wong17 Measuring at peering links (cont.)  Monitoring both ingress and egress links at the peering links can capture a large fraction of the traffic –Ingress links for inbound and transit traffic –Egress links for outbound traffic Problems:  Internal traffic is missing  Ambiguous ingress point for outbound traffic  Duplicate measurement of transit traffic

April 4th, 2002George Wai Wong18 Measuring at peering links (cont.)  Fortunately, an ISP typically knows the IP address of its directly connected customers.  However, for customers that connect to two or more service providers, the first methodology is more appropriate.

April 4th, 2002George Wai Wong19 Experimental results  Flow-level measurements are collected at the peering links of the AT&T IP backbones  The flow-level measurement were collected by enabling Netflow on each router that terminate peering links

April 4th, 2002George Wai Wong20 Traffic Analysis  ~80% of the total traffic is shown  Note that plots are nearly linear on the log-log scale  Rank traffic demands from largest to smallest and plot the percentage of the total traffic attributable to each

April 4th, 2002George Wai Wong21 Traffic Analysis (cont.)  The small number of heavy hitters has important implication for traffic engineering  Since leading heavy hitters account for so much traffic, care in routing just these demands should provide most of the benefit  Significant variation in demand sizes at the highest ranks

April 4th, 2002George Wai Wong22 Conclusion  A model of traffic demands that capture 1. the volume of data 2. the entry point into the ISP network 3. destination reachability information is proposed  A methodology for populating the demand model from flow-level measurement is presented  The measured demands reveals significant variations in demand sizes and popularities by time-of-day

April 4th, 2002George Wai Wong23 References  Anjia Feldmann, Albert Greenberg, Carsten Lund, Nick Reingold, and Jennifer Rexford, “Deriving Traffic Demands for Operation IP Networks: Methodology and Experience,” IEEE/ACM Transactions on Networking, vol 3, no. 3, pp , June 2001  Cisco Netfow (2001). [Online]. Available:

April 4th, 2002George Wai Wong24 Questions?