AGG-NANOG IP Network Traffic Engineering Albert Greenberg Internet and Networking Systems Research Lab AT&T Labs - Research; Florham Park, NJ See (to appear in IEEE Network Magazine, special issue on Internet Traffic Engineering, March 2000). Joint work with Anja Feldmann, Carsten Lund, Nick Reingold and Jennifer Rexford.

AGG-NANOG IP Network Traffic Engineering  Goal? In operational IP networks, improve performance and make more efficient use of network resources, by better matching the resources with traffic demands  How? By integrating –traffic measurement –network modeling –selection and configuration of network management and control mechanisms.  Time Scale? Tens of minutes, hours, days, …  Applications? –Troubleshooting performance problems. »Why is this link congested? –Incremental load balancing »How to tune intradomain (OSPF, IS-IS) routing weights, or interdomain (BGP) import policies? –Capacity planning and optimization »How to estimate facilities cost from forecasted demands and optimal design?  Focus of this talk: ISP backbone networks  (See framework draft of new IETF, Traffic Engineering Working Group)

AGG-NANOG Traffic Engineering in IP Networks  Topology –Connectivity and capacity of routers and links  Demands –Expected load between points in the network  Routing –Tunable rules for selecting a path for each traffic flow  Performance objective –Balanced load, low latency, service level agreements, …  Question: Given the topology and traffic demands in an IP network, how do you decide which routes to use?

AGG-NANOG Short Answer?  The desired detailed, up to date, network-wide views of the topology are unavailable  The prevailing traffic demands are unknown  The network doesn’t adapt path selection to the load  The static routes aren’t necessarily optimized to the traffic These challenges arise because IP networks are  Decentralized  Self-configuring  Connectionless  Operating in loose confederation with peers Attributes that contributed to success and dominance of IP

AGG-NANOG Example: Congested Link  Detecting that a link is congested –Utilization statistics every five minutes from SNMP –Active probes suffer degraded performance –Customers complain  Reasons why the link might be congested –Increase in demand between some set of source-destination pairs –Failed router/link in our network causes change in our routes –Failure or policy change in another ISP changes traffic flow  How to determine why the link is congested?  How to relieve the congestion on the link?

AGG-NANOG Long Answer!  Derive topology from network configuration information  Compute traffic demands from edge measurements  Model path selection achieved by IP routing protocols  Build a query and visualization environment for “what-if” analysis Measurements Configuration Information Model Reporting Configuration Debugging Provisioning Capacity Planning Network Evolution Performance Debugging

AGG-NANOG Analysis/Visualization Routing Model Info Model ConfigurationMeasurements Toolkit Architecture Important to separate models from methods and data used to populate models

AGG-NANOG Configuration  Information –Backbone topology, link capacities, and router locations –Layer 2 and layer 3 links (e.g., ATM PVCs) –Intra-domain and inter-domain routing (e.g., OSPF weights) –Customer location and IP addresses; external IP addresses –Administrative policies, conventions  Construct –Unified views of the network topology, and of customer and peer reachability –Main sources: router configuration files, forwarding tables

AGG-NANOG Measurements  Performance statistics –Impact of traffic demands on the network »delay, loss, throughput from active probes between edge systems »Utilization, loss statistics from passive monitoring of links, nodes –Mapping of statistics onto the network topology  Traffic Demands –An accurate view of the demands themselves is extremely useful for effective traffic engineering –A large fraction of the traffic is interdomain, and a large number of customers are multihomed »Model traffic demands as loads from an edge interface to a set of candidate edge interfaces

AGG-NANOG Information Model  Abstraction of IP networks –Different views »router complexes, router, physical (layer 2), abstract (for routing) –Objects representing »routers, links, and traffic demands –Methods for manipulating objects »finding and selection of objects »linkage of objects, e.g., router complexes to routers »statistics: histogram, tables, etc.  Salient features –Captures important global network properties –Supports routing simulation (e.g., change of OSPF weights) –Trade off between accuracy and simplicity of model

AGG-NANOG Utilization (from passive measurement): link color (high to low) Delay (from active probes): link width (high to low ) Visualization of Link Utilization and Delay in Backbone

AGG-NANOG Routing Model  Capture: selection of shortest paths to/from (multihomed) customers and peers; splitting of traffic across multiple shortest paths; multiplexing of layer 3 links over layer 2 trunks access links Backbone peering links X1X1 X4X4 X3X3 X2X2 Y1Y1 Y2Y2 Y3Y3 Y4Y4 Y5Y5

AGG-NANOG Routing Model (continued)  Intradomain (OSPF) routing emulator –Extract backbone topology and link weights –Compute all shortest paths (Dijkstra’s algorithm) –Split load evenly along all shortest paths –Emulates Cisco-style use of multiple routes

AGG-NANOG Visualization of Traffic Flow in Backbone Color/size of node: proportional to traffic to this router ( high to low ) Color/size of link: proportional to traffic carried ( high to low )

AGG-NANOG Systems  Configuration –construction of network topology: layer 2, 3 connectivity, capacity, OSPF weights, customer and peer IP addresses, router locations  Measurements –Performance (active – delay, loss, throughput; passive – link and node utilization) –Traffic demands  Information model –physical level, IP level, router-complex level, abstract level –router attributes, link attributes  Routing model –shortest-path routing, OSPF tie-break, multi-homing, interdomain routing –bookkeeping to accumulate traffic load on each link  Visualization/analysis environment –querying to subselect links and nodes; histograms; what-if capabilities –coloring and sizing to illustrate link and node statistics

AGG-NANOG Key Ideas  data (configuration, routing, measurement) | models (topology, demands, routing) | analysis –Generate accurate global views of the network, and provide mechanisms to infer network-wide implications of changes in traffic, configuration and control –Architecture—separate systems for measurement, models, methods to populate models, analysis »Can and must evolve with change to underlying infrastructure and network architecture –Interfaces for modules above »E.g., design and optimization (e.g., Bernard Fortz and Mikkel Thorup, "Internet Traffic Engineering by Optimizing OSPF Weights," Proc. IEEE INFOCOM, March  … | (informed) provisioning and reconfiguration –Closing the loop… –Improving performance and making more efficient use of network resources