Investigation of Global Network Routing Behavior BJ Premore Dartmouth College Prof. David Nicol, Advisor December 8, 2000 In collaboration with Jim Cowie, Renesys Corporation Tim Griffin, AT&T Labs-Research Andy Ogielski, Renesys Corporation … and several other colleagues

2 Objectives –better understand inter-domain routing dynamics –explore impact of implementation tradeoffs –explore extensions before deployment –provide a useful tool for researchers Implementation –simulation architecture –BGP functionality –validation Research Applications –convergence (ongoing) –security (ongoing) –timing, policy interaction, proposed extensions, etc. Overview

3 Objectives –better understand inter-domain routing dynamics –explore impact of implementation tradeoffs –explore extensions before deployment –provide a useful tool for researchers Implementation –simulation architecture –BGP functionality –validation Research Applications –convergence (ongoing) –security (ongoing) –timing, policy interaction, proposed extensions, etc. Overview

4 Simulation Architecture SSFNet = SSF Network Models - compositional approach to large network design - not independent SSF = Scalable Simulation Framework - a modern standard for discrete-event simulation of large, complex systems - multiple implementations - the “engine under the hood” DML = Domain Modeling Language - model configuration

5 Simulation Layers SSF standard Simulator API C++CJava CSSFRacewayDaSSF Simulator Implementations implements SSFNet Network Components as Java Classes enhances Model Instances DML Configurations configure

6 Why Another Simulator? Fully Integrated Network Environment –control over more than just BGP –eg: TCP/IP, traffic, router & link hardware Scalability –designed to handle large, complex simulations –tens of thousands of multi-protocol nodes Design Trade-off Toggles –eg: tie-breaking in route selection –eg: apply minAdver timer to withdrawals Explore Impact of New Functionality –before it goes live! –eg: MPLS; protocol extensions

7 Pros and Cons We can’t … –expect to model real-world routers perfectly with every detail We can … –capture the most important characteristics –change and tweak the protocol –explore consequences of fundamental design of BGP –explore proposed and novel protocol extensions –evaluate and analyze collective behavior on a large-scale

8 SSFNet Layer SSF standard DML configurations CSSFRacewayDaSSF Simulator API Simulator Implementations SSFNet C++CJava Network Components as Java Classes Model Instances implements enhances configure

9 Example SSFNet Components physical entities link router host logical containers Net protocol graph protocols BGP OSPF FTP HTTP Sockets IPTCP

10 SSF.OS.BGP Based on RFCs RFC 1771: BGP-4 and latest drafts RFC compliant implementation Includes some RFC-specified extensions (Route Reflection) Has features similar to those used by vendors (policy-based filtering)

11 SSF.OS.BGP4 Functionality –Finite state machine, timers, RIB –TCP transport –Peering: exterior and interior –Route reflection –Messages and path attributes –Policy –filter based on path attribute –attribute modification –Monitoring of protocol operation –gather stats on practically any event of interest

12 Package SSF.OS.BGP4 Organization BGPSession PeerEntry RIBInLocRIBRIBOut RIBIn Policy Rule (inbound) Policy Rule (outbound) Timers ConnRetry KeepAlive Hold MinAdver

13 Validation Methodology No standards, create our own suite Basic behavior in simple topologies –Peering session maintenance (Hold & KeepAlive timer operation) –Route advertisement and withdrawal –Route selection –Reflection –Internal BGP General behavior in complex topologies –End-to-end data delivery –Exercises basic behaviors as well Policy testing –Converging and non-converging gadgets [Griffin 1999]

14 Example: Route Reflection Validation Test Topology

15 Another Test Topology

16 Large Network Example

17 Example With Monitoring Filters

DML Example host [ id 1 interface [ id 1 ] ] router [ id 2 interface [ idrange [ from 1 to 4 ] ] ] link [ attach 1(1) attach 2(1) ] 2 3 4

19 DML: Adding Protocols protocol graph router [ graph [ ProtocolSession [ name bgp use SSF.OS.BGP4.BGPSession ] ProtocolSession [ name ospf use SSF.OS.OSPF.sOSPF ] ProtocolSession [ name tcp use SSF.OS.TCP.tcpSessionMaster ] ProtocolSession [ name ip use SSF.OS.IP ] IP TCP OSPF BGP

20 Objectives –better understand inter-domain routing dynamics –explore impact of implementation tradeoffs –explore extensions before deployment –provide a useful tool for researchers Implementation –simulation architecture –BGP functionality –validation Research Applications –convergence (ongoing) –security (ongoing) –timing, policy interaction, proposed extensions, etc. Overview

21 Interesting Possibilities –Better value for MinAdver timer? –Improved route flap dampening? –Policy studies –How do various configurations affect convergence? –Test effects of policy changes before deployment –EGP-IGP interaction studies –Are there instability side-effects? –Is it safe to convert between different cost metrics? –MPLS –Will it have any unexpected effects on routing? –Security studies

22 A Security Study Black Holes How many networks can/will be included? parameters –severity of misconfiguration or maliciousness –number of misbehaving routers –location of misbehaving routers Other Questions –What is the impact of SBGP on routing efficiency? –Can attacks and misconfigurations be detected? –How can we speed up convergence after an attack?

23

24 A Convergence Study Goals build upon previous work –Labovitz, Ahuja, Bose & Jahanian 2000 –what factors contribute to observed dynamic behaviors? isolate contributions of different parameters –policy, topology, iBGP, timers, etc. make recommendations for implementations (eventually) –what changes can alleviate impact of various factors?

25 A Convergence Study Model Parameters –topology: N ASes each with just 1 router »shape: line, loop, wheel, meshes, grid »size: vary N from 2 to 100 –policy »permit all or typical customer/provider/peer –link delay »all equal or random Advertise, Withdraw, Wait and Watch 1.Wait for system to reach stable state, then … 2.Designated AS advertises a bogus destination to everyone else 3.Wait for system to reach a stable state again, then … 4.Designated AS tells everyone that the bogus route is not reachable through it any more 5.Wait for system to reach a stable state again

26 Simple Topologies line loopwheel emesh imesh grid

27 Line Experiment fixed or random link delays

28 Loop Experiment fixed link delays

29 Wheel Experiment fixed link delays

30 IBGP Full Mesh Experiment fixed link delays

31 EBGP Full Mesh Experiment fixed link delay

32 Grid Experiment fixed link delay, width=10, no policy

33 Preliminary Observations Convergence time related to number of alternate paths a router sees –policy helps reduce Agreement with previous results –full mesh experiments in particular Full external mesh still the most interesting –how many alternate paths are actually “seen” depends a lot on timing –using random link delays reduced convergence time

34 Coming Soon … –Functionality –aggregation –route flap dampening –communities –confederations –and more... –Experiments –look for better timer values –how does policy affect convergence? –can we improve route flap dampening? –test extensions and other proposed modifications –and more …

35 For Further Information SSF/Raceway and SSFNet: SSF.OS.BGP4: (or follow link from

36 Net [ id 1 AS_status boundary router [ id 1 graph [ ProtocolSession [ name bgp use SSF.OS.BGP4.BGPSession autoconfig false connretry_time 120 min_as_orig_time 15 reflector false neighbor [ as 0 address 1(1) use_return_address 1(1) hold_time 90 keep_alive_time 30 min_adver_time 30 infilter [ # give low priority to routes learned from 0 clause [ precedence 1 predicate [] action [ primary permit atom [ attribute local_pref type set value 80 ] ] outfilter [ _extends.filters.permit_all ] ] This sample DML code configures an AS with a single router running BGP. It performs explicit configuration of all BGP attributes. It is taken from the ‘goodgadget’ validation test. (continued next page)

37 neighbor [ as 2 address 1(2) use_return_address 1(2) hold_time 90 keep_alive_time 30 min_adver_time 30 infilter [ # give high priority to routes learned from 2 clause [ precedence 1 predicate [] action [ primary permit atom [ attribute local_pref type set value 100 ] ] outfilter [ _extends.filters.permit_all ] ] neighbor [ as 3 address 1(2) use_return_address 1(3) hold_time 90 keep_alive_time 30 min_adver_time 30 infilter [ # deny all routes learned from 3 clause [ precedence 1 predicate [] action [ primary deny ] ] ] outfilter [ _extends.filters.permit_all ] ] ProtocolSession [ name socket use SSF.OS.Socket.socketMaster ] ProtocolSession [ name tcp use SSF.OS.TCP.tcpSessionMaster ] ProtocolSession [ name ip use SSF.OS.IP ] ] interface [ idrange [ from 0 to 3 ] ] ] host [ id 101 _extends.basic_host ] link [ attach 1(0) attach 101(0) delay ] ]