Internet Routing (COS 598A) Today: Router Configuration Jennifer Rexford Tuesdays/Thursdays 11:00am-12:20pm

Outline Individual router –Router CPU and interfaces –Links, adjacencies, and sessions –Filtering and injecting routes Network routing design –Case study of an ISP backbone –BGP community to convey state Current state of the art –Problems with today’s languages –Static analysis to detect mistakes –Template-based automation

Configuring the Router CPU Loopback interface –IP address for accessing the CPU Access control for router CPU –Command-line interface and SNMP set/get –Id and password, and levels of access permission Default parameters –E.g., TCP parameters Services –E.g., logging, telnet, TFTP, etc.

Access Interfaces: The Basics High-level information –Media (e.g., Serial, Packet-Over-Sonet, and ATM) –Location (e.g., slot 10, port adaptor 1, port 0) –Description (i.e., a comment field) –Capacity (i.e., bandwidth) Diverse communication media at layer 2 –Serial link, ATM, packet over SONET, etc. –Various low-level, media-specific parameters Addressing and routing –IP address and network mask –Static route to map destination prefix to interface

Access Interfaces: Layer-3 Resource Allocation Access control –Filtering of IP packets Based on packet-header bits (e.g., IP addresses, type- of-service bits, port numbers, and protocol) Buffer management –Maximum queue size –Parameters for Random Early Detection (RED) Link scheduling –Mapping of packets to queues Based on packet-header bits –Allocation of bandwidth resources E.g., weights for class-based queuing

Example of Statically-Routed Customer hostname big-fat-router interface Loopback0 ip address ! interface Serial10/1/0/12:0 description Static Customer ip address ip access-group 666 in ! ip route Serial10/1/0/12:0 access-list 666 permit access-list 666 permit Static route for /23 Allow incoming packets with source in /23 or /30 Implicit “deny” at the end…

Link Between Two (or More) Routers A link is just a (very small) network –Single subnet (e.g., /30 ) IP addresses for each interface –One on each router Special IP addresses –Broadcast address –Network address /

Intradomain Routing Protocol Interface participation –Enable interface participate in OSPF –Assign an OSPF weight (and area) Link participation –All interfaces enabled in OSPF, in same area –Triggers them to establish an adjacency Network of OSPF-speakers –Routers flood the link-state advertisements –Each router constructs view of the OSPF topology

BGP Session With a Neighbor BGP session –AS number of the neighbor –TCP connection between two IP addresses Reaching the remote end of the connection –Local router must be able to reach the neighbor router –Need to be able to route to establish the session! Diversity of configurations –One link vs. multiple links –Direct connection vs. intermediate components LR NR LR NR firewall, NAT, etc. multiple access linkssingle access link

Session With Directly-Connected Interface BGP session with the other end of the link –Single link directly to the neighbor’s router –Local router knows how to reach the address LR NR BGP session interface Serial10/1/0/12:0 ip address ! router bgp 7018 neighbor remote-as neighbor … ! AS 7018 AS 18585

Static Routes to Remote BGP Speaker BGP session associated with static routes –Multiple access links to the neighbor’s router –Or, intermediate components on the path BGP session NR interface POS7/0 ip address ! interface POS8/0 ip address ! router bgp 7018 neighbor remote-as neighbor … ! ip route POS7/0 ip route POS8/ LR POS7/0 POS8/0

Filtering BGP Route Advertisements E.g., filter routes not belonging to neighbor E.g., filter routes for martian IP addresses router bgp 7018 neighbor remote-as neighbor distribute-list CUSTOMER in ! ip prefix-list CUSTOMER seq /23 ip prefix-list CUSTOMER seq /16 router bgp 7018 neighbor remote-as 701 neighbor prefix-list MARTIAN in ! ip prefix-list MARTIAN seq /0 ip prefix-list MARTIAN seq /8 le 32 …

Introducing Routes Into BGP interface Serial10/1/0/12:0 description Static-routed customer ip address ip access-group 666 in ! router bgp 7018 network mask redistribute static ! ip route Serial10/1/0/12:0 Introducing routes into BGP –Explicit announcement (“network”) –Redistribution from other protocols

Network Routing Design

Network Routing Design: ISP Backbone Inside the network –OSPF Compute paths between routers –Internal BGP Distribute BGP routes inside Periphery of the network –Packet filters Block incoming packets at entry points –Static routes Learn how to reach non-BGP customers –External BGP Exchange reachability information with BGP neighbors

Inside the Network: Interior Gateway Protocol OSPF: all internal links participate –Area structure driven by Point-of-Presence Backbone area (area 0): inter-PoP links Non-backbone areas: intra-PoP links –Weights driven by topological properties Physical distance Link bandwidth PoP

Inside the Network: Internal BGP iBGP: all routers participate –Route-reflector hierarchy driven by PoP Route reflectors: two RRs per Point-of-Presence RR clients: connect to both RRs in their PoP –Full mesh of top-level route reflectors PoP

Periphery of the Network: Packet Filters Permit valid customer to send (prevent spoofing) –Permit source addresses belonging to customer Permit routing protocols and management to work –Permit source address of remote BGP speaker –Permit customer to “ping” your end of the link Prevent bogus IP addresses –Deny source addresses of “martians” –Deny source address of your own services Prevent access to your own equipment –Deny packets destined to routers from unexpected source Prevent unwanted applications/services –Deny SNMP port number or multicast addresses –Deny BGP port number from unexpected source

Periphery of Network: External BGP External BGP sessions –Session to each BGP-speaking neighbor –Import and export policy for each session Policy mechanisms –Filtering: discard route announcements –Preference: assign high/low preference –Tagging: mark routes with extra state Policy goals –Business relationships –Traffic engineering –Route aggregation

Business Relationships: Assigning Preference route-map INPEER permit 100 set local-preference 80 ! route-map INCUST permit 100 set local-preference 90 ! Prefer-customer policy –Session with peer Import policy: assign local-pref of 80 –Session with customer Import policy: assign local-pref of 90

Business Relationships: Tagging for Memory Tag to remember the type of route –Session with peer: community 0:1000 –Session with customer: community 0:2000 route-map INPEER permit 100 set local-preference 80 set community 0:1000 ! route-map INCUST permit 100 set local-preference 90 set community 0:2000 !

Business Relationships: Filtering Based on Tag Export policy based on tag –Export all routes to customers –Export only customer-learned routes to peers route-map INPEER permit 100 set local-preference 80 set community 0:1000 ! route-map OUTPEER permit 100 match community 0:2000 ! route-map INCUST permit 100 set local-preference 90 set community 0:2000 ! route-map OUTCUST permit 100 match community 0:1000 0:2000 ! Filter routes from peers

Periphery of Network: Static Routes Static route for destination prefix –Reaching destinations of a non-BGP customer –Reaching the remote end of eBGP session Tagging to limit export of static routes in BGP –Local to the router: no injection in BGP Prefix contained in block allocated to provider’s router … and customer connects to Internet in only one place –Just inside the AS: inject with “no export” Prefix contained in provider’s address space … and customer connects to only one provider –Both in and out of AS: inject with no restriction Prefix not contained in provider’s address space … or customer connects to multiple providers

Complexity of BGP Policies Remembering from one router to another –Business relationship for export policy –Scope of prefix for route-aggregation –Geographic location Side information unknown to routers –Business relationship with neighbor AS? –Customer’s prefix falls in provider’s supernet? –Customer has multiple access links or providers? Intrinsic complexity and diversity of policies –Business relationships, traffic engineering, route aggregation, security, etc.

Current State of the Art

Manual Configuration Dangerous –Typo in a routing policy: black hole –Interfaces in different OSPF areas: no traffic on link –Missing a packet filter: denial-of-service vulnerability Expensive –Delays in deploying equipment –Hiring and training skilled engineers –Lock-in to the router vendor Disruptive –Half of network outages (Yankee Group) –Failures of Internet services (USITS’03) –BGP routing anomalies (SIGCOMM’02, NSDI’05)

The Problem With Configuration Languages Heterogeneity –No standard language for the industry –Differences across products and versions by a vendor –Change over time due to new protocols and features Low-level language –Thousands of different commands –Complex inter-relationships between commands –Very little abstraction or nesting –Sometimes no public grammar or parser Poor commit semantics –Command-line interface, one command at a time –Often no support for atomic actions

Problem With Configuration Languages Router-level configuration –Configuration of individual routers, not the network –E.g., configure each end of a link separately Complexity of distributed protocols –Some things are hard to do in distributed fashion –E.g., ensuring complete visibility of routes in iBGP –E.g., the need for BGP communities to pass state Lack of abstractions –Research emphasis on speed and features –… not on simplicity, managability, and clean abstractions –Config languages are no better (though often worse!) than our abstractions for mechanisms, protocols, and practices

Reducing Configuration Errors Configuration checking tools –Dump the running configuration of all routers –Parse, join, and query the data –Generate reports of problems Automated configuration –Templates and rules for generating configuration –Database for storing the values of variables –Software to fill templates and apply commands Better configuration languages –Vendor neutral languages for routing policy (RPSL) –Research on network-wide configuration

Configuration Checking

Configuration Checking: Types of Checks Errors: clear mistakes –Variables used but not defined –Mismatch between two ends of link or session Warnings: risky behavior –Variables defined but not used –Dependence on default configuration values –Violations of best common practices Inconsistencies: pattern mismatches –Same variable defined differently on two routers –Variables with different names defined same way Local policy violations: –Mismatch with the operator’s explicit intent

Configuration Mistakes: BGP Prefix Filtering Martian prefixes –Goal: block announcements for bogus prefixes –Variable: prefix-list define martian prefixes –Instantiation: eBGP sessions apply the prefix-list Configuration mistakes –Error: MARTIAN instantiated but not defined –Warning: MARTIAN defined but not instantiated –Inconsistency: MARTIAN defined differently on different routers –Local policy violation: MARTIAN not defined as /0, /8-32, /8-32, etc.

Configuration Mistakes: iBGP Configuration Internal BGP sessions –Goal: disseminate reachability info inside the AS –Mechanism: iBGP sessions between routers Configuration mistakes –Error: Router A has iBGP session to Router B, but B does not have an iBGP session to A –Warning: iBGP session between non-Loopback addresses –Inconsistency: all routers have two route reflectors (RRs), except one –Local policy violations: all client routers should have two RRs; route RRs should form a full mesh

Automated Configuration

Automated Configuration System DB interface Serial10/1/0/12:0 description CBB Customer ip address ip access-group 666 in ! router bgp 7018 neighbor remote-as neighbor route-map CUST-FACE in neighbor route-map FULL-ROUTES out neighbor distribute-list 13 in neighbor soft-reconfiguration-inbound ! What is your AS number? What export policy do you want? Do you want a dynamic default? What are your address blocks? Do you need to receive communities? query template RULESRULES configlet Technical Questions (TQ) interface description ip address ip access-group in ! router bgp 7018 neighbor remote-as neighbor route-map CUST-FACE in neighbor route-map out neighbor distribute-list in neighbor soft-reconfiguration-inbound [neighbor send-community] ! router Automation through template and database … but doesn’t raise the level of abstraction So building these kind of systems is challenging

Automation Stages Initializing a new router –Base configuration to load on the router –E.g., access control for the router CPU –E.g., enabling of services (e.g., telnet, logging) –E.g., defining MARTIAN lists and routing policies Changing the configuration –Use cases for different activities –… with TQ, database variables, and template –E.g., “add link,” “add BGP-speaking customer,” or “move static-route customer to BGP customer) –Generates a configlet applied to the router

Conclusion Router configuration is hard –Low-level configuration languages –Wide variety of protocols and mechanisms Improving on manual configuration –Configuration-checking tools –Automated configuration systems –Research needed on languages and abstractions Possibility of auto-configuration? –Specify high-level network design –Ship a router and plug it in to the network –Router auto-configured from a server

Next Time: Removing Routing from Routers Three short papers –“Routing as a Service” –“The Case for Separating Routing From Routers” –“Network-Wide Decision Making: Toward a Wafer-Thin Control Plane” Review just of the first paper –Summary –Why accept –Why reject –Directions for future work Plan for a discussion-driven class on Thursday Reminder: no class next week!!!