Link State on Data Center Fabrics Russ White, LinkedIn http://rule11.tech

BGP BGP Past Trend BGP BGP

Why BGP? Requirement OSPF IS-IS BGP Prefix distribution Yes Prefix filtering Limited Extensive Traffic Engineering Traffic Tagging Basic Multi-vendor stability Even more so (think about the Internet) https://forums.juniper.net/t5/Data-Center-Technologists/BGP-in-the-Data-Center-Why-you-need-to-deploy-it-now/ba-p/227547 Why BGP? REQ1: Select a topology that can be scaled "horizontally" by adding more links and network devices of the same type without requiring upgrades to the network elements themselves. REQ2: Define a narrow set of software features/protocols supported by a multitude of networking equipment vendors. REQ3: Choose a routing protocol that has a simple implementation in terms of programming code complexity and ease of operational support. REQ4: Minimize the failure domain of equipment or protocol issues as much as possible. REQ5: Allow for some traffic engineering, preferably via explicit control of the routing prefix next hop using built-in protocol mechanics. https://tools.ietf.org/html/rfc7938

Current Trend Open/R BGP BGP BGP Openfabric BGP Firepath BGP (+ BGP) (maybe + BGP) BGP Firepath (+ others) BGP

What changed? Our perception of policy is changing SDN Complexity of mixing policy and reachability Our perception of configuration is changing “Single source of truth” versus “minimal or no configuration” ZTP/DevOps versus autonomic

Layered Control Plane Controller needs full network view LSDB is easiest Hence link state reachability and topology preferred Overlay is “policy only” Removes filtering, security, and traffic engineering configuration from fabric devices

Shifting Trendlines Requirement OSPF IS-IS BGP Prefix distribution Yes Prefix filtering Limited Extensive Traffic Engineering Traffic Tagging Basic Multi-vendor stability Even more so (think about the Internet) versus Shifting Trendlines Requirement OSPF IS-IS BGP Prefix distribution Yes Prefix filtering Handled by the SDN overlay Traffic Engineering Traffic Tagging Multi-vendor stability Reduced importance through disaggregation Autoconfiguration Can be (largely) modified to work Full View of Topology Can be modified to work

What do I need to do to BGP? Autocompute router ID and AS number Peer on link locals Autodiscover local peers through some secondary protocol Peer with anyone who tries to open Use something like BGP-LS to gather a topology at the controller But… I still Need to configure remote peers, policies, etc. Need to carry policy in the distributed control plane These are made more challenging by the mix of local autoconfiguration and controller based single source of truth

Why am I doing this? BGP gives me policy But… mixing policy, reachability, and topology discovery has proven complex BGP is well known and widely implemented This is valid if you are relying vendor driven software in a mixed vendor environment Disaggregation (white box) changes the game BGP “scales better” Let’s think about this one a little…

Scaling Issues BGP can handle more prefixes Mitigating factors Modern processors Optimized SPF Most changes are leaf only Link state can carry the number of prefixes required in a DC fabric BGP can handle more routers in a single network This comes down to flooding

Why IS-IS? Easier to modify than OSPF Easier incremental/partial SPF Flooding domains are less entangled Packet format is TLV based Easier incremental/partial SPF Does not require link local addressing

The Flooding Challenge B C D E 1 2 The Flooding Challenge 3 4 Each Is receives at least 4, and potentially more than 8 copies, of each modified LSP

Distributed Solution Reverse Flooding Optimzation A1 runs SPF 2 3 4 A B C D E A1 runs SPF C1-4, A2-4 are two hop neighbors B1 chosen as flooder Flooded to B1 in normal LSP Flooded to others in link local LSP (RFC7356) draft-white-openfabric

Distributed Solution Reverse Flooding Optimization 1 2 3 4 A B C D E Do not flood to any neighbor on any shortest path towards the originator draft-white-openfabric

Centralized Flooding Firepath draft-li-dynamic-flooding Jupiter Rising research.googlstatic.googleusercontent.com /media/research.google.com/en//pubs/arc hive/43837.pdf draft-li-dynamic-flooding

The Scaling Problem Route count is not a huge problem There are solutions in this space if needed Node count primarily comes down to flooding The flooding problem can be solved What else do we need? Autoconfiguration bits A policy overlay

Autoconfiguration IS net ID Link local IPv6 through SLAAC Can be calculated From a local MAC address Link local IPv6 through SLAAC Loopback for management Interface addresses are not needed Local label pool for traffic engineering

Fabric Location Openfabric hop count == spf with all metrics set to 1 x = max hop count from my perspective y = max path from the perspective of someone who is T0 location == y – x draft-white-openfabric Advertised in tier TLV from shen-isis-spine-leaf-ext A B C D E

Fabric Location If… draft-shen-isis-spine- leaf-ext …connected to T0, then I must be T1 …connected to T1, then I must be T2 Etc. draft-shen-isis-spine- leaf-ext Advertised in tier TLV from shen-isis-spine- leaf-ext A B C D E

Policy Overlay There are many choices here PCEP I2RS BGP as a southbound Publish/Subscribe transport is (probably) best Every operator and/or vendor could choose a different solution

Summary BGP is a good DC fabric protocol For some specific situations Such as when you need… Policy mingled with reachability Little or no visibility into the topology Strong multivendor support Single source of truth with strong DevOps automation environment

Summary Each of these points is being challenged Policy should be separated from reachability and topology Strong requirements for topology visibility Minimize configuration/autonomic operation Link state protocols fit these requirements better than BGP