CRIO: Scaling IP Routing with the Core Router-Integrated Overlay Xinyang (Joy) Zhang Paul Francis Jia Wang Kaoru Yoshida

highest ranked “problem” scaling the routing system It was clear at the workshop, and probably clearly evident elsewhere, that if there is a highest ranked “problem” in the routing space then it would be that of scaling the routing system unbounded continued growth trigger off some real limitations It appears that unbounded continued growth of the routing and forwarding system in the Internet appears to trigger off some real limitations relating to hardware design and switching centre infrastructure Internet Architecture Board Routing Workshop (Oct 2006) ---- Geoff Huston

Why is Scaling a Problem? A glimpse of current routing system:  Static table size Global IPv4 : ~ 200K entries VPN: ~800K entries And more routes are coming: IPV6, traffic-engineered, etc.  Routing Dynamics BGP update churns Persistent instabilities Long convergence time (due to damping and MRAI timer) Looking into the future:  Can we support a routing table twice (or 10 times) the size of today?  Can we rely on the hardware advances to alleviate the scaling pressure? 200K Active BGP entries (FIB) Date This talk is about the static characteristics of the scaling Validity of CRIO approach

CRIO’s Approach to Scaling Tunneling  Revisit old idea (by Deering)  Decouples addressing from topology Virtual Prefix  Novel approach  Greatly shrink forwarding table

CRIO Tunneling: an Illustration Customer Site C /24 Routing Adv /24 NH=CE2 Customer Site C1 PE2 CE2CE1 PE1 Prefix TE Source / BGP/OSPF Mapping Adv /24 TE=PE2 Prefix TE Source PE BGP /24 PE2 Mapping Provider Networks PE PE /24 PE3 Mapping

CRIO Tunneling: Benefits Separate Mapping from Routing  BGP only computes routes to TE prefixes On the order of one thousand entries Stable ISP provisioned prefixes  Mappings are easy to distribute A mapping entry is the same no matter where it appears  Support multi-homing without burdening the routing system

What about router’s forwarding table? CRIO tunneling can not shrink forwarding information Forwarding table is expected to get larger  Since CRIO supports for fine-grained multi-homing Benefits for having small forwarding tables  Smaller memory requirement on routers’ line cards  Faster transfer for forwarding table updates

CRIO Virtual Prefix: an Illustration A virtual prefix is a super-prefix that spans a large portion of the address space Routers that advertise a given virtual prefix must hold the mappings for every prefix within the virtual prefix Customer Site CE /24 PE2 PE1 PE3 Prefix TE Source PE BGP PE BGP /24 PE2 Mapping Prefix TE Source PE BGP /24 PE2 Mapping Routing Adv / / BGP PE /24 PE4 Mapping

CRIO Virtual Prefix: Trade-off Virtual prefixes provide a tuning knob for the router  trade-off forwarding table size for path length  Per-prefix basis It’s a good trade-off to make  Few prefixes handle most traffic (power law)  Routers could shed most of their prefixes with very little overall increase in traffic volume  Save routers from handling large amount of mapping updates Virtual Prefix is particularly suitable for VPNs

CRIO Evaluation: Static Analysis Evaluate the static performance of CRIO by simulation  Quantify table size vs. path length tradeoff  Simulated both Global Internet and VPN Based on actual Internet topology & ISP traffic matrices  Simulation tool: C-BGP

CRIO Evaluation: Data Collection Global Internet  Topology: POP-level from RocketFuel 23 Tier-1 ISP, 1219 POPs, 4159 inter-POP links  Mappings: Derived mappings from RocketFuel raw traces  Internet Traffic Matrices: Prefix-level, across all POP in our topology Use Netflow records from Tier-1 ISP backbone VPN  Same data is collected for VPN from a large VPN provider and one of its national-sized customers

CRIO Evaluation: Forwarding Table Content Customer Site CE /24 PE2 PE1 PE3 Prefix TE Source PE BGP PE BGP /24 PE2 Mapping Prefix TE Source PE BGP /24 PE2 Mapping Routing Adv / / BGP Prefix TE Source / BGP Direct Entries Virtual Prefix Entries “Extra” Path-Shrinking Entries

CRIO Evaluation: Virtual Prefix Placement Policy Customer Site CE /24 PE2 PE1 PE3 Routing Adv /8 Provider 1 PE4 Provider 2 Inter-ISP (Random) Intra-ISP Intra-ISP shortest customer path Prefix TE Source PE BGP /24 PE2 Mapping Prefix TE Source PE BGP / BGP Prefix TE Source PE BGP / BGP Prefix TE Source PE BGP /24 PE2 Mapping Routing Adv /8 Prefix TE Source / BGP

CRIO Global Internet Simulation Results Path-length vs. Table-size Trade-off Virtual Prefix does increase the path length Average path length converges quickly as the # path-shrinking entries increases Reduce FIB size by 3-5 times with very little path length penalty 99% Traffic uses shortest path Increase the percentage of shortest path traffic by increasing # of Path-Shrinking Entries

CRIO VPN Simulation Results (One Customer) Hub-Spoke nature of VPN traffic exploits the tradeoff better Reduce table size by times with very little path length penalty Cumulative Distribution of PE Routers PE Routers In Hub Sites

Conclusions CRIO is a new routing architecture, aimed to provide  Scalable and stable core routing Reduce BGP RIB by two order of magnitude  FIB size reduction Reduce FIB by one order of magnitude for global Internet, 10-20x for VPN

Future Work Design and implement the mapping distribution infrastructure Study the dynamics aspect of CRIO Study the security aspect of CRIO Explore the use of CRIO to provide traffic engineering for multi-homed site Address (??) new management challenges