NetLord: A Scalable Multi-tenant Network Architecture for Virtualized Cloud Datacenters Jayaram Mudigonda Praveen Yalagandula, Jeff Mogul, Bryan Stiekes, Yanick Pouffary Hewlett-Packard Company

The Goal Build the right network for a cloud datacenter? 12 November 201812 November 2018

The Goal Build the right network for a cloud datacenter? 12 November 201812 November 2018

Cloud Datacenter Provides Infrastructure as a Service Virtualized Shared across multiple tenants Virtualized Tenants run Virtual Machines (VMs) 12 November 201812 November 2018

The Right Network Virtualization + Multi-tenancy Scale Inexpensive A Virtual Network to each tenant Flexible abstraction: No restrictions on addressing or protocols Scale Tenants, Servers: 10s of 1000s, VMs: 100s of 1000s Adequate bandwidth Inexpensive Capital: Cheap COTS components Operational: Ease of management, High bisection BW 12 November 201812 November 2018

The Challenge Basic COTS switching gear  Limited functionality Limited resources: Not enough Forwarding Information Base (FIB) space COTS Switches + Bisection BW  Multipathing Multipathig and/or Scale  More switch resources 12 November 201812 November 2018

The Challenge Assume no switch support and conserve switch resources (FIB) to Simultaneously achieve: Scale, Multi-tenancy, Bisection BW, and Ease of config 12 November 201812 November 2018

State of the Art – Scale Most prior work is limited by one or more of: New protocols Modified control and/or data planes Preferred topologies Resources (such as table space) on switches 12 November 201812 November 2018

State of the Art – Multi-tenancy Traditional VLANs Single tenant No L3 abstraction Cannot scale beyond 4K Commercial: Amazon EC2 and Microsoft Azure Most of recent work is on segregation not virtualization Diverter [WREN 2010] Carves a SINGLE IP address space among tenants Restricts: VM placements and IP address assignment Can be limited by the FIB space in switches 12 November 201812 November 2018

NetLord An encapsulation scheme and A complementary switch configuration  Scalable multi-tenancy Ease of configuration Order of magnitude reduction in FIB requirements High bisection BW 12 November 201812 November 2018

Why Encapsulate? Unmodified VM packets onto the network Excessive FIB pressure and no L3 abstraction Alternative: Rewrite headers Cannot assume L3 header  Rewrite only MAC hdr Rewrite with server MACs  Somewhat reduced FIB Cannot identify the right VM on destination Server 12 November 201812 November 2018

NetLord Encapsulation Two headers: a MAC and an IP Reduced FIB pressure Outer Src MAC = MAC of the Src edge switch Outer Dst MAC = MAC of the Dst edge switch Correct delivery Right edge switch: The outer MAC header Right server: Right port # in the outer dest IP addr Right VM: Tenant-ID frm outer dest IP + Inner dest MAC Clean abstraction No assumptions about VM protocols and/or addressing 12 November 201812 November 2018

NetLord Encapsulation VM1 Tenant 15 Hypervisor Server 1 ES-MAC1 P1 NetLord Agent Hypervisor Server 2 ES-MAC2 P2 NetLord Agent VM2 Tenant 15 Port# TenantID 7 bits 24 bits IP Address Format IP-Range-2 COTS Ethernet ES-MAC2 ES-MAC1 VM1  VM2 P1.0.0.15 P2.0.0.15 ES-MAC1  ES-MAC2 12 November 201812 November 2018

Switch Configuration Outer MAC hdr takes pkt to egress edge switch A switch on MAC Pkt addressed to itself Strips MAC hdr and forwards based on IP hdr inside Standard behavior Correct forwarding  Configure the L3 forwarding tables right Make sure to match the server configs 12 November 201812 November 2018

… … Switch Configuration 1.*.*.* 1.0.0.1 2.*.*.* 2.0.0.1 48.*.*.* Hypervisor Server 1 1.*.*.* 1.0.0.1 NLA Hypervisor Server 2 NLA Hypervisor Server 48 NLA … 2.*.*.* 2.0.0.1 48.*.*.* 48.0.0.1 Prefix Gateway, port 1.*.*.* 1.0.0.1, 1 2.*.*.* 2.0.0.1, 2 … 48.*.*.* 48.0.0.1, 48 … 2 1 48 12 November 201812 November 2018

NL-ARP Builds and maintains the mapping <VM-MAC,Subnet,Tenant-ID>  <SWITCH-MAC,PORT> Query – response. Similar to regular ARP NL-Agent broadcasts on VM boot-up and migration 12 November 201812 November 2018 HP Confidential 12/11/18

Goals Achieved: Abstraction No assumptions about VM-to-VM pkt contents  No restrictions on addressing schemes No restrictions on L3 protocols 12 November 201812 November 2018

Goals Achieved: Scale 24-bit tenant ID  Millions of tenants Order of magnitude reduction in FIB requirements Non-edge switches see only edge-switch MACs Edge switches see their local server MACs in addition Small number of L3 table entries in edge switches One per downlink 12 November 201812 November 2018

Goals Achieved: Ease of Config Automatic Local Same across all switches  Easy to debug Does not change  Easy to debug Easy per-tenant traffic engineering in the network Tenant ID readily available in standard IP header 12 November 201812 November 2018

Bisection Bandwidth Reduced FIB pressure  Utilize SPAIN effectively J. Mudigonda, P. Yalagandula, M. Al-Fares, and J. C. Mogul. SPAIN: COTS Data-Center Ethernet for Multipathing over Arbitrary Topologies. In Proc. USENIX NSDI, 2010. 12 November 201812 November 2018

SPAIN in One Slide Multipath forwarding in container/pod datacenters Few thousand servers (or unique MAC addresses) COTS Ethernet (with VLAN support) Arbitrary topologies A central, off-line manager: Discover topology Compute paths; exploit redundancy in the topology Merge paths into a minimal set of trees Install trees as VLANs into switches End-host (NIC driver): Download VLAN maps from the manager Spread flows over VLANs (i.e., paths) 12 November 201812 November 2018

SPAIN & Extreme Scalability Critical resource: Forwarding table entries (FIB) Each MAC address appears on multiple VLANs Worst case FIB = #VLANS × #MACs Overflowing FIB  Floods and poor goodput 64K Entries  only a few 1000 unique MACs NetLord’s ability to hide MACs helps greatly 12 November 201812 November 2018

Putting It All Together VM1 Hypervisor Server 1 P1.*.*.* NetLord Agent VM2 Hypervisor Server 2 P2.*.*.* NetLord Agent VM1VM2 VM1VM2 IP1IP2 VM1VM2 IP1IP2 macAmacB VM1VM2 IP1IP2 macAmacB macB macA COTS Ethernet 12 November 201812 November 2018

Evaluation Prototype + VM emulation Analytical arguments: Max VMs : ~650 K using 48-port 64K FIB switches NL-ARP overheads: < 1% of link BW, ~0.5% of Server memory Prototype + VM emulation NLA, encaping overheads are ignorable Substantial goodput improvements 12 November 201812 November 2018

Experimental Setup: Topology 74 Servers in a 2-level fat-tree topology 12 November 201812 November 2018

Prototype, VM Emulation NLA Kernel module VM Emulator A thin module above NLA Exports a virtual device Re-writes MAC addresses Unique pair for each TCP cxn Up to 3K VMs / Servers 200K VMs in all User-level Process … Protocol Stack VM Emulator NetLord Agent NIC Driver Physical NIC Network 12 November 201812 November 2018

Experimental Setup: Workload Parallel shuffles Emulating shuffle-phase of Map-Reduce jobs Each shuffle: 74 mappers & 74 reducers, one per server Each mapper transfers 10MB of data to all reducers 12 November 201812 November 2018

Goodput Traditional SPAIN NetLord Goodput (in Gbps) Number of VMs

Flooded Packets (in Millions) Floods Flooded Packets (in Millions) Number of VMs HP Confidential 12/11/18

Summary NetLord combines simple existing primitives in a novel fashion to achieve several out-sized benefits of practical importance: Scale Multi-tenancy Ease-of-use Bisection BW 12 November 201812 November 2018

BACKUP 12 November 201812 November 2018

Putting It All Together VM1 Hypervisor Server 1 P1.*.*.* NetLord Agent VM2 Hypervisor Server 2 P2.*.*.* NetLord Agent VM1VM2 VM1VM2 IP1IP2 VM1VM2 IP1IP2 macAmacB VM1VM2 IP1IP2 macAmacB MAC-A COTS Ethernet 12 November 201812 November 2018