Aled Edwards, Anna Fischer, Antonio Lain HP Labs Diverter: A New Approach to Networking Within Virtualized Infrastructures Aled Edwards, Anna Fischer, Antonio Lain HP Labs

Outline Data Center Networks for Cloud Computing Our Approach: Diverter Evaluation Future Work 18 September, 2018

Data Center Networks for Cloud Computing

Data Center Networks for Cloud Computing Goals (and Challenges!) Multi-tenancy and Security Host multiple customers on a single shared infrastructure Allow each customer to configure their own network topology to suit application needs Data and performance isolation between customers, and the utility Allow controlled and efficient inter-communication between customers if required and permitted “provide rich ecosystem of interacting services” Large scale Automation Flexibility / Programmability Performance 18 September, 2018

Data Center Networks for Cloud Computing Problems with Traditional Approaches Traditional L2 Flat network: isolation, scalability VLANs: configuration, management Encapsulation, Tunneling Explicit routing entities required, e.g. routing VMs Traditional L3 Mobility Routing bottlenecks 18 September, 2018

Our Approach: Diverter

Our Approach: Diverter High-level Overview Isolate customer resources into Cells Cell is a collection of virtual resources Cell has a single owner Each Cell can have its own virtual network topology Cells consist of several Subnets Cell owner can define network policies Security: define who can communicate with VMs QoS: define bandwidth limits for VMs 18 September, 2018

Our Approach: Diverter Virtual Network Topology Subnet C3 Globally managed virtual IP address space representing virtual network topologies IP address format: 10.<CELL>.<SUBNET>.<HOST> (for example) Subnet A1 Subnet C2 Subnet B2 Subnet A2 Subnet B1 Subnet C1 Cell A Cell C Cell B Virtual Router Virtual Router Virtual Router 18 September, 2018

Our Approach: Diverter Realisation as a Distributed Virtual Router As virtual routing functionality is distributed across all servers rather than implemented by particular, traditional routing entities, communication between any endpoints in the infrastructure always involves just a single network “hop”. Virtual routers are realised as Distributed Virtual Router implementation (“VNET”) VNET component running on each server VNET intercepts packets to/from VMs, processes them, eventually forwards them, or discards them VNET takes care of Simulating routing across subnets, or Cells Multicast/broadcast distribution Address discovery 18 September, 2018

Our Approach: Diverter How Does It Work? MAC Rewriting! VNET rewrites packets to simulate routing hop Packets are sent to / received from virtual router interface when crossing subnets Important to emulate behaviour of traditional network topology VNET uses (modified) ARP to discover physical machines hosting a particular VM VNET rewrites packets to send directly to physical machines hosting destination VM VNET rewrites packets to limit VM broadcast/multicast traffic to particular Cell/subnet 18 September, 2018

MAC Rewriting Simplified 1. Packet TX sVMAC dVMAC Physical host A Physical host B Virtual machines 7. Packet RX Direct network hop between any endpoint No virtual MACs leaking onto the physical wire sVMAC dVMAC 2. Packet intercept 6. Packet RW 3. Packet RW 4. Packet TX 5. Packet RX sPMAC dPMAC sPMAC dPMAC Physical network 18 September, 2018

Virtual Router Simulation 3. Packet TX sVMAC RVMAC Physical host A Physical host B Virtual machines 9. Packet RX RVMAC dVMAC DHCP Response with Virtual Router IP 2. ARP Request / Reply for Router IP 4. Packet intercept Virtual MACs do not leak across subnets! 8. Packet RW 5. Packet RW 6. Packet TX 7. Packet RX sPMAC dPMAC sPMAC dPMAC Physical network 18 September, 2018

Our Approach: Diverter Further Benefits Efficiency Use of multicast/unicast ARP instead of broadcast Local DHCP response generation No packet encapsulation Fast tracking of moving VMs/addresses Security Integrated network policy framework Enforcement of fine-grained packet filtering Allow frequent changes of network policies Manageability No programming of physical infrastructure required No synchronization between physical switches and servers Only rely on underlying flat L2 network Separation of concerns: network administrators vs. server administrators Communication possible with non-VNET servers No programming of explicit routing entities required No specific hardware (or hardware modifications) required 18 September, 2018

Evaluation

Traditional L2 vs. Diverter Intra-subnet vs. Inter-subnet Communication Subnet B Subnet A Subnet A Routing VM Traditional L2 Diverter Physical network 18 September, 2018

Performance Evaluation VM Network Throughput 18 September, 2018

Future Work

Future Work Direct Network I/O QoS Virtual Network Cloning Integrate with virtualization-aware HW on server-side, e.g. SR-IOV NICs, blade server networking Integration with new I/O virtualization approaches developed around KVM/Xen QoS Virtual Network Cloning Data Center Network Federation L2 Scalable Data Center Ethernet 18 September, 2018