SecondNet: A Data Center Network Virtualization Architecture with Bandwidth Guarantees Chuanxiong Guo 1, Guohan Lu 1, Helen J. Wang 2, Shuang Yang 3, Chao Kong 4, Peng Sun 5, Wenfei Wu 6, Yongguang Zhang 1 1 Microsoft Research Asia, 2 Microsoft Research Redmond, 3 Stanford University 4 Huawei Technologies, 5 Princeton University, 6 University of Wisconsin-Madison Dec 2, 2010 Philadelphia, USA 1

Outline Background VDC abstraction and service model SecondNet architecture Port-Switching based Source Routing VDC allocation Experimental results Related work Conclusion 2

Background Network virtualization with bandwidth guarantee 3 Source: Microsoft.

SecondNet VDC Fat-tree VDC VDC 1 VDC VDC n VL2 DCell BCube Others DCN Virtualization DCN Infrastructures Topology updates VM, switch, server mgt VDC VDC 0 IP packet PSSR packet Data IF Control IF Data IF

SecondNet VDC Fat-tree VDC VDC 1 VDC VDC n VL2 DCell BCube Others DCN Virtualization DCN Infrastructures Topology updates VM, switch, server mgt VDC VDC 0 IP packet PSSR packet Control IF Data IF Control IF vm 0 vm 1 vm 2 vm 3 vm 4 Virtual Data Center (VDC) – A set of VMs plus a SLA – Every VDC has its own (private) IP address space Service model – Best-effort – Type-1: local egress/ingress bandwidth guarantee – Type-0: bandwidth guarantee between any two VMs 500Mb/s

Challenges in VDC bandwidth guarantee Timely and efficient VDC allocation and expansion – NP-hard problem Scalable VDC state maintenance – State: VM to physical server mapping, bandwidth reservation, routing path – The state number can easily reach tens of millions Practical deployment – Applicable to various topologies (dcell, bcube, fat- tree, vl2) and addressing scheme – Implementable using commodity servers and switches – Failure handling 6

SecondNet Logically centralized VDC manager – Efficient and low time-complexity VDC allocation – Failure handling Put virtualization and bandwidth reservation state into servers – All the state at server hypervisors – Stateless switch-core Port-switching based source routing – Make Secondnet applicable for all network topologies – Deployable with current commodity switches 7

VDC Manager users requests stateless switches stateful servers PSSR path: VDC 0 VM 0 ->VM s0s0 s1s1 VDC 0 VM 0 VDC 0 VM 1 VDC 1 VM 0 VDC 1 VM 1 hypervisor State: v2p, band resv, pssr paths hypervisor State: v2p, band resv, pssr paths trusted domain untrusted VMs

Port switching based source routing (PSSR) Source routing – Pin routing path for bandwidth guarantee – Keep state only at server hypervisors Port-switching – Given the topology is known, port number based forwarding is possible – Simpler switching functionality PSSR Stateless switch-core Addressing agnostic Can be implement using MPLS 9

stateless switches stateful servers s0s0 s1s1 VDC 0 VM 0 (ip 0 ) VDC 0 VM 1 (ip 1 ) hypervisor0 hypervisor trusted domain untrusted VMs data ip1ip0 data ip1ip00221 vdc 0 data ip1ip00221 vdc 0 data ip1ip00221 vdc 0 data ip1ip00221 vdc 0 data ip1ip0 data ip1ip00221 vdc 0 PSSR example

VDC allocation 11 0: Cluster pre-calculation divide servers into clusters of different sizes 1: Cluster selection 2: Min-cost flow 3: Routing path

Simulation: VDC allocation time 12 BCube with 4,096 servers Fat-tree with 27,648 serversVL2 with 103,680 servers

App TCP/IP V-NIC VMSwitch NDIS NIC Driver App Root partitionChild partition TCP/IP VMNIC V2P table Port- switching WMI VMBus secondnet.sys Hyper-v mgr Policy mgr Kernel space V2P table Neigh maint recvsend User space To VDC mgr Implementation

Testbed A BCube testbed – 16 servers (Dell Precision 490 workstation with Intel 2.00GHz dualcore CPU, 4GB DRAM, 160GB disk) – 8 8-port mini-switches (DLink 8-port Gigabit switch DGS-1008D) NIC – Intel Pro/1000 PT quad-port Ethernet NIC – NetFPGA 14

Experiment: bandwidth guarantee 15 Physical topology: fat-tree VDC1 and VDC2 both have 24 VMs Each server has one VM for each VDC VDC 1 VDC 2

Related work DCN virtualization – Seawall, Netshare – VL2 – Amazon VPC, EC2 Virtual network allocation – Simulated annealing – Virtual network embedding Bandwidth guarantee – IntServ, DiffServ – VPN hose model 16

Summary VDC as abstraction and resource allocation unit SecondNet as the network virtualization layer for VDC isolation and performance guarantee – Virtualization and bandwidth guarantee state at server hypervisors – VDC manager for VDC allocation and failure handling – Port-switching based source routing for implementation Future work 17

Q&A 18