† Nagoya Institute of Technology, Japan. Keisuke Matsuo† Ryota Kawashima† Hiroshi Matsuo† † Nagoya Institute of Technology, Japan.

The Goals Proposal Realizing broadcast/multicast in virtual networks Reducing traffic amount in substrate networks Broadcast Multicast Virtual Network VM VM VM VM Proposal PM PM PM PM Physical Network PM : Physical Machine

Outline Backgrounds Related Work Proposal Evaluation 1 Network Virtualization 2 Related Work IP multicast in substrate networks Source-based Unicast Replication 3 Proposal 4 Evaluation

Multi-tenant Datacenter Networks Overlay-based network virtualization Each tenant can have its own virtual networks Virtual Networks Tenant A VM VM Tenant C VM VM Tenant B VM VM VM VM VM Physical Network

Broadcast on Virtual Networks Multiple VMs of different tenant run on same PMs Physical Switch Virtual Switch VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM Sender Physical Machine Generate many wasteful packets VM : Tenant A VM : The Other Tenant

Broadcast on Virtual Networks VM VM VM VM PM PM PM PM PM PM PM PM Physical Network PM : Physical Machine

B/Mcast Support on Major Services Amazon VPC, Microsoft Azure Broadcast/Multicast do not be supported VMware NSX Supporting broadcast/multicast Unicast Replication methods

Major Broadcast Protocols Physical networks use various protocols ARP RARP Gratuitous ARP DHCP Broadcast communication on Virtual Networks is not allowed Functionality of virtual networks is poor

Major Multicast-related Protocols Routing protocol OSPF Construction of routing table Providing High-Availability VRRP Heartbeat messages among servers Without IP Multicast, we can’t construct practical virtual networks

Outline Backgrounds Related Work Proposal Evaluation 1 Network Virtualization 2 Related Work IP multicast in substrate networks Source-based Unicast Replication 3 Proposal 4 Evaluation

IP Multicast in Substrate Networks VXLAN Mapping virtual networks to IP multicast groups Virtual Switch VM VM VM VM VM VM VM Multicast group A Virtual Network A VM VM VM VM VM VM VM VM VM Multicast group B Multicast group C Virtual Network C Virtual Network B Physical Machine

IP Multicast in Substrate Networks Virtual Network VM VM VM VM VM VM VM router router router PM PM PM PM PM PM PM PM Physical Network Introducing and managing multicast routers cost a lot

Source-based Unicast Replication (SUR) Duplicates the packet Replaces dest addresses Hot-spot Controller Hot-spot VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM Sender Hot-spot in both upstream and sender-side links VM : Tenant A VM : The Other Tenant

Source-based Unicast Replication Virtual Network VM VM VM VM VM VM VM Controller Hot-spot Hot-spot PM PM PM PM PM PM PM PM Physical Network

Outline Backgrounds Related Work Proposal Evaluation 1 Network Virtualization 2 Related Work IP multicast in substrate networks Source-based Unicast Replication 3 Proposal 4 Evaluation

(Broadcast, Unknown unicast and Multicast) Our Proposal TE-Cast: Topology Embedded xCast Supporting broadcast/multicast in virtual networks Reducing traffic amount in substrate networks Logically grouped virtual switches Encapsulated BUM packets with topology information On-the-fly packet forwarding on virtual switches (Broadcast, Unknown unicast and Multicast)

Proposal Overview Our proposal can reduce traffic amount Sends network topology information to the Sender-side switch Controller Group1 Group2 Group3 Creates packets containing the topology information Group4 VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM VM Sender Our proposal can reduce traffic amount in both upstream and sender-side links : Representative switch VM : Tenant A VM : The Other Tenant

Proposal Overview Virtual Network Physical Network Controller PM PM PM VM VM VM VM VM VM VM Controller PM PM PM PM PM PM PM PM Physical Network

Key Features of Our Proposal Grouping Virtual Switches Getting Topology Information Forwarding Mechanism Multicast Support

Grouping Virtual Switches Reduction of traffic amount Grouping virtual switches based on network distance Ex. server rack unit based Representative virtual switch Different switch can be elected for each flow Packet processing load can be distributed Group1 Server rack サーバ 5 サーバ1 サーバ2 サーバ3 サーバ4 Representative

Key Features of Our Proposal Grouping Virtual Switches Getting Topology Information Forwarding Mechanism Multicast Support

Getting Topology Information Using OpenFlow protocol Network topology information dl_dst = ff:ff:ff:ff:ff:ff Actions = CONTROLLER Database OFPT_PACKET_IN OFPT_FLOW_MOD OFPT_VENDOR Controller VM VM Tenant ID, IP address, MAC address, switch port number

Key Features of Our Proposal Grouping Virtual Switches Getting Topology Information Forwarding Mechanism Multicast Support

Extracts the original packet Packet Encapsulation Extracts the original packet Encapsulation Physical Network VM VM Receiver Sender Ethernet (Physical) IP Proposal UDP Physical Network Header Ethernet (Virtual) Payload Original Packet

Packet Delivery within a Group Analyzing the topology information 2 PM1 PM2 Representative Non Representative 1 3 Group1 VM1 VM2 VM3 VM4 Ethernet (Physical) IP (Physical) UDP (Physical) Proposal Ethernet (Virtual) Payload Addresses for PM2 Entry for VM1 Total length Tenant ID Type IPPM1 MACPM1 IPVM1 MACVM1 Output port Num IPPM2 MACPM2 IPVM3 MACVM3 Entry for VM3

Key Features of Our Proposal Grouping Virtual Switches Getting Topology Information Forwarding Mechanism Multicast Support

IP Multicast on Virtual Networks Multicast routers are not required Virtual Network VM Manage Multicast Group VM VM VM VM VM VM Controller PM PM PM PM PM PM PM PM Physical Network

How to Manage Multicast Groups Controllers manage multicast groups Snooping IGMP packets IGMP Join message Actions = CONTROLLER Adds this VM to the multicast group Controller OFPT_FLOW_MOD OFPT_PACKET_IN VM VM

VRRP (Multicast) Providing High-Availability function Virtual Network Virtual IP address Multicast address Providing High-Availability function Virtual Network VM VM VM VM VM Multicast VM Backup Master Controller PM PM PM PM Physical Network

OSPF (Multicast) Each router constructs routing tables Virtual Network Multicast address OSPF Virtual Network VM VM VM OSPF Virtual Router VM VM Controller PM PM PM PM Physical Network

Floodlight Controller Implementation Open vSwitch Controller Virtual switches Open vSwitch 2.3.1 ofproto Get topology information datapath Deliver packets in groups Controllers Floodlight 0.90 Management function of topology information Userspace ofproto Kernel Module VM datapath Physical NIC OpenFlow Protocol Internal Services Core Services TE-Cast Applications App. Floodlight Controller

Pros and Cons Pros Supporting Broadcast/Multicast on virtual networks ( Comparing with VXLAN ) Multicast routers are not required On-the-fly packet forwarding on virtual switches ( Comparing with the Source Unicast Replication ) Mitigating hot-spot links Cons Hop counts for packet delivery can increase The processing load of the controller increase

Outline Backgrounds Related Work Proposal Evaluation 1 Network Virtualization 2 Related Work IP multicast in substrate networks Source-based Unicast Replication 3 Proposal 4 Evaluation

Evaluation Comparing with Source Unicast Replication Packet transfer delay Packet amounts in links Physical topology & machine specifications Physical Switch H …… 1GbE network 1GbE network Floodlight Mininet OS CentOS 6.6 (2.6.32) CPU Intel® Core™ i3-530 2.93GHz Memory 16GB OS CentOS 7.0 (3.10.0) CPU Intel® Core™ i5-4400 3.10GHz Memory 16GB

Network Environment Communications on tenant A Group1 Group2 Group3 Bridge Floodlight Bridge Bridge Bridge Group1 Group2 Group3 H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 VM : Tenant A VM : The Other Tenant

Result of Packet Transfer Delay Floodlight Bridge Bridge Bridge Bridge Representative Non Representative H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 Responder Responder Sender ARP response time Pattern 1: Default Responder: Host 1 Pattern 2: Source Unicast Replication, Proposal (Representative) Via controller Pattern 3: Proposal (Non Representative) Responder: Host 5 Via controller + Representative switch [packets] Delay [ms] …… ……

Result of Packet Amounts in Links Floodlight Bridge 1 12 19 Bridge Bridge Bridge 2 3 13 14 20 21 4 5 6 7 8 9 10 11 15 16 17 18 22 23 24 25 H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 The number of BUM packets in each link Every host of tenant A transmits broadcast Pattern 1: Default Pattern 2: Source Unicast Replication Pattern 3: Proposal Relative packet amounts 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ~ 25 Our proposal reduced traffic amount in upstream and sender-side links Ratio of reduced packets SUR 20% Proposal 44% There were no wasteful packets

Conclusion Broadcast/Multicast in virtual networks Waste of the Network Bandwidth Proposal: TE-Cast (Topology Embedded xCast) Supporting Broadcast/Multicast on virtual Networks Reducing packet amounts Realizing on-the-fly virtual switches Future work A cache mechanism of topology information Reducing the network delay Distribution of processing load of the controller Using distributed controllers

Result of Packet Amounts in Links SUR Proposal Bridge 1 12 19 Bridge Bridge Bridge 2 3 13 14 20 21 4 5 6 7 8 9 10 11 15 16 17 18 22 23 24 25 H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 Relative packet amounts 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ~ 25

Result of Packet Amounts in Links SUR Proposal Bridge 1 12 19 Bridge Bridge Bridge 2 3 13 14 20 21 4 5 6 7 8 9 10 11 15 16 17 18 22 23 24 25 H1 H2 H3 H4 H5 H6 H7 H8 H9 H10 H11 H12 H13 H14 H15 H16 Relative packet amounts 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ~ 25

Create Multicast Group Supporting Multicast Controllers construct multicast groups Snooping IGMP packets Controller Create Multicast Group IGMP Join message OFPT_PACKET_IN VM VM 1 Match ! Entry 1 Entry 1 copy Entry 2 …… Entry n Entry list for Tenant A Multicast group 1