Scalable Network Virtualization in Software-Defined Networks Author: Dmitry Drutskoy, Eric Keller, Jennifer Rexford Publisher:IEEE Internet Computing 2013 Presenter: Yuen-Shuo Li Date: 2013/04/24 因為大家多半比較不了解什麼是 SDN ，所 以我會多提一些 Background，裡面包含論文的內容和我的理解。

Background - Software-defined networking (SDN) SDN is an approach to networking in which control is decoupled from hardware and given to a software application called a controller. The administrator can shape traffic from a centralized control console without having to touch individual switches and change any network switch's rules when necessary. Essentially, this allows the administrator to use less expensive, commodity switches and have more control over network traffic flow than ever before. 有些企業用的 Switch 可以識別各種不同的 packet，然後個別處理，但這種的 switch 非常貴。 而且網管需要客製調整網路設定時，必須針對每臺交換器或路由器，逐一設定，不易快速變動網路架構來因應企業建置新系統的需求。而且透過人工逐一設定的方式也有很高的風險。 另一方面廠商各有各的網路管理技術或是網路作業系統軟體，網管軟體彼此之間難以相容，一旦企業購買某一廠牌的設備，未來就必須遷就於該廠牌的網管功能，無法選用其他廠牌的設備。  SDN網路架構為了要解決這些問題，修改了傳統網路架構的控制模式，將網路分為控制層（Control Plane）與資料層（Data Plane）， 將網路的管理權限交由控制層的控制器（Controller）軟體負責，採用集中控管的方式。  由控制器軟體統一下達指令給網路設備，網路設備則只負責於封包的傳遞。讓網管能更靈活地配置網路資源，日後網管人員只需在控制器上下達指令即可。 

Background – SDN Controller Application In SDN, a logically centralized controller manages the collection of switches through a standard interface, enabling the software to control switches from a variety of vendors. With the OpenFlow standard the controller’s interface to a hardware switch is effectively a flow table with a prioritized list of rules. Each rule consists of a pattern that matches bits of the incoming packets, and actions that specify how to handle these packets e.g. dropping the packet, sending the packet to the controller... Vendors of SDN controllers include Big Switch Networks, HP, IBM, VMWare and Juniper. 使用了 SDN，以後企業要採用那一家的 Switch 就不重要了，不需要被商家綁死，只要 Switch 支援 SDN 即可。 至於 Controller 要如何和 Switch 溝通呢？就需要定義一個標準協定來處理，最常見的就是現在很夯的 OpenFlow

Background – OpenFlow OpenFlow is a protocol that allows a server to tell network switches where to send packets. With OpenFlow, the packet-moving decisions are centralized, so that the network can be programmed independently of the individual switches and data center gear. Several established companies including IBM, Google, and HP have either fully utilized, or announced their intention to support, the OpenFlow standard. By early 2012, Google's internal network ran entirely on OpenFlow. OpenFlow 是一項通訊協定，用於控制層和資料層間建立傳輸通道，是目前實現SDN架構最主流的技術。  OpenFlow網路環境三大要素： 用來定義網路封包傳輸路徑的 OpenFlow 路由表（Flow Table） 決定網路封包流向的軟體控制器（Controller） 作為傳輸溝通用的OpenFlow協定（OpenFlow Protocol）。  很多家公司都開始使用或支援了 OpenFlow，如 IBM, Google, HP。另外在 2012 時，Google 已經宣布他所有的內部網路都在跑 OpenFlow

Background – OpenFlow Controller An OpenFlow controller is an application that manages flow control in a SDN environment. All communications between applications and devices have to go through the controller. The OpenFlow protocol connects controller software to network devices so that server software can tell switches where to send packets. The controller uses the OpenFlow protocol to configure network devices and choose the best path for application traffic. Because the network control plane is implemented in software, rather than the firmware of hardware devices, network traffic can be managed more dynamically and at a much more granular level. 網管可在 Controller 設定各項網管功能以及預先建立邏輯網路，來決定封包傳輸方式，例如經過哪些交換器，需要多少的網路頻寬，再將傳輸路徑設定成OpenFlow路由表（Flow Table）。 

Background – OpenFlow Switch An OpenFlow switch consists of three parts : Flow Table: Tells the switch how to process each data flow by associating an action with each flow table entry. Secure Channel: Connects the switch to the Controller, so commands and packets can be sent between the controller and the switch. OpenFlow Protocol: Provides an open, standardized interface for the controller to communicate with the switch. 而 OpenFlow 的 Switch 只需要實作三個部分： Flow Table, Secure Channel, OpenFlow Protocol Controller 會與 Switch 以 SSL 加密的方式溝通，確保傳送之間的安全。  使用了支援 OpenFlow 的 Switch，就可以透過支援OpenFlow技術的控制器軟體來管理。也就是說不論只用哪一家廠商的 Switch 都能由控制器統一管理，被單一網通廠商綁定的問題就可以迎刃而解了。 

Background – Network virtualization Network virtualization is the process of combining hardware and software network resources and network functionality into a single, software-based administrative entity, a virtual network. Network virtualization involves platform virtualization, often combined with resource virtualization. Network virtualization is categorized as either external, combining many networks, or parts of networks, into a virtual unit, or internal, providing network-like functionality to the software containers on a single system. Whether virtualization is internal or external depends on the implementation provided by vendors that support the technology. 基本上是將複數台網路裝置整合成一台虛擬網路裝置的一種技術。資料中心內或大型企業內的網路構成，因考慮到備援分散，存在正、副2主相同的冗長網路架構，更需要虛擬網路的機制，減少網管的負擔。 如此不僅可以減少很多網路線，讓構造簡易容易維護，當網路上的主機或通訊機器故障時，亦能很快速的切換。大幅降低導入、維護、營運的成本。但是性能上、功能上是否滿足需求，必須慎重檢討。

Background – Problem Network virtualization gives each “tenant” in a data center its own network topology and control over the flow of its traffic. SDN is a natural platform for network virtualization. Yet, supporting a large number of tenants with different topologies and controller applications raises scalability challenges.

Background – Problem There are two main performance issues with virtualization in the context of SDN. The Controller must interact with switches through a SSL channel and maintain a current view of the physical infrastructure (e.g. which switches are alive). With virtualization, any interaction between a tenant’s controller application and the physical switches must go through a mapping between the virtual and physical networks. 簡單來說，就是 tenant 量太多的話，就會有效能的議題，其中有兩個主要的問題如下

FlowN – Introduction In order to overcome these, we present FlowN. The FlowN architecture is based around two key design decisions. FlowN enables tenants to write arbitrary controller software that has full control over the address space and can target an arbitrary virtual topology. However, we use a shared controller platform rather than running a separate controller for each tenant. We make use of modern database technology for performing the mapping between virtual and physical address space. This provides a scalable solution that is easily extensible as new functionality is needed.

FlowN – Full Controller Virtualization Running a separate controller for each tenant seems like a natural way to support network virtualization. The virtualization system exchanges OpenFlow messages directly with the underlying switches, and exchanges OpenFlow messages with each tenant’s controller. Controller virtualization system

FlowN – Full Controller Virtualization Using the OpenFlow standard as the interface to the virtualization system has some advantages (e.g., tenants can select any controller platform), but introduces unnecessary overhead. Repeatedly marshalling and unmarshalling parameters in OpenFlow messages incurs extra latency. Running a complete instance of a controller for each tenant involves running a large code base which consumes extra memory. Periodically checking for liveness of the separate controllers incurs additional overhead. Controller virtualization system

FlowN – Container-Based Controller Virtualization Instead, FlowN is a modified NOX controller that can run multiple applications, each with its own address space, virtual topology, and event handlers. Rather than map OpenFlow protocol messages, FlowN maps between the NOX API calls. In essence, FlowN is a special NOX application that runs its own event handlers that call tenant- specific event handlers. Controller virtualization system FlowN Controller Application Application

FlowN – Container-Based Controller Virtualization Each tenant’s event handlers run within its own thread. While we have not incorporated any strict resource limits, CPU scheduling does provide fairness among the threads. 也就是說我們不管資源分配的問題，直接交給 OS 處理。 Controller virtualization system FlowN Controller Application Application

FlowN – Virtual Network Mapping To provide each tenant with its own address space and topology, We need to perform a mapping between virtual and physical resources. A virtual-to-physical mapping occurs when an application modifies the flow table The virtualization layer must alter the rules to uniquely identify the virtual link or virtual switch. e.g., adding a new flow rule. A physical-to virtual mapping occurs when the physical switch sends a message to the controller e.g., when a packet does not match any flow table rule. 這些 mapping 可能是一對一，也可能是一對多。一對一可能是新增一個 rule，一對多是可能發生會影響多個 tenant 的事情

FlowN – Virtual Network Mapping These mappings are based on various combinations of input parameters and output parameters. Using a custom data structure to perform these mappings can easily become unwieldy, leading to software that is difficult to maintain and extend. This custom software would need to scale across multiple physical controllers. Depending on the complexity of the mappings, a single controller machine eventually hits a limit on the number of mappings per second that it can perform.

FlowN – Mapping With a Database Instead of using an in-memory data structure with custom mapping code, FlowN uses modern database technology. Both the topology descriptions and the assignment to physical resources lend themselves directly to the relational model of a database. 不是解決資料量大和複雜度高的問題，直接採用資料庫系統，反正資料庫本來就專門解決這類的問題

FlowN – Mapping With a Database Each virtual topology is uniquely identified by some key, and consists of a number of nodes, interfaces, and links. Nodes contain the corresponding interfaces, and links connect one interface to another.

FlowN – Mapping With a Database FlowN stores mapping information in two tables. The first table stores the node assignments, mapping each virtual node to one physical node. The second table stores the path assignment, by mapping each virtual link to a set of physical links, each with a hop count number that increases in the direction of the path.

FlowN – Mapping With a Database Because many more reads than writes in this database, we can run a master database server that handles any writes to the database. Multiple slave servers are then used to replicate the state across multiple servers. Since the mappings do not change often, caching can then be utilized to optimize for mappings that frequently occur. 使用 Database 可以減少程式的複雜度外，還可以直接套用資料庫相關研究成果，應用相關的技巧。

Experiment – Environment We built a prototype of FlowN by extending the Python NOX version 1.0 OpenFlow controller [4]. The embedder populates a MySQL version 14.14 database. We implement all schemes using the InnoDB engine that running a memcached instance. We run our prototype on a virtual machine running Ubuntu 10.04 LTS given full resources of three processors of a i5-2500 CPU @ 3.30GHz, 2 GB of memory, and an SSD drive (Crucial m4 SSD 64GB). We perform tests by simulating OpenFlow network operation on another VM (running on an isolated processor with its own memory space) using a modified cbench [10] to generate packets with the correct encapsulation tags..

Experiment – test We measure the latency by measuring the time between when cbench generates a packet-in event and when cbench receives a response to the event. 如果在 virtual network 個數少的情況下，FlowVisor 會比較好，不過可以看得出來， FlowN 上升的幅度會比較小。