Network Services for Enhanced Cloud Computing T. V. Lakshman Bell Labs (Jointly with F. Hao, S. Mukherjee, H. Song)

© Alcatel-Lucent All rights reserved. 2 | | 2009 Network Support For Cloud Computing: Scenario 1 VM Data Center The mobile platforms … are so powerful now that you can build client apps that do magical things that are connected with the cloud … dont limit your imagination to this set of problems - Eric Schmidt, Google CEO, Oct User attaches to cloud to get service Cloud service provider creates VM to serve user Transparent VM migration across WAN without losing service continuity

© Alcatel-Lucent All rights reserved. 3 | | 2009 Network Support For Cloud Computing : Scenario 2 Data Center Sudden surge of demand Transparent VM migration across WAN to allow resource sharing VM

© Alcatel-Lucent All rights reserved. 4 | | 2009 Benefits of VM migration across multiple networks/data centers For Cloud Service Provider Service migration across data centers Load balancing within and across data centers Performance optimization Green computing For Cloud Users Faster access Efficient data delivery

© Alcatel-Lucent All rights reserved. 5 | | 2009 All traffic anchored at home agent (HA) Triangular routing increase delay and burdens the network Fine for end user device with low traffic volume, but not for servers MIPv6 requires correspondent nodes to support MIP Transition from IPv4 to IPv6?? End users will be mixture of IPv4 and IPv6 clients Why Not Use Mobile IP to Handle VM Migration? VM 2 VM 1 HA All traffic goes through the anchor point User 1 User 2

© Alcatel-Lucent All rights reserved. 6 | | 2009 Network Architecture with Central Control FE is the first layer-3 access and aggregation point for mobile VM Forwarding Element (FE) Handles all data plane functions Sets up a virtual backplane between each other as necessary Can be distributed across WAN Forwarding element or router with APIs for central control Centralized Controller (CC) Controls routing and signaling for mobile VM IP prefixes Computes and installs forwarding table for each FE Centralized Architectures – SoftRouter (Hotnets 2004), OpenFlow, RCP, 4D) Act as a loosely coupled router with FEs as line cards, and CCs as control plane

All Rights Reserved © Alcatel-Lucent Enabling Seamless Migration Of Virtual Machines Within Cloud Virtual machine (VM) location and migration should be transparent to customers Migrate VMs without losing connectivity Enable seamless migration using central controller architecture Centralized Controller (CC) registers VM location, coordinates VM movement Forwarding elements (FEs) distributed in different cloud data-centers FEs announce public VM prefixes externally from all data centers External packets first reach closest FE, then tunneled to actual destination VM location is made transparent to external network Data Center CC FE

© Alcatel-Lucent All rights reserved. 8 | | 2009 Our Approach Traditional virtualization approach Slice and isolate resources in a physical router Each slice acts as a different router Virtual router with distributed forwarding elements managed by logically centralized controller Similar in concept to [SoftRouter/Openflow/RCP/4D] Logically combine multiple physical devices to form a virtual router A physical device mimics a virtual line card with multiple virtual ports Virtual line cards are interconnected to mimic a virtual backplane Dedicated facilities (e.g.,for data centers of a cloud service provider) MPLS bandwidth-guaranteed paths Tunnels through the public Internet

© Alcatel-Lucent All rights reserved. 9 | | 2009 Routing of Packets to VMs In Centrally Controlled Architecture External Routing Mobile VM IP prefixes announced from all FEs to external network To the external network, all FEs controlled by a CE appear as one sink for all mobile VMs that it supports External routers access centrally controlled router through the FE closest to them Internal Routing VM registers with local FE CC maintains global view of all VM locations Each FE maintains a forwarding table local bindings for locally registered VMs foreign bindings for remotely registered VMs

© Alcatel-Lucent All rights reserved. 10 | | 2009 Packet Forwarding VM 2 VM 1 VM client FE receives a packet destined to an external IP address Packet is directly sent out by looking up external forwarding table Client VM Client sends packet to closest FE FE tunnels packet VMs local FE Local FE strips off tunnel header and delivers packet to VM VM VM Packets for VM with local binding are directly forwarded Packets for VM with foreign binding are forwarded to the current FE Packet discarded if no binding is found

© Alcatel-Lucent All rights reserved. 11 | | 2009 VM Migration across Data Centers 1)Start copy between old and new locations 2)VM sends an ARP 3)Local FE receives the ARP and sends the message to CC 4)CC installs new routing entry in local FE for the VM 5)CC installs new routing entry in the old FE VM Data Center 1 Data Center 2 Data Center 3 Old location New location Mobile VMs must have IP addresses that do not conflict with any other hosts in the cloud VMs with destination NAT-ed addresses are moved by allocating non-conflicting private addresses to the mobile VMs VM migration within a data center is similar in principle but simpler 1

© Alcatel-Lucent All rights reserved. 12 | | 2009 Experimental Prototype Prototype based on Linux (FC 9) All FEs are controlled by CC FE 2 and FE 3 have 4-port NetFPGA GbE card Developed new Openflow controller to support Mobile node registration Layer 3 routing VM Migration VM migrated from Server 1 to Server 2 Ping VM from Server 3 at 0.01 sec interval Packet loss = sec connectivity interruption Same downtime over LAN migration negligible overhead Physical Host Migration Mobile PC changed attachment from AP 1 to AP 2 Ping mobile PC from Server 3 at 0.01 sec interval Packet loss = sec connectivity interruption Details in ACM CCR, ACM SIGCOMM VISA workshop paper

© Alcatel-Lucent All rights reserved. 13 | | 2009 Enterprise Network Home Network Data Center Enterprise need extra computing capacity off-and-on to accommodate variation in demands Home user need extra server to support/ interact with various devices in home network Network Support For Cloud Computing: Scenario 3 Transparent cloud computing service to enable seamless integration of computing resources between cloud and user VM

© Alcatel-Lucent All rights reserved. 14 | | 2009 Transparent Cloud Computing -- Challenges Address mapping: Address space of cloud-based resources must be mapped to enterprise address space Isolation: Customers should only see their network extension in the cloud and should be isolated from other customers using the cloud Location independence: Virtual machines running customer application should movable between customer sites and anywhere in the data center Policy control: each customer can change its policy settings for the cloud resources on the fly Scalability: service scale only restricted by total resources available, not dependent on customer composition A few large enterprises vs. many small business or individual users

© Alcatel-Lucent All rights reserved. 15 | | 2009 Isolation Using VLANs Servers are partitioned into LANs or VLANs, connected by L2 switches LANs and VLANs are connected by routers VLANs can be extended across routers via VLAN trunks (tunnels) To support virtual private network for enterprise: Use VLAN to isolate customers and avoid IP address conflict VLAN trunking to expand one subnet across L3 routers L3 IPSec tunnel used between enterprise edge and data center edge VM Virtual switch in hypervisor VLAN for each customer VPN connecting user to cloud

© Alcatel-Lucent All rights reserved. 16 | | 2009 Central Control Based architecture Partition data center network into smaller domains Use VLANs to isolate customer within a domain No global VLANs VLAN ids reused across domains Use router with central control to glue different domains together FEs forward traffic between domains CC stores mapping between user and their VLANs in each domain Per-user policy control Middleboxes attached to FEs Policy routing enforced by FEs CC stores per-customer policy User can configure their policy on-the-fly

All Rights Reserved © Alcatel-Lucent Transparent Cloud Computing Using Central Control Data Center Customer site network is a special edge domain VPLS IPsec Each edge domain partitioned into different VLANs One VLAN per customer subnet VLAN id reused across domains Uses a controller (CC) in data center that controls a a set of forwarding elements (FEs) FEs resolves addresses, enforce policies, forward packets MAC-in-MAC tunnel between FEs Middleboxes (FW, LB, etc.) attached to FE Core domain transports packets between edge domains No VLAN, flat L2 network CC stores address mapping, policy rules, VM locations Same customer across multiple domains sees one logical network Virtual MAC address for each VM Mapping cloud-based resources into customer networks

© Alcatel-Lucent All rights reserved. 18 | | 2009 Network-Cloud Joint Resource Allocation

© Alcatel-Lucent All rights reserved. 19 | | 2009 Cloud Service Providers Infrastructure services, software Services, platforms …. Cloud Service Providers Infrastructure services, software Services, platforms …. User Resource Request and Allocation Cloud Broker Provide users with all the resources needed for a service (network, computation, storage ….) Cloud brokers offer this service by partnering with cloud and network providers and providing brokering services amongst the providers Network Service Providers User requirement User request Allocation

© Alcatel-Lucent All rights reserved. 20 | | 2009 Example Request Scenario 1: Online Game Networks On Demand A group wants to set up a gaming session on demand from different points of attachments into the network Request for group specifies network service needs, game service, server need Group requests gaming session set up to a broker or service provider The provider sets up a virtual private network for the group on demand The game is provisioned in cloud resident servers and plug-ins are installed on users browsers* if the user does not have the game console The resources are allocated flexibly so that new users can be easily provisioned into the session and/or sessions properties can be modified (e.g., more bandwidth, lower delay, etc) *

© Alcatel-Lucent All rights reserved. 21 | | 2009 Example Request Scenario 2: Supercomputing For The Masses* Application: Offering immense computing power to any interested party User requests computing resources at different locations User specifies locations where data resides and network needs for data access Users require network resources along with computation resources. User request sent to a broker or service provider which coordinates allocation of network and computation resources The provider allocates computing and storage resources The provider allocates network resources to move data around the computing cloud The resources are allocated flexibly so that new tasks can be easily provisioned into the system, resources taken out when not in use * * l Supercomputing for the Masses

© Alcatel-Lucent All rights reserved. 22 | | 2009 Offering Cloud Brokering With Service Provider And Cloud Resources Cloud providers and service providers partner to support service Participating cloud and service providers: Publish their resources to participants using a common language –computing, storage, networking, special services, applications, etc. Need a universal publish subscribe and manage model for specifying resources Brokering service: Matches available resources to user requests dynamically Provides a value-added service by using pricing, congestion, location, traffic information Coordinates provisioning of requested resources and presents an integrated network-IT service to users A user request specifies: Resource category, load and duration Connectivity needs and location constraints Traffic treatment …..

© Alcatel-Lucent All rights reserved. 23 | | 2009 Challenges: How To Choose Resources For Allocation To User Requests? User requirement Cloud and network resources Data Center Mapping Data Center VM Disk User requests network and cloud resources Multiple cloud providers have available resources How to choose resources using dynamic pricing, connectivity, traffic and routing needs? Network state and available resources used for deciding which resources to allocate satisfying service level requirements and best use of network

© Alcatel-Lucent All rights reserved. 24 | | 2009 Resource Mapping Challenges: Mapping one user request How to Optimally Place Allocated Virtual Resources? User input: resource and communicaiton needs Run MapReduce for computation Read from large databases at S1, S2 S S TB S TB btwn S1 and S2 8 2 TB2 anywhere map reduce web server User input Data center CA 50 VMs on one rack, 5 TB storage on same LAN 30 VMs on one rack, 3 TB storage on same LAN Two racks: one with 8 VM, one with 2 VM, 2 TB storage S1 S2 Data center NY Data center IA Connectivity between sites Allocation And Mapping User input used for optimizing choice of resources allocated to user Better performance for user application Least cost network service for user

© Alcatel-Lucent All rights reserved. 25 | | 2009 Mapping Multiple Users: Optimally Mapping Multiple Virtual Networks and Cloud Resources Into Network Physical Network Game servers E-Science Network Game Network E-science cloud network: Traffic matrix: A->C: 10Mbps, B->D: 20Mbps, … Known traffic, low burstiness Very low latency. A C D B E Game network: Traffic demand not known point- to-point. Game server location & capacity to be fixed, receiver location & capacity fixed, … QoS guaranteed session creation Low jitter and low bandwidth Storage Deploy Deplo y Compute servers

© Alcatel-Lucent All rights reserved. 26 | | 2009 Challenges …. How do you take the user request and allocate the requested resources from resources available from multiple cloud providers and connectivity providers Allocation must meet service requirements and maximize resource usage How do you provision and instantiate these networks and resources in very short times scales? How do you handle a large number of set-up and tear-downs when the requests are far more complex than connections? Keeping track of the current state of resources and updating distributedly What happens when different components requested belong to different domains? Standard description of resources offered and customer demand? Processing, memory, storage, location, bandwidth, routing, … Standardized, un-ambiguous and expressive request specification