Software Defined Networking in Apache CloudStack Chiradeep Vittal CloudStack Committer @chiradeep

Agenda Introduction to CloudStack and IAAS What is SDN Why SDN and IAAS? CloudStack’s Network Model Extensible Networking in CloudStack SDN integrations in CloudStack CloudStack’s native SDN approach Future

Build your cloud the way the world’s most successful clouds are built Apache CloudStack History Incubating in the Apache Software Foundation since April 2012 Open Source since May 2010 In production since 2009 Tons of deployments, including large-scale commercial ones Build your cloud the way the world’s most successful clouds are built Need a better slide than this

How did Amazon build its cloud? Amazon eCommerce Platform AWS API (EC2, S3, …) Amazon Orchestration Software Open Source Xen Hypervisor Networking Commodity Servers Commodity Storage

How can YOU build a cloud? Amazon eCommerce Platform Optional Portal AWS API (EC2, S3, …) CloudStack or AWS API Amazon Orchestration Software CloudStack Orchestration Software Open Source Xen Hypervisor Hypervisor (Xen/KVM/VMW/) Networking Servers Storage

SDN Definition Separation of Control Plane from the hardware performing the forwarding function Control plane is logically centralized

SDN Advantages Centralized control makes it easier to configure, troubleshoot and maintain Eliminates ‘box’ mode of configuration Enables control at a high level

Related to SDN API layer over a collection of ‘boxes’ OpenFlow API layer communicates with boxes using box-level APIs / ssh / telnet OpenFlow Standard protocol for the centralized control plane to talk to the forwarding elements. Tunnels / overlays SDN is valuable for virtual topologies Initial target of SDN implementation

Centralized control plane MySQL/NoSQL Controller Cluster API Openflow/ssh/netconf/other Boxes

Defining Cloud Computing (IAAS) Agility Re-provision complex infrastructure topologies in minutes, not days API Automate complex infrastructure tasks Virtualization Enables workload mobility and load sharing Multi-tenancy Share resources and costs

Defining Cloud Computing (IAAS) Scalability Ability to consume resources limited by budget, not by infrastructure Elasticity Scale up and down on demand Reduce need to engineer for peak load Self-service No IT assistance

Cloud Networking Requirements Agile Complex networking topologies created by non-network engineers API Language to talk with the network infrastructure layer (not CLI) Virtualization Hypervisor-level switches work together with physical infrastructure

Cloud Networking Requirements Scalability Usually means L3 in the physical infrastructure Elasticity Release resources when not in use Introduce new resources on demand Self-service Novices deploying, maintaining, troubleshooting virtual networks

IAAS + SDN – made for each other SDN enables agility API to controller enables easy changes to networks SDN works with virtualization / vSwitches Typical of most SDN controllers SDN controllers are designed for large scale SDN enables virtual networking The illusion of isolated networks on top of shared physical infrastructure

SDN issues Discovery of virtual address -> physical address mapping VxLAN = multicast GRE = programmed by control plane L3 isolation = no mapping, no discovery

SDN issues State maintenance Large number of endpoints + flows High arrival rate of new flows Needs fast and scalable storage and processing Differentiator between vendors

SDN issues L4-L7 Service insertion and orchestration How do endpoints get services such as Firewall Load balancers IDS/IPS Service levels and performance Service Chaining

Network Virtualization in IAAS Tenant 1 Virtual Network 10.1.1.0/24 Tenant 1 VM 1 10.1.1.2 Gateway address 10.1.1.1 Tenant 1 VM 2 10.1.1.3 Internet Tenant 1 VM 3 10.1.1.4 Tenant 1 VM 4 10.1.1.5 With VLAN or L2 isolation, each tenant gets a contiguous range of ips in each network they create.

Network Virtualization in IAAS Tenant 1 Virtual Network 10.1.1.0/24 Public Network Public IP address 65.37.141.11 65.37.141.36 Tenant 1 VM 1 10.1.1.2 Gateway address 10.1.1.1 Tenant 1 Edge Services Appliance(s) Tenant 1 VM 2 10.1.1.3 Internet NAT DHCP FW Tenant 1 VM 3 10.1.1.4 Tenant 1 VM 4 10.1.1.5 We can provide NAT, DHCP and FW services for example by starting a virtual appliance to provide gateway services to this network and provide the edge services. The virtual appliance has one NIC on the public VLAN and one nic on the VLAN assigned to the network.

Network Virtualization in IAAS Tenant 1 Virtual Network 10.1.1.0/24 Public Network Public IP address 65.37.141.11 65.37.141.36 Tenant 1 VM 1 10.1.1.2 Gateway address 10.1.1.1 Tenant 1 Edge Services Appliance(s) Tenant 1 VM 2 10.1.1.3 Tenant 1 Edge Services Appliance(s) Internet NAT DHCP FW Tenant 1 VM 3 10.1.1.4 Load Balancing VPN Tenant 1 VM 4 10.1.1.5 If we wanted additional services like LB and VPN, the same virtual appliance or additional appliances or hardware devices can provide services (for example, load balancer and VPN)

Network Virtualization in IAAS Tenant 1 VM 1 Tenant 1 VM 2 Tenant 1 VM 3 Tenant 1 VM 4 Public Network Tenant 1 Virtual Network 10.1.1.0/24 Gateway address 10.1.1.1 NAT DHCP FW Public IP address 65.37.141.11 65.37.141.36 10.1.1.2 10.1.1.3 10.1.1.4 10.1.1.5 Tenant 1 Edge Services Appliance(s) Tenant 2 VM 2 Tenant 2 VM 3 Tenant 2 VM 1 Tenant 2 Virtual Network 10.1.1.0/24 VPN DHCP Tenant 2 Edge Services Appliance Public IP address 65.37.141.24 65.37.141.80 Load Balancing Network Virtualization in IAAS Internet Every network created by any tenant can get its own unique set of services either by sharing hardware devices with other tenants or using dedicated appliances / devices. Each network gets its own VLAN

CloudStack Network Model Tenant 1 VM 1 Tenant 1 VM 2 Tenant 1 VM 3 Tenant 1 VM 4 Public Network Tenant 1 Virtual Network 10.1.1.0/24 Gateway address 10.1.1.1 NAT DHCP FW Public IP address 65.37.141.11 65.37.141.36 10.1.1.2 10.1.1.3 10.1.1.4 10.1.1.5 Tenant 1 Edge Services Appliance(s) Tenant 2 VM 2 Tenant 2 VM 3 Tenant 2 VM 1 Tenant 2 Virtual Network 10.1.1.0/24 VPN DHCP Tenant 2 Edge Services Appliance Public IP address 65.37.141.24 65.37.141.80 Load Balancing Map virtual networks to physical infrastructure Define and provision network services in virtual networks Manage elasticity and scale of network services

CloudStack Network Model: Network Services L2 connectivity IPAM DNS Routing ACL Firewall NAT VPN LB IDS IPS

CloudStack Network Model: Network Services Service Providers L2 connectivity IPAM DNS Routing ACL Firewall NAT VPN LB IDS IPS Virtual appliances Hardware firewalls LB appliances SDN controllers IDS /IPS appliances VRF Hypervisor

CloudStack Network Model: Network Services Service Providers Network Isolation L2 connectivity IPAM DNS Routing ACL Firewall NAT VPN LB IDS IPS Virtual appliances Hardware firewalls LB appliances SDN controllers IDS /IPS appliances VRF Hypervisor No isolation VLAN isolation Overlays L3 isolation

Service Catalog Cloud users are not exposed to the nature of the service provider Cloud operator designs a service catalog and offers them to end users. Gold = {LB + FW, using virtual appliances} Platinum = {LB + FW + VPN, using hardware appliances} Silver = {FW using virtual appliances, 10Mbps}

Service Catalog examples 10.1.1.0/24 VLAN 100 10.1.1.1 DHCP, DNS NAT Load Balancing VPN 10.1.1.2 VM 1 10.1.1.3 VM 2 10.1.1.4 VM 3 10.1.1.5 VM 4 CS Virtual Router L2 network with software appliances 65.37.141.111 65.37.141.112 10.1.1.0/24 VLAN 100 DHCP, DNS CS Virtual Router 10.1.1.112 65.37.141.112 10.1.1.2 VM 1 10.1.1.3 VM 2 10.1.1.4 VM 3 10.1.1.5 VM 4 Netscaler Load Balancer 10.1.1.1 65.37.141.111 Juniper SRX Firewall L2 network with hardware appliances NAT, VPN If some tenants require more performance than that can be offered with a virtual appliance, they can choose a network offering that is backed by more powerful hardware appliances. For example, CloudStack can orchestrate a Juniper SRX and a Citrix Netscaler device together to offer a combination of powerful firewall and load balancing services. Upgrade

Multi-tier virtual networking Internet Loadbalancer (virtual or HW) Customer Premises IPSec or SSL site-to-site VPN Virtual appliance/ Hardware Devices MPLS VLAN Web VM 1 Web VM 2 Web VM 3 Web VM 4 Web subnet 10.1.1.0/24 VLAN 101 DB Subnet 10.1.3.0/24 DB VM 1 VLAN 2724 App subnet 10.1.2.0/24 App VM 1 App VM 2 VLAN 353 Network Services IPAM DNS LB [intra] S-2-S VPN Static Routes ACLs NAT, PF FW [ingress & egress] Additionally you can connect the entire set of networks to a site-to-site VPN using ipsec or an MPLS VLAN.

Orchestration Orchestration describes the automated arrangement, coordination, and management of complex computer systems, middleware and services Wikipedia

CloudStack Architecture Hypervisor Plugins Orchestration Core Plugin Framework Network Plugins Storage Plugins Allocator Plugins

CloudStack Architecture XenServer VMWare KVM OracleVM Hypervisor Plugins Orchestration Core Plugin Framework Nicira Netscaler Brocade MidoNet Network Plugins Allocator Plugins Random User-concentrated Intel TXT Affinity

CloudStack Orchestration Hypervisor Resource 5 4 Hypervisor Plugins Orchestration Core Plugin Framework 6 Network Resource Network Plugins API API 7 API Allocator Plugins Storage Plugins 8 1 2 3 9 Storage Resource Storage Resource Allocator Plugins Allocator Plugins Physical Resources Orchestration steps can be executed in parallel or in sequence

CloudStack and SDN Physical Resources Hypervisor Resource 5 4 Hypervisor Plugins Orchestration core Plugin Framework 6 Network Resource SDN controller Network Plugins API API 7 API Allocator Plugins Storage Plugins 8 1 2 3 9 Storage Resource Storage Resource Allocator Plugins Allocator Plugins Physical Resources Network plugin is the glue that understands the SDN controller’s API

CloudStack SDN Integration Nicira NVP L2 (STT) isolation in 4.0 Source NAT / Logical Router in 4.2 BigSwitch VLAN isolation in 4.1 VNS in 4.2 Midokura L2-L4 network virtualization Coming in 4.2 CloudStack Native Tech preview (since 4.0) Requires XenServer

VM Orchestration Example Hypervisor Resource Call Hypervisor APIs Hypervisor Plugins Orchestration core Plugin Framework Network Resource SDN controller API Network Plugins API API Allocator Plugins Storage Plugins Storage Resource Storage Resource Start 3 VMs Allocator Plugins Allocator Plugins Allocate hypervisors VM 1 Host 1 Host 3 VM 2 VM 3 VR Host 2 Host 4

Built-in (native) controller Create Full Mesh of GRE tunnels (if they don't already exist) between hosts on which VMs are deployed CloudStack SDN controller programs the Open vSwitch (OVS) on XenServer to configure GRE tunnels CloudStack SDN Controller Host 1 (Pod 2) OVS Host 3 (Pod 3) VM 1 GRE Tunnel Host 2 (Pod 4) OVS Host 4 (Pod 2) OVS VM 2 VM 3 VR GRE Tunnel GRE Tunnel

Built-in controller Assign 'Tenant' key for isolation New tenants can share the established GRE tunnels with separate tenant keys Tenant1 Tenant2 Host 1 Host 3 VM 1 VM 1 VM 3 VR GRE Tunnel Host 2 Host 4 VM 2 VM 2 VM 3 VR GRE Tunnel GRE Tunnel

What makes it different Purpose built for IAAS Not general purpose SDN solution Proactive model Deny all flows except the ones programmed by the end-user API Scaling problem is manageable Part of CloudStack ASF project Uses Virtual Router to provide L3-L7 network services Could change

Futures AWS VPC semantics Optimize ARP & DHCP responses Support security groups, ACL Optimize ARP & DHCP responses Cross-zone networks Optimize inter-subnet routing