NETWORKING SOLUTIONS FOR A SERVER VIRTUALIZATION ENVIRONMENT APRICOT 2011 Russell Cooper

2 WHAT YOU WILL GET FROM THIS SESSION 1. Talk: about challenges Server Virtualization technologies brings for the data center networks. 2. Demonstrate: standards based approach, where available, to improve the experience and economics in a virtualized environment.

3 AGENDA 1.Market Drivers 2.Limitations of legacy network 3.Solutions  Simplification  Infrastructure  Enhanced services 4.Summary

4 THE EVOLUTION OF SERVER VIRTUALIZATION Server Consolidation Guiding Principle: Improve utilization of physical resources Driver:  Power and space  Improvements in server utilization  Savings Network had no role Business Agility Guiding Principle: : Improve utilization of a pool of resources Driver:  Adapt quickly to new demands  Heightened compliance & security  Better disaster management  Cloud Based Computing Models Network has a huge role

5 LEGACY NETWORKS RESTRICT AGILITY VM2VM3 SERVER 1 NIC VM2VM3VM1 SERVER 2 NIC VM1 COMPLEX: Too Many Devices to Manage Additional virtual switches INFRASTRUCTURE: LACK OF ADDITIONAL SERVICES: POOR PERFORMANCE Multiple layers Across North-South path PROPRIETARY: Pre-standard protocols MOBILITY: North-south path Scale & scope of L2 adjacencies Across sites SECURITY: Silo’ed, unavailable across domains Intra- VM traffic MANAGEABILITY: Orchestration between the physical and virtual network

6 NETWORK SIMPLIFICATION FOR SUPPORTING SERVER VIRTUALIZATION VM2VM3 SERVER 1 NIC VM2VM3VM1 SERVER 2 NIC VM1 INFRASTRUCTURE: LACK OF ADDITIONAL SERVICES: POOR PERFORMANCE Multiple layers Across North-South path PROPRIETARY: Pre-standard protocols Interoperability Lock-in MOBILITY: North-south path Scale & scope of L2 adjacencies Across sites SECURITY: Silo’ed, unavailable across domains Intra- VM traffic MANAGEABILITY: Orchestration between the physical and virtual network HIGH PERFORMANCE INFRASTRUCTURE THAT IS: OPEN, STANDARDS BASED MOBILITY MANAGEABILITY SECURITY ENHANCED SERVICES NEEDED COMPLEX: Too Many Devices to Manage Additional virtual switches SIMPLIFICATION

7 BEFOREAFTER Fewer devices to manage: 44 -> 4 SIMPLIFICATION NETWORK DEVICE CLUSTERING

8 TECHNOLOGY APPROACHES  Facts  Simplify operations  Behaves as a single node both at L2 & L3 layers so it inherits all benefits found in L2 Table Synch approach Control Plane Unification  Facts  Distributed link aggregation (LAG) plus some L2/L3 protocols enhancements to minimize interchassis link load L2 Table Synch Multiple Devices – One Control Plane Multiple Devices – Enhanced Protocols

9 INFRASTRUCTURE THAT IS: OPEN STANDARDS BASED SIMPLIFICATION HIGH PERFORMANCE MOBILITY MANAGEABILITY SECURITY ENHANCED SERVICES NEEDED OPEN, STANDARDS BASED

10 VM2VM1 NIC VM3VM2VM1 NIC VM3VM2VM1 NIC COMMUNICATION BETWEEN THE VIRTUAL MACHINES 1.In the hypervisor vendor’s switch(e.g. VM Ware vSwitch) 2. In the NIC 3. In the existing external physical switch (VEPA) VM3

11 COMPARING VEPA AND VEB VM2VM1 NIC VM3VM2VM1 NIC VM3 Virtual Ethernet Port Aggregator (VEPA) North – South optimized Full functioned hardware switch Virtual Ethernet Port Aggregator (VEPA) North – South optimized Full functioned hardware switch Virtual Ethernet Bridge (VEB) East – West optimized Limited function software switch Virtual Ethernet Bridge (VEB) East – West optimized Limited function software switch Hypervisor/software switch Physical switch Network services in hardware Network services in software

12 COMPARISON OF OPTIONS Switching done in SoftwareHardware Customer’s Time to adopt solution Low – comes in- built with hypervisor Unknown Low - simple software upgrade Latency for switching Very Low Low vSwitchNICVEPA Industry support (standards based) NAUnknownYes Virtual switching managed by Server adminUnknown Network Admin Customers’ Cost to adopt Low – comes with hypervisor Unknown Free - software upgrade Compatibility with any existing network YesUnknownYes Feature Richness Very LowLowHigh

13 VEPA Virtual Ethernet Port Aggregator  Uses external physical network for intra- server VM to VM communication  It’s an evolving open standard IEEE 802.1Qbg / 802.1Qbh  Supported by almost all the major IT vendors  For more information 009/new-bg-thaler-par-1109.pdf ml 009/new-bg-thaler-par-1109.pdf ml VEPA brings the evolved Ethernet functionality to virtual networking VM2VM1 NIC VM3

14 TOP 3 BENEFITS OF VEPA Features & Scale Switching where it belongs – on the switches Elegant VEPA is a non-disruptive and cost-effective Open Server and hypervisor agnostic, maximum flexibility.

15 INFRASTRUCTURE THAT IS: HIGH PERFORMANCE SIMPLIFICATION OPEN, STANDARDS BASED MOBILITY MANAGEABILITY SECURITY ENHANCED SERVICES NEEDED HIGH PERFORMANCE

16 LATENCY WITH LEGACY NETWORK  Every hop adds additional latency  Increases load on uplinks  Requires VLANs to span multiple access switches to support VM migration BA

17 VIRTUALIZATION WITH CHASSIS CLUSTERING Clustered Access Switches 10x latency improvement by eliminating trip to upper layers  Single-point lookup model  Works with any Hypervisor BA

18 INFRASTRUCTURE THAT IS: MOBILITY SIMPLIFICATION OPEN, STANDARDS BASED MANAGEABILITY SECURITY ENHANCED SERVICES NEEDED HIGH PERFORMANCE MOBILITY

19 NETWORK REQUIREMENTS FOR VM MOBILITY IP network with 622 Mbps is required. The maximum latency between the two servers < 5 milliseconds (ms). Access to the IP subnet & data storage location Access from vCenter Server and vSphere Client. Same IP subnet & broadcast domain  Layer 2 adjacency  VLAN stretch

20 VM MIGRATION SCENARIOS Within Same Data Center Rack A Layer 2 domain across racks Scenario #1 Clustered Access Switches Rack A Data Centers in the same City - two different locations Layer 2 domain across fiber connected data centers Scenario #2 Clustered Access Switches Data Center Layer 2 domain across virtual private LAN Scenario #3 Clustered Access Switches Data Center VPLS Data Centers in different Cities Remember the vMotion Requirements! Bandwidth/Latency/IP Subnet/VLAN

21 Top-of-Rack / End-of- Row Clustered Switches RACK TO RACK RACK 1RACK 2  Managed as a single device  Automatic VLAN update propagation.  Sub 10us latency VM2VM5VM3 NIC VM4VM1

22 VM2VM1VM5VM4VM3 NIC VM2VM1VM5VM4VM3 NIC POD TO POD Core Clustered Chassis  Extends L2 domain across multiple Rows/Pods in a DC  Extends L2 adjacency to over 10,000 1GbE servers  Eliminates STP  Core managed as a single device VM2VM5 NIC POD NPOD 1 Clustered Access Switches VM3VM4VM1

23 ACROSS DC/CLOUDS  Extends L2 domain across DC /clouds  Allows VM Motion across locations.  VPLS can be provisioned or orchestrated using vendor tools and scripts  VLAN to VPLS mapping  DB/Storage mirroring VM2VM1VM5VM4VM3 NIC VM2VM1VM5VM4VM3 NIC VM2VM5VM4 NIC VM2VM1VM5VM4VM3 NIC VM2VM1VM5VM4VM3 NIC VM2VM1VM5VM3 NIC VM6 VPLS Over MPLS Cloud Routers with VPLS Core Switches Access Switches Routers With VPLS VM3VM4 Core Switches Access Switches VM1

24 INFRASTRUCTURE THAT IS: MANAGEABILITY SIMPLIFICATION OPEN, STANDARDS BASED SECURITY ENHANCED SERVICES NEEDED HIGH PERFORMANCE MOBILITY MANAGEABILITY

25 Network Admin Server Admin DC MANAGEABILITY CHALLENGES WITH SERVER VIRTUALIZATION 1.Blurred roles between the server and network admin. 2.No automation/ orchestration to sync-up the 2 networks. 3.VM Migration can fail. 4.Proprietary products & protocols B B A A Virtual n/w Physical n/w P P P P VM1VM2VM3 VM1VM2 A A

26 ONE STEP ORCHESTRATION 1.Clear roles and responsibilities 2.Automated orchestration between physical and virtual networks 3.Scalable solution – allows VMs to move freely 4.Open Architecture Network Admin Server Admin VM1VM2 Orchestration Tools A A A A A A A A Virtual n/w Physical n/w P P P P A A A A VM2VM3VM1

27 INFRASTRUCTURE THAT IS: SECURITY SIMPLIFICATION OPEN, STANDARDS BASED ENHANCED SERVICES NEEDED HIGH PERFORMANCE MOBILITY MANAGEABILITY SECURITY

28 VIRTUAL NETWORK SECURITY IMPLICATIONS OF VIRTUAL SERVERS PHYSICAL NETWORK ESX Host Physical Security is “Blind” to Traffic Between Virtual Machines Firewall/IPS Inspects All Traffic Between Servers HYPERVISOR VM1VM2VM3

29 APPROACHES TO SECURING VIRTUAL SERVERS: THREE METHODS 2. Agent-based Each VM has a software firewall Drawback: Significant performance implications; Huge management overhead of maintaining software and signature on 1000s of VMs ESX Host VM1VM2VM3 FW Agents HYPERVISOR 3. Kernel-based Firewall VMs can securely share VLANs Inter-VM traffic always protected High-performance from implementing firewall in the kernel Micro-segmenting capabilities ESX Host FW as Kernel Module VM1VM2VM3 HYPERVISOR 1. VLAN Segmentation ESX Host Each VM in separate VLAN Inter-VM communications must route through the firewall Drawback: Possibly complex VLAN networking HYPERVISOR VM1VM2VM3

30 Hypervisor Kernel Stateful Firewall Purpose-built virtual firewall  Secure Live-Migration (VMotion)  Security for each VM by VM ID  Fully stateful firewall Tight Integration with Virtual Platform Management, e.g. VMware vCenter Fault-Tolerant Architecture ESX Host KERNEL VF INTRODUCING THE IDEA OF A STATEFUL KERNEL FIREWALL Security Policy Management Data Center Firewall Access Switch Network Security Information And Event Management VM1VM2VM3

31 ESX Host FOLLOW-ME POLICIES Data Centre Firewall Access Switch ESX Host Access Switch  When a VM migrates, the network policies of the VM are migrated to the new server port.  Traffic between VMs still gets re-directed to the same appliance in the Services cluster  No migration of services state is required PolicyPolicy VM2VM3 VM2 KERNEL VF PolicyPolicy VM1

32 SIMPLIFCATION: Few Devices Fewer Devices to Manage SUMMARY OF SOLUTIONS FOR SERVER VIRTUALIZATION INFRASTRUCTURE: ADDITIONAL SERVICES HIGH PERFORMANCE Few layers Clustered Switches OPEN: VEPA Standards Based MOBILITY: VPLS Clustered Switch domains SECURITY: Kernel Stateful Firewalls Integration with DC FWs for follow me policies MANAGEABILITY: VEPA Orchestration Tools Routers Core Switch Clusters Data Center Firewalls Access Switch Clusters VM2VM3 SERVER 1 NIC VM2VM3VM1 SERVER 2 NIC VM1