Virtualization Trends, Challenges and Solutions 4/22/2017 10:31 AM Virtualization Trends, Challenges and Solutions Naresh Sehgal, Ph.D., MBA Lead SW Architect Enterprise Platforms and Services Division Intel Corp, Bangalore Email: naresh.k.sehgal@intel.com © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Robert X. Cringely on Computers… "If the automobile had followed the same development cycle as the computer… A Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year, killing everyone inside.” 

Hardware Virtual Machines (VMs) 4/22/2017 10:31 AM Hardware Virtual Machines (VMs) ... App App App VM0 VM1 ... ... App App App App App App Operating System A new layer of software... Guest OS0 ... Guest OS1 GFX Physical Host Hardware VM Monitor (VMM) Processors Memory Graphics Physical Host Hardware Network Storage Keyboard / Mouse Without VMs: Single OS owns all hardware resources With VMs: Multiple OSes share hardware resources Virtualization enables multiple operating systems to run on the same platform © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Would you believe ~45 - 50 years? 4/22/2017 10:31 AM How long has virtualization been around? Recent development: ~5 years A while: 10 years Older than Microsoft: 30 years A lot longer…..>40 years Would you believe ~45 - 50 years? © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Intel introduces Intel Virtualization Technology Strachey: Time Sharing in Large Fast Computers 4/22/2017 10:31 AM Virtualization Open source Xen is released MIT: Project MAC Connectix is founded Intel introduces Intel Virtualization Technology VMWare is founded Goldberg: Survey of Virtual Machines Research Microsoft acquires Connectix 1950 1960 1970 1980 1990 2000 Today IBM: M44/44X Project 1950’s IBM & MIT collaborate on the Compatible Time Sharing System (CTSS) Christopher Strachey publishes a paper titled Time Sharing in Large Fast Computers in the Int’l Conference on Information Processing 1960’s IBM works on the M44/44X Project @ the IBM Watson Research Center evaluating time sharing system concepts based on virtual machines MIT’s Project MAC begins with a focus on the design and implementation of a better time sharing system 1970’s Robert P Goldberg authors a paper titled Survey of Virtual Machines Research that describes the shortcomings of typical 3rd generation architectures and multi-programming operating systems 1988 Connectix is founded 1998 VMware is founded 1999 VMware delivers VMware Workstation 2001 VMware delivers VMware GSX Server & VMware ESX Server 2003 Microsoft acquires Connectix to offer virtualization solutions VMware offers VMware VirtualCenter with VMmotion University of Cambridge describes Xen in a paper and provides first public release 2004 Intel introduces Intel Virtualization Technology on client & server platforms 2005 IBM & MIT: Compatible Time Sharing System © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Virtualization Challenges 4/22/2017 10:31 AM Virtualization Challenges Complexity CPU virtualization requires binary translation or paravirtualization Must emulate I/O devices in software Functionality Paravirtualization may limit supported guest OSes Guest OSes “see” only simulated platform and I/O devices Reliability and Protection I/O device drivers run as part of host OS or hypervisor No protection from errant DMA that corrupts memory Performance Overheads of address translation in software Extra memory required (e.g., translated code, shadow tables) © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Processor Virtualization Guest OSes run at intended rings 3 Para- Virtualization VMn Modified OS Apps Without VT With VT VM0 Legacy OS Apps VM0 WinXP Apps VMn Linux Apps Ring 3 Ring 1 Ring 0 Ring 3 Ring 0 VMX Root Mode VM Entry VM Exit Ring Compression H/W VM Control Structure (VMCS) VMM VMM Memory and I/O Virtualization VMCS Configuration Binary Translation Cache Binary Translator 2 Binary Translation CPU0 CPUn Processors with VT-x (or VT-i) VT-x CPU0 Processors (Standard IA-32 or IPF) 1 Ring Deprivileging CPUn

..… Intel® Virtualization Technology (VT) 1st VT base SW Solutions OS App OS App OS App OS App ..… Virtual Machine Monitor Processors with Intel® Virtualization Technology and others … Intel® VT First to market with native virtualization support Broadest HW and SW ecosystem support CoreTM 2 Microarchitecture based systems Significant increase in performance and improved VT performance overall segments Mobile - Intel® Core™2 Duo Mobile Processor for Intel® Centrino® Duo Mobile Technology Desktop - Intel® Core™2 Duo Desktop Processor E6000 sequence - Server Dual-Core Intel® Xeon® Processor 5100 series Get More Done On Every Server Get More Capabilities On Client

Enables rapid deployment Today’s Uses – Servers Virtualization addresses today’s IT concerns Server Consolidation Test and Development HWn … HW0 VM1 VMn OS App … HW VM1 VMn VMM OS App VM1 OS App OS App VMM HW 10:1 in many cases Enables rapid deployment Virtualization increases server utilization, simplifies legacy software migration

Emerging Server Usage Models True “Lights Out” Datacenter Dynamic Load Balancing Disaster Recovery VM1a VM2a OS App 2 VM1b VM2b OS App 4 VM1 HW VMM OS App … VMn OS App VM1 … VMn OS App OS App 1 OS App 3 OS App VMM VMM VMM HW HW HW CPU Usage 62% CPU Usage 90% CPU Usage 30% CPU Usage 63% Upholding high-levels of business continuity Balancing utilization with head room Intel® Virtualization Technology will play an integral role on the next generation of VMMs

Intel Platform Software Emerging Business Usage Models Built-in Management Proactive Security Professional Business Platform Energy Efficient Performance Intel Platform Software

Supported by over 45 OEMs, ISVs, & IT Outsourcers vProTM Key Features Remotely Manageability - Repair down systems Securely update systems Audit powered-down PCs Prevents malicious packets from entering the OS HP OpenView Supported by over 45 OEMs, ISVs, & IT Outsourcers More details in the IDF vProTM tracks

Intel® Virtualization and Intel® vPro™ technology Uses Intel® VT for creating a separate independent hardware-based environment inside of the PC Service Partition – Allowing IT administrators to create a dedicated and tamper resistant service environment or partition where tasks can run independently and isolated from the main operating system as well as from the end user User partition - OS and application . VM0 VM1 User Partition Service Partition Stack owned and managed by IT dept… protected from users “Firewall” Application “Management” Application App0 App1 Appn Service OS (WinCE or Linux) User OS (Win2K, XP) Lightweight VMM (LWVMM) Intel® architecture Platform VT AMT Help desk or console access even when user partition is “down” Intel, the Intel logo, and Intel architecture are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

Intel® Virtualization Technology Evolution 4/22/2017 10:31 AM Intel® Virtualization Technology Evolution Standards for IO-device sharing: Multi-context I/O devices Endpoint device translation caching Under definition in the PCI-SIG* IOV Vector 3: I/O Focus PCI-SIG Hardware support for IO-device virtualization: Device DMA remapping Direct assignment of I/O devices to VMs Device-independent control over DMA VT-d Vector 2: Platform Focus Establish foundation for virtualization in the IA-32 and Itanium architectures… VT-x VT-i … followed by on-going evolution of support: Micro-architectural (e.g., lower VM switch times) Architectural (e.g., extended page tables, EPT) Vector 1: Processor Focus VMM Software Evolution Software-only VMMs Binary translation Paravirtualization Simpler and more Secure VMM through foundation of virtualizable ISAs Increasingly better CPU and I/O virtualization Performance and Functionality as I/O devices and VMMs exploit infrastructure provided by VT-x, VT-i, VT-d Past No Hardware Support Today VMM software evolution over time with hardware support *Other names and brands may be claimed as the property of others © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Options for I/O Virtualization 4/22/2017 10:31 AM Hypervisor Shared Devices I/O Services Device Drivers VM0 Guest OS and Apps VMn Hypervisor Model Shared Devices I/O Services Hypervisor Device Drivers Service VMs VMn VM0 Guest OS and Apps Guest VMs Service VM Model Assigned Devices Hypervisor VM0 Guest OS and Apps Device Drivers VMn Pass-through Model Pro: High Performance Pro: I/O Device Sharing Pro: VM Migration Con: Large Hypervisor Pro: Higher Security Pro: I/O Device Sharing Pro: VM Migration Con: Lower Performance Pro: Higher Performance Pro: Rich Device Features Con: Limited Sharing Con: VM Migration Limits VT Goal: Support all 3 Models © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

VT-d Overview VT-d provides infrastructure for I/O virtualization 4/22/2017 10:31 AM VT-d Overview VT-d provides infrastructure for I/O virtualization Defines architecture for DMA and interrupt remapping Common architecture across IA platforms Will be supported broadly across Intel® chipsets CPU DRAM South Bridge System Bus PCI Express PCI, LPC, Legacy devices, … Integrated Devices North Bridge VT-d PCIe* Root Ports *Other names and brands may be claimed as the property of others © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

How VTd works? Each VM thinks it is 0 address based 600 1000 100 200 250 350 700 Each VM thinks it is 0 address based GPA (Guest Physical Address) But mapped to a different address in the system memory HPA (Host Physical Address) VTd does the address mapping between GPA and HPA Catches any DMA attempt to cross VM memory boundary VM2 VM0 VM1 100 300 50 10 260

DMA Remapping: Hardware Overview DMA Requests Memory-resident Partitioning & Translation Structures Device Assignment Structures Address Translation Device D1 Device D2 Bus 255 Bus 0 Bus N Dev 31, Func 7 Dev P, Func 1 Dev 0, Func 0 Dev P, Func 2 4KB Page Frame 4KB Page Tables Device ID Virtual Address Length … Fault Generation DMA Remapping Engine Translation Cache Context Cache Memory Access with Host Physical Address

VT-d Applied to Hypervisor Model Shared Devices I/O Services Device Drivers VM0 Guest OS and Apps VMn Hypervisor Model Improved Reliability and Protection Hypervisor programs remap tables Errant DMA is detected by hardware and reported to hypervisor / device driver Bounce Buffer Support Limited DMA addressability in I/O devices limits access to high memory “Bounce buffer” is a software technique to copy I/O buffers into high memory VT-d eliminates need for “bounce buffer” Above equally useful for standard OSes VT-d does not require a VMM to function Pro: Higher Performance Pro: I/O Device Sharing Pro: VM Migration Con: Larger Hypervisor

VT-d Applied to Service VM Model Shared Devices I/O Services Hypervisor Device Drivers Service VMs VMn VM0 Guest OS and Apps Guest VMs Service VM Model Device Driver Deprivileging Device drivers run above hypervisor as part of a “Service OS” Guest device drivers program devices in DMA-virtual address space Service VM Forwards DMA API calls to hypervisor Hypervisor sets up DMA-virtual to host-physical translation Further Improvements in Protection Guest device driver cannot compromise hypervisor code or data Pro: High Security Pro: I/O Device Sharing Pro: VM Migration Con: Lower Performance

VT-d Applied to Pass-through Model Assigned Devices Hypervisor VM0 Guest OS and Apps Device Drivers VMn Pass-through Model Direct Device Assignment to Guest OS Guest OS directly programs physical device For legacy guests, hypervisor sets up guest- to host-physical DMA mapping For remapping aware guests, hypervisor involved in map/unmap of DMA buffers PCI-SIG I/O Virtualization Working Group Activity towards standardizing natively sharable I/O devices IOV devices provide virtual interfaces, each independently assignable to VMs Pro: Highest Performance Pro: Smaller Hypervisor Pro: Device-assisted sharing Con: VM Migration Limits

DMA Remapping: IOTLB Scaling Address Translation Services (ATS) extensions to PCIe* enable IOTLB scaling ATS endpoint implements ‘Device IOTLBs’ Device-IOTLBs can be used to improve performance E.g., Cache only static translations (e.g. command buffers) Pre-fetch translations to reduce latency Minimizes dependency on root-complex caching Support device-specific demand I/O paging *Other names and brands may be claimed as the property of others

Address Translation Services (ATS) ATS Translation Flows Device issues Translation Requests to root-complex Root-complex provides Translation Response Device caches translation locally in ‘Device IOTLB’ Devices can issue DMA with translated address Translated DMA from enabled devices bypass address translation Root Complex Translation Request Endpoint Device Remap Hardware IOTLB Translate Address Translation Response Translated DMA Request Device IOTLB DMA using Translated Address VT-d supports per-device control of ATS *Other names and brands may be claimed as the property of others

Invalidation Architecture Invalidation enforces consistency of caches Required when software updates translation structures Invalidation primitives Global, domain-selective, and page-range invalidations Support for Device-IOTLB invalidation (through ATS) Invalidation software interfaces Synchronous interface through MMIO registers Queued interface through invalidation queue

ATS Invalidations ATS Invalidation Flow Invalidation details Root-complex issues invalidation request to device Device invalidates specified mappings from Device IOTLB Device issues Invalidation response Invalidation details Invalidation request contains unique Invalidation Tag Invalidation Responses may be coalesced Endpoint Device Root Complex Remap Hardware IOTLB Device IOTLB Invalidation Request Invalidation Response Invalidate Device-IOTLB *Other names and brands may be claimed as the property of others

Mapping to VMM Software Challenges VMn Virtual Machines (VMs) … Apps Apps Apps Apps OS OS OS OS Higher-level VMM Functions: Resource Discovery / Provisioning / Scheduling / User Interface VMM (a.k.a., hypervisor) Processor Virtualization Memory Virtualization I/O Device Virtualization Ring Deprivileging Virtual CPU Configuration EPT Configuration DMA and Interrupt Remapping Configuration I/O Device Sharing VT-x VT-x2 VMDq VT-d2 PCI SIG VT-d Binary Translation Page-table Shadowing I/O DMA Remapping Interrupt Remapping I/O Device Emulation CPU0 CPU0 Storage Physical Platform Resources CPUn CPUn Network Processors Memory I/O Devices

Example 6: Virtualization overhead on Intel® experimental client VMM Example 6: Virtualization overhead on Intel® experimental client VMM* (vs. Native OS) Relatively low Virtualization overheads for client benchmark Targeting <10% overhead with improved SW techniques Further VMM SW optimization and Next generation VT features to reduce virtualization overheads * Pre beta version Source: Intel Corporation Projections and technical specifications are based on internal analysis and subject to change

Summary: A better IA platform 4/22/2017 10:31 AM Summary: A better IA platform First to Market & Massive Ecosystem Support: Choice: Broadest virtualization software support in the industry Robust: First x86 hardware assisted virtualization technology (Intel VT) Innovation: common specification = enhanced virtualization on x86 and will set the standard Flexibility: Leverage Intel® Xeon® processor-based servers widely deployed infrastructure for advanced failover and dynamic load balancing Better Platform Reliability: Critical for more applications on the same server More reliability features Proven Platform Architecture - almost 40X more IA based servers than AMD based since 19961 “Choose the right basket” Performance Headroom Intel® Xeon® processors have key performance features for virtualization: dual-core, hyper-threading, I/O, memory, and larger caches Slide Purpose: Intel based servers are the best solution for virtualization Key Messages: Strong 1st to market message with hardware assisted virtualization Broadest level of ecosystem enablement Leverage the install base of Intel-based servers (Advanced virtualization features, like VMWare’s VMotion* requires a common infrastructure (i.e. cannot mix Xeon and Opteron) set for underlying hardware) When using virtualization for server consolidation, the end user is putting more reliance (more OS/applications) on a single system…therefore the single system (basket) better have the best level of reliability 1 – source: Q4’05 IDC server Tracker, 1996-2005 total system shipped Whitepaper on Virtualization benefits: http://www.intel.com/business/bss/products/server/virtualization_wp.pdf © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Backup Q & A

Example 1: SysBench Running with VMware*’s ESX Server* Figure 1: Normalized SysBench results for the two test servers in the one, two, and four virtual machine environments. Higher numbers are better. Significant performance lead over competition *Source: Principled Technologies (PT) performance report http://www.principledtechnologies.com/clients/reports/Intel/VMSysBench0706.pdf System configuration in backup foils

Example 2: SPECjbb Running with VMware*’s ESX Server** Figure 2: Normalized SPECjbb2005 results for the two test servers in the one, two, and four virtual machine environments. Higher numbers are better. performance lead over competition * *Other names and brands may be claimed as the property of others **Source: Principled Technologies (PT) performance report Comparing Dual-Core AMD Opteron 285 with Dual-Core Intel® Xeon ® Processor 5160

Example 3: Microsoft* Virtual Server* VMM – Microsoft* Virtual Server* 2005 R2 SP1 Java JFT workload Guest OS - Windows 2003* Enterprise Edition R2 (32 bit) Benchmark - JVM BEA WebLogic x`® (build R26.0.0-188-52875-1.5.0_04-2005110-0920-linux-x86_64) Systems – HP DL385 2 AMD Opteron 2.6GHz 2x1MB Intel® Dual-Core Intel ® Xeon® Processor 3.0G SuperMicro SDP 16x1GB Up to 53% gain Source: Intel Corporation Projections and technical specifications are based on internal analysis and subject to change *Other names and brands may be claimed as property of others. System Configuration details in backup. Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of information to evaluate the performance of systems or components they are considering purchasing. For more information on performance tests and on the performance of Intel products, visit http://www.intel.com/performance/resources/limits.htm or call (U.S.) 1-800-628-8686 or 1-916-356-3104.

Example 4: Energy Efficient Performance Intel® CoreTM 2 Duo based system provide Energy Efficient Performance (EEP ) Leadership in virtualized environment Source: Intel Corporation Projections and technical specifications are based on internal analysis and subject to change

Example 5: MS VS SpecJBB 2005* Host OS Microsoft* Server 2003 X64 Enterprise Edition SP1 RTM Virtualization Microsoft* Virtual Server* R2 Beta SP1 ver. 1.1.512.0 EE Drop B1036 vmm.sys Microsoft* Virtual Machine Windows* Guest Editions ver. 13.705 Guest OS RedHat V9 2.4.20-8 kernel (32-bit) Workload SpecJBB 2005 1.18X 1.66X 1.95X Intel® XEON® SW Virtualized guest performance is 1.66x of Opteron Intel® XEON® VT performance is 1.18x of Software (no VT) Intel® XEON® Intel® XEON® VT performance is 1.95x of Opteron SW (no Pacifica) Source: Intel Corporation Projections and technical specifications are based on internal analysis and subject to change