Budapesti Műszaki és Gazdaságtudományi Egyetem Méréstechnika és Információs Rendszerek Tanszék Virtualization Zoltan Micskei Operating system (vimia219)

Virtualization  Central concept in computers  Virtualization: hiding the actual parameters of a resource from its users, e.g. o presenting a resource as separate logical ones, o presenting separate resources as one logical…  Virtual memory, virtual filesystem… 2

The virtualization buzzword Misused termonology Computer virtualization Packaged applications Thin clients Desktop products Hiding the storage structure + OS level virtualization: separate OS containers + OS level virtualization: separate OS containers Dymamic management, lifecycle, templates… 3

* taxonomy ~ structure for presenting relationships between concepts Virtual machine taxonomy* Source: J. Smith and Ravi Nair, “The architecture of virtual machines,” IEEE Computer, vol. 38, 2005, pp VM sees an ABI VM sees a hardware Multiprog- rammed OS Java,.NET… 4

Virtual machine taxonomy (detailed) 5

Platform virtualization  Platform virtualization: virtualizing a full computer, running multiple OS on one hardware o Also known as: server, computer, hardware virtualization..  Concepts: o Host machine = physical computer o Guest machine = virtual computer o Virtual Machine Monitor (VMM): program managing the virtual machines 6

History of platform virtualization  ~ IBM CP-40 system o in the mainframe products  x86 virtualization o Seemed impossible o 1997: Stanford, Disco projects o 1998: VMware solution o Other solutions  Now: o has its own business o becomes commodity 7

Why is platform virtualization good?  Building test systems  HW consolidation  Legacy systems  On-demand architectures  High availability, disaster recovery  Portable applications  … 8

Platform virtualization  Two approaches: Hardware OS Virt. SW App. OS App. Hardware Virt. SW Management OS Management App. OS App. HostedBare-metal Mainly desktop productsL: VMware Workstation, Server, Player, Oracle VirtualBox, MS VirtualPC, KVM, UML Mainly server products: VMware ESX Server, Xen Enterprise, MS Hyper-V Mainly server products: VMware ESX Server, Xen Enterprise, MS Hyper-V HOST GUEST Neve: VMM – Virtual Machine Monitor Hypervisor Neve: VMM – Virtual Machine Monitor Hypervisor Main component: VMM – Virtual Machine Monitor Main component: VMM – Virtual Machine Monitor 9

Use case: mobil virtualization 10

Theoretical background

Requirements Requirements for a virtualization solution:  Equivalence: programs in a VM should perform indistinguishable from running on the hardware  Resource control: the VMM should handle all the physical resources  Efficiency: most of the VM’s instructions should run directly on the hardware Gerald J. Popek, Robert P. Goldberg: Formal Requirements for Virtualizable Third Generation Architectures. Commun. ACM 17(7): (1974) 12

Main problem  The system must be protected from the guests  E.g.: HLT (Halt) instruction o Desirable: only the VM should stop o But all VMs would stop if executed  Solution: VMM monitors the guest instructions o Privileged instructions should be handled 13

Theoretical background CPU virtualization Memory virtualization I/O virtualization

Basic methods – Full emulation HW Emulator Virtual machine Virtual HW Application State of virtual HW Pro: Different CPU can be emulated Con: Slow Pro: Different CPU can be emulated Con: Slow Full state of the virtual hardware is stored in the emulator (registers, flags) Full state of the virtual hardware is stored in the emulator (registers, flags) Every instruction is inspected by the VMM Every instruction is inspected by the VMM Instruction is applied in the emulator, transforms the instruction, executes 15

Basic methods – Trap and emulate Trap: hardware exception handling, which resumes execution after the handler HW Emulator Virtual machine Virtual HW Application State of virtual HW Non-privileged instructions are directly executed on the hardware Non-privileged instructions are directly executed on the hardware Privileged or sensitive instructions cause a trap, VMM handles it 16 HW support is required: protection modes (e.g. x86 ring) VM runs in a lower mode Privileged instructions should case a trap when called from a non-privileged mode HW support is required: protection modes (e.g. x86 ring) VM runs in a lower mode Privileged instructions should case a trap when called from a non-privileged mode

Issues with x86 virtualization  Some architectures can be easily virtualized o x86 cannot  From ~250 instructions 17 violate the classical requirements, e.g.  POPF instruction: modifies EFLAGS register o But if not executed in ring 0, doesn’t throw an exception  Privileged state can be detected o OS can detected whether it’s running in a VM 17 Conclusion: the trap & emulate method cannot be used on the original x86

Solutions for virtualizing x86  Binary translation (software)  Paravirtualization  Hardware-assisted virtualization 18

Binary translation  most of the instructions run directly  privileged instructions translated runtime  doesn’t need source  caches translated code  guest OS not aware of virtualization 19

Binary translation – example 20 vEPC mov ebx, eax cli and ebx, ~0xfff mov ebx, cr3 sti ret mov ebx, eax mov [VIF], 0 and ebx, ~0xfff mov [CO_ARG], ebx call HANDLE_CR3 mov [VIF], 1 test [INT_PEND], 1 jne call HANDLE_INTS jmp HANDLE_RET start Guest CodeTranslation Cache Source: Carl Waldspurger, Introduction to Virtual Machines

Paravirtualization  Modifying the source of the guest OS  Replacing “problematic” instructions  Hypercall: calling the VMM directly 21

Hardware-assisted virtualization  ~2005: Intel Virtualization Technology (VT-x) and AMD AMD-V  HW support: root mode, VMCS o Instructions: VMCALL, VMLAUNCH  trap & emulate now works 22

What is the best?  Answer changes constantly o Depends on the environment, workload o BT used to be more matures, but..  Most products mix several techniques Comparing Hardware Virtualization Performance Utilizing VMmark v1.1Comparing Hardware Virtualization Performance Utilizing VMmark v VMware: Paravirtalization + BT is better than pure BTVMware: Paravirtalization + BT is better than pure BT VMware: BT is better than HW assisted virtualizationBT is better than HW assisted virtualization 23

Theoretical background CPU virtualization Memory virtualization I/O virtualization

Memory virtualization (software) VM1VM2 Guest: virtual memory Guest: „physical” memory Machine: physical memory Guest page tables VM allocation page tables Instead of double translation: shadow page tables ISSUE: synchronization Instead of double translation: shadow page tables ISSUE: synchronization 25

Memory virtualization (paravirtualization)  Also uses shadow page tables  Modifying the guest OS source code  When the OS modifies it’s page tables, it should notify the VMM also 26

Memory virtualization (hardware)  HW support in the recent CPUs o AMD Rapid Virtualization Indexing, Intel Extended Page Tables  Nested page table o Storing guest physical -> machines physical translation o Traversed by HW address translation  Tagging TLB entries  Great performance increase: o , KVM: MMU paravirtualization is deadMMU paravirtualization is dead o 2009., VMware: Performance Evaluation of AMD RVI Hardware Assist, 42% improvement in some casesPerformance Evaluation of AMD RVI Hardware Assist 27

Theoretical background CPU virtualization Memory virtualization I/O virtualization

Handling I/O devices (software) VMM Virtual machine Unmodified driver Emulated virtual HW Emulating the whole real communication Guest OS HW manager I/O device 29

Handling I/O devices (paravirtualization)  Special package installed in the VM: o VMware Tools, Virtual PC Additions o Always install these! VMM Virtual machine Paravirt. driver Guest OS HW manager Simplified calls, communication through shared memory Simplified calls, communication through shared memory I/O device 30 VMM Virtual machine Unmodified driver Emulated virtual HW Guest OS HW manager I/O device

Handling I/O devices (hardware)  Hardware support o Intel VT-d, AMD IOMMU o PCI standard extensions: I/O Virtualization (IOV)  I/O devices o can be shared between VMs o can be directly assigned to one VM 31

Products and companies

Players 33

Players ESXi, vSphere… open source hypervisor XenServer, XenApp Virtual PC, Hyper-V, System Center Solaris Containers, Oracle VM, VirtualBox Kernel based Virtual Machine (KVM) mainframe, powerVM … 34

DEMO  Resource pools  VM maps  Performance graphs  Live Migration – moving VMs between hosts on the fly Centralized management 35

Cloud computing 36 ???

Cloud computing 37 Cloud

Types of cloud computing Getting a VM Amazon EC2, RackSpace… IaaS Getting a runtime environment Java container,.NET, database… MS Azure, Google AppEngine… PaaS Getting a service Google Docs, SalesForce CRM… SaaS 38

Future (?): Mirage OS 39

More information  Ole Agesen et al.: The evolution of an x86 virtual machine monitor, SIGOPS Oper. Syst. Rev. 44, 4 (December 2010)The evolution of an x86 virtual machine monitor  P. Barham et al.: Xen and the Art of Virtualization, SIGOPS Oper. Syst. Rev. 37, 5 (October 2003)Xen and the Art of Virtualization 40

Summary  Virtualization: became commodity  Conflicting terminology  Many competing vendors  Operating systems o Core functions implemented in the hypervisor o Purpose of general OS? 41