1. A Survey on Virtualization Technologies Susanta K Nanda

2. Virtualization is “HOT” Microsoft acquires Connectix Corp. Microsoft acquires Connectix Corp. EMC acquires VMware EMC acquires VMware Veritas acquires Ejascent Veritas acquires Ejascent IBM, already a pioneer IBM, already a pioneer Sun working hard on it Sun working hard on it HP picking up HP picking up  Virtualization is HOT!!!

3. Veritas/Ejascent Veritas Cluster Server Veritas Cluster Server Integrates the Ejascent’s Application Virtualization software Integrates the Ejascent’s Application Virtualization software Enables cluster server users to move data seamlessly across applications without disrupting the transaction state Enables cluster server users to move data seamlessly across applications without disrupting the transaction state

4. Virtualization: What is it, really? Real vs. Virtual Real vs. Virtual Similar essence, effect Similar essence, effect ”Formally” different ”Formally” different A framework that combines or divides [computing] resources to present a transparent view of one or more environments A framework that combines or divides [computing] resources to present a transparent view of one or more environments Hardware/software partitioning (or aggregation)‏ Hardware/software partitioning (or aggregation)‏ Partial or complete machine simulation Partial or complete machine simulation Emulation (again, can be partial or complete) Emulation (again, can be partial or complete) Time-sharing (in fact, sharing in general)‏ Time-sharing (in fact, sharing in general)‏ In general, can be M-to-N mapping (M “real” resources, N “virtual” resources)‏ In general, can be M-to-N mapping (M “real” resources, N “virtual” resources)‏ Examples: VM (M-N), Grid Computing (M-1), Multitasking (1-N)‏ Examples: VM (M-N), Grid Computing (M-1), Multitasking (1-N)‏

5. Virtualization: Why? Server consolidation Server consolidation Application Consolidation Application Consolidation Sandboxing Sandboxing Multiple execution environments Multiple execution environments Virtual hardware Virtual hardware Debugging Debugging Software migration (Mobility)‏ Software migration (Mobility)‏ Appliance (software)‏ Appliance (software)‏ Testing/Quality Assurance Testing/Quality Assurance

6. Virtual Machine Implementation: Issues Only one “bare” machine interface Only one “bare” machine interface Virtualizable Architecture Virtualizable Architecture “A virtualizable architecture allows any instruction inspecting/modifying machine state to be trapped when executed in any but the most privileged mode” - Popek & Goldberg (1974)‏ X86 is not virtualizable (Vanderpool??)‏ X86 is not virtualizable (Vanderpool??)‏ Hard to optimize [from below] Hard to optimize [from below] Unused memory pages Unused memory pages Idle CPU Idle CPU Difficult to know what NOT to do Difficult to know what NOT to do Example: Page faults (VMM), System Calls (OS level)‏ Example: Page faults (VMM), System Calls (OS level)‏

7. Example X86 Instruction: STR (gets security state)‏ X86 Instruction: STR (gets security state)‏ Value retrieved has the Requester Privilege Level Value retrieved has the Requester Privilege Level Thus, behavior depends on the privilege level Thus, behavior depends on the privilege level  Problematic X86 has at least 17 such instructions X86 has at least 17 such instructions

8. HARDWARE KERNEL USER LEVEL LIBRARIES APPLICATIONS API Calls System Calls Instructions User Space Kernel Space Machines: Stacked Architecture

9. Possible Abstraction Levels Instruction Set Architecture Instruction Set Architecture Emulate the ISA in software Emulate the ISA in software Interprets, translates to host ISA (if required)‏ Interprets, translates to host ISA (if required)‏ Device abstractions implemented in software Device abstractions implemented in software Inefficient Inefficient Optimizations: Caching? Code reorganization? Optimizations: Caching? Code reorganization? Applications: Debugging, Teaching, multiple OS Applications: Debugging, Teaching, multiple OS Hardware Abstraction Layer (HAL)‏ Hardware Abstraction Layer (HAL)‏ Between “real machine” and “emulator” (maps to real hardware)‏ Between “real machine” and “emulator” (maps to real hardware)‏ Handling non-virtualizable architectures (scan, insert code?)‏ Handling non-virtualizable architectures (scan, insert code?)‏ Applications: Fast and usable, virtual hardware (in above too), consolidation, migration Applications: Fast and usable, virtual hardware (in above too), consolidation, migration

10. Possible Abstraction Levels cont’d Operating System Level Operating System Level Virtualized SysCall Interface (may be same)‏ Virtualized SysCall Interface (may be same)‏ May or may not provide all the device abstractions May or may not provide all the device abstractions Easy to manipulate (create, configure, destroy)‏ Easy to manipulate (create, configure, destroy)‏ Library (user-level API) Level Library (user-level API) Level Presents a different subsystem API to application Presents a different subsystem API to application Complex implementation, if kernel API is limited Complex implementation, if kernel API is limited User-level device drivers User-level device drivers Application (Programming Language) Level Application (Programming Language) Level Virtual architecture (ISA, registers, memory, …)‏ Virtual architecture (ISA, registers, memory, …)‏ Platform-independence (  highly portable)‏ Platform-independence (  highly portable)‏ Less control on the system (extremely high-level)‏ Less control on the system (extremely high-level)‏

11. Overall Picture ***** *******Degree of Isolation ** ******Ease of Impl ** *******Flexibility ********* *Performance PLLibraryOSHALISA (more stars are better) ‏

12. Instruction Set Architecture Level Virtualization Technologies Technologies Emulation: Translates guest ISA to native ISA Emulation: Translates guest ISA to native ISA Emulates h/w specific IN/OUT instructions to mimic a device Emulates h/w specific IN/OUT instructions to mimic a device Translation Cache: Optimizes emulation by making use of similar recent instructions Translation Cache: Optimizes emulation by making use of similar recent instructions Code rearrangement Code rearrangement Speculative scheduling (alias hardware)‏ Speculative scheduling (alias hardware)‏ Issues Issues Efficient Exception handling Efficient Exception handling Self-modifying code Self-modifying code

13. ISA Level Virtualization: Examples Bochs: Open source x86 emulator Bochs: Open source x86 emulator Emulates whole PC environment Emulates whole PC environment x86 processor and most of the hardware (VGA, disk, keyboard, mouse, …)‏ x86 processor and most of the hardware (VGA, disk, keyboard, mouse, …)‏ Custom BIOS, emulation of power-up, reboot Custom BIOS, emulation of power-up, reboot Host ISAs: x86, PowerPC, Alpha, Sun, and MIPS Host ISAs: x86, PowerPC, Alpha, Sun, and MIPS Crusoe (Transmeta)‏ Crusoe (Transmeta)‏ “Code morphing engine” – dynamic x86 emulator on VLIW processor “Code morphing engine” – dynamic x86 emulator on VLIW processor 16 MB “translation cache” 16 MB “translation cache” Shadow registers: Enables easy exception handling Shadow registers: Enables easy exception handling QEMU: QEMU: Full Implementation Full Implementation Multiple target ISAs: x86, ARM, PowerPC, Sparc Multiple target ISAs: x86, ARM, PowerPC, Sparc Supports self-modifying code Supports self-modifying code Full-software and simulated (using mmap()) MMU Full-software and simulated (using mmap()) MMU User-space only: Useful for Cross-compilation and cross-debugging User-space only: Useful for Cross-compilation and cross-debugging

14. HAL Virtualization Techniques Standalone vs. Hosted Standalone vs. Hosted Drivers Drivers Host and VMM worlds Host and VMM worlds I/O I/O Protection Rings Protection Rings Multilevel privilege domains Multilevel privilege domains Handling “silent” fails Handling “silent” fails Scan code and insert/replace artificial traps Scan code and insert/replace artificial traps Cache results to optimize Cache results to optimize

16. VMware Architecture

17. VMware: I/O Virtualization VMM does not have access to I/O VMM does not have access to I/O I/O in “host world” I/O in “host world” Low level I/O instructions (issued by guest OS) are merged to high-level I/O system calls Low level I/O instructions (issued by guest OS) are merged to high-level I/O system calls VM Application executes I/O SysCalls VM Application executes I/O SysCalls VM Driver works as the communication link between VMM and VM Application VM Driver works as the communication link between VMM and VM Application World switch needs to “save” and “restore” machine state World switch needs to “save” and “restore” machine state Additional techniques to increase efficiency Additional techniques to increase efficiency

19. Paravirtualization Traditional architectures do not scale Traditional architectures do not scale Interrupt handling Interrupt handling Memory management Memory management World switching World switching Virtualized architecture interface Virtualized architecture interface Much simpler architectural interface Much simpler architectural interface Virtual I/O and CPU instructions, registers, … Virtual I/O and CPU instructions, registers, … Portability is lost Portability is lost

20. Examples Denali Denali Simpler customized OS with no VM for network applications Simpler customized OS with no VM for network applications Xen Xen Simpler port to commercial OS Simpler port to commercial OS Exposes some “real” hardware, e.g. clock, physical memory address Exposes some “real” hardware, e.g. clock, physical memory address

21. OS Level Virtualization Containers (operating environments) on top of OS Containers (operating environments) on top of OS Processes, File System, Network resource (IP address), Environment variables, System call interface Processes, File System, Network resource (IP address), Environment variables, System call interface Technologies Technologies chroot(): File system virtualization on Unix chroot(): File system virtualization on Unix Name spaces: Each container is tagged and new entities (fork()) generated from a container remains inside Name spaces: Each container is tagged and new entities (fork()) generated from a container remains inside System call interposition: The only interface with user space, can modify parameters, return values (to expose a different environment) System call interposition: The only interface with user space, can modify parameters, return values (to expose a different environment) Copy-on-write: Enables sharing of files Copy-on-write: Enables sharing of files Applications: Sandboxing, Fine grain access control (root in the container)‏ Applications: Sandboxing, Fine grain access control (root in the container)‏

22. Ensim’s VPS

23. OS Level Virtualization: Examples Jail Jail FreeBSD based virtualization using “chroot()” FreeBSD based virtualization using “chroot()” Scope is limited to the jail Scope is limited to the jail Curtailed access to resources and operations Curtailed access to resources and operations Signals, debugger, IP spoofing, system calls Signals, debugger, IP spoofing, system calls A file-system sub-tree, one IP address, one “root” A file-system sub-tree, one IP address, one “root” Ensim’s “Virtual Private Server” Ensim’s “Virtual Private Server” Supports virtual “boot”, per-VM resource limits Supports virtual “boot”, per-VM resource limits Virtual /proc, IP address-space Virtual /proc, IP address-space Linux “Virtual Environment” (VE)‏ Linux “Virtual Environment” (VE)‏ Tagged VE (VE-id), policy support for the rights of “root” Tagged VE (VE-id), policy support for the rights of “root”

24. Library Level Virtualization Technologies Technologies API interception through DLL hooking API interception through DLL hooking Partial/complete implementation of APIs Partial/complete implementation of APIs Emulate low level kernel implementations in user-space Emulate low level kernel implementations in user-space Useful when the host OS does not provide required support (e.g. Win32 threads vs. pthreads)‏ Useful when the host OS does not provide required support (e.g. Win32 threads vs. pthreads)‏ Mandatory drivers Mandatory drivers Examples Examples WINE: Win32 API implementation on Unix/X WINE: Win32 API implementation on Unix/X POSIX, OS/2 subsystems on Windows POSIX, OS/2 subsystems on Windows Supports Unix and OS/2 like API Supports Unix and OS/2 like API LxRun: Linux API implementation on SCO UnixWare, Solaris LxRun: Linux API implementation on SCO UnixWare, Solaris WABI: Sun’s implementation similar to WINE (not extensive)‏ WABI: Sun’s implementation similar to WINE (not extensive)‏

25. Low-Level Drivers Win9x Kernel Kernel32.DLL Gdi32.DLL, User32.DLL,… Windows DLLs Applications Low-Level Drivers NT Kernel & Executive NTDLL.DLL Kernel32.DLL User32.DLL, … Windows DLLs Executables POSIX, OS/2 Subsystem Windows Architecture

26. Wine Architecture Closely follows NT Closely follows NT Implements all the “core” DLLs (ntdll, user32, kernel32)‏ Implements all the “core” DLLs (ntdll, user32, kernel32)‏ Wine server provides the NT backbone Wine server provides the NT backbone Message passing Message passing Synchronization Synchronization Object handles Object handles Native DLL support for non-core libraries Native DLL support for non-core libraries Hardware access through Unix device drivers Hardware access through Unix device drivers

27. WINE Implementation Wine server Wine server IPC through Unix sockets and shared message queues IPC through Unix sockets and shared message queues Process/Thread management Process/Thread management Simulates Synchronization primitives Simulates Synchronization primitives Native vs. Built-in DLLs Native vs. Built-in DLLs DLLs are implemented as Unix shared libraries (built-in DLLs)‏ DLLs are implemented as Unix shared libraries (built-in DLLs)‏ Supports non-core Windows DLLs (Native DLLs)‏ Supports non-core Windows DLLs (Native DLLs)‏ A fully implemented built-in DLL takes precedence over native DLLs A fully implemented built-in DLL takes precedence over native DLLs Executable Load Executable Load DLL descriptors table maintain the list of loaded DLLs DLL descriptors table maintain the list of loaded DLLs Imports are resolved using DLL descriptor table or on-disk DLLs Imports are resolved using DLL descriptor table or on-disk DLLs Processes/Threads Processes/Threads Windows processes are mapped to WINE/UNIX processes Windows processes are mapped to WINE/UNIX processes Thread-related APIs implemented in user-space and using pthreads Thread-related APIs implemented in user-space and using pthreads

28. Application Level Virtualization Java Virtual Machine (JVM)‏ Java Virtual Machine (JVM)‏ Executes Java byte code (virtual instructions)‏ Executes Java byte code (virtual instructions)‏ Provides the implementation for the instruction set interpreter (or JIT compiler)‏ Provides the implementation for the instruction set interpreter (or JIT compiler)‏ Provides code verification, SEH, garbage collection Provides code verification, SEH, garbage collection Hardware access through underlying OS Hardware access through underlying OS JVM Architecture JVM Architecture Stack-based architecture Stack-based architecture No MMU No MMU Virtual hardware: PC, register-set, heap, method (code) areas Virtual hardware: PC, register-set, heap, method (code) areas Rich instruction set Rich instruction set Direct object manipulation, type conversion, exception throws Direct object manipulation, type conversion, exception throws Provides a runtime environment through JRE Provides a runtime environment through JRE Other Examples:.NET CLI, Parrot (PERL 6)‏ Other Examples:.NET CLI, Parrot (PERL 6)‏

29. Featherweight Virtual Machine (FVM)‏ Motivation Motivation “Trying out” un-trusted programs in a realistic setting “Trying out” un-trusted programs in a realistic setting System Inconsistencies due to System Inconsistencies due to New application installations New application installations Accidental deletion of critical system files Accidental deletion of critical system files through application uninstall or human error through application uninstall or human error System damages due to Virus System damages due to Virus Hard to undo changes made to the System Hard to undo changes made to the System Requirements Requirements Isolation Isolation Easy Manipulation: Create, Destroy, Suspend, Resume Easy Manipulation: Create, Destroy, Suspend, Resume Persistence across reboots Persistence across reboots Processes are killed Processes are killed Other states need to be saved Other states need to be saved Flexibility: Interface to configure a machine’s visibility Flexibility: Interface to configure a machine’s visibility

30. FVM: Architecture Virtualization at the OS Level Virtualization at the OS Level Name-space Name-space Renaming at the System call interface Renaming at the System call interface Each VM starts with a similar environment as the host machine Each VM starts with a similar environment as the host machine VM states VM states VM-ID, IP address, Processes VM-ID, IP address, Processes Logs for deleted registry-entries and files Logs for deleted registry-entries and files Visibility options Visibility options Operations Operations Create/Delete, Suspend/Resume, Copy, Commit, Configure Create/Delete, Suspend/Resume, Copy, Commit, Configure

32. Implementation Registry Registry Registry access prefixed with the FVM’s repository key and the VM-ID along with COW Registry access prefixed with the FVM’s repository key and the VM-ID along with COW Example: \HKCU\X  \FVMRep\VM1\HKCU\X Example: \HKCU\X  \FVMRep\VM1\HKCU\X File System: Similar to registry File System: Similar to registry Processes Processes First process created through CreateVM()‏ First process created through CreateVM()‏ Child belongs to the same VM as parent’s (services?)‏ Child belongs to the same VM as parent’s (services?)‏ Services and SCM Services and SCM Objects Objects Network Network

33. FVM: Applications Secure mobile code execution Secure mobile code execution Automatic clean uninstall Automatic clean uninstall Memory Stick based mobile computing Memory Stick based mobile computing

35. Virtualizations Uncovered Display virtualization (Terminal Service)‏ Display virtualization (Terminal Service)‏ Network stack virtualization Network stack virtualization Grid-computing Grid-computing And many more And many more