1. This document is for informational purposes only, and Tekelec reserves the right to change any aspect of the products, features or functionality described in this document without notice. Please contact Tekelec for additional information and updates. Virtualizing Operating Systems Dr. Dorgham Sisalem

2. 2 | Agenda › What is meant with Virtualization? › Why bother? › Terminology › Classical approach?  Why do not we use it? › Virtualization methods  Full Virtualization  Paravirtualization  Hardware-Assisted Virtualization › Summary › Slides under http://www.iptel.org/~dor/papers/Sisalem_VM.ppt

3. Classical Computer System Machine Operating System User Process User Process User Process Physical hardware: CPU, memory, IO devices Operating System: Mediator between user and machine Distribute resources among processes Access IO on behalf of processes Only one OS at a time Compiled only for a certain ISA User applications: Compiled only for certain OS on certain ISA Each process believes to have access to all HW resources Instruction Set Architecture (ISA)

4. Classical Virtual System Host Platform Virtual Machine Monitor (VMM) Guest OS Guest OS Guest OS Host: Physical hardware: CPU, memory, IO devices Possibly operating system VMM: Hypervisor Mediator between guest and host Distribute resources among guests Access IO on behalf of guests Works on a certain ISA Guest: Assumes certain ISA (and possibly OS) Each guest believes to have access to all HW resources and to be using its native ISA Instruction Set Architecture (ISA) ISA UPUP UPUP UPUP UPUP UPUP UPUP UPUP UPUP UPUP

5. What is a Virtual Machine Monitor?  Classic Definition (Popek and Goldberg ’74)  The VMM provides an environment for programs which is essentially identical with the original machine  Programs run in the environment show at worst only minor decrease in speed  The VMM is in control of system resources

6. Virtualization Benefits  Isolation  Fault isolation  Performance isolation  Portability  Independent of physical hardware  Enables migration of live, running VMs  Efficiency  Idle resources used by multiple users and services  Improved scalability  Add more resources for a customer

7. A short history of virtualization Seventies: Save costs Expensive Hardware Support time-sharing Multiple versions of OS IBM implements virtualization into its mainframes (VM/370) Since 1998: Save costs, clouds Virtualization to save costs by using the same system for multiple users and OS Cheap Hardware High performance Vmware, Xen, Parallels, Microsoft … offer virtualization solutions Eighties, nineties: Virtualization considered exotic technology Moderately expensive Hardware Moderate performance Support windows over mac or old games

8. What‘s in it for the developer? 8 | › Portable software must be tested on different platforms › The Different platforms will remain idle most of the time › Reduce costs, management and space needs BSDLinuxUnix VMM BSDLinuxUnix

9. What‘s in it for hosting provider? 9 | › Data centers with no Virtualization › Multiple customer applications on the same OS › No isolation › Possible security problems › Server/customer › Idle resources › With Virtualization › Each customer on a virtual machine › Reduce costs, management and space needs Customer1Customer 2Customer 3 VMM Cust 1Cust 2Cust 3

10. What‘s in it for users? 10 |  Heterogeneous applications on the user’s host  Windows application on mac  Old games  …  Offer the user a wider range of applications Visio (only for windows= Keynote (only for mac OS) Atari Host VisioKeynoteAtari

11. VM Classification 11 | VM ProcessSystem Emulation High Level Language VM (Java VM) Hosted Type 2 Bare HW (Type 1)

12. Process vs. System VM 12 | Host Platform Virtual Machine Monitor UPUP Instruction Set Architecture (ISA) Machine Operating System (Linux) User process (UP) Windows application Instruction Set Architecture (ISA) Runtime Linux application (ISA2) Runtime Linux UPUP UPUP UPUP Win XP UPUP UPUP UPUP Win Vista UPUP UPUP Process VM: Enable a single process to run on the host System VM: Enable a complete system (OS + applications) to run on the host

13. Virtualizing Software Process VM: Emulation Adapt from one OS to another Adapt from one ISA to another Virtualize the Application Binary Interface (ABI) User ISA System calls to OS Virtualization software intercepts the application’s system calls and translates them into the hosts system calls If guest and host ISA are different then translate these as well Examples: Wine 13 | Machine Operating System Guest Instruction Set Architecture (ISA) ABI

14. Host OS System VM: Hosted vs. Bare HW VMM 14 | › Hosted VM  VMM runs as a user process on top of the host OS  Easy to install An additional application  VMM can use IP device drivers of host OS  VMM scheduled as user process No control on execution and resources  Less efficient  Examples: VM Workstation. Parallels.. Machine Virtual Machine Monitor UPUP Linux UPUP UPUP UPUP Win XP UPUP UPUP Win Vista UPUP UPUP Machine Virtual Machine Monitor User Process (UP) UPUP Win XP UPUP UPUP UPUP Linux UPUP UPUP › Bare HW VM  VMM runs directly on the hardware  Need to install the machine from scratch  VMM offers the IP devices to the guests  VMM controls which guest gets which share and when  Examples: VM ESX, Qemu, Xen

15. Virtualization: What are the Problems? › Isolation: Make sure that one guest does not harm another  Only VMM can use priviliged instructions › Memory management: Each guest believes it has access to physical memory  Map memory locations of guest to actual machine memory › IO: All guests want access to disc, network, USB....  Schedule between guests and prevent one guest from disturbing others Multiple guests share the same NIC but might want to have different IP addresses 15 |

16. Computer Architecture: CPU 16 | Non-Privileged Instruction: User User ISA: Non-Priviliged CPU calculations Memory read-write in user memory space Privileged Instruction: Kernel Kernel ISA: Priviliged IO access Manage user memory space Create/destroy processes Context switch Application Machine OS Instruction Set Architecture (ISA) ABI 1.Run calculation on CPU 2.Raise system call for IO access 3.Kernel takes over and issues privileged instruction 1 2 3

17. Classical Virtualization Approach: CPU 17 | Even less Privileged Instruction: User User: Non-Privileged CPU calculations Memory read-write in user memory space Non-Privileged Instructions: Kernel Kernel: Non-Privileged IO access Manage user memory space Create/destroy processes Context switch Privileged Instructions: VMM VMM: Privileged IO access Manage Guest memory space Create/destroy guests Context switch between guests Need at lest three levels of privileges

18. VMM Classical Virtualization Approach: CPU 18 | Application Machine Guest OS ISA ABI 1.Run calculation on CPU 2.Raise system call for IO access 3.Kernel takes over and issues privileged instruction but in non-privileged mode 4.Command will fail and generate a trap 5.The VMM catches the trap and 1.Checks if the guest is allowed to do the action 2.Issues the command on behalf of the guest 1 2 3 ISA ✗ 4 5

19. Computer Architecture: Memory 19 | Logical page numbers (LPN): Process believes to have access to all memory Physical page numbers (PPN): OS maps LPN to a physical location Process 1 Process 2 LPN PPN

20. Classical Virtualization Approach: Memory 20 | Logical page numbers (LPN) Physical page numbers (PPN): Each guest believes to have access to the entire machine physical memory No longer references physical memory Machine page numbers (MPN): Map physical pages to the machine memory Process 1 Process 2 Virtual Machine 1Virtual Machine 2 VMM MPN Shadow pages PPN Shadow page numbers: Map logical pages to the machine memory Reduces the effort for a two step lookup Increases the complexity as the pages must be kept in sync with the guests

21. Virtualizing X86 › X86 use the IA32 Instruction set › Offers four levels of priviliges › Is not virtualizable using the classical approach  Some proviliged instructione when run in non-proviliged mode fail silently No trap! 21 | Ring3: User Ring2 Ring1: Guest OS Ring0: VMM

22. Full Virtualization: Processor › Emulate x86 environment to the guest OS  Interpretation Fetch one instruction at a time Decode the instruction: get the registers and memory Execute an Interpreter routine that provides the same functionality Highly inefficient ­1 instruction in OS can cause hundreds of instructions in VMM Sometimes used for process vitualization  Binary Translation Translate source binary program to target binary before/during execution ­Go through the binary code and replace problematic instructions ­Generate compiled code Higher startup costs: VMM need to understand what is going on Higher performance: number of additional instructions in the range of tens ­Optimization: »Optimize frequently used parts »Cache compiled segments 22 |

23. Full Virtualitzation: Binary Translation Introduction to Virtual Machines, Carl Waldspurger vEPC mov ebx, eax cli and ebx, ~0xfff mov ebx, cr3 sti ret mov ebx, eax mov [VIF], 0 and ebx, ~0xfff mov [CO_ARG], ebx call HANDLE_CR3 mov [VIF], 1 test [INT_PEND], 1 jne call HANDLE_INTS jmp HANDLE_RET start Guest CodeTranslation Cache

24. Full Virtualization: Memory Management › Use shadow tables for memory management  Block memory segments of Guests  Guest access to memory segments results in a trap  VMM updates shadow tables › VMM needs to keep shadow and LPM in sync 24 |

25. Full Virtualization: IO Devices › IO access  Shared devices: Network card Guest access to a virtual device is intercepted by the VMM VMM conducts the actual access Data received on the device cause an interrupt at the guest OS VMM needs to implement the device drives for all kinds of devices  Dedicated device: display, keyboard … Guest Bypasses VM: No need to virtualize the device Not really that simple as some instance needs to translate between guest and IO memory ­IO devices have their own memory ­IO devices can access machine memory directly 25 | OS Driver Device OS Driver Virtual device Device Driver Device

26. Paravirtualization › VMM offers the Guests a modified ISA  Example: XEN › Guest operating system is modified to use the VMM  In Full Virtualization problematic parts are fixed by VMM  In Paravirtualization problematic parts are fixed by developer of OS Use Hypercalls instead of the native OS calls › Memory management  Guest allocates the pages  Read access is still directly without VMM involvement  Updates to the pages afterwards is only through hypercalls  VMM validates the Guest’s write/delete instructions  Guest can apply changes in batches  Mapping from logical to physical to machine addresses is the responsibility of the Guest 26 |

27. Paravirtualization: IO Access › IO access  Add Paravirtual device drivers to the Guest  VMM and Guest domain share an IO interface XEN uses a circular buffer After placing one or more requests in the buffer the Guest informs the VMM through a hyper call about the data  No need for interrupts 27 | OS Paravirtual Driver IO Interface Driver Device

28. Paravirtualization › Advantages  Higher performance than Full virtualization  Guest still has access to some real resources (time and machine addresses) › Disadvantage  Need to modify the Guest OS ­Annoying even if <1% ­Changes are different for each OS  Paravirtual drivers must be implemented for each OS 28 |

29. Hardware Assisted Virtualization › Add an additional privilige level  Guest OS runs in non-Root level  Instructions that required Emulation or Paravirtualization cause a switch from non- root to root mode › Hardware takes over the logical-2-physical- to-Machine mapping  For every PPN access the hardware checks also the MPN  No need for shadow tables › Better support for dedicated devices  Hardware takes over the mapping of logical address to device address No need for virtual device drivers No need for emulation or paravirtualization 29 | Ring 2: Ring 1 Ring 0 (non-Root): Guest OS Ring 0 (root): VMM Ring 0 : User

30. Summary › What are the benefits?  Isolation, portability, scalability › What are the usage scenarios?  Development, data centers, heterogenous applications › What is the classical way of doing virtualization  Trap and emulate › Why doesn‘t the x86 allow for virtualization in the classical manner  Not all insructions of the IA32 cause a trap › What are the common virtualization techniques  Full virtualization: Uses Binary translation, hides the machine but is complex and can be slow  Paravirtualization: Uses Hypercalls, is faster and simpler than full virtualization but requires changes to the OS  Hardware assisted: Remove some of the limitations of Full Virtualization Remember which? 30 |

31. References › Virtual Machines: Versatile Platforms For Systems And Processes, James E. Smith, Ravi Nair, May 2005, ISBN 1-55860-910-5  “The Architecture of Virtual Machines". Smith, J. E.; Nair, R.(2005) Computer (IEEE Computer Society) “The Architecture of Virtual Machines › “Understanding Full Virtualization, Paravirtualization, and Hardware Assist” “Understanding Full Virtualization, Paravirtualization, and Hardware Assist › “A Comparison of Software and Hardware Techniques for x86 Virtualization”, K. Adams and O. Ageson. Proceedings of ASPLOS 2006, October 2006 “A Comparison of Software and Hardware Techniques for x86 Virtualization › “Xen and the Art of Virtualization”, P. Barham et al. Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), October 2003 “Xen and the Art of Virtualization › "Intel Virtualization Technology: Hardware Support for Efficient Processor Virtualization". G. Neiger et al.; Intel Intel Technology Journal 10 (3); 2006Intel Virtualization Technology: Hardware Support for Efficient Processor Virtualization › “Intel Virtualization Technology for Direct IO”, D. Abramson et al. ; Intel Technology Journal 10 (3); 2006 “Intel Virtualization Technology for Direct IO 31 |

32. 32 | Thank you !! Questions Dorgham Sisalem Director Strategic Architecture Mobile: +49 171 304 2053 E-mail: dorgham.sisalem@tekelec.com