1. Disco Running Commodity Operating Systems on Scalable Multiprocessors

2. Topics Approach Micro-Kernel vs VMM Motivation Disco Overview Interface Virtual CPUs Virtual Physical Memory NUMA Memory Management Experiment Conclusion

3. Approach  Run multiple operating systems on a single scalable multiprocessor –With virtualization, possible to run multiple OS on a uniprocessor  Insert an additional software layer in between the hardware and operating system  Akin to the idea popular in 1970s virtual machine monitors

5. Micro-Kernel vs VMM  How Micro-Kernel and VMM are different?

6. Motivation  Poor performance of shared memory multiprocessors –Difficult to utilize the potential power of the underlying hardware  Inability to provide the expected and promised performance level  How’s Disco’s approach?

7. Disco Overview  Runs different operating systems on a multiprocessor  Turns commodity operating system into high performance software system base  Uses multithreaded shared memory program

8. Can communicate each other using TCP/IP or NFS -- even though virtual machines exist in the same shared memory, they do not think that way

9. Interface  Is there really an interface that higher level software can see?  Processors –The virtual CPUs provide the abstraction of a MIPS R10000 But does not support a complete virtualization of the kernel virtual address space –Extensions; enabling and disabling of CPU interrupts, accessing the privileged registers  Physical Memory –Provide the ability to deal with non uniform memory access time  IO Devices –Operating systems assume exclusive access to their virtualized IO devices –Disco Intercepts all communications to and from the IO devices to translate the operations

10. Virtual CPUs  Emulates the execution of the virtual CPUs by direct execution on the real CPU  Disco runs in kernel mode with full access to the hardware –MIPS has privileged mode, supervisor mode, and user mode  When a virtual machine is in control, it runs in supervisor mode –Can it execute a privileged instruction? –Cat it directly access a physical memory? –Can it access a protected address space?  To schedule a virtual CPU –Sets the registers of the real machine to those of the virtual CPU

11.  Most of the operations runs as the same hardware because of the direct execution of the real CPU  Each virtual CPU maintains a data structure like a process table entry –Maintains the saved registers and state of a virtual CPU –speed as they would on the raw

12.  The process traps caused by a page fault or system call is emulated by updating the privileged registers of the virtual processor  Uses affinity scheduling that increases data locality  How to switch among operating systems?

13.  How to switch among operating system? –Schedule the virtual CPU Set the real machine’s registers to those of the virtual machine Jump to the current PC of the virtual CPU

14. Virtual Physical Memory  Maintains physical-to machine address mapping to emulate a virtual-to physical mapping Physical Pages Machine Pages

15. Virtual machines use physical addresses starting at address zero up to the virtual machine ’ s memory size TLB miss is expensive To lessen the performance impact, maintains a second-level software TLB Disco allocates memory and schedules virtual CPU so that cache misses generated by a virtual CPU will be satisfied from local memory

16. 7 5 7 5 4 6 1 2 0 3 Machine Pages 700-799 600-699 500-599 400-499 300-399 200-299 100-199 0-99 Physical Pages TLB misses? Disco ’ s TLB miss handler An OS tries to map in the Page table VMM invoked

17. NUMA Memory Management  Allocate real memory to virtual machines (NUMA  UMA)  page migration –moves only pages that will likely results in a performance benefit –The VMM changes the physical-to-machine mapping –Invalidate the TLB entries –Copies the data to the local machine page

18.  page replication –mostly read-shared pages are replicated to the nodes most often using them –the monitor sets the access to read-only –Copies the page to the local node –Updates the relevant TLB entries

19. Experiment  used the SimOS machine simulator to develop and evaluate Disco

22. Conclusion  Improved memory locality by page migration and replication  Scalable  As they claimed, when running 8 VMs, the performance got 40% faster than commercial OS  Through memory sharing across virtual machine boundaries, saves the memory foot prints