1. Disco: Running Commodity Operating Systems on Scalable Multiprocessors Presented by: Pierre LaBorde, Jordan Deveroux, Imran Ali, Yazen Ghannam, Tzu-Wei Kuo Paper by: Edouard Bugnion, Scott Devine, Kinshuk Govil, Mendel Rosenblum

2. Introduction Pierre LaBorde

3. Introduction CC-NUMA o Cache-Coherent Non-Uniform Memory Access Coupling with standard distributed protocols o TCP/IP o NFS Global Buffer Cache

4. Introduction Hide NUMA-ness o Page placement o Dynamic page migration o Dynamic page replication

5. Problem Operating systems for innovative hardware o Scalable shared memory multiprocessors Significant changes required o OS typically have millions of lines of code

6. Solution

7. Virtual Machine Monitors Instead of modifying existing OS o Additional layer of software between hardware and OS o Multiple copies of existing operating systems  Support a variety of workloads o Virtualizes all of the resources  Exports conventional hardware interface o Schedules virtual resources on the physical  Processor  Memory

8. Virtual Machine Monitor Monitor and distributed protocols need to scale o Simplicity of the monitor o Fault-containment o NUMA memory management issues Global policies o Fine-grained resource sharing

9. Challenges Overheads o Privileged instructions o I/O Devices Resource Management o Instruction execution stream  Idle loop  Lock busy-waiting Communication and Sharing o Virtual disk

10. Disco: A Virtual Machine Monitor Jordan Deveroux

11. Disco's Interface Processors o Abstraction of MIPS R10000 processor o Does not support complete virtualization of kernel virtual address space o Extends architecture to support efficient access to some processor functions Physical Memory o Abstraction of main memory that resides in contiguous physical address space o Uses dynamic page migration and replication to export nearly uniform memory architecture to the software I/O Devices o Each virtual machine has specified set of I/O devices o Intercepts communication from all of it's I/O devices for translation or emulation o Virtualizes access to the networking devices of the underlying system

12. Implementing Disco Multithreaded, shared memory program Disco vs. Other Systems o NUMA memory placement o cache-aware data structures o interprocessor communication patterns NUMA memory management o Copy DISCO into all memories of FLASH machine Cache-aware data structures o Partitioned so that parts accessed only by a certain processor are in memory near that processor Interprocessor communication patterns o Very few locks o Wait-free synchronization

13. Implementing Disco: Virtual CPU's Emulates virtual CPU's by using direct execution of real CPU's Same execution speed as running on real CPU's Each virtual CPU has a data structure like a process table entry in traditional O.S. o Contains state of virtual CPU Runs in kernel mode with full access Simple scheduler allows virtual processors to be shared

14. Implementing Disco: Virtual Physical Memory Add a level of address translation and maintains physical-to-machine address mappings Translation performed using translation-lookaside buffer Memory references are translated through this mapping from now on Each TLB entry is marked with an address space identifier to avoiding the flushing the TLB on context switches Each miss is more expensive o emulation of trap architecture o emulation of privileged instructions o remapping of physical addresses

15. Implementing Disco: NUMA Memory Management Optimization that enhances data locality Fast translation of virtual-to-physical addresses Allocation of real memory to virtual machines Only moves pages that will have performance benefit Contains a memmap data structure with an entry for each real machine memory page

16. Two different virtual processors of the same virtual machine logically read-share the same physical page, but each virtual processor accesses a local copy

17. Implementing Disco: Virtual I/O Intercepts all device access from the virtual machine and forwards them to the physical devices Each disco device defines a monitor call used by the device driver to pass all command arguements Disks and network interfaces include a map as part of their arguements o list of address pairs that specify the source and destination of I/O operations

18. VM Sharing Imran Ali

19. Copy-on-Write Disks Uses Virtual Memory Addressing to Map Data to physical Memory Multiple Virtual Machines(VM) Share Machine Memory Copy on write means that VM is unaware of Machine Memory being shared

20. VM Sharing Pages

21. Virtual Network Interfaces Virtual Machines are not allowed to communicate with each other Uses Standard Protocols to communicate through Ethernet- type addressing All read only pages can be shared through virtual machines reducing memory overhead Pages are shared whenever possible and are replicated when needed to improve proformance

22. Transparent Sharing of Pages

23. Experimental Results Yazen Ghannam

24. Experimental Setup Experiments are Simulated, not using real hardware Used four different workloads o Software Development (Pmake)  OS, I/O Intensive o Hardware Development (Engineering)  OS light; Large memory footprint o Scientific Computing (Raytrace, Radix)  OS light; uses shared memory regions o Commercial Database  I/O light; Single memory intensive

25. Execution Overheads

26. Memory Overheads

27. Scalability

28. Page Migration and Replication

29. Experiences and Related Work Tzu-Wei Kuo

30. Experiences on Real Hardware Disco was ported to run on a real hardware in order to confirm the simulation test results Run on SGI Origin200 board which forms the basis of the FLASH machine o Single - 180MHz MIPS R10000 processor o 128MB of memory

31. Experiences on Real Hardware Overheads of Virtualization Two workloads o Pmake: compiles Disco itself using the SGI development tools, two files at a time o Engineering: simulates the memory system of the FLASH machine

32. Experiences on Real Hardware This table shows a breakdown of the execution time for the two workloads and a comparison between IRIX and Disco running IRIX. The execution time is broken down into the user, system, and idle time.

33. Related Work System Software for Scalable Shared Memory Machines Virtual Machine Monitors Other System Software Structuring Techniques CC-NUMA Memory Management

34. Conclusion Developing system software for scalable shared memory multiprocessors without massive development effort Experimental results shows that the overhead of virtualization is modest in both processing time and memory footprints Disco provides simple solution for scalability and reliability Lower implementation cost

35. Disco: Running Commodity Operating Systems on Scalable Multiprocessors Presented by: Pierre LaBorde, Jordan Deveroux, Imran Ali, Yazen Ghannam, Tzu-Wei Kuo Paper by: Edouard Bugnion, Scott Devine, Kinshuk Govil, Mendel Rosenblum

36. Title Text