1. Embedded System Lab. 오명훈 snt2426@gmail.com Memory Resource Management in VMware ESX Server Carl A. Waldspurger VMware, Inc. Palo Alto, CA 94304 USA carl@vmware.com

2. 오 명 훈오 명 훈 Embedded System Lab. TABLE OF CONTENTS Introduction Reclamation Mechanism Sharing Memory Shares vs Working sets Dynamic Reallocation I/O Page Remapping Reference

3. 오 명 훈오 명 훈 Embedded System Lab. Introduction ESX Server manages system hardware directly, providing significantly higher I/O performance and complete control over resource management This paper introduces several novel mechanisms and policies that ESX Server 1.5 uses to manage memory

4. 오 명 훈오 명 훈 Embedded System Lab. Reclamation Mechanism Memory Overcommit Page replacement Issues  meta-level page replacement policy, double paging Ballooning  small balloon module is loaded into the guest OS -inflate(alloc), deflate(free), get_free_page() 40 client dbench due to VM data structures

5. 오 명 훈오 명 훈 Embedded System Lab. Sharing Memory Eliminating redundant copies of pages Content-Based Page Sharing  hashing, COW  0.5% space overhead Scanning only during otherwise-wasted idle cycles The CPU overhead due to page sharing was negligible, 1.6% lower to 1.8% higher (increase hit rate in physically-indexed caches) single chain 16bytes

6. 오 명 훈오 명 훈 Embedded System Lab. Shares vs Working Sets The working set of a VM is the set of memory pages which are being “actively” accessed by the VM Shared based allocation (relative resource rights) Measuring, Reclaiming Idle Memory - sampling  minor page fault count Multiple sampling rate ESX VM Shares-per page ratio Default : 0.75 Default : 100pages 30sec

7. 오 명 훈오 명 훈 Embedded System Lab. Shares vs Working Sets 2 Zero Page Thread

8. 오 명 훈오 명 훈 Embedded System Lab. Dynamic Reallocation Most operating systems attempt to maintain a minimum amount of free memory Prevent rapid state fluctuations

9. 오 명 훈오 명 훈 Embedded System Lab. Dynamic Reallocation 2 Only 1GB was available for executing VMS The aggregate VM workload was configured to use a total of 1472 MB  1200MB (result of sharing) Nearly all time is spent in the high and soft states( Shared ↓  Balloon ↑ ) Windows zero min size Long-running query

10. 오 명 훈오 명 훈 Embedded System Lab. I/O Page Remapping Support Remapping (Highmem  Lowmem)  increased latency, reduced throughput, increased CPU load specified threshold It may be desirable to remap some low pages into high memory ( simple random replacement ) Hot page HIGHMEM DMA Repeated I/O operation

11. 오 명 훈오 명 훈 Embedded System Lab. Reference https://labs.vmware.com/vmtj/memory-overcommitment-in-the-esx-server P. Barham et al., "Xen and the Art of Virtualization", SOSP, 2003 Professional Linux Kernel Architecture book