1. Memory Buddies: Exploiting Page Sharing for Smart Colocation in Virtualized Data Centers Timothy Wood, Gabriel Tarasuk-Levin, Prashant Shenoy, Peter Desnoyers*, Emmanuel Cecchet, and Mark D. Corner University of Massachusetts, Amherst *Northeastern University

2. Server Placement in Data Centers Virtualization improves resource utilization by consolidating servers But how to determine which servers to place together? Must consider many resource constraints: –CPU, Disk, Network, Memory

3. Why Memory? CPU scheduling is fine grain –Easily share among many users –Work conserving, so no waste Memory is much less flexible –Allocated on a large time scales –Being wrong (paging) is disastrous Memory is an expensive resource Memory capacity is increasing slower than CPU power

4. Hypervisor Physical RAM FREE D Content Based Page Sharing VM 1 Page Table A B C B C AA D B VM 2 Page Table A D B If two VMs have an identical pages in memory, just keep one copy Supported by VMware ESX platform –Experimental tests in Xen, further support planned Potential benefits –33% in VMware ESX paper –65% with subpage sharing (Difference Engine) 1) Hypervisor detects identical pages 2) Copy-on-Write references created for shared pages

5. But what if…. Pages change over time, breaking sharing If memory is being overcommitted, this can lead to hotspots Physical RAM FREE D B C A VM 1 Page Table A B C VM 2 Page Table A D B A*

6. Physical RAM What’s the problem? VM 4 Page Table A B C B C AFREE D E D F VM 3 Page Table E D F Only get a benefit if VMs on a machine actually have pages to share! Physical RAM VM 2 Page Table A B C B C AFREE D E D F VM 1 Page Table E D F Host 1 Host 2

7. Where to place a VM? How do you figure out which VMs to place together? –Meet resource constraints –Maximize sharing Why placement is hard in large data centers? –Many applications from different clients –Many software stacks / platforms –Workloads change over time Here or there? Or there or there or there…? ?

8. Memory Buddies Goals Efficiently analyze the memory contents of multiple VMs to determine sharing potential Find more compact VM placement schemes Respond quickly to changing conditions to prevent memory hotspots Bonus! Traces released at traces.cs.umass.edu

9. Outline Motivation Memory Fingerprinting & Comparison Sharing-aware Colocation Hotspot Mitigation Implementation & Evaluation Related Work & Summary

10. Memory Fingerprints Hypervisor creates hash for each page –Check hash table to see if page is sharable –Record these hashes to create fingerprint Hash lists are big –32bits per 4K page = 1MB per 1GB of RAM –Need to forward fingerprint to other hosts Comparisons of lists is relatively slow VM 1 A B 0x11223344 0x55667788

11. Bloom Filter Fingerprints Bloom filter is a probabilistic data structure –Stores keys by setting some bits to 1 –False positive chance at lookup from hash collisions Very space efficient Tradeoff between filter size and accuracy 01101…001 M bits Insert(key) --> set h 1 (key)=1 and h 2 (key)=1 VM 1 0x11223344 0x55667788

12. Fingerprint Comparison Hash list comparison –Sort each list and then step through Bloom Filter –Simple method: Dot product of bit vectors –Bloom Sharing Equation Corrects for the expected number of false matches in each filter Impressively accurate! 10000 010011 111 1111 10111 1111 1 = 4

13. Eval: Fingerprinting 4GB RAM VMs –Hash: 4 sec –Sorted: 0.3 sec –Bloom: 0.02 sec Bloom Fingerprint 10% the size, still < 1% error Bloom filters are smaller and 10 to 100 times faster

14. Outline Motivation Memory Fingerprinting & Comparison Sharing-aware Colocation Hotspot Mitigation Implementation & Evaluation Related Work & Summary

15. Sharing Aware Placement Where to place a freshly started VM? Use staging area to find initial placement Find feasible hosts Estimate sharing potential Migrate VM Done! Staging Host 1100011 Host 1 1010101 Host 3 Host 2 1110000 1000010 1110010 + = Compare

16. Consolidation & Hotspot Mitigation Resource usage changes over time –Sharing may not last forever Periodically consolidate servers –Identify candidates (least loaded hosts) –Match to destinations (hosts with best sharing) –Migrate VMs –Disable unnecessary servers Hotspot Mitigation –Monitor memory usage to detect hotspots –VMs may run out of memory if sharing stops –Redistribute VMs to rebalance

17. Offline Planning Tool Host resources Resource Traces Memory Fingerprints Dynamic programming based bin-packing tool Finds subsets of VMs that can be placed together and maximize sharing Host 1 Host N Number of hosts required = X VM to host mapping Estimated sharing per host = Y

18. Outline Motivation Memory Fingerprinting & Comparison Sharing-aware Colocation Hotspot Mitigation Implementation & Evaluation Related Work & Summary

19. Implementation Memory Tracer –Tool used to gather data for trace study –Runs on Linux, OS X, and Windows –Calculates 32bit hashes for each page in memory –Sends either a hash list or Bloom filter to control node Works on physical systems or in VMs

20. Implementation Nucleus –Collects memory fingerprints for each VM –Sends data to control plane Control Plane –Gathers VM statistics and makes migration decisions based on sharing –Interacts with VMware Virtual Infrastructure to manage VMs

21. Eval: Trace Study System Mix Total Memory MB % Sharable Used Mem MB (with sharing) Linux 2.6.9 Darwin 9.0 Windows XP 422313.2%3666 Darwin 9.0 Darwin 9.4 Windows XP 524835.3%3397 Darwin 9.0 Darwin 9.2 Darwin 9.4 Windows XP 627236.8%3966 Darwin 9.4 (3 MacBook + iMac) 819240.0%4917

22. Eval: App Placement Try to place as many VMs onto a set of 4 hosts Sharing Oblivious: Place on first host with sufficient capacity Four app types -- data contents different for each VM instance 1 2 3 4 Host Sharing Oblivious Sharing Aware TPC-W OFBiz RUBiS SpecJBB 17 VMs 20 VMs

23. Outline Motivation Memory Fingerprinting & Comparison Sharing-aware Colocation Hotspot Mitigation Implementation & Evaluation Related Work & Summary

24. Related Work Waldspurger, OSDI 2002 –CBPS in VMware ESX Server Gupta, et al., OSDI 2008 –Increase sharing potential by looking at parts of pages VM Memory provisioning –Zhao & Wang (yesterday) has a good list!

25. Summary Hypervisors already support page sharing… Memory Buddies makes it more useful –Identifies sharing opportunities across data center – Migrates VMs to maximize sharing –Uses efficient memory fingerprinting techniques to scale to large data centers Traces will be online (soon) at: –http://traces.cs.umass.edu –Macbooks, Linux servers, and more! Questions? twood@cs.umass.edu