1. Presenter: Yinon Avraham Advisor: Assoc. Prof. Danny Raz Technion - Israel Institute of Technology

2. Agenda 1.Introduction 2.Background and System Overview 3.Memory Fingerprinting 4.Sharing-aware Colocation 5.Hotspot Mitigation 6.Implementation 7.Experimental Evaluation 8.Conclusions (Placement)

3. 1 Introduction Data centers are increasingly employing virtualized architecture – applications run inside virtual servers which resides on a physical server Hypervisor – responsible to allocate physical resources for the VMs on a physical server Modern hypervisors use the CBPS technique (Content-Based Page Sharing) to reduce the memory footprint of resident VMs The problem: Select which VMs should be colocated onto each physical host within the data center so that page sharing can be maximized The suggested solution: Memory Buddies – a system for intelligent VM colocation within a data center to aggressively exploit page sharing benefits

4. Physical Server 1 2.1 Background The use of CBPS in a hypervisor: VM 1 : M 1 unique pagesVM 2 : M 2 unique pages S – common pages across VM 1 & VM 2 Page sharing reduces the memory footprint to: M 1 + M 2 - S M1 M2 SS

5. 2.1 Background (cont.) Problem formulation: VM colocation problem is one where each VM is colocated with a set of other “similar” VMs with the most redundant pages. Several instantiations of the smart colocation problem arise during: I.Initial placement II.Server consolidation III.Offline planning Physical Server 1Physical Server 2 S VM1 SS VM2 S VM3 ??

6. 2.2 System Overview Memory Buddies detects sharing potential between virtual machines and then uses the low- level sharing mechanisms to realize these benefits Nucleus – runs on each server, calculates: – Memory fingerprint for each VM – Aggregate memory fingerprints – the union of the VMs on the server Control Plane – runs on a distinguished control server, Responsible for virtual machine placement and hotspot mitigation

7. 3 Memory Fingerprinting 3.1 Fingerprinting generation In Memory Buddies, Hsieh’s SuperFastHash algorithm is used to generate 32bit hashes for each 4KB page The set of unique page hashes for a VM’s pages are gathered in a sorted order to generate the raw memory fingerprint Such a fingerprint may be compared against another VM or server to indicate the potential memory sharing between them Cons: – These fingerprints are large – 1MB for each 1GB of VM address space – They need to be sorted in order to be compared efficiently

8. 3 Memory Fingerprinting 3.2 Succinct Fingerprints Bloom filter - a lossy representation of a set of keys, which may be used to test a value for membership in that set with configurable accuracy. Consists of: – m-bit vector – k hash functions: H = h 1, h 2, … h k For each element a: H(a) = 1 if a is part of the set The probability (p e ) of such errors depends on m, k, and the probability that any bit in the vector is 1. n – number of elements stored Does the other way always true?No – false positive result

9. 3 Memory Fingerprinting 3.2 Succinct Fingerprints In practice, neither method will produce an accurate prediction because: – Fingerprints are snapshots in time, while memory contents changes – There is a difference between what could be shared and what the hypervisor identifies as sharable

10. 3 Memory Fingerprinting 3.3 Fingerprint Comparison Page sharing potential = intersection of fingerprints (Takes the errors under consideration) z 1, z 2 – number of zeros in the two Bloom filters z 12 – number of zeros in the AND result m – size of the filter vectors k – number of hash functions

11. 3 Memory Fingerprinting Conclusion Benefits of the succinct fingerprints (vs. hash lists): – Much smaller – Save communication bandwidth – Much faster to compare – Require no sorting before comparison

12. 4 Sharing-aware Colocation Each nucleus computes & transmit fingerprints to the control plane The control plane has an overview of all the data center The control plane uses this knowledge in order to maximize page sharing potential There are 3 supported types of placement decisions: 1.Initial Placement 2.Server Consolidation 3.Offline Planning

13. 4 Sharing-aware Colocation 4.1 Initial placement The goal: Deploy the new VM while allowing more VMs to be hosted on a given number of servers (the greatest amount of sharing) The algorithm: 1.Place the new VM on a staging host – collect usage information (memory, CPU, network, disk) 2.Determine set of feasible hosts (with sufficient resources) 3.Estimate sharing potential (new VM on each feasible host) 4.Choose host with the maximum sharing potential

14. 4 Sharing-aware Colocation 4.2 Server Consolidation The goal: Pack VMs onto servers so as to reduce aggregate memory footprint and maximize the number of VMs that can be housed in the data center. (Save energy, reduce servers’ weariness) The algorithm phases: 1.Identify servers to consolidate (mean memory usage under a threshold) 2.Determine target hosts (start with the largest VM, same as initial placement) 3.Migrate VMs to targets (live migration, limit the number of concurrent migrations)

15. VM 1 4 Sharing-aware Colocation 4.2 Server Consolidation Capacity Server 2Server 1Server 3 VM 4 VM 2 VM 1 VM 3

16. 4 Sharing-aware Colocation 4.3 Offline Planning Tool for Smart VM Colocation The goal: Answer the question: “What if?” Input: – Data centers and their resource capacities – Resource utilization statistics – Memory fingerprints Output: – VM placements that match each VM to a host – total memory consumption – Expected rate of sharing This problem analogous to a bin packing problem where the resource constraints define the size of each bin. Memory Buddies uses a dynamic programming technique to solve it.

17. 5 Hotspot Mitigation Goal: Resolve memory pressure caused by changes in the VM behavior, mitigate the effect by re-balancing the load over the hosts Memory hotspot reasons: – Increasing demand for memory by one or more VMs (application and/or OS) – Loss of page sharing Monitoring: – Level of swap activity (VM’s OS) – Number of shared pages (hypervisor) Solution: – Detect hotspot (swap activity rises, shared pages decreases) – Resolve by re-distributing the VMs – use Initial Placement, start with the VM which provides the highest absolute gain in sharing – No feasible destinations  bring a server to life

18. 6 Implementation Virtualization layer: VMware ESX – Supports VM migration – Supports page sharing, but it is unavailable  the nucleus is deployed on each VM (memory tracing kernel module), not as a part of the hypervisor Testbed A cluster of P4 2.4GHz servers connected over gigabit ethernet Memory Tracer Memory analysis tool (Linux, Windows, Mac OS X), generates 32bit hashes for each page in the memory. The resulting fingerprints are sent to the control plane every few minutes. Control Plane Java based server, communicates with the VMware Virtual Infrastructure management console via WS-API, in order to gather VMs’ information & statistics, and to initiate migrations

19. 7 Experimental Evaluation Memory Trace Analysis Result: 37% of the pages can be shared with one or more systems

20. 7 Experimental Evaluation Case Study: Internet Data Center Result: Effective capacity of the data center increased by 16%

21. 7 Experimental Evaluation Hotspot Mitigation Result: Hotspot detected mitigated by determining a different host with a higher sharing potential

22. 7 Experimental Evaluation Case Study: Desktop Virtualization Result: Memory Buddies can be used offline to compute memory sharing and answer “what if” questions when planning for desktop virtualization

23. 7 Experimental Evaluation Fingerprint Efficiency and Accuracy Result: Employing Bloom filters in large data centers can reduce sharing estimation time by an order of magnitude and can reduce network overheads by over 90%, while still maintaining a high degree of accuracy

24. 7 Experimental Evaluation Sub-page sharing Break each page into n chunks, map each chunk to a 32bit hash. Result: The detected sharing increases The sharing between two 64bit Ubuntu Linux, 2GB of RAM

25. 8 Conclusions Modern hypervisors use CBPS to reduce the footprint of the residing VMs (by ~33%) This technology can be used to intelligently colocate VMs on hosts, decreasing the total footprint, hence reduction of the TCO. (Experiments results show an increase of effective capacity of the data center by 16%) Memory Buddies suggest a solution for 3 types of placement decision: – Initial Placement – Server Consolidation – Offline Planning As well a hotspot mitigation technique

27. Slide Master Your Text here Lorem ipsum dolor sit amet, consectetuer adip iscing elit, sed diam no n u mmy nibh euismod tincidunt ut laoreet do lore magna aliquam er at v olut pat. Ut wisi enim ad mi ni m venia m, quis nostrud exerci tatio n ulla mco rper susc ip it lobor tis nisl ut aliquip ex ea commodo cons equat. Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse mo les tie consequat, vel illum dolore eu feugiat nulla fac ilisis at vero eros et ac cumsan et iusto odio dign issim qui bla ndit praesent lup tatum zzril dele nit augue duis dolore te fe ug ait nulla facilisi

28. Print Master Your Text here Lorem ipsum dolor sit amet, consectetuer adip iscing elit, sed diam no n u mmy nibh euismod tincidunt ut laoreet do lore magna aliquam er at v olut pat. Ut wisi enim ad mi ni m venia m, quis nostrud exerci tatio n ulla mco rper susc ip it lobor tis nisl ut aliquip ex ea commodo cons equat. Duis autem vel eum iriure dolor in hendrerit in vulpu tate velit esse mo les tie consequat, vel illum dolore eu feugiat nulla fac ilisis at vero eros et ac cumsan et iusto odio dign issim qui bla ndit praesent lup tatum zzril dele nit augue duis dolore te fe ug ait nulla facilisi