1 Automated Power Management Through Virtualization Anne Holler, VMware Anil Kapur, VMware

2 Agenda 1 Virtualization and Cluster Basics 2 Distributed Resource Management 3 Future Directions: Integrating Power Budget and Resource Management Across Virtualized Server Cluster

3 Agenda 1 Virtualization and Cluster Basics 2 Distributed Resource Management 3 Future Directions: Integrating Power Budget and Resource Management Across Virtualized Server Cluster

4 What is Virtualization?  Virtualization is a proven software technology that is rapidly transforming the IT landscape and fundamentally changing the way that people compute  Virtualization makes it possible to run multiple operating systems and multiple applications on the same SERVER at the same time Benefit  Virtualization allows you to reduce IT costs while increasing the efficiency, utilization and flexibility of your existing x86 computer hardware

5 For the visually inclined… Hypervisor

6 4 Key Properties  Partitioning  Isolation  Encapsulation  Hardware Independence

7 Partitioning Key Virtualization Properties  Run multiple operating systems on one physical machine  Divide system resources between virtual machines Hypervisor

8 Isolation Key Virtualization Properties  Fault and security isolation at the hardware level  Advanced resource controls preserve performance Hypervisor

9 Encapsulation Key Virtualization Properties  Entire state of the virtual machine can be saved to files  Move and copy virtual machines as easily as moving and copying files Hypervisor

10 Hardware Independence Key Virtualization Properties  Provision or migrate any virtual machine to any similar or different physical server Hypervisor

11 Live Machine Migration with Zero Downtime Enables the live migration of virtual machines from one host to another with continuous service availability. Benefits:  Revolutionary technology that is the basis for automated virtual machine movement  Meets service level and performance goals Hypervisor

12 Agenda 1 Virtualization and Cluster Basics 2 Distributed Resource Management 3 Future Directions: Integrating Power Budget and Resource Management Across Virtualized Server Cluster

13 DRS Overview Manage a Cluster As a Single Large Host 6 hosts CPU = 10 GHz Memory = 64 GB 1 “Giant Host” CPU = 60 GHz Memory = 384 GB

14 DRS Goals  Ease of Host Management  Initial Placement  CPU and Memory Load Balancing  VM-VM Affinity (Anti-Affinity)  VM-Host Affinity (Anti-Affinity)  Host Maintenance Mode Host Cluster vMotion Affinity

15 DRS Load-balancing Measuring Imbalance  Imbalance metric is a cluster-level metric  First for each host, DRS computes VM entitlement is a measure of how much resources a VM deserves (both CPU and memory). A host with higher capacity can have more VMs for the same normalized entitlement.  Imbalance metric is the standard deviation (spread) of these normalized entitlements Imbalance metric = 0.30 Imbalance metric =

16 Distributed Power Management Applying DRS to Power Management  Right-sizes Cluster Capacity  Consolidates virtual machines (VMs) onto fewer hosts & powers hosts off when demand is low  Powers hosts back on when needed to meet workload demand or to satisfy constraints  Optional add-on to Distributed Resource Scheduler DRS Cluster with DPM enabled Response to Reduced Demand Power Off

17 Agenda 1 Virtualization and Cluster Basics 2 Distributed Resource Management 3 Future Directions: Integrating Power Budget and Resource Management Across Virtualized Server Cluster

18 Motivating Problem  Rack power budget cannot be overcommitted or its fuse will trip  Equipment name plate max power draw is much larger than actually required (as much as 40%)  This leads to stranded (paid for) power and underutilizes available space in data center  Untapped power (and space) forces unnecessary DC purchases

19 Current (Partial) Solution  Vendors (e.g. HP, Dell, IBM) support server power capping to allow for safe over provisioning of rack allocated power  Determining appropriate power cap is a static, time consuming process to measure consumed power at each host and set cap  Vendor host power capping techniques do not take into consideration VM workload, which can clip performance so power cap today may impact tomorrow’s performance

20 Cloud Power Cap Solution: Determine Workload Needs Leveraging DRS  Load balance power across hosts in a rack to meet VM constraints and performance demands  Uses BMC setting which has host response rates in < 1 ms

21 Rack Power Budget Resource Trade-off CONFIDENTIAL Consider host comprising 12 CPUs, each 2.9 GHz, as follows: Power Cap(W) Num Hosts CPU Total (GHz) CPU RatioMemory Total (GB) Memory Ratio CPUMemoryNameplatePeakIdle 34.8 GHz96 GB400 W320 W160 W R ack w/8 KW power budget can support different maximum capacities CloudPowerCap manages racks w/host power cap below peak Trade-offs possible in allocating a rack’s 8KW power budget to its hosts:

22 Cloud Power Cap Distribution CONFIDENTIAL 22 VM 1 VM 2 cc VM 3 cc VM 3 VM 2 VM 1 cc Host A Host B Host A Host B Powercap Allocation Provide resources to allow CRM to satisfy VM-VM affinity constraint

23 Cloud Power Cap Entitlement Balancing Powercap Entitlement Balancing Enable entitlement balancing to enhance response to demand bursts (below) Reduce entitlement balancing overhead by avoiding some migrations VM 1 VM 2 cc VM 3 cc Host A Host B Host A Host B VM 1 VM 2 cc VM 3 cc

24 Cloud Power Cap Distribution Examples CONFIDENTIAL VM 1 Host A Host B Host A Host B VM 1 VM 2 cc Stand-by VM 2 CC Powercap Redistribution Provide burst headroom after host power-off during low demand period

25 Cloud Power Cap Design & Implementation Interacts with each CRM phase to support its goals  Operates on host/VM model used by CRM to determine actions  Modifies hosts’ CPU capacity when changing hosts’ power-cap  Issues actions in appropriate order wrt other CRM actions  Implemented w/Distributed Resource Scheduler (DRS)

26 Experiments Cloud Power Cap vs. Rack Budget Table Static High: Each host power cap statically set to its peak of 320W  Maximizes the amount of CPU capacity available for given rack power budget  Doesn’t enable using power for more memory & I/O capacity w/lower CPU Static: Each host power cap statically set to 250W  Allows more servers to be placed in the rack  Allows more memory and I/O capacity w/lower CPU capacity

27 Experiments CloudPowerCap vs. Static Strategies  Headroom balancing  Supporting DRS headroom balancing while avoiding vMotion overhead  Flexible resource capacity  Enabling trade-off between powering CPU and memory capacity at runtime

28 Headroom Rebalancing Experiment

29 Headroom Rebalancing Experiment

30 Flexible Resource Capacity Experiment

31 Flexible Resource Capacity Experiment

32 Questions & Answers