System Analysis and Optimization 1 1 Efficient Resource Provisioning in Compute Clouds via VM Multiplexing Xiaoqiao Meng, Canturk Isci, Jeffrey Kephart, Li Zhang, Eric Bouillet, Dimitrios Pendarakis IBM T.J. Watson Research Center 7th IEEE International Conference on Autonomic Computing, 2010

System Analysis and Optimization 2 2 Credit Modified version of conference presentation slides provided by Xiaoqiao Meng. 7th IEEE International Conference on Autonomic Computing

System Analysis and Optimization 3 3 Outline Background VM multiplexing Related work Design  Performance constraint  VM selection  Joint-VM sizing Applications Summary

System Analysis and Optimization 4 4 Resource provisioning in cloud Network equipments ServersStorage Virtual machine pools  Cloud creates an illusion of “infinite” pool of resources  Cloud manages resources via creating and consolidating Virtual Machines  Efficiency of resource provisioning measured by overall resource utilization  Cloud resource provisioning: decision on VM size and VM placement

System Analysis and Optimization 5 5 VM sizing Static: fixed size upon VM creation  Easy to manage  Lack of elasticity in resource reuse Dynamic: size adapts to demand  High resource utilization  Avoiding over and under-provisioning is challenging CPU utilization for a VM instance in 4 months Peak-load as VM size Chance for capacity violation < 1%

System Analysis and Optimization 6 6 Traditional VM placement CPU8 cores Memory24 GByte Physical host capacityVM size VM 1 VM 3 VM 2 VM 4 CPU12 cores Memory4 GByte CPU4 cores Memory12 GByte CPU4 cores Memory12 GByte CPU2 cores Memory8 GByte CPU2 cores Memory6 GByte CPU2 cores Memory4 GByte VIRTUALIZATION Placement is done separately from sizing  VM size is fixed regardless of where it is placed Target: minimize required physical servers or reduce energy cost Formulated as a multi-dimensional vector packing problem

System Analysis and Optimization 7 7 Outline Background VM multiplexing Related work Design  Performance constraint  VM selection  Joint-VM sizing Applications Summary

System Analysis and Optimization 8 8 VM workload multiplexing  Multiplex VMs’ workload on same physical server  Aggregate multiple workload. Estimate total capacity need based on aggregated workload  Performance level of each VM be preserved Separate VM sizingVM multiplexing s1s1 s2s2 s3s3 We expect s 3 < s 1 + s 2. Benefit of multiplexing !

System Analysis and Optimization 9 9 Example for VM multiplexing Traditional method: Provision VMs separately. Total capacity need = 1.04 cpu Alternative method: Consolidate and provision three VMs as a whole Total capacity need = 0.67 cpu Gain from statistical multiplexing!

System Analysis and Optimization 10 Potential Improvement by VM multiplexing Test on servers in a Global Hosting Services  1325 physical hosts, VMs  Based on 3-month CPU/memory utilization data Average capacity saving per physical server = 39%

System Analysis and Optimization 11 Outline Background VM multiplexing Related work Design  Performance constraint  VM selection  Joint-VM sizing Applications Summary

System Analysis and Optimization 12 Related work Commercial capacity planning tools consider each VM’s need in isolation  IBM WebSphere CloudBurst, VMware capacity planner, Novell PlateSpin Recon, Lanamark Suite Some research work stand on VM-by-VM basis  Low resource utilization. Some prior work consider inter-VM interaction and compatibility to improve resource provisioning  Hotcloud’09: co-place VMs not contending for same resource type  OSDI’08, Hotcloud’09, VEE’09: co-place VMs with memory sharing  MASCOTS’09. Eurosys’09: statistical multiplexing VM to save power  NOMS’08: VM consolidation by formulating an optimization problem

System Analysis and Optimization 13 Outline Background VM multiplexing Related work Design  Performance constraint  VM selection  Joint-VM sizing Applications Summary

System Analysis and Optimization 14 Design of enabling VM multiplexing Describe VM performance constraint  Application performance as a function of VM capacity Construct super-VM by multiplexing VMs  Find VM combinations with complementary workload Joint-VM sizing  Determining total capacity needs for super-VM Describe VM performance constraint (forecast) Construct super-VM by multiplexing VMs Sizing super-VM 2 cpu3 cpu 1.5 cpu1 cpu VM Consolidation  Place super-VM on host  Standard VM placement techniques plugged in

System Analysis and Optimization 15  Basis for computing VM size  Parameters β, T adapt to various application types  Small T and β for time-sensitive, critical applications  Large T and β for time-elastic, long-running background jobs  T=0, β=0 corresponds to peakload-based sizing VM performance constraint time Workload T Size c Performance constraint on VM size Number of intervals with size violation Total interval number < β

System Analysis and Optimization 16 Performance constraint with VM multiplexing Important feature : Knowing individual VM’s performance constraint Total capacity need for super-VM without decreasing individual VM performance easily derive Given n VMs with individual, Let If super-VM size satisfies, each VM must satisfy individual Choose the most stringent performance constraint

System Analysis and Optimization 17 Proof of theorem T: minimum among all T i, all T i is power of 2 Time interval [1, T i ] can be divided into shorter interval Holds for [1, T i ], [ T i +1, 2 T i ], [ 2 T i +1, 3 T i ]….

System Analysis and Optimization 18 Proof (cont.) Because: Thus,

System Analysis and Optimization 19 Construct super-VM Key for multiplexing: complementary workload patterns  Approximately measured by correlation coefficient Solution  Greedy search  Recursively find VM pairs with most strong negative correlation  Clustering  Pairwise distance measured by correlation coefficient Complementary workload! Strong negative correlation Not-so-complementary workload! Strong positive correlation Reference: R.O. Duda, Pattern Classification, 2 nd Edition, Wiley Interscience, 2001

System Analysis and Optimization 20 Greedy search Three-step process  Find the VM pair (i,j) giving the highest correlation coefficient. Output (i,j) as a candidate for joint provisioning  Remove VM i and j  Repeat Step 1 and 2 until all input VMs are removed Correlation coefficient matrix

System Analysis and Optimization 21 Super-VM sizing 1. Aggregation 2. Workload forecasting 3. Determine total size by super- VM performance constraint

System Analysis and Optimization 22 Workload forecasting  Decouple individual VM’s workload into fluctuating and regular components. Forecasting limited to aggregate fluctuating workload  Standard timeseries forecasting methods are applied. Choose the one with smallest forecasting error based on historical data

System Analysis and Optimization 23 Modeling forecast error Modeling forecasting error to compensate for workload variability Applied to any forecasting method  With explicit error model  Compute error statistical distribution by training error model with historical data  Without explicit error model  Use Kernel-Density-Estimation (KDE) to estimate error statistical distribution by historical data

System Analysis and Optimization 24 Outline Background VM multiplexing Related work Design  Performance constraint  VM selection  Joint-VM sizing Applications Summary

System Analysis and Optimization 25 Application to VM consolidation Tested on performance data from four cloud hosts  Use First-fit-decreasing for VM consolidation Performance gain  With VM multiplexing, 28%-62% less physical hosts required compared to no-multiplexing

System Analysis and Optimization 26 Consolidation with various perf. constraint Assume each VM has same performance constraint Observations:  Maximum extra capacity saving is 45% for T=0, β=0  Correspond to peakload-based sizing  Saving shrinks when β increases  Workload dynamics make less different in terms of VM capacity violation  With forecasting, VM consolidation slightly more aggressive  Lead to higher actual β than expected on certain physical hosts  Overall, 88%-96% of hosts have actual β less than expected

System Analysis and Optimization 27 Application to providing VM resource guarantees Apply VM multiplexing to providing resource guarantees Define “joint reservations” instead of individual VM reservations Enforce joint reservations with the “resource pool” abstraction:

System Analysis and Optimization 28 Application to providing VM resource guarantees 16%-75% of more VMs admitted by enabling joint-reservations. Gain subject to performance constraint  Higher gain for more stringent performance constraint

System Analysis and Optimization 29 Outline Background VM multiplexing Related work Design  Performance constraint  VM selection  Joint-VM sizing Applications Summary

System Analysis and Optimization 30 Summary Multiplex VMs with complementary workload patterns to save more overall resource Three design components  Performance constraint  VM selection  Joint-VM size estimation Enormous capacity saving in applications

System Analysis and Optimization 31