RECON: A TOOL TO RECOMMEND DYNAMIC SERVER CONSOLIDATION IN MULTI-CLUSTER DATACENTERS Anindya Neogi IEEE Network Operations and Management Symposium, 2008 Sameep Mehta Presented by: Yun Liaw IBM India Research Lab

Outline  Introduction  ReCon Overview  Mapping Logic  Experimental Validation  Related Works  Discussions  Conclusion and Comments 2015/10/11 2

Introduction  Server virtualization has regained popularity for various reasons  Virtual machines (VMs) support more flexible and finer grain resource allocation  Physical server’s cost of management and total cost of ownership (TCO) has gone up drastically  Virtualization enables consolidation of a number of smaller machines as VMs on a large server  Leads to more efficient utilization of hardware resources  Saving floor spaces, saving management cost 2015/10/11 3

Introduction (cont’d)  ReCon: a tool that uses historical resource usage monitoring data to recommend a dynamic or static consolidation plan on servers 2015/10/11 4 White: high utilizationBlack: low utilization

ReCon Overview  Trace data: a set of measurements taken from the system, typically in a timeseries format  E.g., CPU, memory, etc.  Cost  Static cost: the base cost of running a physical server with associated workload  Dynamic cost: the cost that varies with the utilization  VM migration cost 2015/10/11 5

ReCon Overview (cont’d)  Constraints: to restrict the space of possible mappings between VMs and physical servers  System constraints  Application level constraints  Legal constraints  “What-if” input configuration: For users be able to tweak the input parameters and review the impact of consolidation  Time window size of dynamic consolidation  The period that a server should have no workload to consider turning it off 2015/10/11 6

ReCon Overview (cont’d)  Optimal Mapping Algorithm: To take all parameters, costs, constraints, configurations and process the trace data to generate static or dynamic server consolidation  Consolidation window: the non-overlapping time window to divide the historical data for dynamic consolidation For each time window, a optimal mapping from VM to physical servers are created In static consolidation, the time window is assumed to be the entire trace 2015/10/11 7

Mapping Logic – Basic Notations  Let VM = {VM 1, VM 2,…, VM N }  Each VM i observes and stores K variables O = {O (1,i), O (2,i),…, O (K,i) }  Each VM i is monitored for T time steps, the time series generated by j th sensor of VM i is 2015/10/11 8 Informal Problem Statement Given N application VMs, find n physical machines where n < N such that each VM is assigned to one physical machine while satisfying domain specified constraints Informal Problem Statement Given N application VMs, find n physical machines where n < N such that each VM is assigned to one physical machine while satisfying domain specified constraints

Mapping Logic - Constraints  Virtual machine constraints: Each VM i is associated with a list of M i constraints  Physical server constraints: Each physical server P i is associated with a list of L i constraints  The j th constraint of VM i which should hold in the interval [t 1, t 2 ]  The constraint is said to be satisfied if 9 Where P is the properties of the environment/architecture in time [t 1, t 2 ]

Mapping Logic – Optimization Problem Formulation  Assume that in the initial, each VM (application) is hosted by one physical machine, and each physical machine hosts exactly one VM  |VM| = |P| = N  n is not known a priori, and N is the upper bound of n  A: a N×N matrix, such that A i,j =1 specifies that VM i is assigned to P j  A will be a diagonal matrix in the initial  Y: a |P| bit long vector, such that Y i =1 implies that P i is currently running some VMs  Y will be a vector with all 1 in the initial 2015/10/11 10

Mapping Logic – Optimization Problem Formulation  Cost i : the fixed cost incurred if P i is active  MCost i,j : the cost for migrating VM i to P j  F: a function that calculates the dynamic cost if one or more VMs are assigned to it  Currently this function uses the CPU utilization for computing the dynamic cost  The benefit function attained by the consolidation is as the following function 11 The cost of initial settingFixed cost of running physical servers Cost of VMs migrating to P j

Mapping Logic – Optimization Problem Formulation  the first term of B is fixed and does not change while maximizing the function, therefore the objective function can be transferred to minimize 2015/10/11 12

Mapping Logic – Dynamic/Static Consolidation  Dynamic Consolidation  Assume the consolidation window size is T s  Firstly minimize the optimization function in time interval [1, T s ], and generate the assignment matrix A [1,T s ]  While consolidating for time interval [T s +1, 2*T s ], using the new set of constraints and A [1,T s ] as the starting point for optimization  Static Consolidation  Set all migration costs to zero  Set the consolidation window to cover the whole time period 2015/10/11 13

Experimental Validation – Data Set  The trace data was collected using the Model Driven Monitoring System (MDMS) [8]  186 physical servers  35 clusters with each cluster supporting one application  Approximately 15 parameters are monitored for every server But in this paper, authors use CPU utilization data only  Parameter are sampled at 5 minutes interval  The optimization problem solver: AMPL and CPLEX 14 [8] B. Krishnamurthy, et al., “Data tagging architecture for system monitoring in dynamic environments,” in IEEE NOMS, 2008

Experimental Validation – Evaluation Metrics  Time efficiency  To measure how fast it works given the size of data  The efficiency ε is defined as T R /T S T S : the consolidation window size T R : the time taken by ReCon to generate consolidation plan ε ～ 0 for a highly efficient tool ε ≧ 1 renders the tool useless for all practical purpose  Effectiveness  The percentage of physical machines that can be turned off by packing N VMs onto n i physical machines while satisfying all constraints in the corresponding consolidation window I  The effectiveness S is given by (N - n i )/N S ～ 1 implies most of the physical machines can be turned off 2015/10/11 15

Experimental Validation – Change in recommendations VS migration cost  Recommendation: The merging of two VMs onto the same physical server  Migration cost:  Inter cluster migration cost is normalized to be 100  Intra-cluster migration cost is varied as percentage of inter migration cost 2015/10/11 16

Experimental Validation – Efficiency Results 2015/10/11 17 VMs 1~175 Consolidation Window 10 ~240 (min) Time Taken on

Experimental Validation – Change in recommendations over time period  To study how the recommendations vary with change in the consolidation window size  Results:  As the time window is increased, the number of recommendations decreases More samples makes it difficult to satisfy the constraints  Time Window Size The time window size should not be too big in order to capture the dynamic behavior The time window size should not be too small so that the optimization engine is not used repeatedly without any gain Recommend value: T=300 minutes 2015/10/11 18

2015/10/11 19 X axis: time window Y axis: cost saving

Experimental Validation – Change in Cluster Over Time  To study the effect of recommendations on individual clusters  Based on the mean and standard deviation of saving, the clusters can be categorized into four groups  Low Variation – Low Saving  Low Variation – High Saving  High Variation – Low Saving  High Variation – High Saving 2015/10/11 20

21 Low Variation Low Saving Low Variation High Saving High Variation Low Saving High Variation High Saving X axis: time window Y axis: cost saving

Related Works  There is significant work in capacity planning and runtime resource management domain without bringing in the aspect of virtualization  VMware’s Distributed Resource Scheduler (DRS)  Bobroff et al. describe algorithms for reconsolidation in a dynamic setting while managing SLA violations  In static consolidation several bin-packing heuristics have been used to map VMs to physical servers 2015/10/11 22

Discussion and Future Work  Handling multiple attributes  The implementation cannot exploit the relationships or correlation among attributes E.g., the time lag relation between high CPU utilization and high I/O utilization  Runtime reconfiguration tool  In order to convert the planning tool into a real time decision module, highly efficient implementation and forecasting logic is needed Machine learning and time series forecasting techniques are the candidates for the author’s next step  Extending what-if analysis 2015/10/11 23

Conclusion  A VM consolidation planning tool called ReCon is provided  To analyze the historical resource consumption data  The consolidation problem is formulated in an optimization framework  Time varying constraints are easily incorporate to temporal change in workload characteristics  Different migration cost function, virtualization models can be plugged into the tool 2015/10/11 24

Comment  The problem is well-formulated  But the mentioned cost functions are mysterious 2015/10/11 25