U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science Dynamic Resource Allocation for Shared Data Centers Using Online Measurements Abhishek Chandra Weibo Gong Prashant Shenoy UMASS Amherst

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 2 Motivation Data Centers Server farms Rent computing and storage resources to applications Revenue for meeting QoS guarantees Goals: Satisfy application QoS guarantees Maximize resource utilization of platform Robustness against “Slashdot” effects

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 3 Dynamic Resource Allocation Periodically re-allocate resources among applications Estimate resource requirements for near future Challenges: Reallocation at short time-scales No prior workload profiling/knowledge Low overhead Approach: Online Measurement-based Allocation

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 4 Talk Outline Motivation System Model Dynamic Allocation Techniques Experimental Results Conclusions

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 5 Resource Model Queuing System Generalized Processor Sharing (GPS) scheduler Request classes Different arrival processes, service time distributions QoS Goal: Mean Response Time GPS Resource

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 6 Dynamic Resource Allocation MONITOR System Metrics Resource Shares APPLICATION MODELS Expected Load PREDICTOR Measured Usage ALLOCATOR Rsrc Reqmts RESOURCE

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 7 Dynamic Resource Allocation ALLOCATOR PREDICTOR MONITOR System Metrics APPLICATION MODELS Expected Load Measured Usage RESOURCE

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 8 Monitoring Measure system and application metrics Queue lengths Request response times Monitoring windows Adaptation Window History Time Measurement Interval

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 9 History Adaptation Window Prediction Short-term prediction of workload characteristics Request arrival rate Average service time Use history of measured system metrics Mean Last value AR(1)

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 10 Prediction Accuracy Prediction Error Workload Prediction Time (min)

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 11 Dynamic Resource Allocation PREDICTOR MONITOR APPLICATION MODELS Expected Load Rsrc Reqmts ALLOCATOR Resource Shares RESOURCE

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 12 Measurement-based Model Goal: Relate QoS metric to resource requirement Idea: Model parameterized by online measurements Advantages: Parameters do not need to be computed Allow adaptation to dynamic workload Proposed: Transient Queuing System Description

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 13 Transient Queuing Model Transient queuing behavior over adaptation window Relation between mean response time T ¯ and application share w Little’s Law: Relation is parameterized by the measured workload Arrival rate λ and mean service time s ¯

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 14 Resource Allocation: Utility Model Discontent function: Measures the QoS violations of an application Constrained Optimization problem u1u1 u2u2 Optimization

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 15 Constrained Optimization Formulation Non-linear Optimization Problem: Response Time Discontent D i Goal subject to Solved using Lagrange multiplier method

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 16 Talk Outline Motivation System Model Dynamic Allocation Techniques Experimental Results Conclusions

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 17 Experimental Setup Simulation experiments Soccer World Cup’98 Traces Results based on a 24-hour portion of the trace 755,000 requests Mean req rate: 8.7 req/sec Mean req size: 8.47 KB

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 18 Share AllocationWorkloads Adaptation to Transient Overloads Shares adapt to changing workload characteristics

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 19 Adaptation: System Discontent GPS without reallocationGPS with reallocation System Discontent is lowered substantially

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 20 Conclusions Dynamic Resource Allocation needed for data centers Measurement-based allocation: Monitoring and Prediction gather online state Use this state for application modeling and allocation Future Work: Prediction policies Utility functions

U NIVERSITY OF M ASSACHUSETTS, A MHERST – Department of Computer Science 21 Related Work Prediction Statistical Prediction Models [Zhang00] Application Models Queuing-Theoretic Models [Carlstrom02,Liu01] Control-Theoretic Models [Abdelzaher02,Lu01] Data Centers MUSE [Chase01] COD [Moore02] Oceano [Appleby01]