© 2010 VMware Inc. All rights reserved Distributed Resource Management for Virtualized System Clusters VMware, Inc.

Slides:



Advertisements
Similar presentations
Capacity Planning in a Virtual Environment
Advertisements

High Availability Deep Dive What’s New in vSphere 5 David Lane, Virtualization Engineer High Point Solutions.
Resource Shares Dynamic resource management
© 2009 VMware Inc. All rights reserved Confidential VMware vCenter Capacity IQ overview David Morahan Technical Pre-sales.
Walter Binder University of Lugano, Switzerland Niranjan Suri IHMC, Florida, USA Green Computing: Energy Consumption Optimized Service Hosting.
Protect Your Business and Simplify IT with Symantec and VMware Presenter, Title, Company Date.
VSphere 4 Best Practices/ Common Support Issues Paul Hill Research Engineer, System Management VMware.
© 2011 VMware Inc. All rights reserved High Availability Module 7.
1 Vladimir Knežević Microsoft Software d.o.o.. 80% Održavanje 80% Održavanje 20% New Cost Reduction Keep Business Up & Running End User Productivity End.
VMware Virtualization: The Foundation for a Green Datacenter.
Managing the Capacity and Performance of a VMware Cluster environment Presented by: Pete Weilnau CTO PERFMAN
Virtualization and Cloud Computing Virtualization David Bednárek, Jakub Yaghob, Filip Zavoral.
Xavier León PhD defense
Managing storage requirements in VMware Environments October 2009.
NoHype: Virtualized Cloud Infrastructure without the Virtualization Eric Keller, Jakub Szefer, Jennifer Rexford, Ruby Lee ISCA 2010 Princeton University.
SLA-aware Virtual Resource Management for Cloud Infrastructures
Towards High-Availability for IP Telephony using Virtual Machines Devdutt Patnaik, Ashish Bijlani and Vishal K Singh.
OnCall: Defeating Spikes with Dynamic Application Clusters Keith Coleman and James Norris Stanford University June 3, 2003.
U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Virtualization in Data Centers Prashant Shenoy
Lesson 1: Configuring Network Load Balancing
Virtualization Infrastructure Administration Cluster Jakub Yaghob.
Copyright © 2005 VMware, Inc. All rights reserved. VMware Virtualization Phil Anthony Virtual Systems Engineer
Implementing Failover Clustering with Hyper-V
Windows Server Virtualization Scenarios And Features Jeff Woolsey Lead Program Manager Windows Virtualization Microsoft Corporation.
High Availability Module 12.
VMware vCenter Server Module 4.
Scalability Module 6.
Virtualization Performance H. Reza Taheri Senior Staff Eng. VMware.
Alleviating Constraints with Resource Pools & Live Migration with Enhanced VMotion* Breakout Session# 2823 Raghu Yeluri Sr. Architect Intel Corporation.
VMware vSphere 4 Introduction. Agenda VMware vSphere Virtualization Technology vMotion Storage vMotion Snapshot High Availability DRS Resource Pools Monitoring.
Presented by : Ran Koretzki. Basic Introduction What are VM’s ? What is migration ? What is Live migration ?
About the Presentations The presentations cover the objectives found in the opening of each chapter. All chapter objectives are listed in the beginning.

Department of Computer Science Engineering SRM University
How to Resolve Bottlenecks and Optimize your Virtual Environment Chris Chesley, Sr. Systems Engineer
A Cloud is a type of parallel and distributed system consisting of a collection of inter- connected and virtualized computers that are dynamically provisioned.
Virtualization. Virtualization  In computing, virtualization is a broad term that refers to the abstraction of computer resources  It is "a technique.
Chapter 8 Implementing Disaster Recovery and High Availability Hands-On Virtual Computing.
Appendix B Planning a Virtualization Strategy for Exchange Server 2010.
VSolution Playbook VIRTUALIZED SAN SOLUTION FOR VMWARE SMB.
Cloud Computing Energy efficient cloud computing Keke Chen.
VMware Infrastructure 3 The Next Generation in Virtualization.
Improving Disk Latency and Throughput with VMware Presented by Raxco Software, Inc. March 11, 2011.
Autonomic SLA-driven Provisioning for Cloud Applications Nicolas Bonvin, Thanasis Papaioannou, Karl Aberer Presented by Ismail Alan.
Mesos A Platform for Fine-Grained Resource Sharing in the Data Center Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony Joseph, Randy.
Session objectives Discuss whether or not virtualization makes sense for Exchange 2013 Describe supportability of virtualization features Explain sizing.
Virtualization Infrastructure Administration
Server Virtualization
1 Virtual Machine Memory Access Tracing With Hypervisor Exclusive Cache USENIX ‘07 Pin Lu & Kai Shen Department of Computer Science University of Rochester.
VMware vSphere Configuration and Management v6
Copyright © 2005 VMware, Inc. All rights reserved. How virtualization can enable your business Richard Allen, IBM Alliance, VMware
20409A 7: Installing and Configuring System Center 2012 R2 Virtual Machine Manager Module 7 Installing and Configuring System Center 2012 R2 Virtual.
Cloud Computing Lecture 5-6 Muhammad Ahmad Jan.
Cloud Computing – UNIT - II. VIRTUALIZATION Virtualization Hiding the reality The mantra of smart computing is to intelligently hide the reality Binary->
DPM - IPMI Product Support Engineering VMware Confidential.
Memory Resource Management in VMware ESX Server By Carl A. Waldspurger Presented by Clyde Byrd III (some slides adapted from C. Waldspurger) EECS 582 –
Capacity Planning in a Virtual Environment Chris Chesley, Sr. Systems Engineer
1 Best Practices for Monitoring Databases on VMware Dean Richards Senior DBA, Confio Software.
Module Objectives At the end of the module, you will be able to:
1 SQL Server on VMware? Rob Mandeville Senior DBA, Confio Software 1 Virtualizing Our Environment: Lessons Learned Rob Mandeville.
VMware vSphere 4.0 Preventive & Maintenance. Agenda Preventive & Maintenace Storage/Datastore ESX Host Performance Monitoring ESX Maintenance User Access.
© 2010 VMware Inc. All rights reserved Why Virtualize? Beng-Hong Lim, VMware, Inc.
1 Automated Power Management Through Virtualization Anne Holler, VMware Anil Kapur, VMware.
Virtualization OVERVIEW
Building a Virtual Infrastructure
20409A 7: Installing and Configuring System Center 2012 R2 Virtual Machine Manager Module 7 Installing and Configuring System Center 2012 R2 Virtual.
Specialized Cloud Architectures
Towards Predictable Datacenter Networks
Presentation transcript:

© 2010 VMware Inc. All rights reserved Distributed Resource Management for Virtualized System Clusters VMware, Inc.

2 Outline  Distributed Resource Management for Virtualized System Clusters  Infrastructure Live Migration Architecture for Distributed Communication and Control  Resource Management Basics  Distributed Resource Scheduling  Distributed Power Management  Distributed High Availability  Summary

3 Virtualized Systems Cluster Infrastructure: Live Migration  Example: VMware Vmotion “Hot” migrate VM across hosts Transparent to guest OS, apps Minimal downtime (sub-second) Requirements (current) Globally accessible storage (SAN/NAS) Same subnet (no forwarding proxy) Compatible processors Details Bitmap tracks modified pages Pre-copy iteration sends modified pages Repeatedly pre-copy “diff” until converge Exploit meta-data (shared, swapped)

4 Virtualized Systems Cluster Infrastructure: Example Architecture Host+VM Mgmt clients DB cluster n cluster 1 stats + actions SDK UI ResMgr 1 n

5 Outline Distributed Resource Management for Virtualized System Clusters  Infrastructure  Resource Management Basics Goals Controls  Distributed Resource Scheduling  Distributed Power Management  Distributed High Availability  Summary

6 Resource Management Goals  Performance isolation Prevent virtual machines (VMs) from monopolizing resources Guarantee predictable service rates  Efficient utilization Exploit undercommitted resources Overcommit with graceful degradation Exploit opportunities to reduce power consumption  Easy administration Flexible dynamic partitioning Meet absolute service-level agreements Control relative importance of VMs  Respect availability constraints

7 Resource Controls Overview  Useful Features Express absolute service rates [e.g., 512MHz, 1GB] Express relative importance [e.g., VM A to get 2x CPU of VM B] Grouping VMs for isolation, sharing [e.g., VMs A,B to share 1GHz]  Challenges Simple enough for novices Powerful enough for experts Mapping application-level metrics to physical resource consumption E.g., What MHz is needed to guarantee 100 transactions/second? Scaling from single host to cluster of (say) 32/64/128 servers

8 Basic Resource Controls  Shares Specify relative importance Entitlement directly proportional to shares Abstract relative units, only ratios matters  Reservation Minimum guarantee, even when system overcommitted Concrete absolute units (MHz, MB) Admission control: sum of reservations ≤ capacity  Limit Upper bound on consumption, even when undercommitted Concrete absolute units (MHz, MB)

9 Resource Pools  Motivation Allocate aggregate resources for sets of VMs Isolation between pools, sharing within pools Flexible hierarchical organization Access control and delegation  What is a resource pool? Named object with permissions Reservation, limit, and shares for each resource Parent pool, child pools, VMs

10 Resource Controls: Exploration Areas  Additional controls Real-time latency guarantees Application-level metrics Users think in terms of transaction rates, response times Labor-intensive, requires detailed domain/app-specific knowledge Automate mapping to physical resource controls

11 Outline Distributed Resource Management for Virtualized System Clusters  Infrastructure  Resource Management Basics  Distributed Resource Scheduling Overview Example Exploration Areas  Distributed Power Management  Distributed High Availability  Summary

12 Distributed Resource Scheduling Overview  Useful features Choose initial host for VM power on Dynamic rebalancing by migrating running VMs between hosts Configurable automation and migration threshold levels Provide host evacuation for flexible host downtime Support optional constraints on VM colocation on hosts Preserve resources for failover  Challenges Placement and migration decisions involve multiple resources Resource pools can span multiple hosts Determining appropriate migration threshold controls Assorted failures modes (hosts, connectivity, etc.)

13 Distributed Resource Scheduling Example: VMware DRS  Cluster-wide resource management Hierarchical organization and delegation Flexible grouping, sharing, and isolation Configurable automation levels, migration aggressiveness Configurable VM affinity/anti-affinity rules Preserves unfragmented spare resources for failover  Automatic virtual machine placement and migration Choose initial host when VM powers on Optimize load balance across hosts Dynamic rebalancing using VMotion React to dynamic load changes Evacuate hosts for maintenance and/or power-off

14 Example: VMware DRS Balancing Details  Compute VM entitlements Based on resource pool and VM resource settings VM demand includes usage and unsatisfied demand Don’t give VM more than it demands Reallocate extra resources fairly  Compute host loads Load ≠ utilization unless all VMs equally important Sum entitlements for VMs on host Normalize by host capacity  Consider possible migrations Evaluate cluster balance impact and risk-adjusted cost/benefit Incorporate migration cost for involved hosts  Recommend best moves (meeting specified threshold)

15 Confidential VMware DRS Simple Balancing Example (all VMs in same resource pool with same shares)  Recommendation to improve imbalance: migrate VM2 VM2 VM1 4GHz 3GHz2GHz Host normalized entitlement = 1.25 (5/4) VM3 VM4 4GHz 1GHz Host normalized entitlement = 0.5 (2/4)

16 Distributed Resource Scheduling: Exploration Areas  I/O resource management Quality of service for networking, storage End-to-end control difficult, complex switching/routing fabric Lack of standards, even in non-virtualized environments  Proactive migrations Detect longer-term trends Move VMs based on predicted load while minimizing impact on current load  Large-scale WAN/Grid management

17 Outline Distributed Resource Management for Virtualized System Clusters  Infrastructure  Resource Management Basics  Distributed Resource Scheduling  Distributed Power Management Overview Example Exploration Areas  Distributed High Availability  Summary

18 Distributed Power Management Overview  Useful features Migrate VMs to allow hosts to be powered-off when demand low Power on hosts when demand rises or needed to satisfy constraints Work in concert with Distributed Resource Scheduling and Distributed High Availability goals and constraints Configurable automation and utilization threshold levels  Challenges Non-homogeneous hosts can result in utilization hot or cold spots Sudden unpredicted rise in demand can cause performance impact Benefits depend on demand valleys of non-trivial duration Variety of host wake methods with usability pros/cons

19 Distributed Power Management Example: VMware DPM  Implements useful features: Consolidates virtual machines (VMs) onto fewer hosts & powers hosts off when demand is low Powers hosts back on when needed to meet workload demand or to satisfy constraints Optional add-on to VMware Distributed Resource Scheduler (DRS) DRS Cluster with DPM enabled Response to Reduced Demand Power Off

20 Example: VMware DPM Operation in VC2.5/ESX 3.5  Lightly-used Hosts  Consider Host Power-Off Conservative: considers 40-minutes load history All VMs on selected host are migrated to other hosts Weighs trade-offs between costs and benefits of power-off Host is powered off  Heavily-used Hosts  Consider Host Power-On Responsive: considers 5-minute load history Send wake-on-LAN packet or BMC command [2009] to host Host boots up DRS load-balancing kicks in and some VMs migrated to host  Target Host Utilization Range centered around 63% by default

21 Example: VMware DRS/DPM Interactions  Responsibilities DRS balances load to satisfy service-level agreements DPM reduces running cluster capacity to save power Both DRS and DPM respect any resources needed for failover  Interactions DRS rebalances with DPM-recommended power actions in what-if simulations DPM evaluates the impact of potential power actions based on DRS rebalancing results  Recommendations Final host power actions and VMotions Recommended to user, or applied automatically

22 Distributed Power Management: Exploration Areas  Use VM demand prediction to drive proactive host power-on  Incorporate additional metrics in host off/on selection Examples: host power efficiency, temperature  Operate in cooperation with host-level power management Example: Choose hosts for off/on based on power-management features

23 Confidential Outline Distributed Resource Management for Virtualized System Clusters  Infrastructure  Resource Management Basics  Distributed Resource Scheduling  Distributed Power Management  Distributed High Availability Overview Example Exploration Areas  Summary

24 Distributed High Availability Overview  Useful features Provide various methods to specify resources to reserve to restart VMs upon failures of their hosts in a virtualized system cluster Express whether failover resource reservation is strict or best-effort Decentralized host failure detection and quick VM restart Work in concert with DRS and DPM goals and constraints  Challenges Preserving unfragmented failover resources across hosts Avoiding conservative allocation of spare resources when running VMs have widely different resource reservations and needs Support robust failover operation in many possible failure situations Provide cluster failover status information in a user-friendly way

25 Distributed High Availability Example: VMware HA  Specify resources to be set aside for failover Number of host failures to tolerate Percentage of cluster capacity [2009] Specific hosts to set aside for failover [2009]  Detect failover and respond Cluster hosts send each other heartbeats; when a host fails to do so for some period, failover response action is launched For failed hosts, their running VMs are restarted on other hosts Safe restart supported by locking in ESX server storage system  At failover, want unfragmented powered-on resources CPU,Memory resources for each VM to failover available on 1 host Resources must be on powered-on host DRS/DPM/HA proactively maintain appropriate spare resources

26 Distributed High Availability: Exploration Areas  Provide alternative to VM restart on host failure via continuing the VM from a secondary copy already executing on another cluster host [2009]; enabled by VM snapshot and record/replay  Support DRS/DPM aided failover if decentralized restart fails; use migrations and host power-on if needed to get resources  Explore detecting/responding to partial host failure modes; run on host with diminished working capacity

27 Outline Distributed Resource Management for Virtualized System Clusters  Infrastructure  Resource Management Basics  Distributed Resource Scheduling  Distributed Power Management  Distributed High Availability  Summary

28 Summary  Virtualized System clusters with appropriate supporting infrastructure can benefit from distributed resource scheduling, distributed power management, and distributed high availability.  Each of these technologies has a number of areas open for exploration and innovation.