Multifaceted Resource Management in Virtualized Providers Íñigo Goiri PhD Defense June 14th, 2011 Advisors: Jordi Guitart and Jordi Torres

Motivation Book Store Internet Companies offer their services over the Internet 2

Service Providers over the Internet Book Store Internet Number of users increases Provider requires more resourcesLoad increases 3

Service Providers over the Internet Book Store Internet Number of users decreases Provider requires fewer resources 4

Virtualization on Service Providers Book Store Internet Database Web Server Application Server Database Application Server Web Server Database Web Server DDBB Application Server WebServer Application Server DDBB WebServer Encapsulate tasks in Virtual Machines VM Load is unbalanced 5

Providing Virtual Machines Book Store Internet Virtualized Provider VM Virtualized Provider Virtualized Provider Provider offers its idle Infrastructure as a Service (IaaS) We can also offer applications encapsulated in VMs 6

Managing Virtualized Provider’s Resources Balanced load VMs get enough resources Energy consumption is high 7

Managing Virtualized Provider’s Resources Unbalanced load Some VMs don’t get enough resources Energy consumption is lower 8

Managing Virtualized Provider’s Resources Challenge Efficiently Manage Virtualized Provider’s Resources Maximize Provider’s Profit Every VM gets enough resources Energy consumption is lower 9

Contents Motivation Multifaceted Scheduling – Cost-Benefit Model – Scheduling Policy – Evaluation Multiprovider Scheduling – Capacity Planning – Scheduling – Evaluation Conclusion 10

Multifaceted Scheduling Cost-benefit model – Multiple facets to consider – Aggregate facets into costs – Consider impact of facets on other facets Scheduling policy – Maximize provider profit Evaluation 11

Multiple Facets to Consider 1.Service Level Agreement 2.Virtualization Management 3.Energy Consumption 4.Infrastructure Cost 12

1. Service Level Agreement Contract between user and provider User pays for resources – Pay as you go:SLARevenue(t(VM)) If the provider does not provide the QoS – SLA Penalty:SLAPenalty(VM) Virtualized Provider Virtualized Provider Revenue Penalties 13

1. Service Level Agreement Supported Applications Batch – Deadlines Example – HPC jobs Service – Uptime – Performance Example – Web Servers Task 1 Task 3 Time Task 2 Response Time 14

1. Service Level Agreement Resource Heterogeneity Xeon processor – High energy consumption – High performance Atom processor – Low energy consumption – Low performance Task A Task B Task C Task A Task B Task C 15

1. Service Level Agreement Estimate SLA Penalties – Actions may imply future violations – SLAPenalty’(Host, VM) Factors that might provoke violations – Runtime overhead – Slow host – High host utilization Other facets evaluate penalty estimations 16

Time 2. Virtualization Management Overhead to manage Virtual Machines – Start VMs – Migrate VMs between nodes Time 17

Off 2. Virtualization Management Overhead to manage Virtual Machines – Start VMs – Migrate VMs between nodes Time Off Migration 18

2. Virtualization Management Overhead implies – Extra time to the VM – Extra load to the Host It can imply SLA penalties – Violate deadline – No enough resources to provide performance Estimate SLA penalty for every action (Time, Load) → SLAPenalty’(Host, VM) → € 19

3. Energy Consumption Energy vs. SLA Low Consolidation – Consume a lot of energy – Fulfill SLA High Consolidation – Save energy – Violate SLA 20

3. Energy Consumption vs. SLA Batch Type Low Consolidation High Consolidation Off Time 21

3. Energy Consumption vs. SLA Service Type Low Consolidation High Consolidation Off Time 22

3. Energy Consumption Energy cost Wh → € SLA penalties Host Utilization→ SLAPenalty’(Host, VM) → € 23

4. Infrastructure Cost Provider owns infrastructure – Servers – Air conditioners – Racks… Capital Expense (CAPEX) Cost is amortized over time – Provider has already paid for the hardware €/Period 24

Multiple Facets to Consider Calculate profit of VM at Host 1.Service Level Agreement: SLARevenue(VM)+€ 2.Virtualization Management: Time:SLAPenalty’(VM)-€ Load:SLAPenalty’(VMs in Host) due to VM-€ 3.Energy Consumption: Energy consumed by VM at Host-€ SLA Penalty’(VMs in Host)-€ 4.Infrastructure Cost: Cost of Host running VM-€ TotalProfit€ 25

Scheduling Policy 26 Decide best VM placement Maximize provider profit – Hill Climbing (Greedy) When to schedule? – System changes – Periodically Model Virtualized Provider as a matrix – VM x Host cells – Each is profit of placing VM in Host – Use cost-benefit model

OFF Queue: Scheduling Policy A B C 27

Scheduling Policy 1.5€0.2€ -0.5€ 1.2€ -0.1€0.2€ 1.3€ 1.2€0.7€ -1.5€-∞ -0.5€ -0.3€ 0.1€0.3€ -0.7€0.2€ 1.5€0.2€ -0.5€ 1.2€ -0.1€0.7€ 0.2€ -0.2€0.7€ 1.2€ -∞ -0.5€ -0.3€ 0.1€0.3€ -0.7€0.2€ Calculate cost of allocating Every VM at every Host 1.5€0.2€ -0.5€ 1.2€ -0.1€0.2€ 1.3€ 1.2€0.7€ -1.5€ -∞ -0.5€ -0.3€ 0.1€0.3€ -0.7€0.2€ Recalculate cost of allocating Every VM at every Host Schedule VM in maximum Profit placement When cost is minimized Dispatch VMs A B C 28 After multiple iterations…

Scheduling Policy OFF Queue: A B C 29

Evaluation Multifaceted Scheduling One week heterogeneous workload – Batch: Grid5000 – Service: SPECWeb2005 SLA metrics – Batch: Deadline (Added +20% Base Runtime) – Service: Performance (Response Time) Provider with 65 nodes – Enough to satisfy workload peaks 30

1.Backfilling + Migration – Backfill VMs – Migrate to consolidate 2.Perfect SLA – Analytical (NP) – Perfect SLA fulfillment – Optimal energy consumption 3.Our proposal – Backfilling + Migration – Aggregate multiple facets – Uses cost-benefit model – Maximize profit OFF 1 2 Evaluation Scheduling Policies 3 OFF 4 4

Evaluation Energy consumptionSLA fulfillment 32 + ← Consolidation → -

Evaluation 33

Evaluation Power consumption over time 34

Multifaceted Scheduling Limitations 1.Service Level Agreement 2.Virtualization Management 3.Energy Consumption 4.Infrastructure Cost – Fixed costs 35

Contents Motivation Multifaceted Scheduling – Cost-Benefit Model – Scheduling Policy – Evaluation Multiprovider Scheduling – Capacity Planning – Scheduling – Evaluation Conclusion 36

Multiprovider Scheduling Outsourcing Infrastructure cost – Capital Expenses (CAPEX) Solution: CAPEX → OPEX External Provider 37

Multiprovider Scheduling Outsourcing Reduce provider infrastructure (CAPEX) Multifaceted Scheduling + Outsourcing – Add outsourcing cost (OPEX) – Slower VM creation – Limited VM management External Provider 38

Evaluation Outsourcing Add outsourcing to “Multifaceted Scheduling” Same environment – Reduce local resources: 65 → 20 → 0 nodes – 20 nodes is enough to provide the average External provider: EC2 US – €/hour – 5 minutes to start a VM 39

Evaluation Outsourcing 40

Multiprovider Scheduling Federation How many local resources? – Optimal number of resources – Change capacity planning How to schedule? – New actuators – New trade-offs Characterize provider profitability – Cost-benefit model 41

Federated Provider Model Multidimensional problem Evaluate provider profile – Provider capabilities – Expected workload – VM pricing Evaluate costs – CAPEX: Infrastructure – OPEX: Energy, Cooling,… 42

Multiprovider Scheduling Characterize federation Leverage federated provider model for: Phase 0. Capacity planning – Provider building and setup process – Decide optimal number of nodes Phase 1. Scheduling – Online process – Decide actions to take 43

Phase 0. Capacity Planning Load is variable over time If infrastructure costs are fixed – Underprovision:Cannot support peaks – Overprovision:Underutilized resources 44

Phase 0. Capacity Planning Underprovision Solution: Outsourcing Send peaks to other providers – Reduce provider infrastructure costs – Pay for using external resources External Provider 45

Phase 0. Capacity Planning Overprovision Solution: Insourcing Offer idle resources to other providers – Cheaper price – Enough resources to support peaks Offer to other Providers 46

Phase 1. Scheduling Analyze provider profitability – Decide best actions to perform New actuators to consider – Outsourcing – Insourcing Old actuators change – Turn on/off nodes vs. Insourcing 47

Evaluation Characterization Provider profitability (darker is better) – Offering 80% of the idle resources – Amazon EC2 pricing 48 No InsourcingInsourcing

Evaluation Phase 0. Capacity Planning ISP workload over a week Get optimal capacity – Leverage provider model – Revenue > Costs 49 Overprovision Undeprovision 100 nodes

Evaluation Phase 1. Scheduling 50

Evaluation Phase 1. Scheduling 51

Contents Motivation Multifaceted Scheduling – Cost-Benefit Model – Scheduling Policy – Evaluation Multiprovider Scheduling – Capacity Planning – Scheduling – Evaluation Conclusion 52

Conclusion Multifaceted Scheduling – Efficient Management of Virtualized Providers – Consider multiple facets – Handle effect of multiple facets – Every facet has a significant effect Multiprovider Scheduling – Add new actuators – New policies required – Tradeoff between Insourcing and Turn on/off 53

Future Work Support higher-level SLA metrics Energy efficient datacenter – Reduce PUE, free-cooling… – Energy-aware scheduling: Green availability – Efficient datacenter placement Multiprovider – Apply to real-world scenarios – Enhance provider fault tolerance 54

Summary Developed from a multi-level perspective: – Enhancing Virtualization Fabrics – Managing a Virtualized Host – Multifaceted Scheduling – Multiprovider Scheduling 55

Publications Enhancing Virtualization Fabrics – [NCA09] VM Creation. – [PDP09] VM Migration. – [NOMS10] VM Checkpointing. Managing a Virtualized Host – [CPE09] SLA-driven resource management. – [GECON10] SLA metric for Virtualized Environments. Multifaceted Scheduling – [Cluster10] [FGCS] Multifaceted Scheduling. Multiprovider Scheduling – [Grid10] Outsourcing into Multifaceted Scheduling. – [Cloud10] Characterizing Federation. 56

Multifaceted Resource Management on Virtualized Providers Íñigo Goiri Advisors: Jordi Guitart and Jordi Torres 57

Publications Enhancing Virtualization Fabrics [NCA09] Í. Goiri, F. Julià, J. Ejarque, M. De Palol, R. M. Badia, J. Guitart, and J. Torres. Introducing Virtual Execution Environments for Application Lifecycle Management and SLA-Driven Resource Distribution within Service Providers. Proceedings of the 8th IEEE International Symposium on Network Computing and Applications. [PDP09] Í. Goiri, F. Julià, and J. Guitart. Efficient Data Management Support for Virtualized Service Providers. Proceedings of the 17th Conference on Parallel, Distributed and Network-based Processing. [NOMS10] Í. Goiri, F. Julià, J. Guitart, and J. Torres. Checkpoint-based Fault- tolerant Infrastructure for Virtualized Service Providers. Proceedings of the 12th IEEE/IFIP Network Operations and Management Symposium. 58

Publications Managing a Virtualized Host [CPE09] J. Ejarque, M. de Palol, Í. Goiri, F. Julià, J. Guitart, R. Badia, and J. Torres. Exploiting semantics and virtualization for SLA-driven resource allocation in service providers. Concurrency and Computation: Practice and Experience, 22(5): pages 541–572, [GECON10] Í. Goiri, F. Julià, J. O. Fitó, M. Macías and J. Guitart. Resource-level QoS Metric for CPU-based Guarantees in Cloud Providers. In Proceedings of the 7th International Workshop on Economics of Grids, Clouds, Systems, and Services. 59

Publications Multifaceted Scheduling [Cluster10] Í. Goiri, F. Julià, R. Nou, J. Berral, J. Guitart, and J. Torres. Energy- aware Scheduling in Virtualized Datacenters. Proceedings of the 12th IEEE International Conference on Cluster Computing. [FGCS] Í. Goiri, J. Berral, J. O. Fitó, F. Julià, R. Nou, J. Guitart, R. Gavaldà and J. Torres. Energy-efficient and Multifaceted Resource Management for Profit-driven Virtualized Datacenters. Review process in Future Generation of Computer Systems. 60

Publications Multiprovider Scheduling [Grid10] Í. Goiri, J. O. Fitó, F. Julià, R. Nou, J. Berral, J. Guitart, and J. Torres. Multifaceted Resource Management for Dealing with Heterogeneous Workloads in Virtualized Datacenters. Proceedings of the 11th ACM/IEEE International Conference on Grid Computing. [Cloud10] Í. Goiri, J. Guitart, and J. Torres. Characterizing Cloud Federation for Enhancing Providers’ Profit. Proceedings of the 3rd International conference on Cloud Computing. 61

1. Service Level Agreement SLA Penalty [GECON10] – Low Violation → Low Penalties – High Violation → High Penalties 62

Scheduling Policy Algorithm 63 Model provider as a matrix – Host x VM Calculate best VM placement – Calculate profit for every VM and Host – Take most profitable action – If not converge, start again Perform actions – Create, migrate, cancel…

Cost-Benefit Model Aggregate Different Facets into Costs 1.Service Level Agreement: € 2.Virtualization Management: Virt → SLA →€ 3.Energy Consumption: Wh,SLA → € 4.Infrastructure Cost: € Final Profit of VM vm at Host h: 64

Evaluation Outsourcing 65