1 CMG, 2006 Reno Yiping Ding and Ethan Bolker How Many Guests Can You Serve? - On the Number of Partitions

2 CMG 06 Reno Why Consolidation and Virtualization?

3 CMG 06 Reno It is the power, space, …

4 CMG 06 Reno Common Sense “Figure out how many will fit around your dining room table even before you begin to think about how much they will eat.” Three questions follow: 1. What is the size of the dining room? 2. What is the size of the table? 3. How many tables can fit in the dining room?

5 CMG 06 Reno A huge table in a large dining room: no partition High Utilization

6 CMG 06 Reno Small tables in a large dining room: partition Low Overhead

7 CMG 06 Reno Large tables in a huge dining room: partition High Overhead

8 CMG 06 Reno A large table in … It’s all yours

9 CMG 06 Reno A Basic Computer System without Virtualization Applications Operating System Processors Network Interface I/O Subsystem Memory Hardware

10 CMG 06 Reno Single Server S

11 CMG 06 Reno Two Queues and Two Servers

12 CMG 06 Reno Vocabulary  Partition = guest = virtual partition  Server = processor

13 CMG 06 Reno Two virtual machines on one physical system Memory Processors Hardware Operating System Applications I/O Subsystem Network Interface MemoryProcessors, V1(P) Hardware I/O Subsystem Network Interface Virtualized Layer Hardware Operating System Applications MemoryProcessors, V2(P) I/O Subsystem Network Interface Virtualized Layer Virtualization Manager

14 CMG 06 Reno Memory Processors Hardware Operating System Applications I/O Subsystem Network Interface MemoryProcessors, V1(P) Hardware I/O Subsystem Network Interface Virtualized Layer Hardware Operating System Applications MemoryProcessors, V2(P) I/O Subsystem Network Interface Virtualized Layer Virtualization Manager A Queueing Model for Two Guests S

15 CMG 06 Reno Partition or not? You pay overhead for partition but it is easy to manage and results are more predictable. Without partition, you may serve more

16 CMG 06 Reno Two ways to serve guests  Each guest owns a (virtual) server Separate queues Supermarket model  Guests share the physical processor(s) Single queue Bank model

17 CMG 06 Reno Supermarket Model: (M/M/1) × m

18 CMG 06 Reno Supermarket Model: (M/M/1) × m

19 CMG 06 Reno Bank Model: M/M/m

20 CMG 06 Reno Bank Model: M/M/m

21 CMG 06 Reno How Many Guests Can You Serve? Memory Processors Operating System Applications I/O Subsystem Network Interface Memory Processors Hardware I/O Subsystem Network Interface Virtualized Layer Virtualization Manager Operating System Applications Memory Processors I/O Subsystem Network Interface Virtualized Layer … Operating System Applications Memory Processors I/O Subsystem Network Interface Virtualized Layer

22 CMG 06 Reno The Difference: Virtualized vs. Physical Memory Processors Operating System Applications I/O Subsystem Network Interface Memory Processors Hardware I/O Subsystem Network Interface Virtualized Layer Virtualization Manager Operating System Applications Memory Processors I/O Subsystem Network Interface Virtualized Layer … Operating System Applications Memory Processors I/O Subsystem Network Interface Virtualized Layer

23 CMG 06 Reno Model as M/M/m or M/M/1, with Overhead Memory Processors Operating System Applications I/O Subsystem Network Interface Memory Processors Hardware I/O Subsystem Network Interface Virtualized Layer Virtualization Manager Operating System Applications Memory Processors I/O Subsystem Network Interface Virtualized Layer … Operating System Applications Memory Processors I/O Subsystem Network Interface Virtualized Layer

24 CMG 06 Reno Three types of overhead Consolidate m (identical) separate physical servers each with utilization u as guests on a single machine n times as powerful as the individual servers  Constant Overhead for virtualization management: c 1. Overhead proportional to number m of guests: f0 per partition, m×f0 total 2. Overhead proportional to guest utilization u: f per second per guest, f×m×u total Total Utilization: U = (1+f)×m×u/n + m×f0 + c

25 CMG 06 Reno Example: consolidate 50 servers Server Distribution by Utilization Assuming Processing Power = 1 for each server

26 CMG 06 Reno Processing power needed for supporting the number of partitions (SLA = 2 seconds) Throughput per partition Relative Processing Power Max Number of Partitions Server Distribution by Utilization Assuming Processing Power = 1 for each server

27 CMG 06 Reno Throughput per partition Relative Processing Power Max Number of Partitions

28 CMG 06 Reno Overhead Reduces Number of Supportable Partitions for Same SLA Throughput per Partition Relative Proc. Power Max Number of Partition without Overhead Max Number of Partitions (10% Fixed Overhead)

29 CMG 06 Reno Throughput per Partition Relative Proc. Power Max Number of Partition without Overhead Max Number of Partitions (10% Fixed Overhead)

30 CMG 06 Reno Increase Processing Power to Compensate for Overhead to Support the Number of Partitions Throughput per Partition Relative Processing Power (10% more for Fixed Overhead) Relative Processing Power (without Overhead) Max Number of Partitions (10% Fixed Overhead) (  25) (  4) (  3) (  2) (  1) (  1) (  2) (  1)

31 CMG 06 Reno

32 CMG 06 Reno

33 CMG 06 Reno If we choose one type of sever to support all Relative Processing Power Min Number of Servers Needed How many do we need for each server type ? Server Distribution by Utilization Assuming Processing Power = 1 for each server

34 CMG 06 Reno SLA Matters Too Relative Power Min Number of Servers Needed (SLA = 1 Seconds) Min Number of Servers Needed (SLA = 2 Seconds) How many servers do we need?

35 CMG 06 Reno The impact of fixed overhead and overhead per partition for the Same SLA Throughput per Partition Relative Proc. Power Max Number of Partition without Overhead Max Number of Partitions (10% Fixed Overhead) Max Number of Partitions (10% Fixed Overhead and 2% Overhead per Partition) Max Number of Partitions (10% Fixed Overhead and 5% Overhead per Partition)

36 CMG 06 Reno The relationship between Memory Size and Paging Rate When you ask for what’s not in memory, paging occurs; the larger the memory, the higher the chance what you ask is in memory

37 CMG 06 Reno Example: VMware ESX Memory Management “… With ESX Server it is possible to have 10 virtual machines, each with 3.6GB of RAM available to their operating systems and applications, on a 2 CPU system with 16 GB of physical RAM.” - VMware White Paper: “Virtualization: Architectural Considerations And Other Evaluation Criteria” It that true?

38 CMG 06 Reno When average demand is low, virtualization is better A crossing point: average memory demand of 900MB

39 CMG 06 Reno WMware ESX Virtual Ethernet Switch Virtual Machines / Guests Outbound Adapter Virtual Network

40 CMG 06 Reno Virtual Machines / Guests Outbound Adapter Communication pattern determines how many guests it can support (communicate among guests)

41 CMG 06 Reno Virtual Machines / Guests Outbound Adapter Communication pattern determines how many guests it can support (communicate to outside world)

42 CMG 06 Reno How Many Guests Can You Serve?