Performance Study on Virtual Machine Hypervisors
Xen’s report (1) Comparison of 3 popular VMMs: Xen, VMware and UML Benchmark: The SPEC CPU suite contains a series of long-running computationally- intensive applications intended to measure the performance of a system's processor, memory system, and compiler quality. A full build of the default configuration of Linux on local disk PostgreSQL running the OSDB multi-user Information Retrieval (IR) benchmark PostgreSQL running the OSDB multi-user On-Line Transaction Processing (OLTP) benchmark requires many synchronous disk operations, resulting in many protection domain transitions. The dbench 2.0 file system single user benchmark, It emulates the load placed on a file server by Windows 95 clients. SPEC WEB99 is a complex application-level benchmark for evaluating Web servers and the systems that host them
Xen’s report (2) native Linux (L) Xen/Linux (X) VMware Workstation 3.2 (V) User Mode Linux (U) Linux Build time SPEC INT 2000 OSDB-IR OSDB-OLTP DBench SPEC WEB99
VMware’ report (1) 2 VMMs: VMware ESX Server and Xen Benchmark The integer component of the SPECcpu2000 benchmark suite, available from SPEC® (Standard Performance Evaluation Corporation), was chosen to represent CPU-intensive applications. Passmark, a synthetic suite of benchmarks intended to isolate various aspects of workstation performance, was selected to represent desktop-oriented workloads. Netperf was used to simulate the network usage in a datacenter. The SPECjbb2005 benchmark suite from SPEC was used to represent the Java applications typically used in the datacenters. A compile workload — build SPECcpu2000 INT package — was also added to capture typical IT development and test usage in datacenters.
VMware’s report (2): Spec2000 INT
VMware’s report (3): Passmark
VMware’s report (4): Compile
VMware’s report (5): Netperf
VMware’s report (6): Specjob 2005
Discussion Why different benchmarks lead different results? Why multiple virtual machine products lead different results? Performance is really important? What else could evaluate VMM?