Xen and the Art of Virtualization Ian Pratt University of Cambridge and Founder of XenSource Inc. Computer Laboratory

Outline  Virtualization overview  Xen Today : 2.0 Overview  Architecture  Performance  Live VM Relocation  Xen 3.0 Roadmap (Q2 2005)

Virtualization Overview  Single OS image: Ensim, Vservers, CKRM  Group user processes into resource containers  Hard to get strong isolation  Full virtualization: VMware, VirtualPC  Run multiple unmodified guest OSes  Hard to efficiently virtualize x86  Para-virtualization: UML, Xen  Run multiple guest OSes ported to special arch  Arch Xen/x86 is very close to normal x86

Xen Today : 2.0 Features  Secure isolation between VMs  Resource control and QoS  Only guest kernel needs to be ported  All user-level apps and libraries run unmodified  Linux 2.4/2.6, NetBSD, FreeBSD, Plan9  Execution performance is close to native  Supports the same hardware as Linux x86  Live Relocation of VMs between Xen nodes

Para-Virtualization in Xen  Arch xen/x86 : like x86, but replaces privileged instructions with Xen hypercalls  Avoids binary rewriting and fault trapping  For Linux 2.6, only arch-dep files modified  Modify OS to understand virtualised env.  Wall-clock time vs. virtual processor time Xen provides both types of alarm timer  Expose real resource availability Enables OS to optimise behaviour  MMU virtualisation: direct vs. shadow mode

I/O Architecture  Xen IO-Spaces delegate guest OSes protected access to specified h/w devices  Virtual PCI configuration space  Virtual interrupts  Devices are virtualised and exported to other VMs via Device Channels  Safe asynchronous shared memory transport  ‘Backend’ drivers export to ‘frontend’ drivers  Net: use normal bridging, routing, iptables  Block: export any blk dev e.g. sda4,loop0,vg3

Xen 2.0 Architecture Event Channel Virtual MMUVirtual CPU Control IF Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE) Native Device Driver GuestOS (XenLinux) Device Manager & Control s/w VM0 Native Device Driver GuestOS (XenLinux) Unmodified User Software VM1 Front-End Device Drivers GuestOS (XenLinux) Unmodified User Software VM2 Front-End Device Drivers GuestOS (XenBSD) Unmodified User Software VM3 Safe HW IF Xen Virtual Machine Monitor Back-End

System Performance LXVU SPEC INT2000 (score) LXVU Linux build time (s) LXVU OSDB-OLTP (tup/s) LXVU SPEC WEB99 (score) Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)

TCP results LXVU Tx, MTU 1500 (Mbps) LXVU Rx, MTU 1500 (Mbps) LXVU Tx, MTU 500 (Mbps) LXVU Rx, MTU 500 (Mbps) TCP bandwidth on Linux (L), Xen (X), VMWare Workstation (V), and UML (U)

Scalability LX 2 LX 4 LX 8 LX Simultaneous SPEC WEB99 Instances on Linux (L) and Xen(X)

Live VM Relocation  Why is VM relocation useful?  Managing a pool of VMs running on a cluster  Taking nodes down for maintenance  Load balancing VMs across the cluster  Why is it a challenge?  VMs have lots of state  Some VMs will have soft real-time requirements E.g. web servers, databases, game servers  Can only commit limited resources to migration

Rate Limited Migration

Quake 3 Server migration

Roadmap for Xen 3.0  SMP guest OSes  Prototype now working, undergoing tuning  Required careful design to get good performance and retain security guarantees  Support for Intel VT-x extensions  Run ‘legacy’ unmodified OSes  Other ports : x86/64 and ia64 (ppc)  Both Xen x86/64 and ia64 now boot!

Roadmap for Xen 3.0  Better cluster management tools  Manage pool of VMs across a set of nodes  Better tools for QoS control  New GUI management tool  Improved hardware compatibility  Graphics cards, ACPI, APM

Research Roadmap : 4.0  Cluster load balancing algorithms  Exploit properties of live migration  Software fault tolerance  Exploit deterministic replay  System debugging  Lightweight checkpointing and replay  VM forking  Lightweight service replication, isolation  Secure virtualization  Multi-level secure Xen

Conclusions  Xen is a complete and robust GPL VMM  Outstanding performance and scalability  Excellent resource control and protection  Live relocation makes seamless migration possible for many real-time workloads  