Status Report Ian Pratt University of Cambridge and Founder of XenSource Inc. Computer Laboratory

Overview  Xen Today:  Xen 3.0 Development Update  New benchmark results  Ongoing research

Xen Today : 2.0 Features  Secure isolation between VMs  Resource control and QoS  Only guest kernel needs to be ported  All user-level apps and libraries run unmodified  Linux 2.4/2.6, NetBSD, FreeBSD, Plan9  Execution performance is close to native  Supports the same hardware as Linux x86  Live Relocation of VMs between Xen nodes

Para-Virtualization in Xen  Arch xen_x86 : like x86, but replace privileged instructions with Xen hypercalls  Avoids binary rewriting and fault trapping  For Linux 2.6, only arch-dep files modified  Modify OS to understand virtualised env.  Wall-clock time vs. virtual processor time Xen provides both types of alarm timer  Expose real resource availability Enables OS to optimise behaviour  MMU virtualisation: direct vs. shadow mode

I/O Architecture  Xen IO-Spaces delegate guest OSes protected access to specified h/w devices  Virtual PCI configuration space  Virtual interrupts  Devices are virtualised and exported to other VMs via Device Channels  Safe asynchronous shared memory transport  ‘Backend’ drivers export to ‘frontend’ drivers  Net: use normal bridging, routing, iptables  Block: export any blk dev e.g. sda4,loop0,vg3

Xen 2.0 Architecture Event Channel Virtual MMUVirtual CPU Control IF Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE) Native Device Driver GuestOS (XenLinux) Device Manager & Control s/w VM0 Native Device Driver GuestOS (XenLinux) Unmodified User Software VM1 Front-End Device Drivers GuestOS (XenLinux) Unmodified User Software VM2 Front-End Device Drivers GuestOS (XenBSD) Unmodified User Software VM3 Safe HW IF Xen Virtual Machine Monitor Back-End

Xen 3.0 Architecture Event Channel Virtual MMUVirtual CPU Control IF Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE) Native Device Driver GuestOS (XenLinux) Device Manager & Control s/w VM0 Native Device Driver GuestOS (XenLinux) Unmodified User Software VM1 Front-End Device Drivers GuestOS (XenLinux) Unmodified User Software VM2 Front-End Device Drivers Unmodified GuestOS (WinXP)) Unmodified User Software VM3 Safe HW IF Xen Virtual Machine Monitor Back-End VT-x 32/64bit AGP ACPI PCI SMP

3.0 Headline Features  AGP/DRM in dom0  ACPI/PCI support in dom0  Support for SMP guests  x86_64 support  Intel VT-x support for unmodified guests  Enhanced control and management tools  Optimised inter-VM networking  IA64 and Power support, PAE36

x86_64  AMD Opteron and Intel EM64T  Requires different approach to plain x86  Can’t use segmentation to protect Xen from guest OS kernels  Switch page tables between kernel and user  Large VA space offers other optimisations  Current design supports up to 8TB mem  Call for user testing in ~2-3 weeks

SMP Guest OSes  Takes great care to get good performance while remaining secure  Paravirtualized approach yields many benefits  Avoids many virtual IPMIs  Need for better SMP-aware scheduler  Believed stable, optimisations pending

VT-x / Pacifica  Enables unmodified GuestOSes to be supported  Xen has excellent Shadow page table support  Requires simple platform emulation  Install paravirtualized drivers after booting for high-performance IO

4 th Generation Tools  Controlling Xen is easy, it’s coordinating the rest of the system that’s hard  Driver domains; firewall/routeing rules; shaping  LVM / filesystem image management  VM relocation  Resource measurement, control  Managing clusters of Xen nodes  Replace monolithic xend with tool suite communicating via The Registry

Live VM Relocation  Why is VM relocation useful?  Managing a pool of VMs running on a cluster  Taking nodes down for maintenance  Load balancing VMs across the cluster  Why is it a challenge?  VMs have lots of state  Some VMs will have soft real-time requirements E.g. web servers, databases, game servers  Can only commit limited resources to migration

VM Relocation Strategy

Writeable Working Set

Rate Limited Migration

Iterative Progress: SPECWeb

Iterative Progress: Quake3

Quake 3 Server migration

Research Roadmap  Cluster load balancing  Pre-migration analysis phase  Optimization over coarse timescales  Evacuating nodes for maintenance  Move easy to migrate VMs first  Storage-system support for VM clusters  Decentralized, data replication, copy-on-write  “Internet Suspend Resume”  Just rsync plus IPSec tunnels

Research Roadmap  Cluster load balancing algorithms  Exploit properties of live migration  System debugging and fault tolerance  Lightweight checkpointing, distributed watchpoints, deterministic replay  I/O interposition and replay  VM forking  Lightweight service replication, isolation  Secure virtualization  Multi-level secure Xen

Conclusions  Xen 3.0 release on-target!