Open Source Virtualisation and Consolidation

Whoami ● Senior Linux and Open Source Consultant/ X-Tend ● „Infrastructure Architect“ ● Linux since 0.98 ● IANAKH

Agenda Virtualisation Xen Using Xen Real Life Case Studies

Why Virtualisation Matters ? ● Consolidation ● Security ● Testing Large Scale Infrastructures ● Separating Development/Staging/Production platforms ● Separating Hardware from Operating system – Making Deployments Easier – Making platforms interchangeable ● Higher Availability

3 Flavours of Virtualisation Hardware Virtualisation – Provides virtual hardware to unmodified OS – VMWare, Qemu, Xen on VT OS Virtualisation – One modified kernel for for hosts and guest – “Chroots on Steroids” – eg. OpenVZ, Linux Vserver

ParaVirtualisation ● Provides a Virtual Hardware Platform really similar but not identical to the underlying hardware ● OS Has to be modified (source availbility) ● VMM in Ring 0, OS degrades to1 ring ● Provides some exposure to the actual hardware – Performance increase – OS Needs to be modified – Multiplexes resources at OS granularity (vs Process level granularity) ● VT Chips : Ring -1

What is Xen ? ● Research project started at Cambridge UK ● Backed by Xensource ● Xen is a fast Open Source virtual machine monitor – for x86 (other platforms are in beta) – execution of multiple guest operating systems – unprecedented levels of performance and resource isolation. ● Supported by all the major Linux Vendors ● Fully functional ports of Linux 2.4 and 2.6 running on Xen

Features ● Secure isolation between Vms ● Resource controle and QOS ● Only guest kernel needs to be ported ● Execution performance is close to real speed ● Hardware support = Linux Server Hardware Support ● Live Relocation of VM's between nodes

Terminology ● Dom U : the guest OS ● Domain0 hosts the application-level management software – Creation and deletion of virtual network interfaces and block devices

Xen 3.0 Arch Event Channel Virtual MMUVirtual CPU Control IF Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE) Native Device Driver GuestOS (XenLinux) Device Manager & Control s/w Native Device Driver GuestOS (XenLinux) Unmodified User Software Front-End Device Drivers GuestOS (XenLinux) Unmodified User Software Front-End Device Drivers Unmodified GuestOS (WinXP)) Unmodified User Software Safe HW IF Xen Virtual Machine Monitor Back-End VT-x AGP ACPI PCI SMP

Xen & Networking ● Virtual firewall-router attached to all domains ● Round-robin packet scheduler ● To send a packet, enqueue a buffer descriptor into the transmit ring ● Practically : Bridging – Bonding,.1q support

Xen & Disk Access ● Only Domain0 has direct access to disks ● Other domains need to use virtual block devices – LVM – Loopback – Real Disks – Network Block Devices – iSCSI

Xen and Memory ● Reserved at domain creation ● Statically Partitioned among domains ● Balloon driver enables memory reallocation at runtime

Xen, Live VM Migration ● Why ? – Manage a pool of VM's on a Cluster – Hardware upgrades – Loadbalancing VM's in a cluster ● Why Difficult ? – Migrate State of VM's – Sessions/Response time for databases & webservers ● Requires Shared Storage

From: Wim Coekaerts Cc: Subject: Re: [Xen-devel] databases and xen? testing it has already been fine. I ran 4 databases each in one domain (oracle10g) and it's been amazingly stable. I have not however done performance testing. soon... Early Adopters

From: Moshe Bar Subject: Re: [Xen-devel] databases and xen? Date: Tue, 18 Jan :02: I have run Mysql and Oracle 9i without any problems on and 2.0.1, but I didn't have LVM (don't think that would create problems) Moshe Early Adopters

Managing and Deploying Xen ● Xen Enterprise (XenSource) ● OpenQRM (Qlusters) ● Integrated with SystemImager ● Enomalism ● XenMan ● Distro Specific (yast, debootstrap)

Application Isolation ● Machine with lots of functions, – Mail / Web / Directory Services ● Starts small with each service deployed in a virtual machine. ● Scales out to physical machines when needed.

Solving Netfilter Conflicts ● Building a truly transparant proxy – Integration of LVS and Tproxy fails – Required multiple machines to work – CONNTRACK module conflicts ● Used Xen to build this on 1 machine

Lowering the # of machines ● Telco Environment with maximum 6+16x(2x3+6))=198 machines (actually 6+2x(2x3+6)= 30) – Consolidated already 1 application ● now x(2x2+6) = 166 machines (6+2x(2x2+6) = 24) ● we moved already 2 redundant applications to 1 of the 6 shared machines ● more are following

Cluster Consolidation ● Failover many to 1 – Failover all physical machines to multiple virtual machines on 1 physical machine. ● Virtual to Virtual, with multiple virtual machines ● Application failover in the VM's ● Failing over the full VM

Xen Really Near Future ● :) ● VT Vanderpool Technology is going mainstream ==> No more ports required – Also non free OS’s ● Xen Enterprise Beta is going public ● Infiniband support etc ● GUI stuff going beta, Q1.

Conclusions ● Xen is a complete and robust GPL VMM ● Outstanding performance and scalability ● Excellent resource control and protection ● Vibrant development community ● Strong vendor support ●

Further Reading ● Automating Xen Virtual Machine Deployment, LinuxKongress 2005 ● O'ReillyNet, Getting Started with Xen ● ●

Kris Buytaert Senior Linux Consultant & X-Tend