1. Automating Xen Virtual Machine Deployment Kris Buytaert Kris.Buytaert@x-tend.be

2. Whoami ? ● CTO @ X-Tend.be ● Linux and Open Source Consultant ● „Infrastructure Architect“ ● Linux since 0.98 ● IANAKH

3. Agenda ● Large Scale System Installation theory ● Hybrid Automated Installs ● Xen ● Automated Xen Installations

4. Why Virtualisation Matters ? ● Consolidation ● Security ● Testing Large Scale Rollouts ● Separating Development/Staging/Production platforms ●...

5. Our specific reasons for this work ● Testing Large Scale System Deployment – How to you test bootstrapping a large environment ? – Buy machines for your test platform ? – Interrupt regular services – Take down a chain ? – How do you test upgrade and rollback procedures ?

6. Our specific reasons for this work(2) ● Actually deploying Virtual machines in a large environment – Consolidation of previously deployed machines – Repeated work is boring and error prone – We automated the physical machine deployment already so...

7. Goals ● Hands off Virtual Machine Deployment ● Minimal impact on the current infrastructure ● Fast deployments ● Easy to redeploy / reproducable ● I hate vendor Lock In‘s ● I hate doing the same stuff for different distros

8. The 10 th Floor Test ● Grab a random machine (don’t take a backup before) ● Throw it out a 10 th floor window ● Can you recover it in <10 minutes ? ● Even for Virtual Machines ?

9. Imaging vs Installing Imaging Speed Identical machines Multicasting Installing “slower” Finegrained

10. ● Disclaimer : – Tools are examples, – alternatives exist ● Automated – Fai – Autoyast – Kickstart – System Imager

11. Systemimager Suite ● SystemImager ● Fast deployment ● Golden client based ● Multicast features ● Centrally Stored Images ● Boel framework

12. SystemImager Suite (2) ● SystemInstaller ● Evolved from LUI ● Generates Images based on ● Packagelist ● Distro type ● Partition Definition

13. SystemImager Suite (3) ● SystemConfigurator ● Post install configuration ● Hardware detection / bootloader config ● Distro integration

14. Beyond Installing ● Package management ● Central Repository – Updates of relevant packages ● Yum ● Apt ● current.tigris.net

15. Beyond Installing(2) ● Configuration Management – Isconf – Cfengine – Puppet

16. Hybrid Deployment Keep everything in CVS Image a basic image Generate that image Then install the delta's on a per host basis via packages Use a repository to update systems Keep everything in CVS Use a configuration management tool.

17. Our Environment ● A SystemImager based framework ● System Imager Suite is platform independent, we can use it with Suse/Redhat/Debian etc ● Config data an installation scripts are being stored in CVS ● Base Image of the basic Linux tools and common applications is generated with SystemInstaller ● apt-repository for host specific applications (rpm v3 compliant)

18. Overriding „standard“ configs ● SystemImager Concept ● Over-rides an image per host ● Used for „non packagable“ files ● Contains machine specific information e.g. Package lists

19. Autoinstall, the script / the cd ● Create a dhcp config file ● PXE Boot and downloads the initial kernel and initrd ● Creates an initial ramdisk and asks ip addres, hostinfo, and installserver information. ● download boel image and setup a minimal environment (rsync) ● Based on the hostinfo downloads the host specific script (autoinstallscript)

20. Let‘s talk about Xen ● ParaVirtualisation ● You‘ve already seen Xen enough internals stuff these days ;) ● Going Mainstream real fast ! ● Stable and performant platform ● Scriptable

21. Xen “ParaVirtualization” Provides some exposure to the actual hardware – Performance increase – OS Needs to be modified – Multiplexes resources at OS granularity (vs Process level granularity) ● 100 virtual OS's per machine

22. Xen ● X86 supports 4 levels of privileges – 0 for OS, and 3 for applications – Xen downgrades the privilege of Oses ● Xen exposes a set of simple device abstractions

23. Porting an OS to Xen ● X86 Like ● Priviledged instructions – are replaced with Xen hypercalls – for Linux 2.6 only arch-dep files were modified ● Network Device Driver ● Block Device Driver ● Source code availaiblity ! ● <2% of code-base

24. Porting an OS to Xen ● X86 Like ● Priviledged instructions – are replaced with Xen hypercalls – for Linux 2.6 only arch-dep files were modified ● Network Device Driver ● Block Device Driver ● Source code availaiblity ! ● <2% of code-base

25. Xen & Disk Access ● Only Domain0 has direct access to disks ● Other domains need to use virtual block devices – Use the I/O ring – Reorder requests prior to enqueuing them on the ring – If permitted, Xen will also reorder requests to improve performance

26. Xen 3.0 Arch Event Channel Virtual MMUVirtual CPU Control IF Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE) Native Device Driver GuestOS (XenLinux) Device Manager & Control s/w Native Device Driver GuestOS (XenLinux) Unmodified User Software Front-End Device Drivers GuestOS (XenLinux) Unmodified User Software Front-End Device Drivers Unmodified GuestOS (WinXP)) Unmodified User Software Safe HW IF Xen Virtual Machine Monitor Back-End VT-x AGP ACPI PCI SMP

27. Why Typical Linux Installs don‘t work ● No Xen enabled Distribution CD‘s (yet) ● No „install“ tool (anaconda etc) ● No booting from device X and copying data – Network – Cd – Disk

28. Typical Xen DomainU installations ● „Copy“ an existing image ● Yum –installroot=/path/ -y groupinstall Base ● Debootstrap ● Urpmi –root=/path basesystem urpmi ssh-server ● Yast ● rpmstrap

29. Existing Alternatives ● The alternatives – Preseeding d-i – OLS Paper (snapshotting + containers) ● Issues with them : – Distro Specific – Valid in Isolated environments

30. Building your own ? What do we need ? ● Installing a basic image in a chroot ● Creating „partitions“ ● Creating/Updating configurations ● Booting

31. „Modifying“ an Autoinstall Script create_vhost : { ● Create LVM partitions ● Chroot ● Rsync ● Configure }

32. Generation of scripts ● mksiimage based template ● Creates – lvm create script – xen config – fstab

33. The full monty Install hostX reboot into non xen, lvm enabled kernel if (xen=enabled) then add xen packages from repository overrides include /etc/xen/auto files for $vhost in /etc/xen/auto/* ; create_vhost done reboot into xen enabled / lvm enabled kernel

34. From here to.. ● Management of virtual machine is identical to physical machine ● Deploy new virtual machine is as easy as running create_vhost $hostname ● Cfengine and repositories are being used as within physical machines

35. Future Work ● Eliminate 2 phase bootstrap DONE! ● Integrate with mksiimage ● Look at COW enhancements

36. Conclusions: ● Pass the 10 th floor test ● Hybrid deployments enable best of breed deploying techniques ● Not „Yet Another Systeminstallation Tool“ :), but tools you already use for a physical purpose !

37. Thank you Contact: Kris Buytaert Kris.Buytaert@x-tend.be Further Reading http://www.x-tend.be/~kb/ http://www.infrastructures.org/ http://www.sisuite.org/

38. ? ! ● Questions ?