Lightning talk: Experiences with Opensource Virtualisation and Storage Jon Agland Jisc

Background Sharing some experiences from by time in Jisc RSC Wales / Swansea University – Development Infrastructure/Network Blogged in 2010 SAN and Virtualisation on a Budget of Zero2010 SAN and Virtualisation on a Budget of Zero ESXi and Openfiler – Mostly using Rackmount kit from 2005/06 Since then DRBD Linux box with iSCSI Target and NFS Physically relocated ~2011 Rackmount kit is not suitable for our new dev room (a spare office) Lack of 64-bit support

So we bought some Microservers 5 Microservers – n54l – G8 (£100-£200 each) 2 for Storage 3 for Virtualisation Upgrade each to 16GB RAM 8 x 1 TB Hard Disks Speed and spindles over capacity Re-use of a Gigabit switch ~£1.5-2k

What did we choose? Replaced ESXi with Citrix XenServer Easy in a Development Environment! Kept a single ESXi box in dev Initially used Openfiler Already familiar Openfiler last release 2011 Consume a disk for it’s OS Despite using ~10GB Moved to FreeNAS All software was Free / Gratis / Libre

Citrix XenServer Based on OpenSource Xen Commercial Support Free with limitations Support Automated Updates How often to you update your Hypervisor? No management server All nodes in a Cluster can be used for Management (one is master) Management Data on a shared storage volume Comparable to VMWare Enterprise Plus

FreeNAS Install to a USB Stick Supports NFS, iSCSI, Samba, FTP, Rsync and more… Based on FreeBSD Web interface A Lack of Distributed Storage support e.g. DRBD 

What did we use it for? eduroam – RADIUS Servers (FreeRADIUS, NPS etc..) pfSense Walled garden testing etc.. Config development for Live platform Windows 2012 – Active Directory BigBlueButton

Lessons Storage 20 VMs on a Single (pair of) Hard Disk(s) Doesn’t work well! RAID10 RAM needed on Storage nodes Original NAS nodes had >2GB iSCSI and NFS don’t go via Firewall very well! Stick to single network if possible Jumbo frames

Lessons Virtualisation (with XenServer) Don’t use DHCP for nodes (especially if you cluster!) VM Conversion headaches Never really solved those :\ Physical Conversion was good Use a SAN/NAS or ‘Decent’ RAID controller

Lessons Kit Time to chuck your 32-bit kit (or your SPARC kit) Xen, OpenFiler, FreeNAS only run on 64-bit You could use Linux/Solaris directly and install iSCSI target Running FreeNAS on 1GB of RAM isn’t pretty Running Virtualisation on 4GB also pretty painful. Especially if you like Windows! 8GB you can manage a small number of VMs 16GB is a more realistic minimum.

Backups HP Microserver (n40l ~2012 – G6) 8GB RAM, 4TB Storage with RAID10 - ~£800 Used ZFS under Linux Would probably use FreeNAS today Originally a Solaris product but also available in Linux via FUSE De-Dupe Sounds awesome. Needs lots of memory (so GB+) Maybe even an SSD. Snapshots Just as good for backups Uses less memory 14 days of backup.

Home Another HP Microserver (n40l ~2012) OpenSource Xen on Linux 7 VMs (Linux of course) in 4GB RAM Linux Software MD RAID Highly recommend over any ‘cheap’ RAID LVM - Logical Volume Manager Performance issues with AIO/Tap in Xen (bug?) Built-in 2012 – need to update it! Very quiet

Home – What? Why? What? External Services (available over IPv4 and IPv6) Mail Web – inc OwnCloud Local Services Storage (Samba/SCP) OpenLDAP (User Directory) Nedi / Cacti (Monitoring) Why?

Renting a Virtual Server Bitfolk.com (London) Not a ‘Jisc’ recommendation… but… I’m using privately 480MB Memory, 10GB Storage, 400GB Data - £8.99+VAT per month Xen Virtualisation, Linux VMs The chap who runs it is very Knowledgeable Honest Responsive Gives discounts OVH.co.uk (France, France or the US) Slightly better spec 1024MB Memory, 10GB Storage, 10,000GB Data – £1.69 per month OpenVZ

Ceph Simon told me to mention Ceph Distributed clustered file storage Self-healing Self-managing Supported by OpenStack/CloudStack SuSE Enterprise Storage going to be based on it. SuSE Enterprise Storage

Things I would like to do… Amazon Web Services Computing clusters – Raspberry PIs? Puppet Get my hands on a HP Microserver G8

Things you might like to do.. Install VirtualBox Try it out on your PC or Laptop If your brave and have access to ‘tin’ FreeNAS XenServer Rent a VM in a Datacentre

Questions?