Science Cloud Summer School OpenStack and Eucalyptus on FutureGrid Slides are at Presenters: Javier Diaz Gregor von Laszewski

Science Cloud Summer School Before you can use OpenStack and Eucalyptus Please make sure you have a portal account o Please make sure you are part of a valid FG project o You can either create a new one or o You can join an existing one with permission of the Lead Please make sure the project you have is approved and valid OpenStack and Eucalyptus accounts are automatically created when previous requirements are fulfilled

Science Cloud Summer School OpenStack vs Eucalyptus Both OpenStack and Eucalyptus are IaaS cloud frameworks (similar functionality) We are going to focus on OpenStack during this tutorial We make use of the EC2 interface thought Euca2ools ALL commands explained here work in the same way in Eucalyptus

Science Cloud Summer School OpenStack OpenStack is a collection of open source components to deliver public and private IaaS clouds Components: Nova, Swift, Glance, Keystone, and Quantum IaaS Cloud Services allows users to manage: – VMs, Virtual networks, storage resources

Science Cloud Summer School OpenStack Functionality Amazon AWS Interface Compatibility Flexible Clustering and Availability Zones Access Control List (ACL) with policies management Network Management, Security Groups, Traffic Isolation Cloud Semantics and Self-Service Capability – Image registration and image attribute manipulation Bucket-Based Storage Abstraction (S3-Compatible) Block-Based Storage Abstraction (EBS-Compatible) Hypervisor support: Xen, KVM, VMware Vsphere, LXC, UML and MS HyperV

Science Cloud Summer School OpenStack Testbed OpenStack is available to FutureGrid Users on the India cluster Users can make use of a maximum of 24 nodes on India Instances Types TypeMemoryVCPUsRoot (vda)Ephemeral (vdb) m1.tiny512MB10GB m1.small2048MB110GB20GB m1.medium4096MB210GB40GB m1.large8192MB415GB40GB m1.xlarge16384MB420GB40GB m1.xxlarge16384MB425GB60GB

Science Cloud Summer School Log into India cluster Example: My username is jdiaz $ $ ssh

Science Cloud Summer School Credentials in your $HOME directory under.futuregrid/openstack/ (In Eucalyptus this was.futuregrid/eucalyptus) Load environment variables User Credentials $ unzip ~/.futuregrid/openstack/openstack-essex-jdiaz- india.zip -d ~/openstack $ source ~/openstack/novarc

Science Cloud Summer School Install/Load Euca2ools Euca2ools are the command line clients used to interact with OpenStack. If using your own platform, install euca2ools bundle from ls ls o Instructions for various Linux platforms are available on the download page. On FutureGrid, load the Euca2ools module $ module load euca2ools

Science Cloud Summer School Euca2ools Testing your setup. Use euca-describe- availability-zones to test the setup List the existing images using euca-describe- images $ euca-describe-availability-zones AVAILABILITYZONEindiaavailable $ euca-describe-images IMAGE ami centos2012/euca-centos x86_64.img.manifest.xmavailable public x86_64 machine aki ari instance-store IMAGE ami debian2011/euca-debian x86_64.img.manifest.xml available public x86_64 machine aki e ari finstance-store …

Science Cloud Summer School Key Management Before you instanciate a VM, you need to create at least one key pair. This key pair will be injected into the VM, allowing you to SSH into the VM Create a keypair and add the public key to OpenStack (Replace userkey with your username i.e. jdiazkey) Fix the permissions on the generated private key $ euca-add-keypair userkey > userkey.pem $ chmod 0600 userkey.pem $ euca-describe-keypairs KEYPAIR userkey 0d:d8:7c:2c:bd:85:af:7e:ad:8d: 09:b8:ff:b0:54:d5:8c:66:86:5d

Science Cloud Summer School Image Instantiation Now we are ready to start a VM using one of the pre-existing images. We need the ami-id of the image that we wish to start (euca-describe-images) We use the euca-run-instances command to start a VM with the image ami $ euca-run-instances -k userkey -n 1 ami t m1.small RESERVATION r-zvtbbj8j default INSTANCE i f ami server-1854 server-1854 pending userkey 0 m1.small T15:49:46.000Zunknown zone aki e ari f

Science Cloud Summer School Monitoring euca-describe-instances shows the status of the VMs $ euca-describe-instances i f RESERVATION r-cbcagzl eef90047fbb4eb9ec92f22a1e3 default INSTANCE i f ami server-1855 server-1855 pending userkey 0m1.small T15:49:46.000Z india aki e ari f Shortly after… $ euca-describe-instances i f RESERVATION r-cbcagzl eef90047fbb4eb9ec92f22a1e3 default INSTANCE i f ami server-1855 running userkey 0m1.small T15:49:46.000Z india aki e ari f

Science Cloud Summer School VM Access Please, exit from the VM to continue with the Tutorial The ssh private key that was generated earlier can now be used to log into the VM $ ssh -i userkey.pem # exit

Science Cloud Summer School VM Access VMs also have a private IP that can be used for internal communication between VMs (OpenStack Essex does not show the private IP) On India you can identify the private IP by executing The ssh private key that was generated earlier can now be used to log into the VM $ ssh -i userkey.pem $ nslookup server-1855 novadns

Science Cloud Summer School VM Access Side note – By default, in Ubuntu images you need to login as ubuntu user (NOT our case) – You can change that in your image by editing the file /etc/cloud/cloud.cfg (cloud-init package)

Science Cloud Summer School Create Access Rule (it should not be needed) euca-authorize -P tcp -p 22 -s /0 default

Science Cloud Summer School Nova Volumes Nova-volume provides persistent block storage compatible with Amazon’s Elastic Block Store The storage in the instances is non-persistent and it gets lost when the instance is terminated Persistent volumes keeps data generated during instance life time after the instance is terminated Volumes are accessed via iSCSI

Science Cloud Summer School Create Volumes Create a 1GB Volume in the India zone List available volumes $ euca-create-volume -s 1 -z india VOLUMEvol indiacreating T15:15:47.244Z $ euca-describe-volumes VOLUMEvol indiain-use T21:39:47.000Z ATTACHMENTvol i f/dev/vdc attached VOLUMEvol indiaavailable T21:44:30.000Z VOLUMEvol a 30indiaavailable T21:45:37.000Z

Science Cloud Summer School Attach Volume Attach volume to a running instance A volume can only be attached to one instance. Euca-describe-volumes will show the volume status as “attached” An additional SCSI disk is created in the instance Although we specified the device, it may differ if that device already exists (look into /dev, /var/log/syslog, /dev/log/messages or use fdisk -l to find the new device) $ euca-attach-volume -i i f -d /dev/vdc vol

Science Cloud Summer School Using the New Disk Log into the instance again Format disk (skip this step if you want to reuse data stored) Mount disk Exit from the instance to continue # mkfs /dev/vdc $ ssh -i userkey.pem # mount /dev/vdc /mnt # exit

Science Cloud Summer School Detach Volumes Volumes are automatically detached when instance is terminated Detach volume If you detach the volume while instance is running, it loses access to the disk. Make sure that you umount (umount /mnt) the disk before you detach the volume. If you terminate the instance, the volume is automatically detached $ euca-detach-volume vol

Science Cloud Summer School Volume Snapshots Snapshots are useful to create backups or replicate volumes in different zones Create Snapshot List Snapshots $ euca-create-snapshot -d ’Testing snapshot’ vol SNAPSHOT snap vol creating T14:22:21.728Z 0% Testing snapshot $ euca-describe-snapshots SNAPSHOT snap vol available T14:22:21.000Z 100%

Science Cloud Summer School Create Volume from Snapshot (currently it does not work in OpenStack Essex) The snapshot must be in available status (100% completed) New volume can be bigger if desired Create a 2GB volume from snapshop $ euca-create-volume -s 2 --snapshot snap z india VOLUME vol snap india creating T14:47:07.916Z

Science Cloud Summer School Image Registration (1/4) We will use an CentOS 5 image to test the image registration o Download the gzipped tar ball Uncompress and Untar the archive $ wget i120/test-image/centos5.tgz $ tar vxfz centos5.tgz

Science Cloud Summer School Image Registration (2/4) Next we bundle the image with a kernel and a ramdisk using the euca-bundle-image command – We will use a kernel already registered. – euca-describe-images returns the kernel and ramdisk IDs that we need $ euca-bundle-image -i centos5.img --kernel aki e -- ramdisk ari f Checking image Encrypting image Splitting image... Part: centos5.img.part.00 …….. Part: centos5.img.part.35 Generating manifest /tmp/centos5.img.manifest.xml

Science Cloud Summer School Image Registration (3/4) Upload the image (euca-upload-bundle) Register the image (euca-register) $ euca-register jdiaz-bucket/centos5.img.manifest.xml IMAGEami $ euca-upload-bundle -b jdiaz-bucket -m /tmp/centos5.img.manifest.xml Checking bucket: jdiaz-bucket Creating bucket: jdiaz-bucket Uploading manifest file Uploading part: centos5.img.part.00 …. Uploading part: centos5.img.part.35 Uploaded image as jdiaz-bucket/centos5.img.manifest.xml

Science Cloud Summer School Image Registration (4/4) You cannot run instances until the image is in available status Check status $ euca-describe-instances ami IMAGE ami jdiaz-bucket/centos5.img.manifest.xml available private x86_64 machine aki e ari f instance-store

Science Cloud Summer School Image/Volume Deletion and Instance Termination You can delete your volume You can delete your image (Please do NOT delete other users’ images) You can terminate your VM $ euca-deregister ami $ euca-terminate-instances i f $ euca-delete-volume vol

Science Cloud Summer School More Information We keep this tutorial updated in the FutureGrid portal k k