1. Science Cloud Summer School 2012 https://portal.futuregrid.org OpenStack and Eucalyptus on FutureGrid https://portal.futuregrid.org/tutorials/openstack Slides are at https://portal.futuregrid.org/projects/241 https://portal.futuregrid.org/tutorials/openstack https://portal.futuregrid.org/projects/241 Presenters: Javier Diaz Gregor von Laszewski

2. Science Cloud Summer School 2012 https://portal.futuregrid.org Before you can use OpenStack and Eucalyptus Please make sure you have a portal account o https://portal.futuregrid.org Please make sure you are part of a valid FG project o You can either create a new one or o You can join an existing one with permission of the Lead Please make sure the project you have is approved and valid OpenStack and Eucalyptus accounts are automatically created when previous requirements are fulfilled

3. Science Cloud Summer School 2012 https://portal.futuregrid.org OpenStack vs Eucalyptus Both OpenStack and Eucalyptus are IaaS cloud frameworks (similar functionality) We are going to focus on OpenStack during this tutorial We make use of the EC2 interface thought Euca2ools ALL commands explained here work in the same way in Eucalyptus

4. Science Cloud Summer School 2012 https://portal.futuregrid.org OpenStack OpenStack is a collection of open source components to deliver public and private IaaS clouds Components: Nova, Swift, Glance, Keystone, and Quantum IaaS Cloud Services allows users to manage: – VMs, Virtual networks, storage resources

5. Science Cloud Summer School 2012 https://portal.futuregrid.org OpenStack Functionality Amazon AWS Interface Compatibility Flexible Clustering and Availability Zones Access Control List (ACL) with policies management Network Management, Security Groups, Traffic Isolation Cloud Semantics and Self-Service Capability – Image registration and image attribute manipulation Bucket-Based Storage Abstraction (S3-Compatible) Block-Based Storage Abstraction (EBS-Compatible) Hypervisor support: Xen, KVM, VMware Vsphere, LXC, UML and MS HyperV

6. Science Cloud Summer School 2012 https://portal.futuregrid.org OpenStack Testbed OpenStack is available to FutureGrid Users on the India cluster Users can make use of a maximum of 24 nodes on India Instances Types https://portal.futuregrid.org TypeMemoryVCPUsRoot (vda)Ephemeral (vdb) m1.tiny512MB10GB m1.small2048MB110GB20GB m1.medium4096MB210GB40GB m1.large8192MB415GB40GB m1.xlarge16384MB420GB40GB m1.xxlarge16384MB425GB60GB

7. Science Cloud Summer School 2012 https://portal.futuregrid.org Log into India cluster Example: My username is jdiaz $ ssh @india.futuregrid.org $ ssh jdiaz@india.futuregrid.org

8. Science Cloud Summer School 2012 https://portal.futuregrid.org Credentials in your $HOME directory under.futuregrid/openstack/ (In Eucalyptus this was.futuregrid/eucalyptus) Load environment variables User Credentials $ unzip ~/.futuregrid/openstack/openstack-essex-jdiaz- india.zip -d ~/openstack $ source ~/openstack/novarc

9. Science Cloud Summer School 2012 https://portal.futuregrid.org Install/Load Euca2ools Euca2ools are the command line clients used to interact with OpenStack. If using your own platform, install euca2ools bundle from http://www.eucalyptus.com/downloads/euca2oo ls http://www.eucalyptus.com/downloads/euca2oo ls o Instructions for various Linux platforms are available on the download page. On FutureGrid, load the Euca2ools module $ module load euca2ools

10. Science Cloud Summer School 2012 https://portal.futuregrid.org Euca2ools Testing your setup. Use euca-describe- availability-zones to test the setup List the existing images using euca-describe- images https://portal.futuregrid.org $ euca-describe-availability-zones AVAILABILITYZONEindiaavailable $ euca-describe-images IMAGE ami-00000012centos2012/euca-centos-2012.1.14- x86_64.img.manifest.xmavailable public x86_64 machine aki-00000010 ari- 00000011instance-store IMAGE ami-00000013debian2011/euca-debian-2011.07.02- x86_64.img.manifest.xml available public x86_64 machine aki-0000000e ari- 0000000finstance-store …

11. Science Cloud Summer School 2012 https://portal.futuregrid.org Key Management Before you instanciate a VM, you need to create at least one key pair. This key pair will be injected into the VM, allowing you to SSH into the VM Create a keypair and add the public key to OpenStack (Replace userkey with your username i.e. jdiazkey) Fix the permissions on the generated private key $ euca-add-keypair userkey > userkey.pem $ chmod 0600 userkey.pem $ euca-describe-keypairs KEYPAIR userkey 0d:d8:7c:2c:bd:85:af:7e:ad:8d: 09:b8:ff:b0:54:d5:8c:66:86:5d

12. Science Cloud Summer School 2012 https://portal.futuregrid.org Image Instantiation Now we are ready to start a VM using one of the pre-existing images. We need the ami-id of the image that we wish to start (euca-describe-images) We use the euca-run-instances command to start a VM with the image ami-00000017 $ euca-run-instances -k userkey -n 1 ami-00000017 -t m1.small RESERVATION r-zvtbbj8j default INSTANCE i-0000073f ami-00000017 server-1854 server-1854 pending userkey 0 m1.small 2012-07-09T15:49:46.000Zunknown zone aki- 0000000e ari-0000000f

13. Science Cloud Summer School 2012 https://portal.futuregrid.org Monitoring euca-describe-instances shows the status of the VMs $ euca-describe-instances i-0000073f RESERVATION r-cbcagzl4 461884eef90047fbb4eb9ec92f22a1e3 default INSTANCE i-0000073f ami-00000033 server-1855 server-1855 pending userkey 0m1.small 2012-07-09T15:49:46.000Z india aki-0000000e ari- 0000000f Shortly after… $ euca-describe-instances i-0000073f RESERVATION r-cbcagzl4 461884eef90047fbb4eb9ec92f22a1e3 default INSTANCE i-0000073f ami-00000033 149.165.158.159 server-1855 running userkey 0m1.small 2012-07-09T15:49:46.000Z india aki- 0000000e ari-0000000f

14. Science Cloud Summer School 2012 https://portal.futuregrid.org VM Access Please, exit from the VM to continue with the Tutorial The ssh private key that was generated earlier can now be used to log into the VM $ ssh -i userkey.pem root@149.165.158.159 # exit

15. Science Cloud Summer School 2012 https://portal.futuregrid.org VM Access VMs also have a private IP that can be used for internal communication between VMs (OpenStack Essex does not show the private IP) On India you can identify the private IP by executing The ssh private key that was generated earlier can now be used to log into the VM $ ssh -i userkey.pem root@149.165.158.159 $ nslookup server-1855 novadns

16. Science Cloud Summer School 2012 https://portal.futuregrid.org VM Access Side note – By default, in Ubuntu images you need to login as ubuntu user (NOT our case) – You can change that in your image by editing the file /etc/cloud/cloud.cfg (cloud-init package)

17. Science Cloud Summer School 2012 https://portal.futuregrid.org Create Access Rule (it should not be needed) euca-authorize -P tcp -p 22 -s 0.0.0.0/0 default

18. Science Cloud Summer School 2012 https://portal.futuregrid.org Nova Volumes Nova-volume provides persistent block storage compatible with Amazon’s Elastic Block Store The storage in the instances is non-persistent and it gets lost when the instance is terminated Persistent volumes keeps data generated during instance life time after the instance is terminated Volumes are accessed via iSCSI

19. Science Cloud Summer School 2012 https://portal.futuregrid.org Create Volumes Create a 1GB Volume in the India zone List available volumes https://portal.futuregrid.org $ euca-create-volume -s 1 -z india VOLUMEvol-000000311indiacreating2012-07- 10T15:15:47.244Z $ euca-describe-volumes VOLUMEvol-00000027 100indiain-use2012-06- 06T21:39:47.000Z ATTACHMENTvol-00000027i-0000070f/dev/vdc attached VOLUMEvol-00000028 50indiaavailable 2012-06-06T21:44:30.000Z VOLUMEvol-0000002a 30indiaavailable 2012-06-06T21:45:37.000Z

20. Science Cloud Summer School 2012 https://portal.futuregrid.org Attach Volume Attach volume to a running instance A volume can only be attached to one instance. Euca-describe-volumes will show the volume status as “attached” An additional SCSI disk is created in the instance Although we specified the device, it may differ if that device already exists (look into /dev, /var/log/syslog, /dev/log/messages or use fdisk -l to find the new device) $ euca-attach-volume -i i-0000073f -d /dev/vdc vol-0000031

21. Science Cloud Summer School 2012 https://portal.futuregrid.org Using the New Disk Log into the instance again Format disk (skip this step if you want to reuse data stored) Mount disk Exit from the instance to continue # mkfs /dev/vdc $ ssh -i userkey.pem root@149.165.158.159 # mount /dev/vdc /mnt # exit

22. Science Cloud Summer School 2012 https://portal.futuregrid.org Detach Volumes Volumes are automatically detached when instance is terminated Detach volume If you detach the volume while instance is running, it loses access to the disk. Make sure that you umount (umount /mnt) the disk before you detach the volume. If you terminate the instance, the volume is automatically detached $ euca-detach-volume vol-00000031

23. Science Cloud Summer School 2012 https://portal.futuregrid.org Volume Snapshots Snapshots are useful to create backups or replicate volumes in different zones Create Snapshot List Snapshots $ euca-create-snapshot -d ’Testing snapshot’ vol-00000027 SNAPSHOT snap-00000001 vol-00000027 creating 2012-07- 16T14:22:21.728Z 0% Testing snapshot $ euca-describe-snapshots SNAPSHOT snap-00000001 vol-00000027 available 2012-07- 16T14:22:21.000Z 100%

24. Science Cloud Summer School 2012 https://portal.futuregrid.org Create Volume from Snapshot (currently it does not work in OpenStack Essex) The snapshot must be in available status (100% completed) New volume can be bigger if desired Create a 2GB volume from snapshop $ euca-create-volume -s 2 --snapshot snap-00000001 -z india VOLUME vol-00000032 2 snap- 00000001 india creating 2012-07-16T14:47:07.916Z

25. Science Cloud Summer School 2012 https://portal.futuregrid.org Image Registration (1/4) We will use an CentOS 5 image to test the image registration o Download the gzipped tar ball Uncompress and Untar the archive $ wget i120/test-image/centos5.tgz $ tar vxfz centos5.tgz

26. Science Cloud Summer School 2012 https://portal.futuregrid.org Image Registration (2/4) Next we bundle the image with a kernel and a ramdisk using the euca-bundle-image command – We will use a kernel already registered. – euca-describe-images returns the kernel and ramdisk IDs that we need https://portal.futuregrid.org $ euca-bundle-image -i centos5.img --kernel aki-0000000e -- ramdisk ari-0000000f Checking image Encrypting image Splitting image... Part: centos5.img.part.00 …….. Part: centos5.img.part.35 Generating manifest /tmp/centos5.img.manifest.xml

27. Science Cloud Summer School 2012 https://portal.futuregrid.org Image Registration (3/4) Upload the image (euca-upload-bundle) Register the image (euca-register) $ euca-register jdiaz-bucket/centos5.img.manifest.xml IMAGEami-00000034 $ euca-upload-bundle -b jdiaz-bucket -m /tmp/centos5.img.manifest.xml Checking bucket: jdiaz-bucket Creating bucket: jdiaz-bucket Uploading manifest file Uploading part: centos5.img.part.00 …. Uploading part: centos5.img.part.35 Uploaded image as jdiaz-bucket/centos5.img.manifest.xml

28. Science Cloud Summer School 2012 https://portal.futuregrid.org Image Registration (4/4) You cannot run instances until the image is in available status Check status $ euca-describe-instances ami-00000034 IMAGE ami-00000034 jdiaz-bucket/centos5.img.manifest.xml available private x86_64 machine aki-0000000e ari-0000000f instance-store

29. Science Cloud Summer School 2012 https://portal.futuregrid.org Image/Volume Deletion and Instance Termination You can delete your volume You can delete your image (Please do NOT delete other users’ images) You can terminate your VM $ euca-deregister ami-00000034 $ euca-terminate-instances i-0000073f $ euca-delete-volume vol-00000031

30. Science Cloud Summer School 2012 https://portal.futuregrid.org More Information We keep this tutorial updated in the FutureGrid portal https://portal.futuregrid.org/tutorials/openstac k https://portal.futuregrid.org/tutorials/openstac k