1. Amazon Web Services Justin DeBrabant CIS 570 - Advanced Systems - Fall 2013

2. Cloud Computing A “pay what you use” service for hardware/software resources – analogy: utilities Amazon Web Services (AWS) is a cloud computing provider – there are many others CIS 570 - Advanced Systems - Fall 2013

3. Benefits of Cloud Computing elasticity – scale out or scale up easily* *can be difficult depending on your application – scale down if demand decreases no upfront capital – IT costs can be high, prohibitively so for a startup CIS 570 - Advanced Systems - Fall 2013

4. *Slide Courtesy Jeff Barr, Amazon

5. Available Services S3 (Simple Storage Service) – store and retrieve data EC2 (Elastic Computer Cloud) – scalable compute capacity EBS (Elastic Block Store) – predictable storage volumes that can be attached to a running EC2 instance – highly available, highly reliable CIS 570 - Advanced Systems - Fall 2013

6. Available Services (cont’d) Databases – Relational: RDS (Relational Database Service) – NoSQL: DynamoDB – Scale-Out Caching: ElastiCache EMR (Elastic MapReduce) – Hadoop-based – uses S3 and EC2 CIS 570 - Advanced Systems - Fall 2013

7. Other Important Terms vCPU (virtual CPU) – 1 hardware thread ECU – Amazon’s compute unit instance – EC2 virtual machine instance storage – amount of persistent storage for an instance AMI (Amazon Machine Image) – bundled software to be deployed on an instance CIS 570 - Advanced Systems - Fall 2013

8. Various Storages S3 – available to instances within same region – analogy: DropBox Instance Storage – available during the life of a specific instance, not shared across instances – analogy: local disk EBS – persistent block storage that can be added to local instances, but is preserved across instance lifetimes can be attached to only one running instance – analogy: removable storage media (e.g. external hard drive) CIS 570 - Advanced Systems - Fall 2013

9. AWS Regions Amazon maintains different clusters worldwide – helps provide locality as well as improved geographic redundancy each AWS service is available in specific regions – e.g. S3 us-east-1 (Northern Virginia) us-west-1 (Northern California) us-west-2 (Oregon) eu-west-1 (Ireland) ap-southeast-1 (Singapore) ap-southeast-2 (Sydney) ap-northeast-1 (Tokyo) sa-east-1 (Sao Paolo) http://docs.aws.amazon.com/general/latest/gr/rande.html CIS 570 - Advanced Systems - Fall 2013

10. S3 data organized into buckets – buckets names must be unique across S3 – data in bucket can be organized any way you wish direct mapping between buckets and AWS subdomains – e.g. bucket name: mybucket, file: myfile.txt http://mybucket.aws.amazon.com/myfile.txt must specify region of bucket CIS 570 - Advanced Systems - Fall 2013

11. S3 (cont’d) Guarantees: – 99.999999999% durability – 99.99% availability – designed to be recoverable against two simultaneous data losses SLAs – if above guarantees are not met, partial refunds are given CIS 570 - Advanced Systems - Fall 2013

12. S3 Pricing storage used – pay per GB tiered, firstTB is most expensive – can reduce cost by reducing redundancy – http://aws.amazon.com/s3/pricing/ http://aws.amazon.com/s3/pricing/ data transfer – all data transfer in is free – data transfer out is not pay per GB CIS 570 - Advanced Systems - Fall 2013

13. EC2 machines classified as instances each instance is a virtual machine every instance type has different resources – vCPU – ECU – network bandwidth – memory – instance storage CIS 570 - Advanced Systems - Fall 2013

14. Instance Families General Purpose – balanced, all-purpose machines with up to 8 vCPUs and 30 GB of memory Memory Optimized – up to 244 GB of memory Compute Optimized – up to 32 vCPUs and 10GBit network perf. Storage Optimized – SSD and big disk backed instance storage CIS 570 - Advanced Systems - Fall 2013

15. Instance Software Instances are virtual machines, and can support a wide range of operating systems – Linux (free) – Windows (have to pay for additional license) Can create or use existing AMIs – prepackaged software bundle includes OS and key software – e.g Oracle, MySQL, Apache WebServer – for this class, we’ll use existing AMIs CIS 570 - Advanced Systems - Fall 2013

16. Buying Instances On-Demand Instances – request an instance, use it, terminate it Reserved Instances – akin to buying a machine – permanently own instance Spot Instances – bid for unused EC2 capacity – price can vary greatly – can set maximum bid and wait for available machines at or below that price CIS 570 - Advanced Systems - Fall 2013

17. On-Demand Instance Pricing billed per instance hour – rounded up at per instance granularity! – e.g. 2 instance for 20 mins = 2 instance hours free tier – 750 hours with micro instance – 30 GB EBS storage – 1 GB of data transfer http://aws.amazon.com/ec2/pricing/ CIS 570 - Advanced Systems - Fall 2013

18. EMR easy interface to launch Hadoop jobs using EC2 – just upload runnable jar containing Hadoop job pricing – standard instance pricing + EMR instance pricing – e.g. 8 node cluster of m1.small for 1 hour (8 *.06) + (8 *.015) CIS 570 - Advanced Systems - Fall 2013

19. Accessing AWS Resources Console – easy interactive GUI – can launch EC2 instances, create S3 buckets, and start EMR jobs (among other things) – demo! CLI (command line interface) – must install tools http://docs.aws.amazon.com/AWSEC2/latest/UserGuid e/SettingUp_CommandLine.html http://docs.aws.amazon.com/AWSEC2/latest/UserGuid e/SettingUp_CommandLine.html CIS 570 - Advanced Systems - Fall 2013

20. Accessing a Running Instance use SSH – must have private key file for key specified at instance launch – ssh –i ec2-user@public-dns once SSH’d in, can use like any other machine use SCP to copy in and out files – Warning: will be stored on ephemeral instance storage! CIS 570 - Advanced Systems - Fall 2013

21. Questions? CIS 570 - Advanced Systems - Fall 2013