Amazon Web Services Justin DeBrabant CIS Advanced Systems - Fall 2013

Cloud Computing A “pay what you use” service for hardware/software resources – analogy: utilities Amazon Web Services (AWS) is a cloud computing provider – there are many others CIS Advanced Systems - Fall 2013

Benefits of Cloud Computing elasticity – scale out or scale up easily* *can be difficult depending on your application – scale down if demand decreases no upfront capital – IT costs can be high, prohibitively so for a startup CIS Advanced Systems - Fall 2013

*Slide Courtesy Jeff Barr, Amazon

Available Services S3 (Simple Storage Service) – store and retrieve data EC2 (Elastic Computer Cloud) – scalable compute capacity EBS (Elastic Block Store) – predictable storage volumes that can be attached to a running EC2 instance – highly available, highly reliable CIS Advanced Systems - Fall 2013

Available Services (cont’d) Databases – Relational: RDS (Relational Database Service) – NoSQL: DynamoDB – Scale-Out Caching: ElastiCache EMR (Elastic MapReduce) – Hadoop-based – uses S3 and EC2 CIS Advanced Systems - Fall 2013

Other Important Terms vCPU (virtual CPU) – 1 hardware thread ECU – Amazon’s compute unit instance – EC2 virtual machine instance storage – amount of persistent storage for an instance AMI (Amazon Machine Image) – bundled software to be deployed on an instance CIS Advanced Systems - Fall 2013

Various Storages S3 – available to instances within same region – analogy: DropBox Instance Storage – available during the life of a specific instance, not shared across instances – analogy: local disk EBS – persistent block storage that can be added to local instances, but is preserved across instance lifetimes can be attached to only one running instance – analogy: removable storage media (e.g. external hard drive) CIS Advanced Systems - Fall 2013

AWS Regions Amazon maintains different clusters worldwide – helps provide locality as well as improved geographic redundancy each AWS service is available in specific regions – e.g. S3 us-east-1 (Northern Virginia) us-west-1 (Northern California) us-west-2 (Oregon) eu-west-1 (Ireland) ap-southeast-1 (Singapore) ap-southeast-2 (Sydney) ap-northeast-1 (Tokyo) sa-east-1 (Sao Paolo) CIS Advanced Systems - Fall 2013

S3 data organized into buckets – buckets names must be unique across S3 – data in bucket can be organized any way you wish direct mapping between buckets and AWS subdomains – e.g. bucket name: mybucket, file: myfile.txt must specify region of bucket CIS Advanced Systems - Fall 2013

S3 (cont’d) Guarantees: – % durability – 99.99% availability – designed to be recoverable against two simultaneous data losses SLAs – if above guarantees are not met, partial refunds are given CIS Advanced Systems - Fall 2013

S3 Pricing storage used – pay per GB tiered, firstTB is most expensive – can reduce cost by reducing redundancy – data transfer – all data transfer in is free – data transfer out is not pay per GB CIS Advanced Systems - Fall 2013

EC2 machines classified as instances each instance is a virtual machine every instance type has different resources – vCPU – ECU – network bandwidth – memory – instance storage CIS Advanced Systems - Fall 2013

Instance Families General Purpose – balanced, all-purpose machines with up to 8 vCPUs and 30 GB of memory Memory Optimized – up to 244 GB of memory Compute Optimized – up to 32 vCPUs and 10GBit network perf. Storage Optimized – SSD and big disk backed instance storage CIS Advanced Systems - Fall 2013

Instance Software Instances are virtual machines, and can support a wide range of operating systems – Linux (free) – Windows (have to pay for additional license) Can create or use existing AMIs – prepackaged software bundle includes OS and key software – e.g Oracle, MySQL, Apache WebServer – for this class, we’ll use existing AMIs CIS Advanced Systems - Fall 2013

Buying Instances On-Demand Instances – request an instance, use it, terminate it Reserved Instances – akin to buying a machine – permanently own instance Spot Instances – bid for unused EC2 capacity – price can vary greatly – can set maximum bid and wait for available machines at or below that price CIS Advanced Systems - Fall 2013

On-Demand Instance Pricing billed per instance hour – rounded up at per instance granularity! – e.g. 2 instance for 20 mins = 2 instance hours free tier – 750 hours with micro instance – 30 GB EBS storage – 1 GB of data transfer CIS Advanced Systems - Fall 2013

EMR easy interface to launch Hadoop jobs using EC2 – just upload runnable jar containing Hadoop job pricing – standard instance pricing + EMR instance pricing – e.g. 8 node cluster of m1.small for 1 hour (8 *.06) + (8 *.015) CIS Advanced Systems - Fall 2013

Accessing AWS Resources Console – easy interactive GUI – can launch EC2 instances, create S3 buckets, and start EMR jobs (among other things) – demo! CLI (command line interface) – must install tools e/SettingUp_CommandLine.html e/SettingUp_CommandLine.html CIS Advanced Systems - Fall 2013

Accessing a Running Instance use SSH – must have private key file for key specified at instance launch – ssh –i once SSH’d in, can use like any other machine use SCP to copy in and out files – Warning: will be stored on ephemeral instance storage! CIS Advanced Systems - Fall 2013

Questions? CIS Advanced Systems - Fall 2013