AWS: EC2, S3 and Other Services B. Ramamurthy 11/16/2018

References Cloud Application Architectures: building Applications and Infrastructure in the cloud http://aws.amazon.com https://console.aws.amazon.com/ec2/home (you will have to login to the console for this) 11/16/2018

An overview of AWS AWS is Amazon’s umbrella description of all of their web-based technology services. Mainly infrastructure services: Amazon Elastic Compute Cloud (EC2) Amazon Simple Storage Service (S3) Amazon Simple Queue Service (SQS) Amazon CloudFront Amazon SimpleDB ….they keep on adding services And they are hiring http://aws.amazon.com/ec2-jobs/

Amazon EC2 EC2 provides web services API for provisioning, managing, and deprovioning virtual servers inside amazon cloud. Applications anywhere on the Internet can launch a virtual server in the amazon cloud with a single web services call (either REST or SOAP WS call) Where is SOAP API? For example, http://docs.amazonwebservices.com/AmazonEC2/dg/2006-06-26/using- soap-api.html

Overview of EC2 Components 1 1 1 0..* 1 0..* 1 0..* 11/16/2018

EC2 concepts Instance: virtual server running a guest os of your choice AMI: Pristine image of your server so that you can launch any number of instance from this. Minimally machine image will operating systems + pre-installed tools. Elastic IP address: Not static not dynamic but elastic; an IP reserved for your use; disappears once the server is terminated.

Access to an instance Access to an instance is (for ssh) is through a key pair. Another enabling technology is PKI You generate a private-public key pair, store the private key in your local hard drive, the public key is passed to the instance when it is launched. EC2 instance is configures such that root account is accessible to any user with the private key. 11/16/2018

Instance Purchase Model http://aws.amazon.com/ec2/purchasing- options/ On-demand instances: On-Demand Instances let you pay for compute capacity by the hour with no long-term commitments or upfront payments. You can increase or decrease your compute capacity depending on the demands of your application and only pay the specified hourly rate for the instances you use. 11/16/2018

Instance Purchase Model (contd.) Reserved instances: Reserved Instances let you make a low, one-time, upfront payment for an instance, reserve it for a one or three year term, and pay a significantly lower rate for each hour you run that instance. You are assured that your Reserved Instance will always be available for the operating system (e.g. Linux/UNIX or Windows) and Availability Zone in which you purchased it. 11/16/2018

Instance purchase model (contd.) Spot instances: (you bid for the price of an instance) Spot Instances provide the ability for customers to purchase compute capacity with no upfront commitment and at hourly rates usually lower than the On-Demand rate. Spot Instances allow you to specify the maximum hourly price that you are willing to pay to run a particular instance type. The Spot Price fluctuates based on supply and demand for instances, but customers will never pay more than the maximum price they have specified. 11/16/2018

Static IP By default when you launch a new instance amazon dynamically assign a private and a public IP. While this is fine for development purposed, for a real launch of a web accessible service, we need static IP. Amazon makes available what are classes elastic IPs for this purpose. Up to 5 elastic IPs can be assigned to an instance. Elastic IPs cost money even if you don’t use them; assigning and reassigning strains the system; so it costs money Allocate elastic ip and associate it with an instance. 11/16/2018

Storage Instance –store : disappears with the instance (transient) Block storage: SAN-like, persists across time S3 is independent of an instance: for archival purposes: vault: store it now and retrieve it at a later date Amazon: SimpleDB: Relational database better than MySQl or Oracle for reliability.

Amazon Cloud-front Cloud-based content distributing network enables you to place the content at the edges of the network for rapid delivery. Place the contents in S3 and run the application from anywhere and the content is moved to where the application is (to the edges).

S3 Amazon web services API support the ability to: Find buckets and objects Discover their meta data Create new buckets Upload new objects Delete existing buckets and objects When manipulating the buckets you can optionally specify where they should be stored. Use REST API preferably something that abstracts out even that: Jets3t; s3cmd (command line) BitTorrent access to S3 is also available

Bucket Naming Flat namespace Names may contain only lowercase letters, numbers, periods, underscores, and dashes, and must start with a number or letter Create your own namespace with your own bcukets

Typical Use in an organization Maintain a library of AMIs for the business May contain legacy systems or company-specific systems Harden AMIs using such software as Bastille Launch instances for non-technical but domain experts to work with. What will you provide them so that they can start working? Use S3 for archival storage and shared storage

Understand the various components S3: accessing s3, working with S3: (i) command line, (ii) through aws console (iii) programmatic access You can create directories: your own namespace You can transfer with a click of a button data in and out of S3 You can simply continue to shove data into Amazon S3 without having to worry about ever running out of space! Short-term and long-term backup facility. Access to S3 is via web services and not via file system: so relatively slow. 11/16/2018

Security Group You want to create an “instance” of a server from an already established “AMI: Amazon Machine Image” It has a elastic IP for the whole world to interact with it. How to control access to this? While creating the instance, create a new security group that will specify the policy or “rules” about the access methods Security group is somewhat similar to network segment protected by a firewall Once the server is started you cannot change the security group: so plan ahead 11/16/2018

Availability Zones Availability zones are analogous to a physical data center. Amazon keeps adding availability zones: US East (Virginia), US East (N. California), EU West (Ireland), Asia (Singapore), Asia Pacific (Japan) Am important feature is that zones have different characteristics that no two zones will be down at any time. You can spread your data between two zones or replicate it in 2 or more zones for survivability. Traffic (bandwidth) between zones cost money. You may want to launch in the same zone if bandwidth is your concern; but if redundancy is your quest, you may need different zones 11/16/2018

Snapshot Snapshot is for saving a volume (of storage) is a feature of Amazon’s elastic block storage. You can take a snapshot as often as needed. EC2 automatically saves the snapshots to S3, thus enabling a quick and powerful backup scheme. You can replay it by creating a volume from snapshot. 11/16/2018

AMI Management You can use already existing AMI to bootstrap your system AMI contains root file system for your image. You will have clean up all your files and store them in S3 or snap shot them so that they can be replayed. Stop the sql server Name it and create it with a description. 11/16/2018

More about AMI http://aws.amazon.com/amis An Amazon Machine Image (AMI) is a special type of pre-configured operating system and virtual application software which is used to create a virtual machine within the Amazon Elastic Compute Cloud (EC2). It serves as the basic unit of deployment for services delivered using EC2. You can create an AMI of interest to your workgroup and make it available for them to create instances from. 11/16/2018

Building an AMI http://docs.amazonwebservices.com/Amazo nEC2/dg/2006-06-26/bundling-an- ami.html Sharing an AMI: http://aws.amazon.com/articles/530 Can be private or public 11/16/2018