Amazon Web Services Primer

William StricklandCOP 6938 Fall 2012

University of Central Florida

AWS Overview

● Amazon Web Services (AWS) is a collection of varying remote computing provided by Amazon.com.

● These services are provided using Amazon's extensive infrastructure.

● Together these services provide the infrastructure and services required to create robust, scalable web applications.

● Some example services provided in by AWS are:

– Elastic Compute Cloud (EC2) – private virtualized servers

– Simple Storage Service (S3) – scalable, high-availability file storage

– SimpleDB – no-SQL keypair-value database service

– CloudFront – Content delivery service

– ElastiCache – In-memory caching for web servers

AWS Major Locations

High cost region

Medium cost region

Very high cost region

Low cost region

Asia Pacific(Singapore)

EU(Ireland)

Asia Pacific(Tokyo)

US East(N. Virginia)

US West(Oregon)

US West(N. California)

South America(San Paulo)

AWS Management

● After creating an account, users will need to connect with amazon to request services.

● Amazon provides a REST API to do this.

● Four common ways to interact with and manage Amazon cloud resources using this API:

– Web management console

– Command-line tools

– AWS SDK libraries.

– Raw REST requests

Web Management Console

● Easy to use web interface. http://aws.amazon.com/console

● Updated as new services and features are released.

● Simplified options, not all options shown.

Command-line Tools

● Command-line executables for Window and Unix/Linux

● Leverages JavaVM, JRE required.

● http://aws.amazon.com/developertools/

AWS SDK Libraries and Toolkits

● Implementation and abstraction of the AWS APIs are provided for a variety of popular programming languages.

– Java, Ruby, C#, Obj C, PHP and more.● Additionally, integration is provided for several IDEs.

REST API

● It is always possible to call the REST API directly using your choice of HTTP libraries.

● Detailed specifications are provided for the format and content of HTTP requests and responses defined in the API.

● Also, a SOAP API is available with some restrictions.

● E.g. http://docs.amazonwebservices.com/AmazonS3/latest/API/APIRest.html

Authentication

● The AWS API has several different authentication schemes for its many services and tools.

● The web management console always utilizes username/password authentication.

● The command-line, SDK and other rest tools require an amazon generated key pair for authentication. This key pair is used to sign message digests on the HTTP requests.

● CloudFront requires RSA certificates for the signed URL functionality.

● The SOAP API requires x.509 certificates for authentication for most services.

IAM

● Identity and Access Management provides more granular control over access to AWS resources for an account.

● IAM allows the creation of users, groups and roles associated to the account.

● IAM users access a custom URL to login which is directly tied to the account. Each user can have a key pair password of both for authentication.

● Very fine grain permissions are possible for most services.

AWS Billing

● Current charge break down is always available in the console.

● In general, charges are computed by the hour.

● Billing periods are broken into calendar months.

● At the end of each month the credit card selected as the payment method is charged for the costs incurred.

● Customers with larger infrastructure have other options

– Large customer (usually monthly costs exceeding $10,000) may request payment by invoice.

– Consolidated billing allows multiple accounts to roll costs up together and be payed on one charge/invoice.

EC2 Overview

● Amazon Elastic Compute Cloud (EC2) is classified as an Infrastructure-As-A-Service (IAAS) solution.

– Infrastructure provided, managed and maintained by Amazon.● Internet connection & network, storage, virtual machine.

– Virtualized server running in Amazon’s datacenter.

– Additionally, OS licensing provided by Amazon.

OS / Runtimes

Application

Users

Application

Users Users

EC2 Instances

● What are EC2 Instances?

– Virtual servers hosted on amazon's infrastructure.

– Placed in user selected location (availability zone).

– Allocated CPU and memory as selected by the user.

– Created from predefined or custom machine images.

– Loaded with user selected operating system, runtime environments, libraries and applications.

– With I/O access to local storage (in VM instance) or virtual high-reliability block storage (EBS).

– Networked behind Amazon's routers and NAT.

– With network/VM level inbound packet filtering for security.

Instance Operations

● Important Operations on EC2 instances

– Launch – create a new instance from an existing AMI.

– Create Image – shutdown the instance and make an AMI, will reboot when done.

– Change Instance Type – resize hardware provisioned for instance.

– Start – restart instance from stopped state (resume billing).

– Reboot – perform 'hardware' reboot of operating system (does not pause billing).

– Stop – shutdown the instance and leave it off (stops instance hours billing, but will still be charged for storage).

– Terminate – Deletes instance and all associations. Also deletes EBS volumes that are were created with the instance if they are not disconnected from the instance first.

– Add/Edit Tags – allows the user to set tags (such as name) to help distinguish instances.

AWS Network

● EC2 Instances hosted inside Amazon’s data centers.

● Network Address Translation (NAT) used between data center and Internet.

● Private IP address for each instance (relatively static).

● DNS provided to get current public IP address of instance.

EC2 Availability Zones

● EC2 services are divided into regions

– Regions roughly geographical.

– Billing rates determined by region.

– No resources shared between regions.● EBS volumes, key pairs, security groups, elastic IPs, etc

● Each region contains several availability zones

– Physically distinct data centers in nearby each other.

– EBS resources automatically replicated inside availability zone.

– While isolated, each zone has excellent bandwidth to other zones within that region.

AMIs

● Amazon Machine Images (AMIs) are the virtual machine images that are used to create new EC2 instances.

● Many AMIs are provided out of the box for EC2.

● Users may also create AMIs from exist instances or EBS volumes (requires downtime).

● AMI images will create an exact copy of the original machine the image was created from except for a couple instance dependent configurations such as hostname (if enabled) and hardware properties (such as MAC address). Which configurations change and to what is defined in the AMI.

● Once Created, AMIs are stored in S3 for high-availability.

EBS

● Elastic Block Storage (EBS) provides high-reliability block storage to EC2 instances.

● Since introduced, EBS has become the default storage method for EC2. Most common AMI define EBS as the storage IO for the root disk.

● EBS volumes are replicated inside the availability zone and can be separated and swapped between instances (but only within that availability zone).

● EBS volumes can be backed up using the snapshot function of the EC2 API. Snapshot images are stored as delta files in S3 for increased redundancy. Creating a snapshot does not require downtime or interrupt running instances.

Instance Pricing Models

● There are three pricing models available for instances:

– On-Demand Instance

– Reserved Instance

– Spot Instance

● On-Demand instances

– Easiest and most common pricing model.

– User flat hourly rate for the time the instance is running.

– No commitment and always available.

– Easy to estimate and good for users that need to match demand when it happens but may be able to leave instances off most of the time.

● Reserved instance

– A one time fee is paid up front to lock-in a much lower hourly rate.

– Additional capacity is still available at the standard rate.

– Good for users that anticipate a some large constant level of usage.

● Spot instance

– Users bit on current unused capacity.

– Users set buy and sell thresholds.

– Instances start only when thresholds met.

– Good for users that can delay computation to get lowest possible prices.

Security Groups

● Security groups control the inbound filtering rules for each instance.

● Packets are filtered according to the rules defined for the security group before they are seen by the networking stack of the instance operating system.

● On instance creation, the instance is assigned to a security group. This group assignment cannot be changed.

● The rules defined for a group can be changed at any time.

● Security group rules allow traffic on a port or port range for source. By default traffic is blocked.

● Sources can either be a CIDR address range or another security group.

● Best practice with security groups is to use them to define dynamically to define groups of instances and allow only the necessary communication between them.

Elastic IPs

● Between the public Internet and the private AWS network the addresses of all instances must undergo Network Address Translation (NAT).

● On start, each instance is dynamically assigned a private IP address which other instances can communicate with it on and a public DNS name.

● The DNS name provided will resolve to the public IP address that will NAT to the instance.

● Elastic IPs provide a way to assign a static public IP address to an instance.

● Elastic IPs are reserved from the available pool of the availability zone. The user cannot pick the IP address directly (though one could acquire and release many to get different IP addresses).

● Elastic IPs are are not released when the instance is stopped or terminated.

● Elastic IPs not assigned to an instance will accumulate a small fee per hour

● Elastic IPs must be released when no longer needed.

CloudWatch

● Useful to monitor health and create work-flow around instance utilization.

● Alarms can be set to alert administrators or automated routines.

● Tracks metrics such as

– CPU usage

– Disk I/O

– Network traffic

● Three Pricing plans for EC2

– Basic: included for free, 5 minute interval with 10 predefined metrics.

– Detailed: basic metrics plus 7 predefined metrics at 1 minute interval.

– Custom: pay per metric per month with 1 minute interval.

S3

● Simple Storage Service (S3) is a High Availability data storage service provided by AWS.

● S3 replicates data within and between availability zones in the region specified by the user.

● Billing is by storage and I/O traffic

● There is also a lower cost reduced reliability option.

● Data can be retrieved HTTP or Bittorrent.

● Can store an unlimited number of files and supports folders for file hierarchy.

SQS

● Simple Queuing Service (SQS) provides a means for applications and instances to communicate and share workload in the AWS environment.

● SQS messages are replicated to prevent message loss.

● Messages are limited to 64KB of text and are purged from the queue after 14 days.

● Messages are locked while being read, but will automatically become unlocked if not processed successfully. This prevents duplicate processing and messages being lost because of processing failure.

● The security settings on SQS have flexibility to specify allowed accounts, IP addresses and time of day.