Embed Size (px)
Citation preview
Introduction to Amazon Web Services for developers
Roman Gomolko
July 2015
Ciklum Speakers Corner
Let’s get acquired
UserReport
● Developing products that allow to learn the audience
● Started using AWS more than 5 years ago
● Fully migrated to AWS more than 1.5 years ago
● Processing 3 billions requests monthly
● Generating reports based on 8 billions of requests with batched reports
● Online reports on 300 millions of records
● Used ~50% of services provided by AWS
● Totally happy regarding using AWS
Cloud Computing
Cloud computing is a model for enabling access to a shared pool of configurable computing resources
Typical Cloud benefits
● Pay for what you are using
● High scalability - retrieve as many resources as you need
● High availability
● Gartner's Hype cycle – cloud computing has reached a maturity
Cloud models
● IaaSInfrastructure like physical (rarely) or virtual servers, storage, networking, load balancers, IP addresses allocation etc
● PaaSHosted databases, pre-configured web-servers that can run your application, platforms that can run your code
● aPaaSTools and services that automates and facilitates utilizing Cloud Services. Providing tools for development, deployment, monitoring etc
● SaaSGmail, MailChimp, Highrise, Google photo, Jira Cloud etc
Amazon Web Services
IaaSPaaS
aPaaSSaaS
Naming convention
● Everything is Elastic
● Everything is Simple
● DRY
Elastic Compute Cloud = ECC = EC2
Simple Storage Service = SSS = S3
Elastic Load Balancer = ELB
AutoScale Group = ASG
...
AWS is regional
AWS is regional
● Almost all services are region specific
● Pricing of services depends on region
● Not all services are available in all regions
● New services are launching in N. Virginia (US East 1) first
http://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/
Regions are not solid
● Splitted into Availability Zones (AZ)
● AZs have independent power supply
● Cross-AZ network speed is blazing fast
● Some services belong to AZ, some - not
Iaas basics
● EC2Virtual computers managed by XEN running on Intel CPUs
● EBSSSD or magnetic drives dynamically attached to EC2 instances with configurable size (up to 16Tb) with configurable IOPs
● ELBLoad balancing before your EC2 instances
● VPCCreating network and group your instances/services into “Private Cloud”
PaaS basics
● CloudFrontCDN for downloading and streaming
● S3Infinite storage for objects (files) with security and HTTP access
● RDSHosted databases like MySQL, MS SQL, PostgreSQL, Oracle, Aurora with licenses included
● DynamoDBNoSQL database designed for high performance and availability
PaaS basics
● Route53Fully managed and DNS and domain registration
● ElastiCacheFully managed hosted memcache and Redis
● SQSQueue designed for high availability and millions of messages
Some more PaaS
● CloudSearchSearch engine for free-text search among millions of documents
● EMRFully managed Elastic MapReduce and Spark clusters
● Elastic Transcoder Encoding media-files in different formats with different quality
● … and more
aPaaS
● CloudWatchMonitoring of your infrastructure and applications
● Elastic BeanstalkAutomated web-applications and workers hosting
● CloudFormationManage your cloud services in one place with help of text file
● CodeDeployAutomates deployment of your application
● OpsWorksChef cooked by AWS to automate provisioning and deployment
SaaS basics
● SESSending e-mails from Cloud
● SNSPush notifications
● WorkMailHosted exchange server for organizations
● WorkSpacesVirtual office became reality
Migration of existing web-application to AWS
DBWebServer
Migration to Cloud
● One bite at a time - migration to Cloud itself is a big deal
● Cloud may have similar services to what you are using in your app. Similar is not the same.
● Usually you have application which works somehow try to keep your structure unchanged
● Less efforts, faster migration
Hosting your web-application
1. Create EC2
2. Connect to it
3. Configure it
4. Deploy code to it
5. Access website
Creating EC2. Choose operating system image
Creating EC2. Choose instance type
Creating EC2. Instance details
Creating EC2. Storage
Creating EC2. Tags
Creating EC2. Security group
Creating EC2. Review. Almost done
Creating EC2. Selecting key-pair
Connecting to your instance. Wait until launched
Connecting to your Linux instance
Linux / Mac OS X
ssh -i "<keypair>.pem" ec2-user@<instance IP>
Windows
● convert .pem to .ppk using Putty KeyGen
● connect to instance using Putty
● refer help for more details
Connecting to your Windows instance
● Wait 4-20 minutes after starting instance
● Retrieve password using you keypair
● Use Remote Desktop to connect to your instance. Use Administrator as name
● Password will expire, better create permanent user
Configuring your instance as you like
sudo apt-get install apache2
sudo apt-get install php5
sudo apt-get install libapache2-mod-php5
sudo /etc/init.d/apache2 restart
import-module servermanager
add-windowsfeature Web-Server,
Web-WebServer, Web-Security
Deploy your application
Accessing your web-application
● Your public IP is not static. It will be changed after reboot
● You can request IP address and attach it to instance dynamically
● In future you can assign this IP to another instance
● Put this IP to your DNS or write to host file
Database. Let’s try RDS
Multi-AZ deployments for reliability
multiply hosting cost by number of AZ zones
Configure your instance and DB master credentials
Secure your database
Take advantage of managed solutions
Wait. Cloud? I need more redundancy
1. Create EC2
2. Connect to it
3. Configure it
4. Stop it
5. Create AMI
6. Launch required number of instances using this AMI
7. Deploy code to them
8. Access website
Launching from you AMI
Horizontal scaling of application checklist
● Data-less
● Stateless
● Shared session
● Same encryption keys
Distribute traffic between instances
● DNS round-robin
● Load balancer
Elastic Load Balancer
● Accept incoming traffic and distribute among instances
● Monitors instance availability
● Can accept HTTPS traffic and map to HTTP endpoint of web-server
● Allows to not expose web-servers to worldwide
● Allows Connection Draining
● Provides useful metrics like latency, requests per minute, count of 2XX, 4XX and 5XX response codes
I’ve become popular. Need to scale
Instance roles:
● Web/front-end
● Worker/back-end
● DB
Scaling strategies:
● Vertical scaling
● Horizontal scaling
Vertical scaling
● Good if your bottleneck is CPU, RAM or disk
● Can be done with downtime or without it
Vertical scaling of EC2.
● Simple - downtime
○ Stop
○ Change instance type
○ Start
● Complex - no downtime
○ Launch new instance with required instance type
○ Switch traffic from old instance to new one
Vertical scaling of RDS
● Can be done without downtime in case of Multi-AZ deployment
● Not that fast in any way
Where is fun stuff?
I want to be Elastic
AutoScale groups
Launch configuration - how your instance will look like
AutoScale group - control how many instances will be in your group
Launch Configuration - customize instance bootstrap
AutoScale group
AutoScale Group - scaling policies
Finally
● AWS can scale our application automatically looking on CPU load
● AWS can scale-down instances when load decreased
Small catch-up
● We know what is EC2 and how to deploy application first time
● We know how to horizontally scale up and down your application
● It looks pretty cumbersome
● We don’t know where we should store users’ files
● We don’t know how to improve website performance
● We don’t know how to update your application
Bad reasons to migrate
● Migrating everything by just virtualizing existing stuff and expecting savings
● Expecting magical performance boost and “Cloud will solve all problems”
Good reasons to migrate
● Extremely flexible
● Pay for what you use
o cheap to fail
o cutting cost of testing and applying new technologies
● Fast changes - get what you need when you need
● Allows to focus on your product - not on CDN, Mail sending, file storages,
File storage
● DIY - Distributed File System
● Let’s Amazon handle it - Elastic File System (Preview)
● Ready to change code - S3
S3, CloudFront, Beanstalk demo time
