Upload
saravanaraajaa
View
222
Download
0
Embed Size (px)
Citation preview
7/27/2019 CloudCheckr Aws Economics
1/33
NYC Cloud Computing Group
AWS Economics:Learn how to efficiently optimize your
resource utilization and control your costs
Aaron C. NewmanCEO/Founder, CloudCheckr
7/27/2019 CloudCheckr Aws Economics
2/33
Agenda:
Overview of Costs in AWS
Going Reserved
Going Spot
Optimizing Resources
Economic Denial of Sustainability
Conclusion, Resource, and Questions
7/27/2019 CloudCheckr Aws Economics
3/33
Overview of AWS Costs
7/27/2019 CloudCheckr Aws Economics
4/33
State of Cloud Computing Cost
10 years ago The datacenter was a (mostly) fixed cost High cost for even the most basic data center You paid for your peak capacity
Co-location/ISPs as an alternative Still buying your own equipment/building for peak capacity
High margins were the norm
About 2010 Public Cloud Turns the Corner Technology matures Becomes the de facto for getting a startup off the ground Amazon starts compressing the high margin IT business
Over 20 pri ce reductions in Amazon AWS by 2013Prices continue to drop. But what is the reality of those 20 price reductions?
EC2 Linux Small Instance (On-Demand Per Hour):
Aug 2006 = $0.10, Nov 2009 = $0.085, May 2013 = $0.06
Extrapolate that to 2015 = somewhere between $0.035 and $0.05
7/27/2019 CloudCheckr Aws Economics
5/33
Cost is a product of usage
In old data center, cost was fixed Once you bought the equipment, little could be done to reduce
your cost. No advantage to ever scale down.
The public cloud is heavily weighted to variable costs If you can use less, you spend less
Optimizing Resource Utilization in the Cloud matter Computational engines run as close to 100% as possible
Interactive components need a cushion for peak usage
Auto scaling - important tool for optimizing cloud usage Scale down as much as scaling up
7/27/2019 CloudCheckr Aws Economics
6/33
Cloud Costs
(Bytes of Data Transferred) * Price
+
(Size of Compute Resource) *
(Price of Compute Resource) * (Number of Hours)
+
(Storage Used) * Price * (Time Stored)
+
(Transactions Processed) * Price
7/27/2019 CloudCheckr Aws Economics
7/33
Architecting Applications
In the past, architecture was typically a large, multi-threaded executable talking to a single database runningon the biggest boxes you could afford. Sat idle a lot.
Moving old apps into the cloud Can you resize your resources different times of the day?
Use load balancers or Multi AZ capabilities to resize
Important to design your apps to scale horizontally
Design you application into components That can be added or removed dynamically
7/27/2019 CloudCheckr Aws Economics
8/33
Going Reserved
7/27/2019 CloudCheckr Aws Economics
9/33
What is Reserved?
Pay up front, get a lower variable cost
ROI as high as 500% over 3 years, 60% savings in cost
Types of Reserved Resources Available EC2 Instance
RDS DB Instance Elasticache Node
S3 Reserved Storage (for GovCloud)
Types Utilization light, medium, heavy
Commitments 1 or 3 year
7/27/2019 CloudCheckr Aws Economics
10/33
Calculating ROI on Reserved Instances
7/27/2019 CloudCheckr Aws Economics
11/33
Picking Type of Reserved Instances
7/27/2019 CloudCheckr Aws Economics
12/33
7/27/2019 CloudCheckr Aws Economics
13/33
Going Spot
7/27/2019 CloudCheckr Aws Economics
14/33
What is Spot Pricing
Bidding for unused instances Supply and demand dictates current price
Place your max bid, your instance shuts down if max bid exceeded by others
Spot is almost always cheaper
But you need to consider < 99% availability Spot prices spike frequently
Slightly slower to spin up Wait for spot request to be fulfilled before instance can start
More complex to manage Using EBS/Instance store
7/27/2019 CloudCheckr Aws Economics
15/33
What Does Spot Pricing Look Like
This morning spot pricing:
US East, Linux, M1 Small (1 ECU): 0.007 - 0.010 SA, Linux, M1 Small (1 ECU): 0.011 On demand: US East = 0.06, SA = 0.08
US East, Linux, M1 Extra Large (8 ECU): 0.055 - 0.64 SA, Linux, M1 Extra Large (8 ECU): 0.084 On-demand: US East = 0.48, SA = 0.64
US East, Linux, M3 Double Extra Large (26 ECU) 0.115 SA, Linux, M3 Double Extra Large (26 ECU) 0.185
On-demand: US East = 1.00, SA = 1.36
Spo t Pricing is typic al ly 10-20% of On-Demand
Bu t can easi ly sp ike HIGHER than On -Demand
7/27/2019 CloudCheckr Aws Economics
16/33
Spot Strategies
Most people dont understand spot pricing, afraid to use it As more people understand and use it, pricing will be driven up
Very tempting but dangerous to run exclusively on Spot
From GigaOm: A sudden spike in the price of m2.2xlarge servers (normally$.44/hour) drove the price briefly up to $999/hour, causing a site-wise outage.
If you follow this strategy, use a variety of instance sizes, Availability Zones, andeven regions to minimize the risk
Hybrid Reserved/Spot strategies Run as many spot instances as possible
But maintain a base level of Reserved Instances
Switch to On-Demand if Bid Price Exceeds On-Demand Price
This is a manually intensive strategy
7/27/2019 CloudCheckr Aws Economics
17/33
Optimizing Resources
7/27/2019 CloudCheckr Aws Economics
18/33
Overview
Keep track of what you are using Find and eliminate idle instances
Find and reduce under-utilized resources
Unused EBS drives, ELB, multiple snapshots of same EBS drive
Horizontally scale Find smallest instance type that can handle your transactions
Find your bottle necks (network, disk I/O, CPU util, memory util)
Turning off resources when they arent used
Turn off over the weekend, overnight
Use only what you need E.g. dont check multiple copies of buckets in S3
7/27/2019 CloudCheckr Aws Economics
19/33
Optimizing Instance Types
Picking the optimal Instance Type:
Comparing ECU (EC2 Compute Units) M1 Small (1 ECU) On-Demand in US East = $0.06 ($0.06 per ECU)
M3 Double Extra Large (26 ECU) On-Demand in US East = $1.00 ($0.0385 perECU)
Comparing the cost of Memory M1 Small (1.7 GiB memory) On-Demand in US East = $0.06 ($0.035 per GiB)
M3 Double Extra Large (30 GiB memory) On-Demand in US East = $1.00
($0.033 per GiB)
But you need to compare Resource Type, Pricing Type (on-demand/spot/reserved), Region, AZ, etc for your circumstance
7/27/2019 CloudCheckr Aws Economics
20/33
S3, Glacier, and RRS
S3 Pricing about 10 cents per gigabyte (starts at 9.5c in US East)
Reduced Redundancy Storage AWS doesnt store as many copies of your S3 objects
Typically about 20% cheaper (US East $0.095 reduced to $0.076) Ideal if you are storing terabytes or petabytes of songs, movies, documents that
can be recovered
How much of your S3 storage can you convert to RRS?
Glacier Pricing about 1 cent per gigabyte Pricing difference from S3 decreases as the price goes up
Takes up to 4 hours to retrieve files, and cost to retrieve
7/27/2019 CloudCheckr Aws Economics
21/33
S3, Glacier, and RRS
S3 Pricing about 10 cents per gigabyte (starts at 9.5c in US East)
Reduced Redundancy Storage AWS doesnt store as many copies of your S3 objects
Typically about 20% cheaper (US East $0.095 reduced to $0.076) Ideal if you are storing terabytes or petabytes of songs, movies, documents that
can be recovered
How much of your S3 storage can you convert to RRS?
Glacier Pricing about 1 cent per gigabyte Pricing difference from S3 decreases as the price goes up
Takes up to 4 hours to retrieve files, and cost to retrieve
7/27/2019 CloudCheckr Aws Economics
22/33
Economic Denial of
Sustainability Attacks
7/27/2019 CloudCheckr Aws Economics
23/33
EDoS Attacks
Variation of Distributed Denial of Service Attack Goal is not to overload and crash an application
Instead to cause the server hosting costs to overwhelm
the victims budget
the infrastructure allows scaling of service
beyond the economic means of the vendor
to pay their cloud-based service bills-http://rationalsecurity.typepad.com
7/27/2019 CloudCheckr Aws Economics
24/33
Worst Case Scenario AWS CloudFront
http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/
Author calculated maximum possible charge Used default limit of 1000 requests per second and
1000 megabits per second
At the end of 30 days a maximum of 324TB of data
could have been downloaded (theoretically) $42,000 per month for a single edge location
CloudFront has 30 edge locations
http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/http://www.reviewmylife.co.uk/blog/2011/05/19/amazon-cloudfront-and-s3-maximum-cost/7/27/2019 CloudCheckr Aws Economics
25/33
Stories and Lessons Learned
Anecdotal user experience Personal website hacked by file sharers
Received bill for $10,000
Note: AWS only charges for data out All data transfer in is at $0.000 per GB
Mitigates costsif you dont respond to requests, doesnt costyou anything
Use pre-paid credit cards or credit card with appropriatecredit limit Not sure if this limits your liability legally
7/27/2019 CloudCheckr Aws Economics
26/33
Solutions?
Amazon limits/caps have been in the workssince 2006 Each year Amazon talks about intention of releasing
the feature
May 2012 Amazon announces Billing Alerts http://aws.amazon.com/about-aws/whats-
new/2012/05/10/announcing-aws-billing-alerts/ Helps alert you when this starts happening to you
Could still be a costly few hours
http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/http://aws.amazon.com/about-aws/whats-new/2012/05/10/announcing-aws-billing-alerts/7/27/2019 CloudCheckr Aws Economics
27/33
Misconfigured Security Settings
Scanning Amazon S3 to identify publiclyaccessible buckets http://cloudcheckr.com/2012/05/aws-s3-buckets-
bucket-finder/
Open source tool Bucket Finder script launches a dictionary attack on the names of
S3 buckets and interrogates the bucket for a list ofpublic and private files
Searching out EDoS
http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/http://cloudcheckr.com/2012/05/aws-s3-buckets-bucket-finder/7/27/2019 CloudCheckr Aws Economics
28/33
Resources, Conclusion,
and Questions
7/27/2019 CloudCheckr Aws Economics
29/33
5 Strategies To Optimize
Keep a close handle on what you are running in the cloud
Measure what you are spending
Calculate Return On Investment
Minimize what you dont need
Protect yourself from EDoS
7/27/2019 CloudCheckr Aws Economics
30/33
Its Not About the Price
Cloud Computing is not about the cost Its about accelerating business, moving faster
IaaS is following in SaaS footstep SalesForce.com pioneered the movement
Hard to imagined a third-party controlling your entire customer list (one of yourmost valuable assets)
They proved it was secure, prudent, and effective
Still see some of the slower moving Corporate types claiming
production work loads cant be run on the cloud While their competitors leveraging the cloud eat their lunch
7/27/2019 CloudCheckr Aws Economics
31/33
Resources
Further reading:
How AWS Pricing Works http://media.amazonwebservices.com/AWS_Pricing_Overview.pdf
AWS Service Pricing Overview http://aws.amazon.com/pricing/
CloudCheckr Whitepaper Cost Series http://www.cloudcheckr.com/whitepapers
AWS Simple Monthly Calculator http://calculator.s3.amazonaws.com/calc5.html
7/27/2019 CloudCheckr Aws Economics
32/33
Questions?
Questions on:
Cloud Computing Resource Utilization
Optimizing Your Costs
CloudCheckr
7/27/2019 CloudCheckr Aws Economics
33/33
Thank You for Attending
Get your FREEMIUM account to
check your public cloudat www.cloudcheckr.com
Aaron Newman is the Founder
of CloudCheckr (www.cloudcheckr.com)
Please contact me with additional questions at:
http://www.cloudcheckr.com/mailto:[email protected]:[email protected]://www.cloudcheckr.com/