View
1.513
Download
0
Category
Preview:
DESCRIPTION
Whether you're a startup getting to profitability or an enterprise optimizing spend, it pays to run cost-efficient architectures on AWS. Building on last year's popular foundation of how to reduce waste and fine-tune your AWS spending, this session reviews a wide range of cost planning, monitoring, and optimization strategies, featuring real-world experience from AWS customer Adobe Systems. With the massive growth of subscribers to Adobe's Creative Cloud, Adobe's footprint in AWS continues to expand. We will discuss the techniques used to optimize and manage costs, while maximizing performance and improving resiliency. We'll cover effectively combining EC2 On-Demand, Reserved, and Spot instances to handle different use cases, leveraging auto scaling to match capacity to workload, choosing the most optimal instance type through load testing, taking advantage of multi-AZ support, and using CloudWatch to monitor usage and automatically shutting off resources when not in use. Other techniques we'll discuss include taking advantage of tiered storage and caching, offloading content to Amazon CloudFront to reduce back-end load, and getting rid of your back end entirely, by leveraging AWS high-level services. We will also showcase simple tools to help track and manage costs, including the AWS Cost Explorer, Billing Alerts, and Trusted Advisor. This session will be your pocket guide for running cost effectively in the Amazon cloud.
Citation preview
November 13, 2014 | Las Vegas, NV
Mimi Jiang, Adobe
Constantin Gonzalez, AWS
• Speed, agility, and innovation
• Focus on your business
• Go global in minutes
Ecosystem
Global Footprint
New Features
New Services
More AWS
Usage
More
Infrastructure
Lower
Infrastructure
Costs
Reduced
Prices
More
CustomersInfrastructure
Innovation
45 price
reductions
since 2006Economies
of Scale
aws.amazon.com/premiumsupport/trustedadvisor/
Free with Business or Enterprise Support
Monday Friday End of Vacation Season35% saved
"LaunchConfig": {
"Type" : "AWS::AutoScaling::LaunchConfiguration",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
… packages, sources, files, services …
}
}
},
"Properties": {
"ImageId" : "ami-149f7863",
"InstanceType" : "m1.small",
"SecurityGroups" : [ {"Ref" : "WebServerSecurityGroup"} ],
"KeyName" : "MySSHKey",
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -v\n",
… your user data script …
]]}}
}
}
"WebServerGroup" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"AvailabilityZones" : [
"us-east-1a","us-east-1b","us-east-1c",
],
"LaunchConfigurationName" : { "Ref" : "LaunchConfig" },
"MinSize" : “3",
"MaxSize" : “6",
"DesiredCapacity" : “3",
"LoadBalancerNames" : [ { "Ref" : "ElasticLoadBalancer" } ]
}
}
{ adobe.io}Running Lean
Architectures
in AWSMimi Jiang | Performance & Reliability, adobe.io
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
New era of creativity
Adobe Creative SDK
Adobe Ink & Slide
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
API Gateway
API Management
Services
API Console
{ adobe.io }: Adobe’s API platform
3rd party apps
Adobe apps
{adobe.io}
API services
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
{ adobe.io }: AWS deployment architecture
API Consumers
Availability Zone 1
Private Subnet
Availability Zone 2
Public Subnet
Private Subnet
Auto Scaling Group
Auto Scaling Group
Region (US East, EU, AP, US West)
AmazonRoute
53
Public Subnet
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Instance type sizing
Recommended steps:
Define application characteristics
Shortlist instance types
Analyze – performance, cost, & resiliency
Architect
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
{ adobe.io } use case
Application
characteristics
:
Instance
Type
On-Demand
Cost
vCPU Memory
(GB)
Storage
(GB)
Network
Performance
cc2.8xlarge $2.40 32 60.5 4x840 10 Gigabit *
m3.2xlarge $0.56 8 30 2x80 High
C3.2xlarge $0.42 8 15 2x80 High
CPU intensive traffic
Minimum 8 core CPU
Support for heavy payloads
Architect
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
19168 2165528037
2329230003 3191928357
3269938962
2000 users 4000 users 6000 users
CPU TestSingle node (requests/sec)
m3.2xlarge c3.2xlarge cc2.8xlarge
{ adobe.io } performance analysis
3539
113
Network TestSingle node (requests/sec)
cc2.8xlarge c3.2xlarge m3.2xlarge
CPU: 30%CPU: 90-95%
Architect
Total bandwidth
= Network In +
Network Out
5 4 1
3:1
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
{ adobe.io } cost analysis
$307.44
$409.92
$1,756.80
c3.2xlarge
m3.2xlarge
cc2.8xlarge
Monthly on-demand cost per node (US East)
$2.40/hr
$0.56/hr
$0.42/hr
Architect
0
500
1000
1500
2000
2500
3000
3500
4000
4500
30K req/sec 60K req/sec 100K req/sec 200K req/sec
Co
st
$/Month vs. Throughput
cc2.8xlarge
m3.2xlarge
c3.2xlarge30%
16%
30%
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
{ adobe.io } resiliency analysis
AZ1 AZ2Region
cc2.8xlarge
AZ1 AZ2Region
c3.2xlarge /
m3.2xlarge
VS
Architect
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
{ adobe.io } Decision & takeaways
Instance type selected: c3.2xlarge
What we learned• Match instance type with expected workload
• Smaller instance types provide better granularity
• Enhanced networking is available in c3, r3, & i2 instance
types
• Boot time varies between different instance types
• Amazon CloudWatch detailed monitoring gives better
granularity
• Instance type differs by region (cost, availability)
• Set a cadence to repeat
Architect
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Scaling mechanisms
Event-based scaling• Overprovision for sharp traffic increase
** Prewarm your load balancer
• Observe traffic pattern & scale down
Schedule-based scaling• Match scaling with workload at different times
• Scale down dev & test environments
Architect
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Auto Scaling
• Use for less steep traffic increase
• Scale up fast, scale down slow• ** Remember you are charged hourly
• Set triggers: CPU, network, memory
• Avoid multiple scaling triggers per Auto
Scaling group
Architect
Auto Scaling policy example:
Scale up: +2 instances if CPU utilization > 70% for 2 minutes
Scale down: -2 instances if CPU utilization <= 40% for 20
minutes
Cooldown: 300 seconds
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Monitoring tips
• Tailor monitoring strategy to application
characteristics
• Monitor both cost & usage
• Tag your instances
• Establish thresholds/alerts
• Analyze & adjust resource usage often
• Tools make it easier
Architect
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Open source—Netflix ICE
• Cost & usage reporting
• Uses AWS Programmatic Billing
Access
• Maintains knowledge of:
• Accounts, regions, services,
usage types, cost/usage
categories
https://github.com/Netflix/ice
Architect
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Commercial tool—Cloudability
usage comparison
underutilized instances
daily summary
Architect
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Wrap-up
• Cost, performance, & resiliency have equal
weight
• Embrace failure, architect for it
• Turn off unused capacity, automate it!
• Benefit from auto-scaling, but be careful!
• Monitor & frequently evolve your architectureCheck out these other sessions:• SDD402 – Amazon ElastiCache Deep Dive (Frank Wiebe, Principal Scientist,
Adobe)
• ENT207 – Creating a Culture of Cost Management in Your Organization (J.R.
Stoment, Chief Customer Officer, Cloudability)
© 2014 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Contact me: yjiang@adobe.com
Thank You!
1y RI
Break even
3y RI
Break even
On-Demand:
$0.24
$0.028 (11.7%) $0.026 (10,8%)
$3.28
(1367%)
20%
64%
Caching/Optimization:
80% saved
Cache
flush
Dynamic
DynamoDB:
20% saved
Growth +
new features
Simple, more reliable, lower cost
Recommended