Upload
new-relic
View
124
Download
1
Embed Size (px)
Citation preview
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agility and control while operating at hyper
scale with AWS
Douglas Lee, Head, Solution Architecture, Strategic ISV Partners,
Amazon Web Services
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Agility and control while operating at
hyper scale with AWS
Nov 15, 2016
Douglas Lee
Head, Solution Architecture, Strategic ISV Partners
Software moves faster today
Software creation and distribution is
easier and faster than ever
• Startups can now take on giants with little to
no funding ahead of time
• Getting your software into the hands of
millions is a download away
• Your ability to move fast is paramount to your
ability to fight off disruption
Old software delivery model
The software delivery model has drastically changed
New software delivery model
Disruption is everywhere and its faster today!
iPhone<8 years
iPad<6 years
iWatch<2 years
Echo<2 years
Tesla Model S<4 years
Illumina DNA Sequencer
<2 years
Netflix Streaming<9 years
Airbnb<8 years
Uber<7 years
Square<7 years
Amazon Prime
~10 years
Slack<3 years
What tools do you need to move fast?
Releasing software in this new software-driven world
requires a number of things
• Tools to manage the flow of your software development
release process
• Tools to properly test and inspect your code for defects
and potential issues
• Tools to deploy your applications
• Tools to monitor your applications and performance
Software release processes is a lot closer to donut making than one might think
© Steve Jurvetsonhttps://www.flickr.com/photos/jurvetson/5201796697/
https://creativecommons.org/licenses/by-nc-sa/2.0/legalcode
You build it, you run it
You build it, You run it
How does this apply to
• Commercial Off The Shelf software (COTS)
• Orphaned or Tiny applications
• Custom/Critical/Integrated applications
DevOps = efficiencies that speed up this lifecycle
developers customers
releasetestbuild
plan monitor
delivery pipeline
feedback loop
OpsYou build it, You run it
Dev Ops
Collaboration
DevOps
Embedded
Dev Ops
DevOps-as-a-service
DevOps Dev OpsDevOps
Temporary DevOps team
http://www.slideshare.net/matthewskelton/devops-patterns-team-topologies
You build it, You run it
Dev Ops
Build
Deploy
Test
Mo
nit
or
Automation
Optimization
Compliance
You build it, You run it
A look back at development at Amazon
© Craig Moreyhttps://secure.flickr.com/photos/pixelthing/15806918992/
https://creativecommons.org/licenses/by-nc-sa/2.0/legalcode
2001
Development transformation at Amazon: 2001–2009
2009
monolithic application + teams
microservices + 2 pizza teams
We built tools to automate our software release process
© Lindsey G
https://secure.flickr.com/photos/lindseygee/5894617854/https://creativecommons.org/licenses/by/2.0/legalcode
This has continued to work out really well
• Thousands of service teams across Amazon
• Building microservices
• Practicing continuous delivery
• Many environments (staging, beta, production)
• CI/CD + continuous monitoring
~50 million deployments / year
Things went much
better under this
model and teams
were developing
features faster than
ever, but we felt that
we could still improve
:) Invent and Simplify
We continue to survey our
software developers every year
and results found only one
development tool or service
could be correlated statistically
with happier developers:
Our pipelines service!
continuous delivery ==
happier developers!
© Barbara M.https://www.flickr.com/photos/cannnela/4614340819/https://creativecommons.org/licenses/by/2.0/legalcode
DevOps – Don’t Take It Just From Amazon:
• Oscar Health: 2 systems engineers, 45+
Developers, self service infrastructure tools, HIPAA
requirements
• AirBNB: 5 Operations people for 1000+ instances
• Gilt: several hundred microservices, self service
tools extending AWS, almost entirely on t2 instances
• Intuit/TurboTax: “deployed over 40 simultaneous
experiments during the peak filing season”
• MLB: scale up massively during Minor League
games and events, turn it off later
DevOps – Don’t Take It Just From Amazon:
In September 2015 Phil Calcado, ex-
SoundCloud, wrote in “How we ended up
with microservices.” how SoundCloud
reduced product development lead times
from 66 days to 16!*
*http://philcalcado.com/2015/09/08/how_we_ended_up_with_microservices.html
DevOps – Don’t Take It Just From Amazon:
Capital One’s moved from traditional waterfall to DevOps:
• 100s of code commits per day,
• Integration from once a month to every 15 minutes
• QA from once per month to 4 times per day
• Deployment from manual to completely automated
• Production release from monthly/quarterly to once per
sprint
Our customers should
NEVER be
provision/managing/
updating resources by hand
You build it, You Run It!
Automated actions and transitions; from check-in to production
Development benefits
• Faster
• Safer
• Consistent and standardized
• Visualization of the process
Pipelines
Infrastructure as Code is a practice
by where traditional infrastructure
management techniques are
supplemented and often replaced by
using code based tools and software
development techniques.
AWS ResourcesOperating System and
Host ConfigurationApplication Configuration
Amazon Virtual Private Cloud (VPC)Amazon Elastic Compute Cloud (EC2)AWS Identity and Access Management (IAM)Amazon Relational Database Service (RDS)Amazon Simple Storage Service (S3)AWS CodePipeline…
Windows RegistryLinux Networking OpenSSHLDAPAD Domain RegistrationCentralized loggingSystem MetricsDeployment agentsHost monitoring…
Application dependenciesApplication configurationService registrationManagement scriptsDatabase credentials…
AWS CloudFormation
AWS OpsWorks
AWS CodeDeploy
AWS ResourcesOperating System and
Host ConfigurationApplication Configuration
Amazon Virtual Private Cloud (VPC)Amazon Elastic Compute Cloud (EC2)AWS Identity and Access Management (IAM)Amazon Relational Database Service (RDS)Amazon Simple Storage Service (S3)AWS CodePipeline…
Windows RegistryLinux Networking OpenSSHLDAPAD Domain RegistrationCentralized loggingSystem MetricsDeployment agentsHost monitoring…
Application dependenciesApplication configurationService registrationManagement scriptsDatabase credentials…
AWS CloudFormation
AWS OpsWorks
AWS CodeDeployallOfThis == $Code
https://secure.flickr.com/photos/jasoneppink/499531891
Can’t move fast if you can’t measure what's going on.
Metrics/Monitoring/Logging/APM
Traditionally most of our customers are weak at this.
• You can’t make data driven decisions without data
• MTTR (Mean time to recover) is often affected by having
the right understanding of failures
• Without visibility into your applications and infrastructure,
you can’t responsibly run them.
“What tools are you using for metrics, monitoring,
logging and application performance today?”
Deployments at amazon.com
Innovate faster than ever before
Builders don’t want to be blocked
IdeaCompleted
Project
Not Possible
We don’t support thatMaybe next
year
That’s a change order
ENTERPRISE APPS
DEVELOPMENT & OPERATIONSMOBILE SERVICESAPP SERVICESANALYTICS
DataWarehousing
Hadoop/Spark
Streaming Data Collection
Machine Learning
Elastic Search
Virtual Desktops
Sharing & Collaboration
Corporate Email
Backup
Queuing & Notifications
Workflow
Search
Transcoding
One-click App Deployment
Identity
Sync
Single Integrated Console
PushNotifications
DevOps Resource Management
Application Lifecycle Management
Containers
Triggers
Resource Templates
TECHNICAL & BUSINESS SUPPORT
Account Management
Support
Professional Services
Training & Certification
Security & Pricing Reports
Partner Ecosystem
Solutions Architects
MARKETPLACE
Business Apps
Business Intelligence
DatabasesDevOps Tools
NetworkingSecurity Storage
RegionsAvailability Zones
Points of Presence
INFRASTRUCTURE
CORE SERVICES
ComputeVMs, Auto-scaling, & Load Balancing
StorageObject, Blocks, Archival, Import/Export
DatabasesRelational, NoSQL, Caching, Migration
NetworkingVPC, DX, DNS
CDN
Access Control
Identity Management
Key Management & Storage
Monitoring & Logs
Assessment and reporting
Resource & Usage Auditing
SECURITY & COMPLIANCE
Configuration Compliance
Web application firewall
HYBRIDARCHITECTURE
Data Backups
Integrated App Deployments
DirectConnect
IdentityFederation
IntegratedResource Management
Integrated Networking
API Gateway
IoT
Rules Engine
Device Shadows
Device SDKs
Registry
Device Gateway
Streaming Data Analysis
Business Intelligence
MobileAnalytics
Deploy faster wherever you like - AWS Global Infrastructure
12 regions, 38 availability zones, 4 new regions coming soon
DevOps Technology Partners DevOps Consulting Partners
Accelerate DevOps Projects with AWS Marketplace
Reduce procurement & deployment from days to minutes…Enterprise Supported / Paid Solutions Open Source Solutions
Over a 142M hours of EC2 deployed via AWS Marketplace in Oct 2015
DevOps + Dynamic Infra = Agility & Control at Scale
1. Hard for customers to succeed on AWS without DevOps
2. Customers that ignore DevOps will be unable to handle
disruption in the next ~10 years
3. DevOps feeds compute, storage, and big data services
4. Easy to start small right now, even on-prem
5. Applies across all IT, Development, QA, Security etc.
5 Key technology areas to focus on:
• Continuous Integration/Delivery
• Infrastructure as Code
• Monitoring/Metrics/Logging/APM
• APIs/Microservices Management
• Communication & Collaboration
Thank you!