Upload
molly-park
View
217
Download
2
Tags:
Embed Size (px)
Citation preview
v
~11.6sMean time between
deployments (weekday)
~1,079Max number of deployments
in a single hour
~10,000Mean number of hosts
simultaneously receiving a deployment
~30,000Max number of hosts
simultaneously receiving a deployment
Deployments at Amazon.com
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management
Conclusion
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management
Conclusion
v
Version Control CI Server
Package Builder
Commit to Git/master
Dev
Get / PullCode
Send Build Report to DevStop everything if build failed
Distributed BuildsRun Tests in parallel
CodeConfigTests
Push
Config
Repo
Continuous Integration
v
• Test driven promotion (of development change)• Increasing velocity of feedback cycle through iterative change• Bugs are detected quickly• Automated testing reduces size of testing effort
What does CI give us?
v
Continuous Delivery/Deployment
Version Control CI Server
Package Builder
Deploy ServerCommit to
Git/masterDev
Get / PullCode
AMIs
Send Build Report to DevStop everything if build failed
Distributed BuildsRun Tests in parallel
Staging Env
Test Env
CodeConfigTests
Prod Env
Push
Config InstallCreate
Repo
CloudFormation Templates for Env
Generate
v
• Automated, repeatable process to push changes to production• Hardens, de-risks the deployment process• Immediate feedback from users• Supports A/B testing or “We test customer reactions to features in
production”• Gives us a breadth of data points across our applications
What does CD give us?
v
Continuous Delivery versus Continuous Deployment
v
Example CI/CD pipeline
Dev
IT Ops
Version Control
Build/Compile
Code
Unit TestApp Code
DR Env
Test Env
Prod Env
Dev Env
Application
WriteApp Code
Infrastructure
CloudFormation
tar, war, zipyum, rpmDeploy
AppPackage
Application
Deploy application only
Deploy infrastructure only
BuildAMIs
ValidateTemplates
WriteInfra Code
DeployInfras
AutomateDeployment
Artifact Repository
AMIs
vHOST METRICS
SERVICEMETRICS
LOG ANALYSIS EXTERNAL SITE METRICS
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management
Conclusion
v
• How fast do we need to do this?• Across how many instances?• How do we roll back (or forward)?
Delivery approaches
v
AMI building/deployment methods
vFully Functional AMI OS-Only AMI
Partially Configured AMI
Most amount of post-boot work
Least flexibleto maintain
Try and find a happy medium here
Delivery approaches
vDeploy in place • Deploy all at once (service outage)• Rolling updates
Red-Black deployment• Discrete environment• Multiple environments from branches• Support A/B testing• Use auto scaling group
Deployment approaches
v
Deploy in place – Rolling update
OpsWorks CodeDeploy
v
Deploy in place – Rolling update
v
Deploy in place – Rolling update
v
Deploy in place – Rolling update
v
Deploy in place – Rolling update
v
Deploy in place – Rolling update
v
Deploy in place – Rolling update
v
Deploy in place – Rolling update
v
Red-Black deployment
EC2 Instances
ELB
DynamoDB MySQL RDSInstance
ElastiCache Cache Node
Auto Scaling Group
V1
v
Red-Black deployment
EC2 Instances
ELB
DynamoDB MySQL RDSInstance
ElastiCache Cache Node
Auto Scaling Group
V1 EC2 Instances
UAT
Auto Scaling Group
V2
v
Red-Black deployment
EC2 Instances
ELB
DynamoDB MySQL RDSInstance
ElastiCache Cache Node
Auto Scaling Group
V1 EC2 Instances
Auto Scaling Group
V2
v
Red-Black deployment
ELB
DynamoDB MySQL RDSInstance
ElastiCache Cache Node
EC2 Instances
Auto Scaling Group
V2EC2 Instances
Auto Scaling Group
V1
v
Red-Black deployment
ELB
DynamoDB MySQL RDSInstance
ElastiCache Cache Node
EC2 Instances
Auto Scaling Group
V2EC2 Instances
Auto Scaling Group
V1
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management
• CodeCommit• CodePipeline• CodeDeploy
– Application Management
Conclusion
Version Control
Build/Compile
CodeDev
Unit TestApp Code
IT Ops
DR Env
Test Env
Prod Env
Dev Env
Application
WriteApp Code
Infrastructure
DeployApp.
PackageApp.
BuildAMIs
ValidateTemplates
WriteInfra Code
DeployInfras
AutomateDeploy
Artifact Repository
vA secure, highly scalable, managed source control service that hosts private Git repositories
Eliminates the need to operate your own source control system or worry about scaling its infrastructure
Built-in encryption support
Fully integrated with AWS Identity and Access Management (IAM)
Basically, managed Git
AnnouncedAWS CodeCommit
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management
• CodeCommit• CodePipeline• CodeDeploy
– Application Management
Conclusion
Version Control
Build/Compile
CodeDev
Unit TestApp Code
IT Ops
DR Env
Test Env
Prod Env
Dev Env
Application
WriteApp Code
Infrastructure
DeployApp.
PackageApp.
BuildAMIs
ValidateTemplates
WriteInfra Code
DeployInfras
AutomateDeploy
Artifact Repository
vA continuous delivery and release automation service that aids smooth deployments
You can design your development workflow for checking in code, building the code, deploying your application into staging, testing it, and releasing it to production
Able to be used stand-alone as an end-to-end solution, or can be integrated with your existing source control system, test framework or build tools (like Bamboo, Jenkins, etc) Similar to Bamboo or Jenkins
AnnouncedAWS CodePipeline
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management
• CodeCommit• CodePipeline• CodeDeploy
– Application Management
Conclusion
Version Control
Build/Compile
CodeDev
Unit TestApp Code
IT Ops
DR Env
Test Env
Prod Env
Dev Env
Application
WriteApp Code
Infrastructure
DeployApp.
PackageApp.
BuildAMIs
ValidateTemplates
WriteInfra Code
DeployInfras
AutomateDeploy
Artifact Repository
v
Limited regions
CodeDeploy workflow
CodeCommit
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management
• Elastic Beanstalk• Opsworks• CloudFormation• EC2 Container Service (ECS)
Conclusion
vAWS Elastic Beanstalk
Automated resource management – web
apps made easy
AWS OpsWorks
DevOps framework for application lifecycle management and
automation
DIY / On Demand
DIY, on demand resources: EC2, S3, custom AMI’s, etc.
Convenience Control
Deployment and Management
AWS CloudFormation
Templates to deploy & update infrastructure as
code
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management
• Elastic Beanstalk• OpsWorks• CloudFormation• EC2 Container Service (ECS)
Conclusion
Version Control
Build/Compile
CodeDev
Unit TestApp Code
IT Ops
DR Env
Test Env
Prod Env
Dev Env
Application
WriteApp Code
Infrastructure
DeployApp.
PackageApp.
BuildAMIs
ValidateTemplates
WriteInfra Code
DeployInfras
AutomateDeploy
Artifact Repository
v• Easily deploy, monitor, and scale three-tier web applications and services.
• Infrastructure provisioned and managed by EB – but you maintain complete control.
• Preconfigured application containers that are easily customizable.
• Support for these platforms:
Java PHP Python Ruby .NET Node.js Docker Go
AWS Elastic Beanstalk (EB)
vApplication
Environments• Infrastructure resources (such
as EC2 instances, ELB load balancers, and Auto Scaling groups)
• Runs a single application version at a time for better scalability
• An application can have many environments (such as staging and production)
Application versions• Application code• Stored in Amazon S3• An application can have many
application versions (easy to rollback to previous versions)
Saved configurations• Configuration that defines how
an environment and its resources behave
• Can be used to launch new environments quickly or roll-back configuration
• An application can have many saved configurations
Elastic Beanstalk model
v• Two types:• Single instance
• Load balancing, auto scaling
• Two tiers (web server and worker)• Elastic Beanstalk provisions necessary
infrastructure resources such as load balancers, auto-scaling groups, security groups, and databases (optional)
• Configures Amazon Route 53 and gives you a unique domain name
(For example: yourapp.elasticbeanstalk.com)
Elastic Beanstalk environment
vYour code
HTTP server
Application server
Language interpreter
Operating system
Host
• Elastic Beanstalk configures each EC2 instance in your environment with the components necessary to run applications for the selected platform
• No more worrying about logging into instances to install and configure your application stack
Focus on building your application
On-Instance configuration
v
Saved configurationsSave these for easy duplication for A/B testing or non-disruptive deployments
Application versionsAll versions are stored durably in Amazon S3. Code can also be pushed from a Git repository
Application versions and saved configurations
vVia the AWS Management Console Via EB CLI
Via the AWS Toolkit for Eclipse and the Visual Studio IDE
$ eb deploy
Deployment options
1
2
3
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management
• Elastic Beanstalk• OpsWorks• CloudFormation• EC2 Container Service (ECS)
Conclusion
Version Control
Build/Compile
CodeDev
Unit TestApp Code
IT Ops
DR Env
Test Env
Prod Env
Dev Env
Application
WriteApp Code
Infrastructure
DeployConf + App.
PackageApp.
BuildAMIs
ValidateTemplates
WriteInfra Code
DeployInfras
AutomateDeploy
Artifact Repository
v
On-instance execution viaChef client/zero
CommandJSON
CommandLog+StatusOpsWorks
AWS OpsWorks architecture
1
23
v
• Supports Chef 11.10• Built-in convenience cookbooks / bring your own• Chef run is triggered by lifecycle event firing: push vs. pull• Event comes with stack state JSON
Chef integration
v
A stack represents the cloud infrastructure and applications that you want to manage together
A layer defines how to setup and configure a set of instances and related resources. Eg Java App server layer, PHP layer, RDS layer, MySQL Layer, HAProxy layer etc
An instance represents an Amazon EC2 instance and defines how to scale: manually, 24/7 instances, or automatically, with load-based or time-based instances
Each applicationis represented by an app, which specifies the application type and contains the information that AWSOpsWorks needs to deploy the application from the repository to your instances
OpsWorks components
stack layer instance app
v
Instance lifecycle commands
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management
• Elastic Beanstalk• OpsWorks• CloudFormation• EC2 Container Service (ECS)
Conclusion
Version Control
Build/Compile
CodeDev
Unit TestApp Code
IT Ops
DR Env
Test Env
Prod Env
Dev Env
Application
WriteApp Code
Infrastructure
DeployApp.
PackageApp.
BuildAMIs
ValidateTemplates
WriteInfra Code
DeployInfras
AutomateDeploy
Artifact Repository
v• Infrastructure as Code
• Integrates with version control
• JSON format
• Templates
• Stacks
• Supports all AWS resource types
AWS CloudFormation
v
Architecting on AWS – Overview of Services for Web Applications
Template File Defining Stack
GitSubversion
Mercurial
The entire infrastructure can be represented in an AWS CloudFormation template.
Use the version control system of your choice to store and track changes to this template Build out multiple
environments, such as for Development, Test, and Production using the template
Application stack example
Test
Dev
Prod
v{ "Description" : "Create an EC2 instance.”, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : “my-key-pair”, "ImageId" : "ami-75g0061f”, “InstanceType” : “m1.medium” } } }}
Template anatomy
v{ "Description" : "Create an EC2 instance.”, "Parameters" : { "UserKeyName" : { "Description" : "The EC2 Key Pair to allow SSH access to the instance", "Type" : "String" } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { “Ref” : “UserKeyName”}, "ImageId" : "ami-75g0061f”, “InstanceType” : “m1.medium” } } }}
Template anatomy
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management
• Elastic Beanstalk• OpsWorks• CloudFormation• EC2 Container Service (ECS)
Conclusion
Version Control
Build/Compile
CodeDev
Unit TestApp Code
IT Ops
DR Env
Test Env
Prod Env
Dev Env
Application
WriteApp Code
Infrastructure
DeployContainers
PackageContainers
BuildAMIs
ValidateTemplates
WriteInfra Code
DeployInfras
AutomateDeploy
Artifact Repository
v• Cluster management made easy• Flexible scheduling• High performance• Resource efficiency• Extensible• Security• Programmatic control• Docker compatibility• Monitoring• AWS integration
EC2 Container Service (ECS)
vI have a docker image I want to run in a cluster
Push images
Create task definition
Run instances Use custom AMI with docker support and ECS agent. ECS agent will register with default cluster
Describe cluster Get information about cluster and available resources
Similar to fig template
Customer
Customer
Customer
Customer
Customer
User workflow
1
2
3
4
5
v
Run task
Describe cluster
Customer
Customer
User workflow
6
7
Initial cluster state
Run task
8 New cluster state
v
Agenda
Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management
Conclusion
v“Build your datacenter in 5 minutes.”
Infrastructure as Code
v“Treat your instances as cattle!” Feel free to create
and terminate instances
v
“If it moves, plot it.”
Measure everything
v“If it hurts, do it more often.”Automate
everything(for security, efficiency and
business agility)
v
Questions?