Continuous Delivery on AWS Stephan Hadinger (hadinger@amazon.fr) Rudy Krol (rudykrol@amazon.fr)

Continuous Delivery on AWS

Stephan Hadinger (hadinger@amazon.fr)

Rudy Krol (rudykrol@amazon.fr)

v

~11.6sMean time between

deployments (weekday)

~1,079Max number of deployments

in a single hour

~10,000Mean number of hosts

simultaneously receiving a deployment

~30,000Max number of hosts

simultaneously receiving a deployment

Deployments at Amazon.com

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management

Conclusion

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management

Conclusion

v

Version Control CI Server

Package Builder

Commit to Git/master

Dev

Get / PullCode

Send Build Report to DevStop everything if build failed

Distributed BuildsRun Tests in parallel

CodeConfigTests

Push

Config

Repo

Continuous Integration

v

• Test driven promotion (of development change)• Increasing velocity of feedback cycle through iterative change• Bugs are detected quickly• Automated testing reduces size of testing effort

What does CI give us?

v

Continuous Delivery/Deployment

Version Control CI Server

Package Builder

Deploy ServerCommit to

Git/masterDev

Get / PullCode

AMIs

Send Build Report to DevStop everything if build failed

Distributed BuildsRun Tests in parallel

Staging Env

Test Env

CodeConfigTests

Prod Env

Push

Config InstallCreate

Repo

CloudFormation Templates for Env

Generate

v

• Automated, repeatable process to push changes to production• Hardens, de-risks the deployment process• Immediate feedback from users• Supports A/B testing or “We test customer reactions to features in

production”• Gives us a breadth of data points across our applications

What does CD give us?

v

Continuous Delivery versus Continuous Deployment

v

Example CI/CD pipeline

Dev

IT Ops

Version Control

Build/Compile

Code

Unit TestApp Code

DR Env

Test Env

Prod Env

Dev Env

Application

WriteApp Code

Infrastructure

CloudFormation

tar, war, zipyum, rpmDeploy

AppPackage

Application

Deploy application only

Deploy infrastructure only

BuildAMIs

ValidateTemplates

WriteInfra Code

DeployInfras

AutomateDeployment

Artifact Repository

AMIs

vHOST METRICS

SERVICEMETRICS

LOG ANALYSIS EXTERNAL SITE METRICS

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management

Conclusion

v

• How fast do we need to do this?• Across how many instances?• How do we roll back (or forward)?

Delivery approaches

v

AMI building/deployment methods

vFully Functional AMI OS-Only AMI

Partially Configured AMI

Most amount of post-boot work

Least flexibleto maintain

Try and find a happy medium here

Delivery approaches

vDeploy in place • Deploy all at once (service outage)• Rolling updates

Red-Black deployment• Discrete environment• Multiple environments from branches• Support A/B testing• Use auto scaling group

Deployment approaches

v

Deploy in place – Rolling update

OpsWorks CodeDeploy

v

Deploy in place – Rolling update

v

Deploy in place – Rolling update

v

Deploy in place – Rolling update

v

Deploy in place – Rolling update

v

Deploy in place – Rolling update

v

Deploy in place – Rolling update

v

Deploy in place – Rolling update

v

Red-Black deployment

EC2 Instances

ELB

DynamoDB MySQL RDSInstance

ElastiCache Cache Node

Auto Scaling Group

V1

v

Red-Black deployment

EC2 Instances

ELB

DynamoDB MySQL RDSInstance

ElastiCache Cache Node

Auto Scaling Group

V1 EC2 Instances

UAT

Auto Scaling Group

V2

v

Red-Black deployment

EC2 Instances

ELB

DynamoDB MySQL RDSInstance

ElastiCache Cache Node

Auto Scaling Group

V1 EC2 Instances

Auto Scaling Group

V2

v

Red-Black deployment

ELB

DynamoDB MySQL RDSInstance

ElastiCache Cache Node

EC2 Instances

Auto Scaling Group

V2EC2 Instances

Auto Scaling Group

V1

v

Red-Black deployment

ELB

DynamoDB MySQL RDSInstance

ElastiCache Cache Node

EC2 Instances

Auto Scaling Group

V2EC2 Instances

Auto Scaling Group

V1

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management

• CodeCommit• CodePipeline• CodeDeploy

– Application Management

Conclusion

Version Control

Build/Compile

CodeDev

Unit TestApp Code

IT Ops

DR Env

Test Env

Prod Env

Dev Env

Application

WriteApp Code

Infrastructure

DeployApp.

PackageApp.

BuildAMIs

ValidateTemplates

WriteInfra Code

DeployInfras

AutomateDeploy

Artifact Repository

vA secure, highly scalable, managed source control service that hosts private Git repositories

Eliminates the need to operate your own source control system or worry about scaling its infrastructure

Built-in encryption support

Fully integrated with AWS Identity and Access Management (IAM)

Basically, managed Git

AnnouncedAWS CodeCommit

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management

• CodeCommit• CodePipeline• CodeDeploy

– Application Management

Conclusion

Version Control

Build/Compile

CodeDev

Unit TestApp Code

IT Ops

DR Env

Test Env

Prod Env

Dev Env

Application

WriteApp Code

Infrastructure

DeployApp.

PackageApp.

BuildAMIs

ValidateTemplates

WriteInfra Code

DeployInfras

AutomateDeploy

Artifact Repository

vA continuous delivery and release automation service that aids smooth deployments

You can design your development workflow for checking in code, building the code, deploying your application into staging, testing it, and releasing it to production

Able to be used stand-alone as an end-to-end solution, or can be integrated with your existing source control system, test framework or build tools (like Bamboo, Jenkins, etc) Similar to Bamboo or Jenkins

AnnouncedAWS CodePipeline

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management

• CodeCommit• CodePipeline• CodeDeploy

– Application Management

Conclusion

Version Control

Build/Compile

CodeDev

Unit TestApp Code

IT Ops

DR Env

Test Env

Prod Env

Dev Env

Application

WriteApp Code

Infrastructure

DeployApp.

PackageApp.

BuildAMIs

ValidateTemplates

WriteInfra Code

DeployInfras

AutomateDeploy

Artifact Repository

v

Limited regions

CodeDeploy workflow

CodeCommit

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management

• Elastic Beanstalk• Opsworks• CloudFormation• EC2 Container Service (ECS)

Conclusion

vAWS Elastic Beanstalk

Automated resource management – web

apps made easy

AWS OpsWorks

DevOps framework for application lifecycle management and

automation

DIY / On Demand

DIY, on demand resources: EC2, S3, custom AMI’s, etc.

Convenience Control

Deployment and Management

AWS CloudFormation

Templates to deploy & update infrastructure as

code

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management

• Elastic Beanstalk• OpsWorks• CloudFormation• EC2 Container Service (ECS)

Conclusion

Version Control

Build/Compile

CodeDev

Unit TestApp Code

IT Ops

DR Env

Test Env

Prod Env

Dev Env

Application

WriteApp Code

Infrastructure

DeployApp.

PackageApp.

BuildAMIs

ValidateTemplates

WriteInfra Code

DeployInfras

AutomateDeploy

Artifact Repository

v• Easily deploy, monitor, and scale three-tier web applications and services.

• Infrastructure provisioned and managed by EB – but you maintain complete control.

• Preconfigured application containers that are easily customizable.

• Support for these platforms:

Java PHP Python Ruby .NET Node.js Docker Go

AWS Elastic Beanstalk (EB)

vApplication

Environments• Infrastructure resources (such

as EC2 instances, ELB load balancers, and Auto Scaling groups)

• Runs a single application version at a time for better scalability

• An application can have many environments (such as staging and production)

Application versions• Application code• Stored in Amazon S3• An application can have many

application versions (easy to rollback to previous versions)

Saved configurations• Configuration that defines how

an environment and its resources behave

• Can be used to launch new environments quickly or roll-back configuration

• An application can have many saved configurations

Elastic Beanstalk model

v• Two types:• Single instance

• Load balancing, auto scaling

• Two tiers (web server and worker)• Elastic Beanstalk provisions necessary

infrastructure resources such as load balancers, auto-scaling groups, security groups, and databases (optional)

• Configures Amazon Route 53 and gives you a unique domain name

(For example: yourapp.elasticbeanstalk.com)

Elastic Beanstalk environment

vYour code

HTTP server

Application server

Language interpreter

Operating system

Host

• Elastic Beanstalk configures each EC2 instance in your environment with the components necessary to run applications for the selected platform

• No more worrying about logging into instances to install and configure your application stack

Focus on building your application

On-Instance configuration

v

Saved configurationsSave these for easy duplication for A/B testing or non-disruptive deployments

Application versionsAll versions are stored durably in Amazon S3. Code can also be pushed from a Git repository

Application versions and saved configurations

vVia the AWS Management Console Via EB CLI

Via the AWS Toolkit for Eclipse and the Visual Studio IDE

$ eb deploy

Deployment options

1

2

3

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management

• Elastic Beanstalk• OpsWorks• CloudFormation• EC2 Container Service (ECS)

Conclusion

Version Control

Build/Compile

CodeDev

Unit TestApp Code

IT Ops

DR Env

Test Env

Prod Env

Dev Env

Application

WriteApp Code

Infrastructure

DeployConf + App.

PackageApp.

BuildAMIs

ValidateTemplates

WriteInfra Code

DeployInfras

AutomateDeploy

Artifact Repository

v

On-instance execution viaChef client/zero

CommandJSON

CommandLog+StatusOpsWorks

AWS OpsWorks architecture

1

23

v

• Supports Chef 11.10• Built-in convenience cookbooks / bring your own• Chef run is triggered by lifecycle event firing: push vs. pull• Event comes with stack state JSON

Chef integration

v

A stack represents the cloud infrastructure and applications that you want to manage together

A layer defines how to setup and configure a set of instances and related resources. Eg Java App server layer, PHP layer, RDS layer, MySQL Layer, HAProxy layer etc

An instance represents an Amazon EC2 instance and defines how to scale: manually, 24/7 instances, or automatically, with load-based or time-based instances

Each applicationis represented by an app, which specifies the application type and contains the information that AWSOpsWorks needs to deploy the application from the repository to your instances

OpsWorks components

stack layer instance app

v

Instance lifecycle commands

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management

• Elastic Beanstalk• OpsWorks• CloudFormation• EC2 Container Service (ECS)

Conclusion

Version Control

Build/Compile

CodeDev

Unit TestApp Code

IT Ops

DR Env

Test Env

Prod Env

Dev Env

Application

WriteApp Code

Infrastructure

DeployApp.

PackageApp.

BuildAMIs

ValidateTemplates

WriteInfra Code

DeployInfras

AutomateDeploy

Artifact Repository

v• Infrastructure as Code

• Integrates with version control

• JSON format

• Templates

• Stacks

• Supports all AWS resource types

AWS CloudFormation

v

Architecting on AWS – Overview of Services for Web Applications

Template File Defining Stack

GitSubversion

Mercurial

The entire infrastructure can be represented in an AWS CloudFormation template.

Use the version control system of your choice to store and track changes to this template Build out multiple

environments, such as for Development, Test, and Production using the template

Application stack example

Test

Dev

Prod

v{ "Description" : "Create an EC2 instance.”, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : “my-key-pair”, "ImageId" : "ami-75g0061f”, “InstanceType” : “m1.medium” } } }}

Template anatomy

v{ "Description" : "Create an EC2 instance.”, "Parameters" : { "UserKeyName" : { "Description" : "The EC2 Key Pair to allow SSH access to the instance", "Type" : "String" } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { “Ref” : “UserKeyName”}, "ImageId" : "ami-75g0061f”, “InstanceType” : “m1.medium” } } }}

Template anatomy

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management

• Elastic Beanstalk• OpsWorks• CloudFormation• EC2 Container Service (ECS)

Conclusion

Version Control

Build/Compile

CodeDev

Unit TestApp Code

IT Ops

DR Env

Test Env

Prod Env

Dev Env

Application

WriteApp Code

Infrastructure

DeployContainers

PackageContainers

BuildAMIs

ValidateTemplates

WriteInfra Code

DeployInfras

AutomateDeploy

Artifact Repository

v• Cluster management made easy• Flexible scheduling• High performance• Resource efficiency• Extensible• Security• Programmatic control• Docker compatibility• Monitoring• AWS integration

EC2 Container Service (ECS)

vI have a docker image I want to run in a cluster

Push images

Create task definition

Run instances Use custom AMI with docker support and ECS agent. ECS agent will register with default cluster

Describe cluster Get information about cluster and available resources

Similar to fig template

Customer

Customer

Customer

Customer

Customer

User workflow

1

2

3

4

5

v

Run task

Describe cluster

Customer

Customer

User workflow

6

7

Initial cluster state

Run task

8 New cluster state

v

Agenda

Introduction to Continuous Integration (CI) and Continuous Delivery/Deployment (CD)CD strategiesCI-CD on AWS– Application Lifecycle Management– Application Management

Conclusion

v“Build your datacenter in 5 minutes.”

Infrastructure as Code

v“Treat your instances as cattle!” Feel free to create

and terminate instances

v

“If it moves, plot it.”

Measure everything

v“If it hurts, do it more often.”Automate

everything(for security, efficiency and

business agility)

v

Questions?