Continuous Integration and Deployment Best Practices on AWS

CONTINUOUS INTEGRATION AND DEPLOYMENT BEST PRACTICES ON AWS

Danilo Poccia – Solutions Architect

Compute & Networking

Storage & Content Delivery

AWS Global Infrastucture

Database

App Services

Anal

ytic

s

Deployment & Management

API

API

API

API

API

API

Regions Availability Zones Edge Locations

EC2 VPC Route 53 Direct Connect

S3 Glacier CloudFront Storage Gateway

RDS DynamoDB ElastiCache Redshift

SQS SNS SWF SES AppStream CloudSearch Elastic Transcoder

IAM CloudTrail CloudWatch CloudFormation Elastic Beanstalk OpsWorks

EMR

Dat

a Pi

pelin

eK

ines

is

WorkSpaces

Regions Availability Zones Edge Locations

S3 Glacier CloudFront Storage Gateway

RDS DynamoDB ElastiCache Redshift

SQS SNS SWF SES AppStream CloudSearch Elastic Transcoder

IAM CloudTrail CloudWatch CloudFormation Elastic Beanstalk OpsWorks

EMR

Dat

a Pi

pelin

eK

ines

is

EC2 VPC Route 53 Direct Connect WorkSpaces

Regions

Availability ZonesEdge Locations

EC2

VPC Route 53 Direct Connect

S3Glacier

CloudFront

Storage Gateway

RDSDynamoDB

ElastiCache

RedshiftSQS

SNS

SWF

SES

AppStreamCloudSearch

Elastic Transcoder

IAM

CloudTrail

CloudWatch CloudFormation

Elastic

Beanstalk

OpsWorks

EMR

Data Pipeline

Kinesis

WorkSpaces

http://commons.wikimedia.org/wiki/File:Lego_bricks.jpg


BUILDING BLOCKS TO IMPLEMENT YOUR IDEA

FILE SYNC

DATA ANALYSIS

DATA ANALYSIS COSTS < VALUE

AMAZON EC2 SPOT INSTANCES

2013

280

2011

82

2010

612009

48

2008

24

2007

9 Amazon FPS Red Hat EC2

SimpleDB CloudFront EBS Availability Zones Elastic IPs

Relational Database Service Virtual Private Cloud Elastic Map Reduce

Auto Scaling Reserved Instances

Elastic Load BalancerSimple Notification Service

Route 53 RDS Multi-‐AZ

Singapore Region Identity Access Management

Cluster Instances

Elastic Beanstalk Simple Email Service CloudFormation RDS for Oracle ElastiCache

Redshift Kinesis

WorkSpaces Route 53 Health Check

OpsWorks CloudTrail

number of released features, sample services described

2012

159Glacier

DynamoDB Simple Workflow

CloudSearch Storage Gateway

Route 53 Latency Based Routing

NEW SERVICES + NEW FEATURES = NEW IDEAS

AMAZON KINESIS REAL-TIME PROCESSING

OF STREAMING DATA AT MASSIVE SCALE

!Data!Sources!

App.4%!

[Machine!Learning]!

!!!!!!!!!!!!!!!

! ! !AW

S!En

dpoint!

App.1%%

[Aggregate!&!De:Duplicate]!

!Data!Sources!

Data!Sources!

!Data!Sources!

App.2%%

[Metric!Extrac=on]!

S3

DynamoDB

Redshift

App.3%[Sliding!Window!Analysis]!

!Data!Sources!

Availability Zone

Shard 1

Shard 2 Shard N

Availability Zone

Availability Zone

CLICKSTREAM ANALYTICS INTERNET OF THINGS METERING & BILLING EVENT PROCESSING SOCIAL NETWORKS LOG COLLECTION …

AMAZON SQS + AUTO SCALING + SPOT INSTANCES !

= FREEMIUM

AMAZON SNS MOBILE PUSH

NOTIFICATIONS

EVE

Online

ANY NEW

IDEA TO BUILD?

THE JOURNEY OF CODE TO

PRODUCTION

WRITTEN TESTED

CONSUMED DEPLOYED

~ MANUFACTURING ~

LEAN MANUFACTURING

Transport Inventory Motion Waiting

Over Production

Over Processing Defects Talent!

& Skills

INFORMATION TECHNOLOGY


Over Production


& Skills

OPTIMIZE WITH AWS


Over Production


& Skills

OPTIMIZE WITH AWS

OPPORTUNITY STRATEGY

USE ADJUST

= PROGRAMMABLE

PLATFORM

!

IF YOU CAN DO ITYOU CAN AUTOMATE IT

“Improving daily work is even more important than doing daily work.”

!

!

Gene Kim, Kevin Behr, George Spafford, The Phoenix Project: A Novel About IT, DevOps,

and Helping Your Business Win

CONTINUOUS INTEGRATION

DEVELOPER

SOURCE CODE REPOSITORY


PROJECT MANAGEMENT SERVER



CONTINUOUS INTEGRATION SERVER




PICK TASKS




SUBMITCODE




SCHEDULEBUILD




AUTOMATIC BUILDS




CODEFETCH




CODE QUALITY TESTS

TESTRESULTS




BUILD OUTPUT




DOCS

BINARIES& PACKAGES

TDD TEST-DRIVEN

DEVELOPMENT

BDD BEHAVIOR-DRIVEN

DEVELOPMENT

PAIN POINTS • UNIT TESTS INCOMPLETE • MOCK TESTS MAINTENANCE • EXPENSIVE TEST ENVIRONMENT • TEST ENVIRONMENT ≠ PRODUCTION • DEPLOYMENT CYCLES

“Complexity arises when the dependencies among the elements

become important.” !

!

Scott E. Page, John H. Miller,Complex Adaptive Systems:

An Introduction to Computational Models of Social Life

REAL USERS =

REAL FEEDBACK

CLOSE THE FEEDBACK

LOOP AS QUICKLY

AS POSSIBLE

CONTINUOUS INTEGRATION

CONTINUOUS DEPLOYMENT

SOFTWARE DEPLOY

≠PRODUCT LAUNCH


SMALL, FREQUENT CHANGES CONSTANTLY INTEGRATING INTO

PRODUCTION

KEY = ITERATION

ITERATION =

MODIFY THE SYSTEM TO BETTER MEET THE EXPECTATIONS OF

YOUR USERS

FEATURE SWITCH FEATURE FLIPPER FEATURE TOGGLE

FEATURE FLAG CONDITIONAL FEATURE

// To totally enabled a feature !$server_config[‘foo'] = ‘on’;!!

// To totally disabled a feature!$server_config['foo'] = ‘off’;!!

// Feature with variant turned on for everyone!$server_config['foo'] = 'blue_background';!!

// Enabled for 10% of regular users and all admins!$server_config[‘foo'] = array(! 'enabled' => 10,! 'admin' => 'on',!!);

https://github.com/etsy/feature

http://www.php.net/array


!if (Feature::isEnabled('my_feature')) {!! // do stuff!!}!! !!if (Feature::isEnabled('my_feature')) {!! !! switch (Feature::variant('my_feature')) {!! case 'foo':!! // do stuff appropriate for the foo variant!! break;!! case 'bar':!! // do stuff appropriate for the bar variant!! break;!! }!!}



A / B TESTING CONTENT

// Single-variant feature in 50/50 A/B test!!

$server_config['foo'] = array('enabled' => 50);!!

// Multi-variant feature in A/B test!// with 20% of users seeing each variant!// (and 40% left in control group)!!

$server_config['foo'] = array(!! 'enabled' => array(!! 'blue_background' => 20,!! 'orange_background' => 20,!! 'pink_background' => 20,!! ),!!);






AWS OPSWORKS MODEL AND MANAGE

THE ENTIRE APPLICATION

14 BILLION REQUESTS/MONTH50 000 DATABASE UPDATES / SEC

NO CACHE

CASE STUDY

A / B TESTING NOT JUST FOR CONTENT

USER INTERFACE ALGORITHMS

ARCHITECTURES COST OPTIMIZATION

AWS CLOUDFORMATION INFRASTRUCTURE IS CODE

CLOUDFORMATION TEMPLATE

APPLICATION VERSIONS

+INFRASTRUCTURE

VERSIONS

CLOUDFORMATION TEMPLATE

AWS ELASTIC BEANSTALK

QUICKLY DEPLOY AND MANAGE APPLICATIONS

Production Environment

Test Environment

master branch

test branch

Git Repository

git commit

git aws.push

git aws.push

Swap URLs

AMAZON EC2 API / SDK / CLI

for ip in $(!!

aws ec2 describe-instances! --filter Name=instance-state-name,Values=running,! Name=tag:aws:autoscaling:groupName,Values=XXX! --query Reservations[].Instances[].PublicDnsName! --output text!!

); do ssh -t ec2-user@$ip "/app/update.sh"; done!!

ALL TOGETHER NOW! !

!

!

!

!

DEPLOYMENTS AT AMAZON.COM

!

!

!

11.6s

Mean time between deployments (weekday)

1,079

Max number of deployments in a

single hour

10,000

Mean number of hosts

simultaneously receiving a deployment

30,000

Max number of hosts

simultaneously receiving a deployment


= CONTINUOUS

EXPERIMENTATION


= CONTINUOUS INNOVATION



« Want to increase innovation? Lower the cost of failure »

!

Joi Ito

CONTINUOUS INTEGRATION AND DEPLOYMENT BEST PRACTICES ON AWS

Danilo Poccia – Solutions Architect

Technology

Continuous Integration and Deployment Best Practices on AWS