L A S T U P D A T E D F E B R U A R Y 6 , 2 0 1 7

DEVELOPING DEVOPS ON AWSBy Eric Johnson

INTRODUCTION 3

CONTINUOUS INTEGRATION AND CONTINUOUS DELIVERY (CI/CD) 4

Continuous Integration 4

Continuous Delivery 4

The Bottom Line 5

THE RIGHT TOOLS 5

AWS CodeCommit 5

AWS CodePipeline 6

AWS CodeBuild 7

AWS CodeDeploy 8

WHAT TO WATCH FOR 9

CONCLUSION: PUTTING IT ALL TOGETHER 10

W H I T E P A P E R

DEVELOPING DEVOPS ON AWS

2 / 11

3 / 11DEVELOPING DEVOPS ON AWS

I N T R O D U C T I O N

Put 20 people in a room and ask them to define DevOps; chances are you will

get 15–20 different responses. In its most basic form, DevOps is a merging of the

words “Development” and “Operations.” But the truth is, DevOps is more than a

word; it is a process or even a culture. In her article on automic.com, Courtney

Glyph explains DevOps this way:

DevOps defines a set of practices that, when applied correctly,

“emphasizes the collaboration and communication of both software

developers and other IT professionals while automating the process of

software delivery and infrastructure changes. It creates a culture and

environment where building, testing and releasing software can happen

rapidly, frequently and more reliably.” [read full article]

This white paper focuses on the collaboration and automation aspects of

DevOps and how they are accomplished in AWS. By the end, you will have a

conceptual idea of how collaboration and automation come together to provide

a continuous integration and continuous delivery (CI/CD) process that can

benefit your development process and, furthermore, your bottom line.

W H I T E P A P E R

DEVELOPING DEVOPS ON AWS

4 / 11DEVELOPING DEVOPS ON AWS

CONTINUOUS INTEGRATION AND CONTINUOUS DELIVERY To understand what it takes to create a proper CI/CD process on AWS, it

important to establish what that process looks like and why it’s needed. First,

examine at the process. A solid CI/CD process offers the ability to version and

integrate code, build code specific to an environment, allow for manual approval

or rejection, and deploy code with no downtime as well as the ability to rollback

when needed.

Visualizing the process (above) enables the breaking down of the CI/CD process

into separate parts and allows an explanation of their benefits individually and

then as a whole.

Continuous Integration

The first part of the equation is the process of continuous integration - a

development process that encourages developers to code in very small chunks.

As a developer adds a feature or fixes a bug, they should then commit that

code to the shared repository. When that code is committed, there is then an

automated process that rebuilds the entire code set with the new changes

included and updates the testing location with that new code. The end result is

that the code is continuously integrated and made available for testing.

The advantages of a continuous integration system are numerous. The first

advantage is the ability to test early and test often. When a QA team can test

code early in the process, they can flag issues in the code before the developer

has moved on too far. A second advantage is the ability for the business to also

catch issues in the code direction. As code is updated early and often, testers

can verify that what the developer built matches the scope that has been

laid out. This also allows the business to make early adjustments if what they

envisioned does not work in real life.

Continuous Delivery

The second part of the equation, continuous delivery, builds on top of the

process of continuous integration. When a single piece of code passes all tests,

and it gets the stamp of approval from the business, the logical next step is to

ship it. Continuous delivery is the automated process of shipping code through

multiple environments, with it eventually ending up in production. This is a

paradigm shift from the two-to-three-week sprints culminating in a manual

deploy that many businesses still use today.

5 / 11DEVELOPING DEVOPS ON AWS

The Bottom Line

A properly architected CI/CD process affects a company’s bottom line in

several different ways. Foremost, by being able to deploy with a push of a

button, a company can be incredibly responsive to the customer. For example,

if code is deployed and a company finds that it is not performing well, they

can immediately rollback. Also, with the ability to change code frequently and

quickly, a company can make good use of multi-variant (A, B, (C)) testing to

help improve the user experience. Any time the user is helped to better use a

company’s product, the bottom line is directly affected.

T H E R I G H T T O O L S The DevOps process of CI/CD is not an entirely new concept. There are quite a

few tools on the market such as TeamCity, Jenkins, Hudson, Octopus Deploy,

and others that play in this space. However, AWS has a full suite of native tools to

accomplish this task. Let’s take a look at the different tools available.

AWS CodeCommit

The first tool worth examination is AWS CodeCommit. AWS CodeCommit is a

fully managed source control service that provides private GIT repositories within

your AWS account. Now you may be asking yourself, “Why not just use GitHub?”

This is a great question. over. While there are some advantages to using GitHub,

such as the advanced tooling GitHub adds, there are two clear advantages to

using AWS CodeCommit over GitHub. First of all, and most importantly, AWS

CodeCommit enables greater security. By using AWS CodeCommit your code

never leaves your AWS environment, with the exception of when it is on the

developer’s machine. Furthermore, it is protected by the many layers of security

provided by AWS and the granular access control that Identity and Access

Management (IAM) provides. The second advantage of AWS CodeCommit is

pricing. AWS CodeCommit gives you five users free; each additional user after

that is one dollar.

Within the AWS CI/CD process, AWS CodeCommit is where developers check

their completed code. The repository uses GIT to handle merging, conflicts, and

versioning of code.

Learn more about AWS CodeCommit.

6 / 11DEVELOPING DEVOPS ON AWS

AWS CodePipeline

The second tool in AWS’s DevOps suite is AWS CodePipeline. Notice (above)

that AWS CodePipeline deals with build, manual interface, and deploy. AWS

CodePipeline is the automation tool designed to detect code changes and then

move the code through the other phases. The beauty of AWS CodePipeline is

that it is very configurable. Different jobs can be added to your pipeline and

decisions based on the outcomes of that job.

For example, you can grab the latest code from AWS CodeCommit and ship it

off to a builder like Jenkins. Jenkins can then build and test your code. If all goes

well, Jenkins will notify AWS CodePipeline to move on. AWS CodePipeline can

then send an email to your QA team. The QA team can then test the latest code.

On approval, the user has the choice to approve or reject the code by clicking

on the appropriate button in the email. If the approval is given, then AWS

CodePipeline moves on to the next step. If not, then the pipeline is exited.

Learn more about AWS CodePipeline.

7 / 11DEVELOPING DEVOPS ON AWS

AWS CodeBuild

AWS CodeBuild is the latest tool in the AWS DevOps arsenal. Announced at

AWS Re:Invent in 2016, AWS CodeBuild closes a large gap in the tooling suite by

handling the process of building and testing your code. Before the introduction

of AWS CodeBuild, organizations were forced to look outside of AWS to third-

party tools like Jenkins and Solano. Additionally, when using third-party tools,

organizations were also forced to maintain the infrastructure the tool ran on in

the form of an instance or container, or they had to pay a separate provider to

maintain it and ship the code externally to be built and tested. CodeBuild is

serverless and empowers companies to build and test code while keeping that

code inside of their preferred AWS environment and pipeline.

In looking at how AWS CodeBuild works, it is important to understand where

AWS CodeBuild runs: AWS CodeBuild runs in a pre-built container provided by

AWS. AWS offers many different container flavors like Node, Golang, Python, and

Java, however you can also create your own container for AWS CodeBuild to

launch.

If you’re saying, “Wait a minute, that’s not serverless!” please note that it is as

serverless as any other serverless technology AWS offers. AWS CodeBuild is what

is known as a “backplane” technology, meaning they manage the infrastructure

and it, therefore, is serverless to your organization.

Using AWS CodeBuild to build your project then becomes very easy. The beauty

of the tool is that anything you can do on a command line, you can do on AWS

CodeBuild. When running a project, AWS CodeBuild will look in the root of your

code for a buildspec.yml file. In the buildspec file you define five phases and

what to do in them. Let’s break those out and explain them.

Install Phase: This phase allows you to install any external libraries that may

not be included in AWS CodeBuild. For example, you might install “mocha,”

a node test library.

Pre-Build Phase: This phase allows you to run any pre-build commands

or scripts. This would be a good place to set environment parameters and

create mock data for tests.

Build Phase: This phase allows you to actually build out your code. In a node

world, this is where you would transpile and pack code using tools like Babel

and WebPack.

8 / 11DEVELOPING DEVOPS ON AWS

Post-Build Phase: This phase allows you to run commands after your code

is built. Here you might run unit and regression tests against prepared code.

Artifacts Phase: This phase allows you to define what files should be

returned from AWS CodeBuild back to AWS CodePipeline for the next job in

the pipeline.

AWS CodeBuild is a very simple and very powerful tool. It allows developers to

use the same command line tools they utilize locally, to build and test code in

an automated fashion to push to other environments.

Learn more about AWS CodeBuild.

AWS CodeDeploy

Up until this point, it’s been possible to merge and version code, build and

test that code in the cloud, and get manual approval from your QA team. The

final piece of the puzzle is the ability to deploy the code. This is where AWS

CodeDeploy becomes important. AWS CodeDeploy provides the ability to

deploy and rollback code to multiple servers at a time, plus allows you to deploy

to servers outside of AWS in addition to AWS servers. For example, if you are

running a hybrid architecture, you can deploy to your cloud servers and your

servers on-premises as well.

It’s easiest to get a full understanding of how AWS CodeDeploy works by talking

through a deployment scenario and the components. To begin, you need

to understand what code you are going to deploy and where it exists. One

requirement of AWS CodeDeploy is to have deployment code in a versioned

format. This can be either in the form of an Amazon Simple Storage Service

(Amazon S3) bucket with versioning on, or a GIT repository. When using AWS

CodePipeline and AWS CodeBuild together, you can drop your final artifacts

in the bucket or repository. By using a versioned source, AWS CodeDeploy can

rollback to an earlier version if necessary.

Much like AWS CodeBuild, AWS CodeDeploy will be looking for a file called

appspec.yml. This file is also broken down into several parts.

Files: In the files section, you can indicate your source and destination

locations for deployment. This might include configuration files in one place,

and application files in another.

9 / 11DEVELOPING DEVOPS ON AWS

Hooks: In the hooks section, you can call scripts to be run before installation,

after installation, and on application startup. When all is done, you can also

run a script to validate the service.

AWS CodeDeploy also offers the ability to define how you want your code to

deploy. At a top level, there are two types of deployment, in-place and blue/

green. An in-place deployment keeps the same servers and updates the code

on these servers. A blue/green deployment places the new code on new servers

and re-routes traffic to the new servers.

Within these deployment types there are three approaches. The first approach,

All at Once, attempts to deploy to as many instances at one time as it can. The

second approach, Half at a Time, deploys to half of the instances at a time with

fractions rounding down. The final approach to deployment is One at a Time, in

which AWS CodeDeploy deploys code to one instance at a time.

So, what happens if a deploy fails or needs rollback? If automatic rollbacks have

been configured, then rollbacks happen automatically. In the case of a failed

deploy, or if a monitoring threshold is met, AWS CodeDeploy will automatically

rollback to the last know good version. If automatic rollbacks are not configured,

a manual rollback can be initiated by creating a new deploy using a previous

revision.

Learn more about AWS CodeDeploy.

W H A T T O W A T C H F O R At the time of this writing, AWS CodeBuild does not support Microsoft Windows

build tools. However, AWS is quickly working to include a Windows tool set and

are collecting emails for notification when they release it.

Be aware that these tools are very configuration heavy. Configuring the AWS

DevOps tools can be done from the AWS dashboard, but it’s better managed

and maintained when using the command line interface, or better yet, AWS

CloudFormation. AWS CloudFormation truly lets you experience “infrastructure

as code” and all your build configurations can be included.

10 / 11DEVELOPING DEVOPS ON AWS

C O N C L U S I O N : P U T T I N G I T A L L T O G E T H E R With the combined power of AWS CodeCommit, AWS CodeBuild, and AWS

CodeDeploy working in concert with AWS CodePipeline orchestrating, you can

build out a very powerful CI/CD process.

This example (above) gives a full CI/CD process, including deployment to

multiple environments. Take a look at the scenario:

1. Multiple developers are able to merge their completed code into AWS

CodeCommit.

2. On a completed merge and commit, AWS CodePipeline then grabs the

latest code and passes it to AWS CodeBuild.

3. AWS CodeBuild then compiles, minifies (compresses), and tests the

code. If all goes well and all tests are passed, the final code is then

dropped into the proper bucket and the process is passed back to AWS

CodePipeline.

4. AWS CodePipeline then notifies AWS CodeDeploy.

5. AWS CodeDeploy then deploys the latest version of code to a beta

environment. Upon completion, AWS CodeDeploy then passes the

process back to AWS CodePipeline.

6. AWS CodePipeline then sends an email to a user asking them to

manually verify the code. If the user verifies the code and accepts the

change, AWS CodePipeline then notifies AWS CodeDeploy to continue.

7. AWS CodeDeploy then repeats the process for staging and, finally,

production.

As you can see, using the AWS DevOps tools, it’s possible

to automate versioning, compiling, testing, approval and

deployment to all environments. Several advantages

come with this automation. Primarily, it is a repeatable

process that is monitored. If anything in the process needs

changing, it is then changed every time. Secondly, this

process verifies that all automation tests are passed and

that the project has also been manually tested. While

the process cannot prove that a person has tested the

code, it can prove they acknowledged that they did,

which promotes accountability. An added bonus is that

the entire pipeline is serverless, meaning there is no

infrastructure to maintain.

Learn more about how Rackspace can help you get the

most out of AWS at https://www.rackspace.com/en-us/

managed-aws.

Eric Johnson is the AWS Evangelist at Rackspace. He has

been working with AWS technologies as a developer and

an architect for the last six years. His passion for all things

new in cloud technologies drive him to be a lifetime

student and fanatical speaker on all things cloud. Connect

with him on Twitter and LinkedIn.

© 2017 Rackspace US, Inc. Rackspace®, Fanatical Support® and other Rackspace marks are either registered service marks or service marks of Rackspace US, Inc. and are registered in the United States and other countries. All other trademarks, service marks, images, products and brands remain the sole property of their respective holders and do not imply endorsement or sponsorship.

This case study is for your informational purposes only. RACKSPACE MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS CASE STUDY. All customer examples and the information and results illustrated here are based upon the customer’s experiences with the referenced Rackspace services and are not necessarily indicative of the future performance of Rackspace services. Rackspace detailed services descriptions and legal commitments are stated in its services agreements. Rackspace services’ features and benefits depend on system configuration and may require enabled hardware, software or additional service activation. Actual cost of specific hosted environment and performance characteristics will vary depending on individual customer configurations and use case.

About AWS: For 10 years, Amazon Web Services has been

the world’s most comprehensive and broadly adopted cloud

platform. AWS offers over 70 fully featured services for compute,

storage, databases, analytics, mobile, Internet of Things (IoT) and

enterprise applications from 35 Availability Zones (AZs) across 13

geographic regions in the U.S., Australia, Brazil, China, Germany,

Ireland, Japan, Korea, Singapore, and India. AWS services are

trusted by more than a million active customers around the

world – including the fastest growing startups, largest enterprises,

and leading government agencies – to power their infrastructure,

make them more agile, and lower costs.

To learn more about AWS, visit aws.amazon.com.

Rackspace, the #1 managed cloud company, helps businesses

tap the power of cloud computing without the complexity and

cost of managing it on their own. Rackspace engineers deliver

specialized expertise, easy-to-use tools, and Fanatical Support®

for leading technologies developed by AWS, Google, Microsoft,

OpenStack, VMware and others. The company serves customers

in 120 countries, including more than half of the FORTUNE

100. Rackspace was named a leader in the 2015 Gartner Magic

Quadrant for Cloud-Enabled Managed Hosting, and has been

honored by Fortune, Forbes, and others as one of the best

companies to work for.

Learn more at www.rackspace.com or call us at 1-800-961-2888.