Continuous Delivery Seminars...DevOps, Continuous Delivery and Automation DevOps (Philosophy)...

cdSummitContinuous Delivery Seminars

Automating the deployment process9th September 2014

A bit about me…

• Co-founder and Director

• Worked in Deployment Automation Solutions in

regulated environments for 10 years

• With MidVision since 2008

• A toolsmith at heart

Gartner think’s we’re cool

• Born in a Bank – regulation, risk and compliance

• Application Release Automation Platform, RapidDeploy deploy

environments AND applications at scale

• Cross industry customer success – proven in large complex

regulated enterprises

• Core Platform – with ability to integrate to Open Source and

Commercial tool chains

• Gartner DevOps Cool Vendor

Customers

Agenda

• Relationship between DevOps, CD and Automation

• Desired state, Packaging and versioning

• Deployment models

• Toolchains

• Automation and self service

DevOps, Continuous Delivery and Automation

DevOps

(Philosophy)

Continuous Delivery

(Practice)

Application Release

Automation (ARA)Application Release Automation

a.k.a. Deployment Automation

“The process of packaging and deploying an

application or update of an application from

development, across various environments,

and ultimately to production”

Application Deploy

Infrastructure Automation

Image Provisioning

Release Management

Application Configuration

APPLICATION DEPLOYMENT

Continuous Delivery / Release Pipelines

Automated Deployment Principles

• Version control and build configuration

• Release versions of configuration change in the

same way as code

• Allows linking of configuration releases and

application releases – together comprising a

business application

• “Build Once, Deploy Anywhere”

8

Model Driven Deployment

• Model defines complete configuration for a logical

unit of automation

• Define similarities and differences

• Always contain the complete configuration for a

target technology

• Deltas become part of the template – so

configuration changes are not made in isolation

• Idempotent

9

Properties / Template

MQSC Template

------<QMGR_NAME>--------

------<QMGR_HOST>--------

------<QMGR_PORT>--------

DevTst01

QMGR_NAME SALESDEVQM01

QMGR_HOST lnxdev01

QMGR_PORT 1414

10

Properties / Template

MQSC Template

------<QMGR_NAME>--------

------<QMGR_HOST>--------

------<QMGR_PORT>--------

DevTst01

QMGR_NAME SALESDEVQM01

QMGR_HOST lnxdev01

QMGR_PORT 1414

SysTst01

QMGR_NAME SALESTSTQM02

QMGR_HOST lnxtst01

QMGR_PORT 1415

11

Properties / Template

MQSC Template

------<QMGR_NAME>--------

------<QMGR_HOST>--------

------<QMGR_PORT>--------

------<DB_USER>--------

DevTst01

QMGR_NAME SALESDEVQM01

QMGR_HOST lnxdev01

QMGR_PORT 1414

DB_USER devtstuser

SysTst01

QMGR_NAME SALESTSTQM02

QMGR_HOST lnxtst01

QMGR_PORT 1415

DB_USER syststuser

Business Appliationdev_area \DeployDesc

\database(s)\source

Infrastructure ‘Desired State’release_area \jee

\scripts\static\database

Baseline File: SALESAPP_BUILD_01_02_003

DEPLOY PACKAGE

SALESAPP_BUILD_01_02_003.zip

Creating the Deploy Package

Implementation-Version=SALESAPP_BUILD_01_02_003

DevOps Toolchain

`

Release Manager

Directed Deployment IHS

WAS

MQ

cluster/ds/jms

httpd.conf

qmgr/queue

httpd.conf

IHS

WAS

MQ

MQ

WAS

IHSIHS

WAS

MQ

cluster/ds/jms

httpd.conf

qmgr/queue

cluster/ds/jms

httpd.conf

qmgr/queue

cluster/ds/jms

qmgr/queueImplementation Manager selects:

• Application

• Environment

• Version

SALESAPP_BUILD_01_02_003.tarSALESAPP_BUILD_01_02_004.tar

Composite Deployment

DevOps Toolchain

17

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

18

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

x.mqsc Alter channel

19

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

x.mqsc Alter channel SALES_MQ_01_02_033.zip

20

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ

21

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ

x.py JMS Queue Dest

22

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ

x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip

23

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ

x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip SALES_JBOSS_01_02_046 JBoss

24

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ

x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip SALES_JBOSS_01_02_046 JBoss

x.ddl Column, View, etc

25

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ

x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip SALES_JBOSS_01_02_046 JBoss

x.ddl Column, View, etc SALES_DB_01_02_012.zip

26

Version

Control

Configuration

Management

Build Process Release Mechanism Targets

Code

Configuration

Commandline /

WebUI / REST /

Eclipse

Generate Versioned Release

Components

Ideally web based, self

service

Web / Application /

Database / etc

x.mqsc Alter channel SALES_MQ_01_02_033.zip SALES_MQ_01_02_033 MQ

x.py JMS Queue Dest SALES_JBOSS_01_02_046.zip SALES_JBOSS_01_02_046 JBoss

x.ddl Column, View, etc SALES_DB_01_02_012.zip SALES_DB_01_02_012 Oracle

27

AppServer - products

Apache HTTP

Jboss 6.2

RabbitMQ

RHEL 6 Update 5

Database - products

Oracle 11.1.0.7.0

RHEL 6 Update 5

Provisioning New EnvironmentsProducts

28

AppServer - products Configuration / App

Apache HTTP WEB_CONFIG_SALES_1_2_056

Jboss 6.2 JBOSS_CONFIG_SALES_1_2_087

RabbitMQ MQ_CONFIG_SALES_1_2_032

RHEL 6 Update 5 STANDARD ORG IMAGE

Database - products Configuration / App

Oracle 11.1.0.7.0 ORA_CONFIG_SALES_1_2_042

RHEL 6 Update 5 STANDARD ORG IMAGE

Provisioning New EnvironmentsProducts > Configuration

29

SalesRelease 1.2

AppServer - products Configuration / App

APP_CONTENT_SALES_1_2_124

Apache HTTP WEB_CONFIG_SALES_1_2_056

Jboss 6.2 JBOSS_CONFIG_SALES_1_2_087

RabbitMQ MQ_CONFIG_SALES_1_2_032

RHEL 6 Update 5 STANDARD ORG IMAGE

Database - products Configuration / App

Oracle 11.1.0.7.0 ORA_CONFIG_SALES_1_2_042

RHEL 6 Update 5 STANDARD ORG IMAGE

Provisioning New EnvironmentsProducts > Configuration > Code

Continuous Delivery Overview

Packaging Principles, Build Once, Deploy Anywhere… when does Continuous Beer start!

Packaging PrincipleBuild Once, Deploy Anywhere

• Build once, deploy to any (defined) environment in the pipeline.

• A package should be a single compressed, versioned file.– Package integrity across all environments

– Check-summed and labeled

• The package should contains all deployment instructions, environment configuration and resources (or address locations to remote resources).

• Package should be runnable as a standalone process.- not dependent on the deployment tool.

• Store and retrieve deployment artifacts in a DSL.

• Use Jenkins and to build your deployment package and store it in the DSL.

Packaging Model for Deployment Artifacts

• Topology of different environments can define the packaging model to use.

• Single App / Single Cluster – Deploy application and configuration together

• Multiple App / Multiple Cluster– Deploy applications and configurations separately

Deployment PackageApp Config

Deployment Package 1App 1

Deployment Package 2ConfigApp 2

Prerequisite Resource & Environment Testing

• Define which environment resources need testing:– file permissions– disk space– port availability– user accounts

• Do not continue with the deployment if the prerequisite tests fail.• Automating prerequisite testing of resources saves a lot of pain and time.• Keep adding tests to the deployment as you get a better understanding of

the environment.• Sometimes known as a “Deployment Dry Run”.

Automated Testing Strategy

• Use Jenkins post build plugins to initiate the tests after successful a deployment.

• Building pipelines can be quick, converting your manual tests to automated tests can be laborious and take months…..stick with it.

Rollback Strategy• Rollback strategy depends on the technology of the

deployment.• Database changes cannot be rolled back automatically

if business data has been added or amended since the deployment took place. Need a DBA.

• Make sure DB changes are backwards compatible.• Generally with databases, use a fix forward strategy.• Middleware can be rolled back if the code is deployed

with the configuration.

Metrics, Analytics & Feedback Loop

• Measure the release process.– Deployment times– No. of deployments (successful & failed).– Velocity

• Report on the release process measurements.• Show management the improvements CD is making to your organization.• Fully automating CD is not cheap. Use the metrics to get management on-

side with investing in CD.• Make use of the CI and CD metrics from Jenkins to help with the analytics.

What are we trying to achieve?

Faster(Acceleration)

We automate processes carried out by IT

Faster deployments through whole lifecycle

Bring products to market quicker

Safer (Lower Operational Risk)

Eliminate outages

Workflow-scheduling

Audit & compliance controls

Cheaper(Reduce TCO)

More resource time for value-creation

Why outsource when you can automate

Continuous Delivery ….e2e Pipeline

• Continuous Integration

• Automated Unit Testing

• Artifact creation

• Automated Acceptance testing /

Approval

• Automated Deployment

AutomationAutomate your tool chainempower your team

III