How do you implement Continuous Delivery? Part 3: All about Pipelines

How do you implement

Continuous Delivery?

Part 3: All about Pipelines

Pipeline for

progressive testing

With CD, every build is a release candidate Pipeline for progressive testing

þ þ ý ý þ þ ý þ þ

The Testing Pyramid

Pipeline for progressive testing

Faster Feedback

Broader Scope

The Testing Pyramid

Pipeline for progressive testing

The Delivery Pipeline

Faster Feedback

Broader Scope

Faster Feedback

Increasing Confidence

The system being developed Pipeline for progressive testing

Other Services

Application A

Application B

Pipeline for progressive testing

Unit Tests

Each unit test covers one app component

Other components are stubbed

Pipeline for progressive testing

The first stage of the pipeline runs the unit tests

Unit Tests

Pipeline for progressive testing

Integration Tests

Integration tests cover a group of related components within the application

Pipeline for progressive testing

Application Tests

Application tests cover an application running in a container, mocking connections to other applications

Pipeline for progressive testing

System Tests

System tests cover the set of applications developed by the team, running together, mocking connections to external services

Pipeline for progressive testing

System Integration

Tests

Systems integration tests exercise all parts of the system, including external dependencies

Pipeline

Design Patterns

Pipeline Design Patterns

Application Pipeline

Automated tests

Automated tests

Automated tests

Tested Build

Manual tests Live

Unattended Process

One-click deployments

Typical Pipeline Stages

Pipeline Design Patterns

Typical Commit Stage

Commit Stage

1.  Developer commits code to SCM

Typical Pipeline Stages

Pipeline Design Patterns

Typical Commit Stage

Commit Stage

1.  Developer commits code to SCM

2.  Build server checks out & compiles code

Typical Pipeline Stages

Pipeline Design Patterns

Typical Commit Stage

Commit Stage

1.  Developer commits code to SCM

2.  Build server checks out & compiles code

3.  Unit tests

Typical Pipeline Stages

Pipeline Design Patterns

Typical Commit Stage

Commit Stage

1.  Developer commits code to SCM

2.  Build server checks out & compiles code

3.  Unit tests

4.  Code analysis

Typical Pipeline Stages

Pipeline Design Patterns

Typical Commit Stage

Commit Stage

1.  Developer commits code to SCM

2.  Build server checks out & compiles code

3.  Unit tests

4.  Code analysis

5.  Create build artefact

Typical Pipeline Stages

Pipeline Design Patterns

Typical Deployment Testing Stage

Deployment Testing Stage

1.  Prepare environment

Typical Pipeline Stages

Pipeline Design Patterns

Typical Deployment Testing Stage

Deployment Testing Stage

1.  Prepare environment

2.  Retrieve artefact

Typical Pipeline Stages

Pipeline Design Patterns

Typical Deployment Testing Stage

Deployment Testing Stage

1.  Prepare environment

2.  Retrieve artefact

3.  Deploy

Typical Pipeline Stages

Pipeline Design Patterns

Typical Deployment Testing Stage

Deployment Testing Stage

1.  Prepare environment

2.  Retrieve artefact

3.  Deploy

4.  Smoke Test

Pipeline Design Patterns

Typical Deployment Testing Stage

Deployment Testing Stage

1.  Prepare environment

2.  Retrieve artefact

3.  Deploy

4.  Smoke Test

5.  Automated UI tests

Typical Pipeline Stages

Pipeline Design Patterns

Other pipeline stages

q  Performance testing

q  Security testing

q  Exploratory testing

q  User acceptance testing

Typical Pipeline Stages

Best practices for pipeline stages

Keep everything in source control

Only build artefacts once

Replicate production constraints

Deploy the same way every time

Pipeline Design Patterns

Typical Pipeline Stages

Keep it simple, implement only those stages that make sense for your system

Pipeline Design Patterns

Build Progression

þ þ þ r101

Typical Pipeline Stages

Pipeline Progression

Pipeline Design Patterns

Build Progression

þ þ þ r101 ý

If tests fail, work stops until the issue is resolved

Typical Pipeline Stages

Pipeline Progression

Pipeline Design Patterns

Build Progression

þ þ þ r101 ý

The fix is committed, and a new build goes through the pipeline from the

beginning

þ þ r102 þ þ Typical Pipeline Stages

Pipeline Progression

Pipeline Design Patterns

Build Progression

þ þ þ r101 ý

Multiple builds may go through test without being deployed to production

þ þ r102 þ þ þ þ r103 þ þ

Typical Pipeline Stages

Pipeline Progression

Pipeline Design Patterns

Build Progression

þ þ þ r101 ý

Deployment to Production

þ þ r102 þ þ þ þ r103 þ þ þ þ r104 þ þ þ þ

Typical Pipeline Stages

Pipeline Progression

Joining multiple applications Pipeline Design Patterns

Application A

Application B

Typical Pipeline Stages

Pipeline Progression

Joining multiple applications

Pipeline Design Patterns

Application A

Application B

Application Testing Complete

Application Testing Complete

Joining multiple applications

Typical Pipeline Stages

Pipeline Progression

Joining multiple applications

Pipeline Design Patterns

Application A

Application B

Application Testing Complete

Application Testing Complete

Integration Testing

Joining multiple applications

If tests fail for application A, application B can still proceed

using the last “good” version of A

Typical Pipeline Stages

Pipeline Progression

Joining multiple applications

LIVE

Pipeline Design Patterns

Provider

Consumer

Federated Pipelines

Pre-release testing

Typical Pipeline Stages

Pipeline Progression

Joining multiple applications

Provider team could host instances for testing…

Pipeline Design Patterns

Provider

Consumer

Federated Pipelines

Pre-release testing LIVE

Consumer contract tests

…or provide packages for deployment by the Consumer team Typical Pipeline Stages

Pipeline Progression

Joining multiple applications

Provider team could host instances for testing…

Automated Testing

Stay tuned for Part 4…

Deploy a great product faster. Agile teams deliver working software early and often. Go automates and streamlines the build-test-release cycle for worry-free, continuous delivery of your product.

Learn More See how Go can help you in your CD journey

goContinuous Delivery