Continuous Delivery Maturity Model

1

Continuous DeliveryMaturity Model

Eric Minick, Lead Consultant

2

Eric MinickLead Consultant

• 8 Years at Urbancode• Commits on three

generations of UC products• Experience as a Dev, Tester

and build guy• Currently focused on best

practices and industry trends

3

The plan for today

• Short Intro to Continuous Delivery• The Maturity Model– Building, Deploying, Testing, and Reporting

• Applying the Model: Personas– El Emeno Investments: Balancing agility and control– All-Green United Systems: Large scale Scrum– Diversity Handhelds: Dozens of platforms

• Additional Resources and Q&A

4

The plan for today

• Short Intro to Continuous Delivery• The Maturity Model• Appling the Model: Personas• Additional Resources and Q&A

5

CI: it’s about Quality Determination

• Integrating frequently is less painful than infrequently.– But, If we commit frequently blindly, we have a

broken code base

• Implied condition: each integration will not decrease the code quality. – This is the key to CI – The rest is implementation details.

http://www.urbancode.com/blogs/anthillpro-blog/2011/09/07/continuous_integration_why_you_dont_really_get_it.html

6

Continuous Delivery: Beyond the Team Build

• Continuation of the Quality Determination• Functional tests require deployment.• Progression through environments on a march

to production, with “the build” subjected to more exhaustive tests in each environment.

• Beyond the functional team, but also across projects

7

Why a CD maturity model?

• How mature are our Build, Deploy, Test practices?

• Where can we get the most improvement based on our specific problems and needs?

• How are other organizations solving these same problems?

“The future is here. It’s just not evenly distributed yet.”– William Gibson

8

Maturity Model: Parts and personas

Building, Deploying, Testing, and Reporting– Base, Beginner, Intermediate, Advanced, Extreme

• Industry Norm vs. Target– What do we see most people doing today?– A general target to shoot for (and how to pick a good target

for your own team)

9

Poll: Our CI or CD system handles:• Building our code (at least daily)• … and deploying after build• … and testing after deployment• … and reporting across building, deploying and

testing• … and more!• We have no CI system

10

Building

11

Deploying

12

Testing

13

Reporting

14

Standard Target

Based on this model, the standard target is:• Continuous builds on a distributed grid using

dependency management

• Builds automatically deployed to first test environment

• Build runs extensive unit tests; extensive functional tests automatically executed in test environments

• Test results and trends are widely visible and available

• Information from the build system informs manual risk-based testing

• Self-service deployments to other environments– Those deployments coordinated between tiers / services

15

The plan for today

• Short Intro to Continuous Delivery• The Maturity Model• Appling the Model: Personas• Additional Resources and Q&Aof platforms

16

Apply the maturity model: Personas

El Emeno Investments: Balancing agility & control

All-Green United Systems: Large-scale Scrum

Diversity Handhelds: Dozens of platforms

17

Poll: Our main problem we hope CD can help with is:

• Improving our delivery speed• Improving quality and compliance• Supporting multiple platforms or devices• Multiple of these• We have other problems• We have no problems

18

El Emeno Investments

Trading system for securities– Time to market is a key competitive advantage– Strict legal requirements & audits

Problem: slow heavyweight process– Numerous sign-offs, handheld deployments,

lengthy manual test cycle– Pressure to work overtime to compensate– Late mistakes in deployment– High turnover

“How do we balance agility and control?”

19

El Emeno Investments: Deploying

Needed automated, traceable deployments– Secure push-button deployments– Deploy to production

Benefits– Self-service deployments to test take pressure off

release engineers– Using uniform automated deployments reduces

errors & delays– Security and traceability make auditors happy

Advanced level deployments

20

El Emeno Investments: Building

Needed Audit Proof Builds– Automatic, Controlled Build Process– Builds stored in an artifact repository

(ITIL: Definitive Software Library)

Benefits– Clear understanding of what is in each build lowers risk– Supports accurate deployments by guaranteeing no changes

to tested artifacts– Faster approvals and audits with lower risk of failure

Beginner level builds with some Intermediate practices

21

El Emeno Investments: Testing

Needed Rapid Regression Testing and Security– Automated unit tests of core functionality on all builds– Automated security scans of select builds

Benefits– Less time on manual regression testing allows quicker

detection of problems in new functionality– Security scans lower business risk, increasing compliance

Beginner level testing with advanced security scans

22

El Emeno Investments: Reporting

Needed historic release information, consolidated reports– Audit log of who deployed what build where– Produce test, security scans, approval logs for releases

Benefits– Consolidating reports yields faster, easier audit

compliance– Easy review of a build’s history reduces deployment errors– Risk is more easily assessed

Intermediate level reporting

23

El Emeno Investments: Results

“How do we balance agility and control?”– Time to market is a key competitive advantage– Problem: slow, heavyweight process

“Our continuous delivery system provides us an end-to-end automation across our build lifecycle. We now have a faster cycle time, fewer manual errors and an easier time in audits.”

24

All-Green United Systems

Large global IT with business critical applications– Large scale scrum with cross-functional teams– Traditional integration QA & release engineering– Recent move to SOA

Problem: releases are a bottleneck– Legacy “big bang” release process– Lack of synchronization across applications creates backlog– QA finding cross-team problems in integration testing– Apps fail in product, after appearing to work in QA

“How do we realize the benefits of Scrum in a large organization?”

25

All-Green United Systems: Building

Needed unification of build systems across scrum teams– Consolidate from numerous build systems to one build grid– Dependency management between components– Scrum of Scrums yields Build of Builds

Benefits– Build grid more efficiently uses hardware, lowering costs and

improving response time– Build of Builds more quickly detects cross-team integration

problems and improves component traceability– Single build system lowers learning curve when switching scrum

teams

Intermediate level building

26

All-Green United Systems: Deployments

Needed to speed the release process to keep pace– Automated release across teams and environments– Coordination of SOA releases – Deploy what was tested– New applications with extensive testing using continuous

deployment to production

Benefits– Unified deployment scripts avoid errors in production– Self-Service deployments remove burden from release engineers – Continuous deployment give some systems maximum

throughput

Advanced and Extreme level deployments

27

All-Green United Systems: Testing

Needs vary by scrum team and application– Investment in expanded automatic testing focused on

mission critical, frequently changing, and new applications– Component interfaces gain automated regression tests

Benefits– Fewer breakages in shared components improves

developer productivity– Mission critical applications gain stability, lowering risk of

costly defects

Varying levels of testing

28

All-Green United Systems: Reporting

Needed better visibility between cooperating small teams– Consolidated build systems put all build information in one place– Include test and deployment reports in CD system– Trend report results

Benefits– Reduced coordination overhead between scrum teams– Single report repository improves coordination between

development and testing– Trending improves the scrum team’s understanding of how done

they are

Advanced level reporting

29

All-Green United Systems: Results

“How do we realize the benefits of Scrum in our organization?”– Large scale scrum with cross-functional teams– Problem: releases are a bottleneck

“Our continuous delivery system allowed our small, globally distributed scrum teams to cooperate in large system builds automatically. By keeping the code integrated we eliminated late errors, improved coordination across teams, and eliminated our release bottleneck.”

30

Diversity Handhelds

Software platform for mobile devices– Partners work with device manufacturers– Each product ported across a wide range of platforms

Problems: breaking existing ports & tracking build content – Porting to a new platform might break an inactive port;

hard to track down cause later– Tracking deliverable contents manually is time consuming;

hard for support to find information when needed

“How can we reduce the cost of adding a new platform?”

31

Diversity Handhelds: Building

Needed regular builds of many build configurations– Diverse, multi-platform, multi-tool chain build farm– Regular, automatic builds for all configurations– Upload of build product into build artifact repository

Benefits– Building all configurations at least daily saves developer time with

quick detection of platform specific compile problems– Support has easy access to builds delivered to customers– Automated Bill of Materials tracking reduces manual effort and

errors.

Intermediate level building

32

Diversity Handhelds: Testing

Needs automated cross-platform testing– Extensive unit testing run regularly– Automated functional tests run in available, representative device

simulators– Static analysis and security checks regularly run

Benefits– Faster defect detection with shorter QA cycles leading to improved

time to market– Lower risk of potentially expensive platform specific defects– Manual testing can find deep bugs rather than focus on smoke

testing dozens of variants

Advanced level testing

33

Diversity Handhelds: Deployments

Needed deployments to simulators to support testing– Scheduled deployments integrated to build

management and testing

Benefits– Advanced testing that provides rapid feedback from

functional testing is dependant on basic deployments

Intermediate level deployments

34

Diversity Handhelds: Reporting

Needed to understand stability and historical information– Gather build, various testing and release logs centrally– Align reports to individual builds– Maintain reports related to released builds through that

version’s end of life

Benefits– Provides data to support and engineering regarding what was

tested on old releases customers have problems with– Lowers effort in understanding test results for wide array of

platforms / configurations

Advanced level reporting

35

Diversity Handhelds: Result

“How can we reduce the cost of adding a new platform?”– Partners work with device manufacturers– Problems: breaking existing ports & tracking build

contents

“Our continuous delivery system handles our wide-range of platforms and prevents our ports from going stale. Our bill-of-materials is now generated automatically and our support people can access historical information any time.”

36

Key Takeaways

• Technology built for “extreme” uses can be used by the rest of us

• Select a final goal and immediate first steps

• Interact across functional groups to project teams to determine larger needs

37

The plan for today

• Short Intro to Continuous Delivery• The Maturity Model• Appling the Model: Personas• Additional Resources and Q & A

38

Four part Maturity Model

39

More on CI, CD, and DevOps

Webinars and more: http://urbancode.com/resources• Deployment Automation Basics• Lean Techniques for Build & Deploy Automation• Enterprise DevOps: Scaling Build, Deploy, Test, Release

You should follow us:Urbancode.com/blogs/Twitter.com/UrbanCodeSoftFacebook.com/UrbanCodeSoftSlideshare.net/UrbanCode

40

Yes, UrbanCode is a CD vendor

• uBuild (AnthillPro)– Continuous integration and simple CD

• uDeploy– Deployment and release management

• uCloud (soon)– Environment provisioning from your private cloud

41

Questions?

Request a copy of the slides or ask more questionseric@urbancode.com

mbz@urbancode.com