Jonny wooldridge DevOps Large and Small

jmwpro jmwpro

jmwpro

Web Master / Lead Java Developer

Lead Developer / Head of Development

Director of Platform Development

Head of Web Engineering

Chief Technology Officer 2014

2011

2007

2003

1999

Background

DevOps Large

DevOps Small

DevOps Small

? Small

???? Small

jmwpro

DevOps Large <aka DevOps Hard/>

jmwpro

•  2011-‐2014 introduced Con,nuous Integra,on, DevOps & Behaviour Driven Development to M&S on £150 Million project.

•  650 Member project team, mostly outsourced. • Project touched 65 new or modified applicaIons, mostly legacy. • Devops Capability team now in place to kick start other teams • BDD iniIaIves taken off on and offshore with

4 years to re-‐plaNorm – ongoing….

DevOps Large: Marks & Spencer

Founded 1884, 85,000 staff £10.3 Bn group revenues

jmwpro

“It’s all about the code” Application code, Test code, Configuration code, Script code, infrastructure code, 3rd Party Binaries

jmwpro

DevOps Large: The Good

So;ware Factory Efficiency Ensure there is only one place to put

code, and one way to build it

So;ware Scanning Quality and Maintainability

Ensure all code meets agreed coding standards. Scan for Security vulnerabiliIes a[er every commit

Route to Produc,on Simplicity and Control All code had to go through the correct

environments first

Regular Release train(s) Team ProducIvity Have a schedule and sIck to it.

Configura,on Management Code control SVN misused, sorted then moved to GIT

Behavior Driven Development

Beber soluIon. Automated tesIng.

Try BDD iniIaIves within in-‐house teams-‐ then roll out offshore.

Ini,a,ve Benefit What to look out for?

jmwpro

DevOps Large: The Bad

Architecture Too coupled Too many dependencies between systems making it difficult to move at speed or test independently

Vendors and Systems Integrators

No Unit tesIng, test automaIon, DevOps

Out of the box Vendor code poor with no or limited unit tests. Teams aligned to build features not an efficient pipeline.

Automated Tes=ng MulIple frameworks not fit for pipeline

Ensure your automated test ‘frameworks’ can run externally (ie via jenkins or similar) and can be configured to run subsets of test. SCM.

Legacy Systems Legacy Systems!! Legacy Systems!!! Add a legacy system being built by a legacy Team and the whole code delivery process can grind to a standsIll.

Test Environments Complex and Expensive

Do all you can to build your apps independently of the dreaded “System IntegraIon Environment”

Ini,a,ve Benefit Explana,on

jmwpro

DevOps Large: The Ugly

Contracts Fixed Price No menIon of operaIonal consideraIons; code process, build mechanisms, frequency of deployment, code quality, automaIon.

Hos,ng Partners No Access to hardware

Ensure you’re not handing over full control of infrastructure to 3rd ParIes with no incenIve to give dev/test teams access. What’s the incenIve to collaborate.

Opera,ng Model Waterfall based Internal processes/gates/sign-‐off based on waterfall. DevOps iniIaIves cut across all phases of standard projects.

Architecture & Test Data

Impossible to get baseline data

With mulIple batch jobs and internal reference tables virtually impossible to take co-‐ordinated snapshot of DBs and recreate test environments at will

Ini,a,ve Benefit Explana,on

jmwpro

Lies, Lies and StaIcs <the forgotten teams/>

jmwpro

The team’s level of agile/lean

working practices

Level of Independently testable and deployable software

Low High

Low

High

Slow

Fast

Continuous Delivery

Daily/Weekly Independent

Monthly Coordinated

Quarterly Enterprise

TEAM

TECHNOLOGY

The cool kids in DevOps

Soon ;-‐)

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Legacy Zone

Cool Zone Team So;ware

Agile/Lean pracIces

Great So[ware

Good pracIces

Good So[ware

Poor working pracIces

Poor So[ware

Bad working pracIces

Bad So[ware

Delivering better software, faster.

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

The real cost of not doing it right

$$$$

$$$

$$ $ Understand the cost to the

organisaIon of slow releases

Cost of rework

Cost of delay and hand off

Cost of building the wrong thing

Cost of poor technology choice

IntegraIon test costs

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Legacy Zone

Cool Zone

Excelling at So[ware Delivery

Delivering better software, faster.

Corporate Complexity

So where on the scale are you?

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Legacy Zone

Cool Zone

DevOps Big: Enterprise Reality

DevOps / BDD Culture on a few new iniIaIves

Outsourced Standard Methods long integraIon test phases,

some test automaIon

Increasing use of BDD, Lean within digital teams

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

DevOps Large: The Team Ceiling

Do you have the team in place to excel? Designed for Con,nuous Delivery? •  Training •  Skills •  Mindset •  Goal alignment •  Contracts •  Empowerment

No maber how good your technology ensure your team is

up to it!

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Team Policy based on DevOps

Do you have the team in place to excel? Designed for Con,nuous Delivery? •  Training •  Skills •  Mindset •  Goal alignment •  Contracts •  Empowerment

No maber how good your technology ensure your team is

up to it!

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

DevOps Large: The Technology Ceiling

Can you really do ConInuous Delivery with your current architecture and systems? Designed for Con,nuous Delivery? •  Vendor systems from the 90s •  No unit tesIng •  Slow/No build automaIon •  Monolithic code bases/systems •  Poor monitoring / logging

No maber how good your team is your technology sucks!

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Procurement strategy based on DevOps

Can you really do ConInuous Delivery with your current architecture and systems? Designed for Con,nuous Delivery? •  Vendor systems from the 90s •  No unit tesIng •  Slow/No build automaIon •  Monolithic code bases/systems •  Poor monitoring / logging

No maber how good your team is your technology sucks!

jmwpro

DevOps Small <aka DevOps Fun/>

jmwpro

•  Selling in over 100 countries. Flagship store in Covent Garden • Hand made in Great Britain in our own factory.

• Opportunity to re-‐invent the technology, teams and ways of working • Autumn 2014 -‐ Website in urgent need of a re-‐plaNorm • DevOps and BDD fundamental to the way we work and technology strategy

DevOps Small: Cambridge Satchel

Founded 2008 by Julie Deane OBE, 120 Staff

£600 ini,al investment in 2008 has led to £10M+ total revenues

jmwpro

What if…. <clean sheet of paper/>

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

App Strategy based on DevOps

Clean sheet of paper. Design and Build Team and Technology from the ground up with DevOps in mind. Designed for Con,nuous Delivery: •  End to End Test AutomaIon •  Build and Deploy AutomaIon •  DevOps culture •  Code Reviews •  API focus / API Mocks •  Behaviour Driven Development •  Cloud Dev Tooling •  Cloud Infrastructure •  So[ware

API

Web

App

jmwpro

Cambridge Satchel

2014

28th November Black Friday

, “Can’t be done!” ;-‐)

arrive

Q: How do you develop, test & deploy so;ware? •  Tell me about DevOps and BDD? •  How much of my budget is for automaIon? •  Will I be le[ with legacy? Where’s the code? •  Is my architecture decoupled?

Crazy!

jmwpro

Cambridge Satchel – dream team please!

2014


20% Service Integrator Developers 20% Internal Engineers 20% API & Middleware Developers 20% BDD & Exploratory TesIng 20% DevOps/AWS Skills

35K in 2013

jmwpro

Cambridge Satchel

2014


20% Service Integrator Developers 20% Internal Engineers 20% API & Middleware Developers 20% BDD & Exploratory TesIng 20% DevOps/AWS Skills

launch

35K in 2013 87K in 2014 On new plaNorm

jmwpro

Some Ips <thoughts on how to start/>

jmwpro

2011+ Head of Web Engineering

Which needs to be fast moving? Which are Ied to corporate release cycles Which have automated tests?

Define & decouple your ‘Pace Layers’

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Fight the right bables with your legacy

Where you invest your $$ is criIcal. Invest in DevOps where it

mabers.

You can always start somewhere

DevOps without legacy is easy. Front End UI

FinanceSystems Payment

Order Mgt

CoreEcomm

DigitalAsset

Cust.Mgt

Apps

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Enterprise equilibrium tends to push your DevOps adopIon

backwards

Watch out for corporate equilibrium

Make the wrong choice and the forces may be working against your goal of faster delivery.

Wrong technology choice

Wrong hiring policy

Wrong contractual & financial frameworks

Wrong 3rd Party Suppliers

Wrong team objecIves & rewards

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Fight the right bables with your legacy

Where you invest your $$ is criIcal. Invest in DevOps where it

mabers.

You can always start somewhere

DevOps without legacy is easy. Front End UI


Order Mgt

CoreEcomm

DigitalAsset

Cust.Mgt

Apps

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Reduce size and complexity of slow moving applicaIons

Reduce your legacy and create new capability

Build New ini,a,ves the right way

E.g. consider creaIng a Front End separaIon layer enabling parts to

be independently released

NEW

Legacy Zone

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

Put all interacIons between applicaIons through APIs

Understand the pace layers of your apps and governance needed

How good are the major vendor Ecommerce and Finance/ERP

systems?

Kill Dependencies. Focus on APIs

Front End UI


Order Mgt

CoreEcomm

DigitalAsset

Cust.Mgt

Apps

API

API API

API

API

API

API

API

Centralise your API access (ie Muleso[)

BDD is great for defining and tesIng APIs as well as Uis

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

When a new iniIaIve comes along and a new team is built to deliver it set the bar high with DevOps operaIonal requirements and ways of working. Encompass: •  Behaviour Driven Development •  ConInuous IntegraIon •  ConInuous Delivery •  Full automaIon •  Robust configuraIon management

Set the bar high for new iniIaIves / programmes

NEW

Legacy Zone

Don’t create new ‘legacy’

jmwpro


working practices


Low High

Low

High

Slow

Fast

Continuous Delivery


Monthly Coordinated


TEAM

TECHNOLOGY

API

Shadowy IT, API Economy and DevOps

AcceleraIng Revenue opportuniIes by providing robust and agile data services needed throughout the business. MarkeIng, Retail and Wholesale teams able to self-‐serve data and own their own integraIon with relevant partners delivered with external teams.

Web

App

Finance My Team

Everyone else…

jmwpro

The Dream Setup BDD Driven software pipeline – fully integrated end2end

jmwpro

Prepare

Slave Controller Build

Deploy

Code

Master Controller

Provision

Setup Run Tests

Source Code & Binary Store

Applica,on Tooling

Requirements, Issues/Bugs and

Workflow

BDD GUI Wiki / DocumentaIon

Live Dashboards

Plans

Data

Test Lab Tests

✖✔✔✔✔✔

Scripts

Test Environment images Base apps

results

Store Prepare

images

Base apps

Build Code Run Tests

Scan

Slave Controller

Unit test

Tests

Binary

Binary

Build a ‘so;ware factory’ ©

jmwpro


Plans Scripts

images

Base apps

Code

Tests

Binary

Master Controller

Applica,on Tooling


Workflow


Live Dashboards

Data

Build a ‘so;ware factory’

©

Fully Integrated tools:

•  Requirements/Wiki •  Source Code Mgt. •  Binary Store •  Code check-‐in/build •  Code Quality scan •  Environment Mgt. •  Deployment •  Test •  Log Storage

jmwpro

Code Source Code & Binary Store

Plans Scripts

images

Base apps

Tests

Binary

Master Controller

Applica,on Tooling


Workflow


Live Dashboards

Data




jmwpro


Code

Master Controller


Applica,on Tooling


Workflow


Live Dashboards

Plans

Data

Scripts

images

Base apps

Build Code

Scan

Unit test

Tests

Prepare




Store

Binary

results

jmwpro

Slave Controller

Prepare


Deploy

Code

Master Controller

Provision

Setup Run Tests


Applica,on Tooling


Workflow


Live Dashboards

Plans

Data

Test Lab Tests

✖✔✔✔✔✔

Scripts

Test Environment images Base apps

results

Store

images

Base apps

Build Code Run Tests

Scan

Unit test

Tests

Binary

Binary

Prepare


jmwpro

Code

Master Controller Test Lab Tests


Applica,on Tooling


Workflow


Live Dashboards

Plans

Data

Scripts

images

Base apps

Tests

Binary

Deploy Provision

Setup Run Tests

Prepare

Run Tests Slave

Controller

Prepare


Store

Build Code

Scan

Unit test

Test Environment images Base apps Binary


jmwpro

Master Controller

Applica,on Tooling


Workflow


Live Dashboards

Data

Code Source Code & Binary Store

Plans Scripts

images

Base apps

Tests

Binary


©

jmwpro

Test Environment

Test Lab


Applica,on Tooling

✖✔✔✔✔✔

Build Code Provision env & Run Tests


jmwpro

What is you outsource everything? Perfect:

• Have insight into your offshore suppliers like never before • Have control of your offshore suppliers like never before

• Software Delivery data and information in one place

Build a Software factory

jmwpro

#jmwpro

My blog, these slides and other musings available at:

Thanks for listening

Internet

Jonny wooldridge DevOps Large and Small