Better ways of developing softwareor, coding like a unicornGovernment Edition

@coteMarch 2016Slides: http://cote.io/unicorn-coding/ 1

2

• @cote – Director, Technical Marketing at Pivotal for Pivotal Cloud Foundry

• Former industry analyst at 451 Research and RedMonk

• Corporate strategy & M&A at Dell

• Former software developer• Podcasts: cote.io/podcasts• More: http://cote.io or

cote@pivotal.io

Hello!

3

It’s not much of an IT department, but I’m sorta attached to it…

4

Source: “The Agile Advantage,” MeriTalk/Accenture, May 2015; Cutter Consortium, July 2015.

And everyone knows it…What is your IT organization's

role in business innovation?87% of Feds say

it takes too long to deploy new systems per requirements

5

The IT Department of Slow/No• 6-12 months to release features• Business struggles to have software match market opportunities• IT is a cost center

Clipart from Gerd Altmann.

The IT Department of Slow/No• 6-12 months to release features• Business struggles to have software match market opportunities• IT is a cost center

Clipart from Gerd Altmann.

Best Practices in Software Development & Delivery

8

• Beyond Agile – Lean and DevOps

• Automation all the things - “cloud” and continuous delivery

• Programming - Microservices, Polyglot, and 12 factor apps

Transforming the donkey

9

• Tops-down change management

• Working with legacy portfolios

• Examples of government & large, often regulated organizations that have pulled it off

Best Practices:

10

Beyond Agile

After 17 years, “Agile” is not as wide-spread as we think it is

Source: “Town Hall: Agile in the Enterprise,” Mike West, Nathan Wilson, Thomas Murphy, Dec 2015, Gartner AADI US conference.11

“We spent a lot of time talking about technology, but technology really isn't the meat of the conversation that needs to occur.” Matt Curry, Allstate

12Source: Matt Curry quote from Feb 2016 TheNewStack interview; “Agile Transformation in the Age of the 3rd Platform,” Al Hilwa , IDC, Feb 2016 (unpublished)

Discovering & delivering the software your customer actually wants

• Most organizations emphasize building to plan

• In reality, the “plan” constantly evolves

• Learning organizations explore & experiment

How most businesses think they’re doing

How smart businesses know they’re doing

13

The “Lean Startup” Approach, Hypothesis Driven Development, aka, “The Scientific Method

14

• Deploy to production each time, avoid waterscrumfall

• Relies on:- Product approach- Small batch oriented

pipeline and process- Cloud native automation

Inception

Hypothesize

Validate w/real use(r)s

Did it work?

Sources: The Lean Startup, Eric Ries, 2011. The Lean Enterprise, Barry O'Reilly, Jez Humble, and Joanne Molesky. See also overview of this approach at the IRS from Dec 2015.

DevOps, the word defines itself

15

(1.)DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support.

(2.)DevOps is also characterized by operations staff making use of many of the same techniques as developers for their systems work.

Source: “What Is DevOps?,” The Agile Admin, Dec 2011. For more: Effective DevOps, Jennifer Davis, Katherine Daniels, May, 2016; see The Practice of Cloud System Administration, chapter 8 for an in-depth definition and discussion. Also, see Damon Edward's 2012 piece "Use DevOps to Turn IT into a Strategic Weapon.” Also, The Phoenix Project for a story of DevOps in the style of The Goal.

Best Practices:

16

Automate All the Things

17

It actually is better this time

18 Source: "Operations is a competitive advantage... (Secret Sauce for Startups!)" Jesse Robbins, O'Reilly Radar, 2007.

Continuous Delivery – the pipeline

19

Build Test/Verify Package repository

VersionControl

InfrastructurePlatform

(IaaS, PaaS, VMs)

ProductionConcerns

(monitoring, scaling, etc.)

Feedback Loop

Specify Code

Development CI/CD Ops

Magic in the pipeline

CommitTests

IntegrationTests

Deployable Artifact

Deploy to CF Test Env

Deploy to Prod

• Developer crafted (before impl.)• Stubs/mocks external services• Developer executed (before commit)• CI executed (after commit)

(after commit)• Developer or QA crafted• CI executed• Runtime context (buildpack applied)• Binds to test services (DB, messaging, etc.)

• “cf push”• Runtime context (buildpack applied)• Binds to test services (DB, messaging, etc.)• Periodic smoke tests

• “cf push”• Runtime context (buildpack applied)• Binds to prod services (DB,

messaging, etc.)• Periodic smoke tests• Monitoring

? ?

20

Best Practices:

21

Programming

The nature of software is changing, yet again

22 Source: "Application Modernization, Service By Microservice," Kurt Bittner and Randy Heffner, Dec 2015, Forrester.

Polyglot is the new normal

23

01 JavaScript02 Java03 PHP04 Python05 C#05 C++05 Ruby08 CSS09 C010 Objective-C…15 Go...17 Swift

Source: “The RedMonk Programming Language Rankings: January 2016,” RedMonk, Jan 2016.

Tied

A microservices approach gives you organizational agility, at the expense of architectural complexity

24

Benefits• Decoupling service

releases, teams• Faster innovation cycles• Composite applications

vs. monolithic apps

Problems• The one about the

murder mystery• Incompatible APIs• You still need other

people to do work

It’s worth solving the problems if you want the benefits…and it’s possible!

The Application Framework promises

Codebase Dependencies Config Backing Services

Build, Release,

RunProcesses Port

BindingConcurrenc

y

DisposabilityDev/Prod

Parity Logs Admin Processes

Source: “The Twelve-Factor App.”

• A set of best practices for developing and deploying cloud-native software.

• Practices translate into platform features and workflow requirements.

25

Cloud Platforms

26Source: "Technical Dive into Cloud Native Application Platforms," Brian Gracely, Wikibon, Sep 2015. See also "The cloud-native future," Casey West, O'Reilly Radar, Aug 2015

Transforming:

27

Wide-spread change has to start at the top

28

• Leading change management

• Gradual organization change• Communicating your

strategy, principals, goals• Start small, get larger• Converting “failure” to

“learning”

Sources: Leading the Transformation, 2015; “Management’s Job is orchestrating the ‘why,’” 2015; The Concise Executive Guide to Agile, 2010. See also “The Real Meaning of Software Transformation for Businesses Today,” Siobhan McFeeney, 2015.

Dealing with legacy: the stuff that makes all the money

29

Quarantine

• Low impact applications

• Lacks good testing, risky to change

• Change resistant priorities

Migration Strategies

• Virtualize to control costs and maximize management

• API gateways combined with the strangler pattern

• Portfolio management

Transforming:

30

Don’t worry, we got your happy trees!

Staffing: There is no talent shortage

F100 CTO: “But Netflix has a superstar dev team, we don’t!”

@adrianco:“We hired them from you.”

Source: @blueboxjesse, Oct 2014. See also “There is no talent shortage.”32

18F: ATO from 9 months to 2 days, how to transform docs

Source: “Deployments We Can Believe In,” Diego Lapiduz, 18F, June 2015; see also “Barriers to DevOps in Government,” Coté, Oct 2015. 33

Once understood, auditors tend to like this approach

34

"There have obviously been culture shocks. What is more interesting though is that the teams that tend to have the worst culture shock are not those typical teams that you might think of, audit or compliance. In fact, if you’re able to successfully communicate to them what you’re doing, DevOps and all of the associated practices seem like common sense. [Auditors] say, 'Why weren’t we doing this before?’”

Source: "DevOps Enterprise Adoption at E*TRADE," InfoQ/DOES, Jan 2016.

A small batches approach is more successful

35

"When we were doing big design upfront, downstream changes had to go through a rigid change control process. We wound up being busy with our own process rather than delivering value, and either we didn’t deliver or we delivered late.”

Large European retail bank

“We did an analysis of hundreds of projects over a multiyear period. The ones that delivered in less than a quarter succeeded about 80% of the time, while the ones that lasted more than a year failed at about the same rate. We’re simply not very good at large efforts.”

Large financial institution

Source: "Application Modernization, Service By Microservice," Kurt Bittner and Randy Heffner, Forrester, Dec 2015; "Best Practices For Agile-Plus- Architecture," Randy Heffner, Forrester February, 2015. More case studies in Brian Gracely’s "Evolving Organizational Dynamics for Cloud Native Applications," Wikibon, Nov, 2015.

For more: transforming to “cloud native”

(1) GreenfieldProjects

(2) Dealing with

Legacy

(3)Corporate

Transformation

Get the free booklet at http://cote.io/pivotal36

37

DevOpsDays DC 2016

June 8th & 9th, 2016http://www.devopsdays.org/events/2016-washington-dc/

Thanks!@cote | cote@pivotal.ioSlides: http://cote.io/unicorn-coding/

“We are uncovering better ways of developing software by doing it and

helping others do it.”- The Agile Manifesto, 2001

38