Analyst Keynote: Continuous Delivery: Making DevOps Awesome

World®’16

AnalystKeynote:ContinuousDelivery:MakingDevOpsAwesomeNicoleForsgren - PhD– CEO- ChiefScientistDORA:DevOpsResearch&AssessmentDO5T14S

DEVOPS

2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD

©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.

Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.

ForInformationalPurposesOnlyTermsofthisPresentation


Abstract

Foryears,therehavebeenstoriesofcontinuousdeliverymakingteamsawesome…butcancontinuousdeliverymakeallteamsawesome?Andhow?

Dr.NicoleForsgrenwillpresentdatafromover25,000technicalprofessionalsshowingthecentralrolethatcontinuousdeliveryplaysinsoftwaredevelopmentanddelivery.Shewillshowyouhowdoingcontinuousdeliverycandrivekeyorganizationaloutcomeslikeprofitability,productivity,andmarketshare.NicolealsopresentsthekeyaspectsofcontinuousdeliverythatmakeyourDevOpsawesome,liketrunk-baseddevelopment,testdata,andtestautomation,andprovidesexamplesofsuccessfromteamsundergoingtheirowntechnologytransformations.

ThepresentationalsoincludesotherimportantdriversofDevOpssuccess,likeleanproductmanagementandteamculture.Attheendofthistalk,youwillhavetheinformationtohelpyouproveyourcase(tomanagementorevenyourself)aboutwhycontinuousdeliveryandDevOpsareessentialtowinning,aswellasgreatstoriesandexamplestoreallybringtheseconceptstolife.You’llleavewithtipsyoucantakebacktogetstartedonyourownDevOpsinitiative.

NicoleForsgren,PhDDevOpsResearch&AssessmentCEO,ChiefScientist


Agenda

INTRODUCTION

ITDOESMATTER

THERE’SMORE…

DEVOPSIS

CONTINUOUSDELIVERY

ITPERFORMANCEDRIVERS

1

2

3

4

5

6

Continuous Delivery:Making DevOps Awesome

Nicole Forsgren, PhD

CEO and Chief Scientist, DevOps Research and Assessment (DORA)Research Affiliate, Clemson University, Florida International University

“IT doesn’t matter.”-- Nicholas Carr, 2003

@nicolefv

IT Does Matter• Times – and IT – have changed•DevOps is good for Organizations•DevOps is good for IT•And then some detail: What drives this change?• Technical practices (hint: Continuous Delivery)• Management practices (hint: from Lean Management)• Culture and identity

@nicolefv

@nicolefv

@nicolefv

DevOps isTechnical practices

seen in Continuous Delivery,Management practices

seen in Lean Management principles, and Organizational Culture

@nicolefv

Research shows that these drive Organizational Performanceand IT Performance

DevOps is good for organizations

@nicolefv

High Performing IT organizations

2xMore likely to exceed Profitability, Market share, andProductivity goals

50%Higher market capgrowth over 3 years*

The 2014 and 2015 State of DevOps Report

@nicolefv

Devops is good for IT

Measuring DevOps and IT Performance- Deploy frequency (when business demands)- Lead Time for Changes- Mean Time to Recover (MTTR)- Change Fail Rate

@nicolefv

High Performing DevOps teams

More agile

The 2016 State of DevOps Report

@nicolefv

200x

200x more frequent deployments

2,555x

2,555x shorter lead times

High Performing DevOps teams

More reliable


@nicolefv

24x24x faster

recovery from failures

3x

3x lowerchange fail rage

DevOps promises – and delivers More throughput

More stability

In tandem. Without the tradeoffs that ITIL calls for.

Let’s talk about what this means for us@nicolefv

High Performing DevOps teamsMore agile What does this mean for:

200xMore frequent deployments


New content deliveryValue/savings around A/B testingValue around speed to marketCompliance / regulatorySecurity2,555x

Faster lead times@nicolefv

Evaluating well-designed and executed experiments that were designed to improve a key metric, only about 1/3were successful at improving the key metric!

Online Experimentation at Microsoft, Kohavi et al http://stanford.io/130uW6X@nicolefv

High Performing DevOps teamsMore reliable What does this mean for:

3xFewer deployfailures


Value/savings around reliabilityValue/savings around uptimeComplianceSecurityReputation around uptime,

compliance & security24xFaster MTTR

@nicolefv

@nicolefv

"Fixing it as soon as possible or having compensating controls in place

days before could have saved this entire breach from occurring in the

first place."

We know:

• IT Performance is comprised of throughput and stability, and both are possible without tradeoffs• IT Performance contributes to org performance

(profitability, productivity, market share)

So:What drives IT and Organizational Performance?

@nicolefv

IT Performance Org Performance???

@nicolefv


1.

@nicolefv


2. 1.

@nicolefv


2. $$$1.

@nicolefv

IT Performance Org Performance

Continuous Delivery

@nicolefv

Continuous Delivery


Continuous Delivery

@nicolefv

Continuous Delivery

• Effective test data management

• Test & Deployment automation

• Trunk-based development

• Continuous integration

• Version control• Shifting left on

security


Continuous Delivery

@nicolefv

Continuous Delivery


• Test & Deployment automation

• Trunk-based development

• Continuous integration

• Version control• Shifting left on

security

Test Data Management

Analysis showed that data was creating bottlenecks in the process.

Key activities in TDM include:• Test data masking and

subsetting• Test data automation• Integration with tooling

Outcomes:• Faster time to market• Improved quality and customer

experience• Significant productivity improvements

and savings• Improved testing efficiencies

@nicolefv

Test & Deployment Automation

“By getting new releases and services out the door quickly, we can provide a better experience to millions of car buyers and sellers and continue to differentiate ourselves in a competitive market.” -- Adam Mills, Senior Manager of Application Development, AutoTrader.com

Using a DevOps approach to unify teams, they have been able to automate their test, build, and deployment

Using these principles, Autotrader.com has been able to:• Reduce test, build, and deploy times from

weeks to just minutes• Decrease software defects by 25%


Continuous Delivery

@nicolefv

Continuous Delivery also makes our work feel better


• Deployment automation

• Test automation• Trunk-based

development• Continuous

integration• Version control• Shifting left on

security Decreases burnout and deploy pain

Microsoft Engineering: DevOps LessonsThiago Almeida -- DevOps Days London, 2016

@nicolefv

Work/Life ScoresBefore CD

38%After CD:

75%

Source: https://vimeo.com/165184757

But what else drives IT Performance?

The 2015 DevOps Survey of Practice and its resulting database are the property of Puppet Labs, Inc. and Gene Kim and Associates, LLC. All rights reserved.

@nicolefv


Lean Practices

@nicolefv

Lean Management Practices


Lean Practices

@nicolefv


• Effective WIP limits

• Visual displays• Monitoring tools• Gathering,

montoring, implementing customer feedback

• Small batches• Making work

visible through the value stream


Lean Practices

@nicolefv





• Small batches• Making work

visible through the value stream

@nicolefv

Visual Displays

“If it moves, graph it.” - Michael Rembetsy, Former VP Operations, Etsy

Making Work Visible Through the Value Stream

“One of the greatest effects… is that we are now asked much earlier by the business to participate in the requirements gathering process.” -- Project Manager at a.s.r.

By creating flowcharts to show work as it moved throughout the value stream, a.s.r. was able to leverage insights from all stakeholders including business, test, and IT. Leveraging these tools, teams could immediately understand changes made to requirements and their impact.

Using this principle, a.s.r. was been able to:• Reduce test time preparation time

significantly• Release software with no incidents• Deliver on time and

on budget


Lean Practices

@nicolefv

Lean Management Practices also make work feel better




• Small batches and making work visible

Decreases burnout

“I was trying to figure out why my team was working themselves to death but not getting anything done… By implementing WIP limits, we were able to focus on our work. Finishing work feels better than sprinting and feeling like a hero in the moment, because that’s only a moment.”

- Julia Wester, Development Manager for Turner Sports, Turner Broadcasting

@nicolefv

But wait… there’s more

IT Performance

Org Performance

Continuous Delivery &

Lean Practices

@nicolefv

CD and Lean Management practices also improve our culture and identity, which improve IT & Org Performance

Westrum Org Culture

Identity

Westrum Organizational Culture

@nicolefv

Pathological Power-oriented

BureaucraticRule-oriented

GenerativePerformance-oriented

Low cooperation Modest cooperation High cooperation

Messengers shot Messengers neglected Messengers trained

Responsibilities shirked Narrow responsibilities Risks are shared

Bridging discouraged Bridging tolerated Bridging encouraged

Failure leads to scapegoating Failure leads to justice Failure leads to inquiry

Novelty crushed Novelty leads to problems Novelty implemented

Identity• I am glad I chose to work for this organization rather than another company.• I talk of this organization to my friends as a great company to work for.• I am willing to put in a great deal of effort beyond what is normally expected to

help my organization to be successful.• I find that my values and my organization's values are very similar.• In general, the people employed by my organization are working toward the

same goal.• I feel that my organization cares about me.

Adapted from Atreyi Kankanhalli, Bernard C.Y. Tan, and Kwok-Kee Wei (2005), “Contributing Knowledge to Electronic Knowledge Repositories: An Empirical Investigation,“ MIS Quarterly, 29, 113-143.

@nicolefv

Google Team Performance

Surprise! No magicformula for what makes the perfectteam

@nicolefv

Intuit

“By installing a rampant innovation culture, we performed 165 experiments in the peak three months of tax season.

Our business result? Conversion rate of the website is up 50%. Employee result? Everyone loves it, because their new ideas can make it to market. ”

- Scott Cook, Intuit founder

@nicolefv

Amazon

“I think building this culture is the key to innovation. Creativity must flow from everywhere. Whether you are a summer intern or the CTO, any good idea must be able to seek an objective test, preferably a test that exposes the idea to real customers. Everyone must be able to experiment, learn, and iterate.”

- Greg Linden

@nicolefv

IT Does Matter• Times – and IT – have changed•DevOps is good for Organizations•DevOps is good for IT•And then some detail: What drives this change?• Technical practices (hint: Continuous Delivery)• Management practices (hint: from Lean Management)• Culture and identity

@nicolefv

For more information:

@nicolefv

Sign up for our ROI whitepaper & get peer-reviewed researchdevops-research.com

Thank you

[email protected]@nicolefv

nicolefv.com


RecommendedSessions

SESSION# TITLE DATE/TIME

DO5T18SManheim'sAutomatedTestingStrategyWithCATestDataManager

11/17/2016at11:30am

DO5T04SAmericanFamilyInsuranceleveragesCAServiceVirtualizationtoacceleratecontinuousintegrationindevelopment.

11/17/2016at12:45pm

DO5T22SINGDeliversUnprecedentedGlobalContinuousDeliveryasaService

11/17/2016at03:00pm


MustSeeDemos

ModernizeAppDeliveryIntegratedCDTheater5- DOV501P

DeliverTestDataFasterCATestDataManagerTheater5- DOV511P

DeliverBetterAppsCAServiceVirtualizationTheater5- DOV507P

OrchestrateYourReleaseCAReleaseAutomationTheater5- DOV513P


Stayconnectedatcommunities.ca.com

Thankyou.


DevOps– ContinuousDelivery

FormoreinformationonDevOps– ContinuousDelivery,pleasevisit:http://cainc.to/PiTFpu

Technology

Analyst Keynote: Continuous Delivery: Making DevOps Awesome