DevOps Journeys

2.0

DevOps: On The Way Up 3

DevOps Goes Mainstream 6

DevOps: The Learning Curve 12

Measuring Success 17

The Future Of DevOps 21

Contributors 26

Contents

DevOps: On the way up

4

DEVOPS: ON THE WAY UP

DEVOPS JOURNEYS 2.0

DevOps is a complex term, and perhaps therewill never be an uncontested definition of whatit really represents, and how it should work, butcrucially it has proven itself as a working practiceover and over again, and businesses of allshapes and sizes are taking notice.

DevOps is really opening up what can beachieved – alongside impressive technicaladvancements, the practice is helping to alignpeople, both across the historical void betweenOps and Dev functions, and out into the widerbusiness and its customers. Grant Smith, authorof Next Gen DevOps, puts it simply – “DevOps ishow I solve engineering problems. There aren'tany alternatives that don't also force me to solvecommunication, organisation, prioritisation andstrategic direction problems before I can get tothe engineering problems”.

DevOps opens up a new approach to solvingproblems. Engineering is no longer a goal untoitself, but part of a wider toolkit to add value.Shabe Razvi, experienced DevOps consultant,talks about seeing DevOps as a way of workingthat “optimises the business value stream bybreaking down barriers between Developmentand Operations to enable them to work togethereffectively. Highly aligned Development andOperations teams, centred around shared goals,can lower the cost of changes, increaseoperational excellence, and result in shorterfeedback loops so that businesses can buildtremendous, high-quality experiences thatdelight customers.”

Aymen El Amri, CTO at Weblib, seconds this,seeing DevOps as “a culture that requires a newvision with a common goal of unifying peopleand departments around unique goals byestablishing the right communication processand a cross-functional collaboration. One ofthese culture characteristics is being customer-oriented: Being focused on customers is the bestway to align teams towards the same valuablegoals without creating an inter-departmentwar.”

Tony Chapman, Director at LinuxRecruit agrees,pointing out how remarkable the adoption ofDevOps has been, and why it has been sosuccessful. For him, while technologicaladvancements are important, behind it all is anemphasis on shared goals – both between devand ops teams, and between employees andcustomers. “That is, in my opinion, at the heartof how DevOps has been able to have suchextraordinary success – it helps everyone dowhat they do best, which makes everyone moreeffective, and most importantly, happier”. Opsand Dev teams are working together to smooththe way for each other to release higher qualitysoftware faster and with fewer issues. Tony viewsDevOps as how “customers get the service theywant - and how the CEO gets the growth theylook for”.

Richard Haigh, Head of Delivery Enablement atPaddy Power Betfair, talks about thistransformation as “a cultural change, a way ofthinking”.

5

DEVOPS: ON THE WAY UP

DEVOPS JOURNEYS 2.0

He also draws attention to the potential ofDevOps being able to “unlock an increase in paceand agility within the business”. DevOps workingpractices have really opened up the capacity ofbusinesses to try new things, and try these thingsfaster and with more confidence. Daniel Bryantagrees, defining DevOps as “a superset ofcontinuous delivery, which strives to encouragethe delivery of business value through theincrease of shared (end-to-end) responsibilitythroughout the organisation”. This value “istechnically realised through the automation ofsoftware release processes and systemobservability.”

Mike Dilworth, experienced DevOps practitioner,talks about this emphasis on the increased speedthat can be achieved with the technologicaladvancements that we’ve seen in recent years.“Fundamentally DevOps is all about getting stufffrom the backlog and into production as quicklyas possible and to the highest possible level ofquality …To help achieve this, the DevOpsmovement has adopted many agile processes, forexample, short iterations with fast feedbackloops, and employs state of the art technologies,such as cloud, containerisation and serverlessarchitecture”.

Rob Elkin, CTO at Busuu, talks about the future,seeing it as “somewhat inevitable that theinfrastructure role would evolve to abstract away

the need to deal with physical hardware, and thathas certainly precipitated the advent of DevOpspractices, allowing businesses to act faster andevolve infrastructure in ways that would not havebeen possible before”.

While tooling has provided routes to thisevolution, the real change has come from achange of focus. Operations are increasinglyremoved from manually configuring servers orattempting to get code running as it did on thedeveloper’s machine. Most of these manualproblems are solved, or heavily abstracted. ForRob, DevOps practices mean that we can focus onhow our infrastructure serves the business needsmore and more. “We try to move as fast aspossible in other areas of software engineering,and Infrastructure is no different. DevOps meanswe can deal with business needs in the mosteffective way, with the most effective technology,quickly and cost effectively”.

Tooling and the abstraction it offers is a greatenabling force for DevOps practitioners to focuson processes that add value. While the right toolsdo play an important role in a true DevOpsculture, tooling alone is not enough, unlesscombined with the right thinking. As Aymen ElAmri puts it very succinctly, DevOps starts withprocess: “Adopting DevOps needs the rightprocess done by the right people using the righttools”.

Adopting DevOps needs

the right process done by the right

people using the right tools.

“ ”

DevOps Goes Mainstream

7

DEVOPS GOES MAINSTREAM

DEVOPS JOURNEYS 2.0

While not quite everyone is adopting DevOps asa working practice yet, it’s definitely reached apeak in popularity, with RightScale’s State ofthe Cloud survey highlighting that adoption hasincreased from 66 percent to 74 percent*, with asteep increase in the enterprise space. WithDevOps playing a role in companies right acrossthe technology landscape, probably fair to saythat DevOps has become mainstream.

Grant Smith is clear that “it can only be a goodthing that DevOps has gone mainstream.DevOps improves the lives of engineers andleads to better solutions”. Daniel Bryant, ChiefScientist at OpenCredo, agrees that goingmainstream is a good thing - “the core conceptsof increasing shared responsibility, automatingoperations, and increasing observability are allfundamentally good things for our industry”.Mike Dilworth similarly sees the wideningapplication of DevOps principles as a positivedevelopment, arguing that “common senseshould not be the exclusivity of the few but theright of the many”.

Tony Chapman argues that a few years back theDevOps community was, to some extent, quiteexclusive – seen as “something for the trendystart ups, or for the most skilled engineers usingthe bleeding edge tech. While that attitudeperhaps hasn’t quite fully died out, I definitely

think it’s on its last legs. What we’re seeing nowis that DevOps has an incredibly wide scope, andit is being utilised in highly regulated industriesand in companies with huge legacy code basesand technology stacks. It’s no longer thepreserve of the technology elite, and that’sdefinitely a good thing”.

David Gildeh, CEO of Outlyer, sees DevOps goingmainstream as not only positive, but necessary.He argues that “the reality is, if you're not doingDevOps, you're essentially doing oldertraditional software models that just aren'tcompetitive and agile enough for today'senvironment. That means you won't attract thebest developers, you won't write the bestsoftware and you'll always be behind the rest ofthe competition that can and will move fasterfrom you…it’s a critical thing if you want yourbusiness to stay relevant and successful”.

GROWING COMMUNITYAs DevOps has grown, so have the meetups,conferences and resources available. RichardHaigh talks about enjoying “the explosion inconferences, meetups and communities thatoccur when a movement goes mainstream”.Getting exposure to DevOps is easier than everbefore. Alongside technical forums, meetupsand conferences, DevOps is now more digestiblefor the wider business world. Non-technical

business leaders have easy access to contentand explanations around DevOps that they canunderstand. It is now relatively easy to be ableto see how IT automation translates into betterdelivery for their customers, helping to promoteDevOps even further into the mainstream.

This sense of community begins internally – asRob Elkin points out that “an engineering teamcan't just shut themselves in a room anymore,they have to deal with marketers, operationspeople, and the rest of our businesses to worktogether to achieve things. Infrastructureengineering is no different, engineers need towork with the rest of the business to delivervalue and as a result will become moremainstream. More people understanding eachothers roles, strengths, and weaknesses, canonly be a good thing”. He sees that, astechnology becomes a bigger aspect of our livesand our businesses - “software engineering isevolving from a focus on just the code, to asocial endeavour”.

Aymen El Amri talks about his early experiencesworking on DevOps projects, working on“difficult problems, (fantastic but) immaturetechnologies and some misconceptions”.

* rightscale.com/blog/cloud-industry-insights/new-devops-trends-2016-state-cloud-survey

8

DEVOPS GOES MAINSTREAM

DEVOPS JOURNEYS 2.0

“But I was not alone, a growing community wasworking on similar problems, these people builtthe foundations and made possible theintroduction of DevOps to all types of companiesfrom startups to corporations”.

DevOps is a community movement, and the fastgrowth of the network conferences, meetups andforums make for great opportunities to findsolutions to problems, and more and moreengineers can take inspiration back to theirorganisations. Richard Haigh also considers thepositive impact of mainstream understanding ofDevOps beyond engineering communities,remembering “years ago struggling tocommunicate the sort of person needed for aDevOps role to my recruiters. It is certainly easiernow”.

Tony Chapman points out that “the mostimportant piece of the DevOps puzzle is thepeople. Good people are hard to find, and if thegrowing can help educate and connect companieswith these good people, then it’s a good thing thatmore people are joining the DevOps community”.

CULTUREAs more and more organisations are striving tocreate an optimal ‘DevOps culture’, there are thosethat are failing to understand what that reallylooks like, focusing instead on ’quick wins’ that areeasy to implement and look nice on a job advert.

Tony reflects that “in recent years, technologyteams who are ‘jumping on the DevOpsbandwagon’ have really ramped up their culturalsales pitch, declaring their foosball table and beerfridge as if this automatically equates to a forwardthinking working culture. Or, more troublingly, yousee an identikit office of engineers with the sameeducation and background – and anyone not inthat mould is ‘not right for the culture’. While Ithink trying to build a culture is usually better thannot trying, many companies – and indeedindividuals – seem to think about the culture fromthe wrong perspective, in that they talk about it asa fixed entity”.

The Business Dictionary defines culture as the‘values and behaviours that contribute to theunique social and psychological environment of anorganisation*. Culture is basically a reflection ofhow and why people go about their jobs. In thecontext of DevOps specifically, Tony considers thatit’s “relatively rare to hear many organisations talkin much detail about this”.

“More commonly, especially in organisations whoare new to DevOps, they talk about culture in fairlyoblique terms - ’the culture needs to change’, ‘shewasn’t a good cultural fit’, when in fact theseorganisations should be thinking about what this‘culture’ actually is, and how each team memberfits into, and adds (or detracts) from this".

* businessdictionary.com/definition/organizational-culture.html

9

DEVOPS GOES MAINSTREAM

DEVOPS JOURNEYS 2.0

David Gildeh makes a great point when hediscusses his changing understanding ofDevOps, from when he first started out, seeingit "as a team that "automated things", andcoming to understand that real change comesfrom a shift in responsibilities. For him, the keyto enabling this shift was monitoring. "Bymaking monitoring self service, givingownership of alerts and metric collection todevelopers so they had full visibility into theirstack, it completely changed the behaviour ofdevelopers and also the types of conversationswe were having between developers andOperations. Instead of opinions, they were databacked discussions and this had a hugeimpact”.

Rob Elkin makes a good assessment of a lackof “situational awareness” within a lot oforganisations. They know that their outputs -delivering value at the speed and reliabilityrequired are not right, but they “can’t identifytheir current issues” accurately. This situationalawareness is what’s vital to ensuring DevOpscan be successful. Organisations need tounderstand the technical challenges, but alsothe cultural challenges they have to overcome.

For Mike Dilworth, a common failing for teamsadopting DevOps is where there is "no attemptto change how they work". There are somegreat organisations who have spent timethinking and how and why their team operates

the way they do, and how changes, like self-service monitoring, can be used, not only toevolve understanding and insight, but to ignitea cultural change.

But, as DevOps continues to take root intechnology teams across the world – and intoorganisations with a higher degree of legacyand entrenchment, it’s more important thanever that organisations take the time to thinkabout their culture, and how they can change itfor the better. Tony argues that "unfortunatelysome organisations who look to adopt DevOpssee declaring a 'cultural change' as the lowhanging fruit of their transformation strategy.Their focus is on upgrading their tooling andautomating things, without giving enoughconsideration to the how and why of theprocesses and behaviours of their team".

AUTHENTICITYWhile the community is great to see, morepeople than ever are claiming to ‘do’ DevOps,which does open up a number of potentialpitfalls. Shabe Razvi argues that, while it’sgreat that the “industry is recognising the valueof delivering faster, through organisationalalignment, removing silos, optimising the valuestream and automation, there is a significantflood of tools on the market that claim to solve‘the DevOps’ that are diluting and confusing analready abstract definition”.

The mostimportant

piece of the DevOps

puzzle is its people.

“ ”

10

DEVOPS GOES MAINSTREAM

DEVOPS JOURNEYS 2.0

Daniel Bryant agrees, pointing to the problemof ‘over-commercialisation – “particularlyaround the 'peak of inflated expectation' timepoint on the Gartner hype curve. A quick lookat the software development market showsyou every person and their dog are rebrandingor "DevOps-washing" their products”.

Organisations who aren’t already looking atDevOps now seeing their competition pullaway, so are not only interested, but pressuredto look at DevOps as a solution. Business valuehas been quantified and proven over and overagain, and it is more and more common to seeDevOps transformations not only supported bythe C-Suite, but initiated. This is, in theory,positive, as C-Level buy-in is important tosuccess, but ultimately DevOps needs to startin the tech team. Enforced DevOps is notDevOps. Shabe points out that “in some casessenior leaders are missing the point that toolsand automation alone do not create a goodDevOps culture”.

As we saw earlier, an over focus on tooling andtechnology was a common mistake whenstarting out, and it’s still a common mistake tofocus on ‘observed practices’ - after all, loadingup with the latest tools and saying you doDevOps a lot is a great way to show you’redoing DevOps right?

Mike Dilworth agrees that “right now, everyoneis a DevOps Engineer and everyone is doingDevOps, but this is very very far from thetruth”, but, with DevOps being a communitydriven initiative, “I predict as this communitygrows and becomes more mature it will exposethe individuals and organisations who simplyjust don’t get it”.

HIRING CHALLENGESShabe Razvi argues that a true DevOps culture“requires everyone to up-skill and cross-skillwhich is causing huge hiring challenges. Withthe changing landscape, it's essential to have agood blend of specialists as well as expertgeneralists who are able to consume, adoptand apply the flood of emerging technologies.He also makes the important point that, insuch a landscape, “it's important to ensurethat emerging technologies are treated in thecorrect way, and adopted in a sustainablemanner by the whole team. Engineers cancome and go easily, so senior leaders need tobe wary of CVDD - CV Driven Development”.

Tony Chapman has seen this from the otherside, referring to the ‘buzzword bingo’ that hesees in some CVs. “There are organisationsthat use DevOps as a branding tool to draw inapplications. Similarly we see sysadminsthrowing in every buzzword they can onto their

CV so they are seem employable in a marketwhere seemingly nothing but DevOps will do.Many hiring companies are still focusing toomuch on tooling and are requiring very specificexperience – experience that nine times out oftime could be acquired by an intelligentengineer in a short time frame”.

“While, of course, in the practical world, if youneed a project doing yesterday, you may needsomeone with experience in tool X, but we arestill seeing a lot of rejection, particularly at themore junior end of the market that isbeginning to create a bit of a divide betweenthose who are ‘DevOps’, and everyone else whowants to make the move but perhaps lack thespecific experience, whether that be in aDevOps environment generally, or with thelatest tool-of-the-day”.

Tony continues, arguing that “everyone isgoing after the same pool of experiencedpractitioners and demanding ‘ready made’experts. However, some businesses approachhiring as a search to find someone who hasextensive experience with every tool andtechnology they will use in the role, effectivelyasking engineers to make a sideways move –doing the same role they already do, in adifferent company, with limited scope to learnmuch of anything new”.

11

DEVOPS GOES MAINSTREAM

DEVOPS JOURNEYS 2.0

"This is also evident at the junior end of themarket, with relatively few organisationscommitting to hiring inexperienced people andtraining them up. As I see it, academic studygenerally does not prepare graduates especiallywell for a commercial DevOps environment, andalternative efforts like the DevOps Institute are notpopular with many in the community".

"What concerns me, is that the visionaries leadingtoday’s DevOps community started out as juniorsysadmins that had help and support to developtheir skills. Of course, not every organisation maybe well placed to help- if you’re a startup you maynot be able to offer the support and guidancerequired, but across the industry, there is thesense that there is a bit of a ceiling for a lot ofpeople without the right experience”.

"On the other hand, we're also noticing a newbreed of entry-level Engineers coming through,who have only used Cloud, who have learnt toAutomate everything and know little else.Engineers who don't understand the basics ofTCP/IP, DNS or Linux Fundamentals and have onlyworked in a fully containerised, fully automated,shiny cloud environment. In most companies,where there is legacy based infrastructure still inproduction, this can also lead to issues when theseengineers join organisations with a stack thatpredates 2017”.

Shabe Razvi makes an important point about the

“superiority complex” that he has seen in DevOpsengineers over their Developer colleagues with“phrases like RTFM still commonplace. Thecollaborative, human empathy aspect is key toensuring the right culture flourishes”. He goes onto say that “identifying these collaborative traitsare a lot harder than technical prowess, however,one without the other means barriers will be morechallenging to break down than a Mexican borderwall”.

As DevOps has grown, it’s important not to losesight of the collaborative mindset that shouldunderpin any team that claims to do DevOps.While Tony emphasises that “there are people andorganisations out there that are doing wonderfulwork on opening up and demystifying DevOps, butI do think that the community could do more tocultivate talent. As the saying goes, it takes avillage to raise a child – and anyone claiming towork in a DevOps environment should be someonewho drives efforts towards adopting DevOps. Getjunior people thinking in the right way, give them achance to learn what DevOps is all about and howbest to apply the practice in a legacy stack. Makesure your lead engineers have the scope to keepdevelop their skills”.

“Everyone has something to learn, and as DevOpsreaches into every organisation in the world, weshould be working together to make sure that wecan create a stable market for it to grow and growand grow”.

DevOps: The Learning Curve

13

THE LEARNING CURVE

DEVOPS JOURNEYS 2.0

Worrying about specific named tooling. A signof a mature approach is if the teams areagnostic to specific tools. Tools should beassessed and selected based on currentrequirements and can be swapped frequently ifrequired. It is too easy to gravitate to themainstream names purely because they arethe flavour of the day with certain groups orare getting air time at conferences.Richard Haigh

I jumped straight into trying to solveconfiguration management problems ratherthan focusing on the outcomes I wanted toachieve. I'm in good company most people whotackle a DevOps transformation do somethingsimilar whether it's configurationmanagement, continuous integration,containers or orchestration.Grant Smith

Personally it was ascribing too much value tothe technical side of the movement. I like tothink of myself as an engineer/technologist atheart, and when I heard about DevOps five orsix years ago, the technology side of what Ilearned about DevOps resonated with me a lot- at the time I was scripting configuration ofJava and MySQL on Windows Server boxes,having initially learnt how to configure these

machines manually (and with greatfrustration!). The drive towards automatingoperations just made sense, and from mysomewhat limited (technical) perspective atthe time, I thought that getting the technology'right' was the most important thing.

However, in the previous five years (andparticularly in my work at OpenCredo) I'veworked with many different clients, rangingfrom one person startups to multi-departmententerprise and governments, and I've seen upclose the impact that culture, process andpeople have on the success of any project.

In combination with the fact that with age alsocomes wisdom, I've realised that my initialthoughts were slightly back-to-front, andculture and leadership are actually the mostimportant things to focus on with DevOps (andbusiness in general). Without clear goals,situational awareness and strong leadershipthroughout an organisation it is nearimpossible to drive lasting change andsustainable success.Daniel Bryant

My biggest mistake was trying to build teamsof engineers comprised solely of permanent

employees. The way people want to work haschanged and contracting is a way of life formany. It is important to realise that to scaleyour capability you will need to use both permand contract resources. Furthermore, you haveto treat perms and contractors the same.

Contractors should not be given the dull andless challenging pieces of work. Rather theyshould be expected to make valuedcontributions to the team, both in terms ofdeliverables but also in terms of improving theactual process of work, to deliver on lean waysof working. Diversity is an important elementof an innovative team and contractors oftenbring this with them. So, embrace contractorsand use them wisely.Mike Dilworth

If I think back to when I was hands on withInfrastructure, the immediate example thatsprings to mind is when I accidentally droppeda production database because I thought that Iwas connected to a staging environment. OnceI had quickly fixed that, I became an expert indatabase permissions and environmentsegregation!Rob Elkin

We spoke to some leading DevOps practitioners about some of the early mistakes and misunderstandings they made when starting out.

14

THE LEARNING CURVE

DEVOPS JOURNEYS 2.0

I think the biggest mistake was not fullyunderstanding the above, seeing it more as ateam who automated things, rather than ashift in responsibilities across both teams.Once we had that epiphany it completelychanged our approach to how we built ourorganisation, culture and tools. They key thatreally helped was monitoring, and making it aself service tool for developers to usethemselves, which is why we launched acompany in that space (Outlyer).

By making monitoring self service, givingownership of alerts and metric collection todevelopers so they had full visibility into theirstack, it completely changed the behaviour ofdevelopers and also the types ofconversations we were having betweendevelopers and Operations. Instead ofopinions they were data backed discussionsand this had a huge impact on our culture.David Gildeh

DevOps is an exciting field with a lot ofchallenges like zero down-time deployment,self-healing and auto-scalability. Working ona DevOps transformation is being inconnection with several teams within thesame organization and mastering severaltechnologies.

Given this fast growing environment, thegreat number of tools, architectures andmethodologies, I found solutions to manyproblems but I realized that I was doing someover-engineering and tried to solve simpleproblems with complicated solutions thatsome people in the same organizationcouldn't really master or understand.

I think that many DevOps beginners are doingthe same thing but I realized with time thatsimplicity is the key to DevOpstransformations.

In order to stop doing over-engineering andeven under-engineering, I simply compare thecost of improving a solution to its benefits.My advice to Devops beginners: DevOpsshould be neither over-engineered nor under-engineered. Don't settle with the simplestsolution and don't find the most perfect onebut consider “right-engineering yourtransformation” and iterating improvementsover your solutions.

Set up only what is needed, only when it isneeded and only where it is needed.Aymen El Amri

15

THE LEARNING CURVE

DEVOPS JOURNEYS 2.0

TOOLING ISN’T EVERYTHINGThere’s certainly a common thread throughthese answers - that tooling does not equalDevOps. Richard Haigh argues that you canspot a team with a mature approach if they are“agnostic to specific tools”, and arenot gravitating instinctively to the mostesoteric “flavour of the day” for theirrequirements.

Tooling in DevOps has always been seen as avital component of striving to automate“everything”. As Daniel Bryant remembers thatthe automation “just made sense” to him, andspent some time believing that “getting thetechnology right was the most importantthing”. Automation is, of course, a centralrequirement for a DevOps working practice, butmature practitioners have moved on to a morenuanced assessment.

It’s now more of a consideration of whethersomething is ‘boring’ - you do it often enoughthat you know how it works, and surprises areunlikely. You also need to consider whether thecustomer will see relative value from suchautomation, whether directly, or in freeing uptime to invest in other things. Aymensuccinctly suggests that you “simply comparethe cost of improving a solution to itsbenefits”.

Daniel Bryant talks about the importance of

having “situational awareness”, and thinkingabout what is going to work for yourcustomers. Daniel talks about his realisationthat “culture and leadership are actually themost important things to focus on withDevOps (and business in general). Withoutclear goals, situational awareness and strongleadership, it is near impossible to drive lastinggoals”.

David Gildeh points out that a lot of people stillthink DevOps is just about faster deployments,which is only one aspect of DevOps. He seesthat it’s wholly possible to be using the latestautomation tools, but still have developersdistanced from production. For David, the realcultural shift happens when you makedevelopers responsible for their code inproduction – “they're the ones that have towake up in the middle of the night to fix issueswith their software, they can't just kick the candown the road to Operations”. This is wherethe mindset really changes. developers beginto put more thought into non-functionalrequirements.

Grant Smith shares this view, noting that thereis still not enough emphasis on these non-functional requirements, with “softwareengineering rarely taking non-functionalrequirements into account until it’s too late”.As David puts it – “end users don't care aboutthat shiny new button if it doesn't work half

the time or takes minutes to respond”.

David sees this shift towards non-functionalrequirements as marking a real turning point,and you soon “start the journey to whereOperations becomes more of a platforms team(or DevOps team) providing common tools andprocesses to the development teams, anddevelopment teams are empowered to buildand run their entire stack themselves, frominfrastructure to the software running on top”.

“This also means monitoring has to move fromjust a tool for Operations to a self-service toolused by development teams - and those teamsget the alerts to go fix things not Operations.Once you enable that shift you will see yourdevelopers behaviour change to focusing onthe whole user experience for your customers,not just the visible ones such as new features,and your software will become significantlybetter written and reliable”.

START FROM THE SOLUTIONGrant Smith recalls when starting out withDevOps that he spent too much time “trying tosolve configuration management problemsrather than focusing on the outcomes”. Whileof course, a problem needs a solution, goodDevOps is routed in solutions thinking, ratherthan finding the most cutting edge way ofdoing something, but it is an easy mistake tomake to overcomplicate your thinking.

16

THE LEARNING CURVE

DEVOPS JOURNEYS 2.0

Aymen remembers that, in the face of a fastgrowing environment and a great number oftools and methodologies, “I found solutions tomany problems but I realised that I was doingsome over-engineering and tried to solve simpleproblems with complicated solutions. I realisedwith time that simplicity is the key to DevOpstransformations”.

With the benefit of of hindsight, Aymen advisesthat “DevOps should be neither over-engineerednor under-engineered. Don't settle for thesimplest solution and don't find the most perfectone but consider ‘right-engineering yourtransformation’ and iterating improvements overyour solutions”. This is a lean approach, withcontinuous amelioration and strong feedbackloops that “stimulate productively and helppeople focus on the real problems”. Engineersare expert problem solvers - the challenge issometimes stepping back enough to focus onthe end solution they are looking for.

THE HUMAN TOUCHDaniel Bryant reflects on the “impact thatculture, process and people have on the successof any project. In combination with the fact thatwith age also comes wisdom, I've realised thatmy initial thoughts were slightly back-to-front,and culture and leadership are actually the mostimportant things to focus on with DevOps (andbusiness in general)”. Part of this focus on thehumans of tech comes from thinking about your

team as people – with personal lives andresponsibilities beyond their jobs. While someorganisations are becoming more flexible,offering remote working and more varied hours,it’s still relatively uncommon, and a lot ofbusinesses are missing out on finding greatengineers who don’t want to, or can’t commit tothe 9-5 in the office. Mike Dilworth also makesan important point about contractors: “The waypeople want to work has changed andcontracting is a way of life for many.” They canbe a vital part of a team, both in terms ofdeliverables but also in terms of improving theactual process of work.

Tony Chapman adds that “many people look forflexibility - whether that is to run a side project,spend time with family, or travel, and we arebeginning to see more openness from clients ineither taking on contractors in lieu of apermanent employee, or offering reduced hoursor remote working, but these are still few and farbetween. While some organisations may have agenuine business need for onsite full timepeople, I do think there’s a massive gap to findgreat people that is not yet being takenfull advantage of”.

While the community is definitely attuned to theimportance of people in any successful DevOpstransformation, we’re perhaps still on thelearning curve when it comes to unpacking howDevOps can work in the working world.

Consider ‘right-engineering your

transformation’ and iterating improvements

over your solutions.

“ ”

Measuring Success

18

MEASURING SUCCESS

DEVOPS JOURNEYS 2.0

The monitoring space has definitely seenhuge changes. Traditionally a fairly static toolto alert the sysadmins if disaster struck,monitoring has transformed. Withcompanies continuously pushing newreleases of software, monitoring needs to beresponsive, intelligible and actionable.

Aymen El Amri talks about how, as DevOpshas gone mainstream, “CEOs have increasedtheir focus on DevOps metrics and success,with organisations giving increasing value tothese metrics”.

This is an important shift, and one whichDavid Gildeh has seen over and over, seeingimproved, self-service monitoring having thepower to change conversations betweendevelopers and Operations: “instead ofopinions they were data backed discussionsand this had a huge impact”.

MEASURE ALL THE THINGSDavid sees that “ultimate success comesfrom understanding that you're trying tobuild better software for your users. Fasterdeployments of a crappy service, that is slow,unreliable and no one uses, won't enableyour business to be successful. You need tostart collecting business metrics (how manyusers logged in, user retention metrics,funnel metrics) all the way down to reliabilitymetrics (how often does my code crash inproduction) to performance metrics (howquickly does the service load for end users) to

get a complete picture of how good yoursoftware really is.”

As environments have complexified, a raft ofmonitoring services that tap into thedifferent layers of the application areavailable. But for some there are almost toomany options to measure, creating a little bitof an ‘analysis paralysis’. David argues thatwhile “you need to measure everything”, youalso need to focus on the end user success ofyour software to ensure your efforts inmonitoring your services are effective.

Aymen sees that effective monitoring comesfrom understanding “the ‘why’ and not onlythe ‘how’ of the developed product, in orderto be able to choose the right metrics”. Forhim, it’s important sure people working onDevOps are partners of the productengineering and design.

Grant Smith argues that “if you're about toembark on anything transformative establishyour metrics, measure them before thechange and afterwards. The last largeDevOps transformation initiative I ledreduced the time taken to provideinfrastructure to project teams from threeweeks to four hours, (including design time) Imade sure we had good metrics definedbefore we released the solution”. You needreal situational awareness of where you are,and what you’re trying to achieve at alltimes.

19

MEASURING SUCCESS

DEVOPS JOURNEYS 2.0

Mike Dilworth talks about some of the commonKPIs used when talking about DevOps, most ofwhich “concern the velocity, frequency andquality of software delivery. These areimportant measures of how well anorganisation is doing in terms of its ability toinnovate at speed, realise value quickly – andhence compete”.

Tony Chapman reflects on the role of velocityand frequency in the context of the DevOpsmovement. “A couple of years back, we heard alot about Amazon deploying to productionevery 11.6 seconds, and this sort of speedbecame a bit of a competitive benchmark.What I don’t remember as much of is manyconversations about the percentage of usersaffected by bugs, median application responsetime etc”.

“Of course I’m not suggesting that everyonewas ignoring these issues, but rather perhapsgetting slightly carried away in the hype! Theconcept of Continuous Delivery wasgroundbreaking - software has always hadbugs, so is perhaps not quite as exciting. But,encouragingly, one thing I have noticed asexperiences have matured is a movement awayfrom talking so much about velocity. Insteadthere’s an increasing focus on quality, andultimately putting the user front and centre”.

WHO CARES?While velocity is an important measure of anyproficient DevOps working practice, ShabeRazvi points to the importance of lookingultimately at business performance, “which canbe underpinned by more concrete, well-knownmeasurements such as the lead-time,deployment frequency and the reduction ofdevelopment waste. Measuring user-journeyavailability is also a great one that implies thatyour systems are designed with operationalconcerns in mind, such as zero-downtimereleases”.

He continues that, in today’s globalenvironment, “having a 4am deploymentwindow of 1 hour with downtime does not giveany competitive advantages! Someonesomewhere is being impacted, and for globalbusinesses that's the equivalent of closing thefront doors. Demonstrating an increase in thedelivery of changes, whilst maintaining orimproving the customer experience shows yourDevOps initiatives are working!”

Rob Elkin shares this view. “As DevOpspractices become more prevalent in a business,it should move engineers closer to impactingdirectly on customers and hence the business,so we should consider that when measuringsuccess”.

Tying it back to the effect on a customer is always a good measure.

“ ”

20

MEASURING SUCCESS

DEVOPS JOURNEYS 2.0

Rob reflects on some of the challengingproblems that need to be solved in managingglobal data – “Can taking an operation multi-region increase the page load speed for yourhomepage for users across the world? Canengineers understand traffic patterns andmeasure successfully so that an infrastructurecan scale up and down based on need, so thatcustomer get the best experience but thebusiness only spends the money oninfrastructure that it needs?”. For him “it reallydepends on where the pain points are for yourbusiness, however tying it back to the effect ona customer is always a good measure for me”.

PUTTING PEOPLE FIRSTAs DevOps has matured, it’s exciting to see anincreased focus on customer experience, but asGrant Smith puts it, it is important not to forgetthat DevOps “is a working practice”. Rob Elkinconsiders a measure on success in looking athow “an infrastructure engineer helped otherengineers to deliver faster, to work togethermore effectively, and to understand the state oftheir code in production”.

Mike Dilworth says that while quality ofsoftware delivery is vital, for him “the mostimportant measure of success is jobsatisfaction. DevOps is unique in that it has theability to actually improve the working

environment and to make the thing most of usdo every day an enjoyable and worthwhileexperience. We should be striving to achieve thisas one of the top deliverables in any DevOpstransformation.”

Daniel Bryant also highlights areas such asemployee retention (and happiness), andamount of collaboration as valuable measures.This is where a HR focus, especially in largerorganisations, can be useful, collecting insight inline with transformational projects. TonyChapman argues that this doesn’t need to beintensive, but an active process of reviews withemployees, and a productive ‘open door’ policyto circumvent issues early can be more useful tokeeping a team motivated and enjoying workthan any new tool.

Daniel also makes an important point about themeasurements of success he has discussed –“they are 'lagging' indicators, and as aconsultant I typically try and look for thesomewhat fuzzier 'leading' indicators such asamount of pain being experienced, attitude tochange, and commitment of organisationalresource”. For him, areas like MTTR andcustomer feedback are important - but he alsorefers to the amount of learning undertaken bystaff. This is where effective feedback loops andregular iteration play a central role – the mark of

an effective DevOps function is failing (andrecovering) fast, and learning a lot.

The tools and thinking around how we canquantify success have improved exponentiallysince DevOps first surfaced, and it’s importantthat engineering teams take full advantage ofthe wealth of ‘lagging indicators’ available, andcontinue to work at better unpicking andquantifying the ‘fuzzier measures’.

Richard Haigh makes a great point in arguingthat we should ultimately be measuring thesuccess of DevOps by “its ability to change andadapt as time goes by”, and perhaps this is agood conclusion – measuring success in DevOpshas innumerable possibilities, and there’s no setway of doing it well. Any attempt to do so needsto be wholly routed in understanding whatchanges you’re making and how to adapt tothis.

And, in a more general sense, we might say that,if DevOps continues to flex and change, and canremain a driving force behind forward thinkingorganisations the world over – then perhaps it’sfair to say it is a success.

The future of DevOps

22

THE FUTURE OF DEVOPS

DEVOPS JOURNEYS 2.0

The future of DevOps is an interesting one.It’s been proven over and over again and looksset to continue its dominance inconversations around software delivery. Thatsaid, there are challenges ahead.

SECURITY AND DATAIt’s rare to meet someone in technology whohasn’t seen a security issue at close quarters.Shabe Razvi talks about the many recenthigh-profile security breaches as “the scourgeof our generation. The pace of delivery has leftmany traditional InfoSec teams playing catch-up, and a "trust but verify" approach has beentaken in many cases. We can do better thanthat and as with quality, we should aim tobuild security in. Tools have taken huge leaps,and so I expect DevSecOps will becomemainstream over the next year or two”.

Tony Chapman has definitely seen an uptakein security focused roles in the last fewmonths, but reports like the HPE ApplicationSecurity and DevOps Report* suggest there’sa lot to be done. While 99 percent ofrespondents saw adopting a DevOps cultureas the opportunity to improve applicationsecurity, but in reality, only 20 percent statethat secure SDLC testing is done throughoutdevelopment. Most organisations are relyingon the technologies downstream, such as pre-production penetration testing and networksecurity.

Rob Elkin also considers how Infrastructureengineering will respond to some of thepolitical and data protection challengesahead. “There are a number of regions withtheir own vigorous data protection lawscoming into effect in the EU, Russia, UK andUS, and how we build global services whiletrying to balance these concerns is going to beinteresting. From an infrastructureperspective, how would we segment ourdatabases and services to maintain Russianuser data only in Russia, EU data in the EU,while still allowing the service to runeffectively and those users to potentiallyinteract with each other?”

Grant Smith seconds this – “the next bigproblem for DevOps is data. We can nowmanage software with software, networkswith software, infrastructure with softwarebut data is much harder to manage. I imaginethe solution will be a large scaleabandonment of data monoliths in the sameway that we've abandoned softwaremonoliths”.

TECHNOLOGY AND TEAMSAymen El Amri sees microservices as beingmore adopted during 2017 so new practiceswill arise in development, architecture,delivery, integration and networking, but alsoin terms of team management andcollaboration.

* https://www.hpe.com/h20195/v2/GetPDF.aspx/4AA6-8302ENN.pdf

23

THE FUTURE OF DEVOPS

DEVOPS JOURNEYS 2.0

David Gildeh shares this prediction seeing that,as more organisations move to DevOps, “theywill have to move to microservices, breaking uptheir teams and services across the organisationinto smaller, more agile parts. This willultimately lead to an explosion of technologiesentering the enterprise, an organisation thatmay just use Java today may also see Pythonand other languages creep in, multipledatabases creep in as teams are empowered tobuild their microservices using the besttechnology for the job. This means as a whole,online services will become even more complex,and tools like monitoring and Docker to easilydeploy and monitor them will be even morecritical in future”.

Daniel Bryant sees the future of microservicesand containers as the status quo, that will“blend with the typical way of working”. Wewon't be talking about microservices andcontainers, and will instead be deployingfederated software architectures onto PaaS-likecloud environments.”

It would seem a safe bet to say that containersare definitely here to stay, with increasinglysophisticated orchestration technologieshelping organisations of all shapes and sizes toensure environment consistency andoperational efficiency. That said, Shabe Razviargues that “ultimately the aim of the game isto serve requests to users, they don't care if it’s

served from bare metal, a VM or a Container.Despite the advances in containers andorchestration technologies, there is still a hugecost to build and maintain your own PaaS. Thesimplicity and price-points will mean that theserverless model will suit a large percentage oforganisations and will be huge in 2017,especially with IoT. The future will be a robust,self-healing world where Developers andSystems Engineers will enjoy a good (alert-free)night’s sleep, knowing that together they’vebuilt highly-available, fault tolerant systems. Toquote the leader of the free world, it's going tobe tremendous, huge”.

Rob Elkin talks about the continued evolutionof areas like Hashicorp tooling andconfiguration management options, “which willmake automation and scaling easier and moreeffective. We'll also see the big cloud platformplayers evolve their product offerings (just lookat how many things come out of AWSRe:invent!) that will make both Infrastructureengineering and other software engineeringroles easier and somewhat more abstracted”.Rob is slightly more cautious around theserverless movement – while he sees it growing,he’s yet to see it “being used truly in anger inproduction, and I think it will take a year or twofor real adoption to start to come out in theform of full APIs and services being written onit, or entire businesses relying on it”.

The future will be a robust, self-healing world where Developers and Systems

Engineers will enjoy a good (alert-free) night’s sleep.

To quote the leader of the free world, it's going to be

tremendous, huge.

“ ”

24

THE FUTURE OF DEVOPS

DEVOPS JOURNEYS 2.0

WIDENING SCOPEAymen El Amri talks about the sprawl ofDevOps as it reaches into other fields withinthe organisation, giving rises to terms likeBizOps ChatOps, DevSecOps, DevQAOps andIoT. Shabe Razvi agrees, seeing that“organisational alignment to enable a DevOpsway of working will become more widelyrecognised, expanding the feedback loopsbeyond just Dev, Sec and Ops to makeBizDevSecOps more mainstream”.

Looking to the coming months, Aymen sees2017 as a “big year for machine learning andartificial intelligence application to solve real-world problems. I think that DevOps is whatcan make these kinds of "scientific projects"open to the public. Improving the developmentof software that supports scientific projects isa must for AI startups to be competitive andreach more satisfied customers - that's why‘ResearchOps’ will be more in demand during2017”.

Tony argues that, “despite the obsession withreducing all these great ideas into pithyportmanteaus, the rise of these titles showsthat areas across the business are seeing whata good DevOps approach can achieve forrelationships between Dev and Ops, andultimately, how this can transform a businessand the experiences of its customers”.

DEVOPS BACKLASH?Mike Dilworth predicts we’ll see a “DevOpsbacklash. While DevOps is definitely here tostay, it is far from being a mature practice. Ifyou believe what you read and what people say,right now everyone is a DevOps Engineer andeveryone is doing DevOps. However, this isvery, very, far from the truth with many DevOpscharlatans, both individuals and organisations,out there”.

We saw earlier that commercialisation can beproblematic. Aymen, though positive about thegrowth of DevOps, is aware that “goingmainstream can ruin the thing we care about,this is what happened to proven frameworkslike ITIL: technologists tends to forget about itsince it became "too mainstream". Anythingthat can be termed a fad is vulnerable to over-saturation. Aymen talks about important anduseful frameworks such as ITIL have somewhatfaded into mundanity - and certainly notbecause they are no longer relevant or useful,but rather it’s become ‘too mainstream’. Thatsaid, he remarks that other methodologiessuch as agile and lean are complementingDevOps well, and remain an important part ofeffective software development.

The balance is a fine line – the success ofDevOps for software development lies in a lackof strict definitions and rules.

25

THE FUTURE OF DEVOPS

DEVOPS JOURNEYS 2.0

Mike Dilworth doesn’t see “the need forcertification or governance to formalise DevOps,to me this is another DevOps anti-pattern. DevOps is a community driveninitiative and I predict as this community growsand becomes more mature it will expose theindividuals and organisations who simply justdon’t get it”.

“We may see a move towards platformengineering, SRE, or infrastructure developers,as skilled practitioners try to distancethemselves and it is to these groups that weshould look for the innovations in tooling andprocesses which help to underpin the lean waysof working that truly is DevOps”.

Daniel Bryant shares this thinking, imaginingthat, ”in larger enterprises, the 'operations'team will most likely focus on delivering andmaintaining some form of platform andproviding guidelines and technical sysadminexpertise. This is very similar to Google's notionof site reliability engineering (SRE)”.

Richard Haigh similarly sees the future ofDevOps as ultimately merging with a lot of theSite Reliability work that is seeing massivegrowth. ”I believe the automation drive ofDevOps and the relentless drive for uptime of

SRE will combine to make a standard approachadopted by many. It will offer an overallinternal consulting, specialist support and R&Dfunction which will underpin the delivery side ofany tech organisation”.

David Gildeh sees DevOps as a “businessimperative to stay relevant and compete”. Tonyagrees, arguing that the future of DevOps isthat it will disappear. “Not as a concept, but itwill just become the norm, the standard way ofstructuring technology teams. DevOps will justbe how everyone works”.

WHAT LIES AHEADFor Daniel Bryant, “the future of DevOps is thatit will become the status quo and blend withthe typical way of working. IT will become lessseparate from the 'business' and we'll beworking in cross-functional teams to solvebusiness problems (rather than simply buildingand deploying supposed 'solutions’)”. He hopesto see developers becoming more operationallyaware, “and develop increased 'mechanicalsympathy’, and also take more responsibilityfor the code they are running, particularly withinsmall organisations”.

Tony looks ahead to DevOps as the future ofsoftware – “everyone, literally every company

with any kind of technology will work to DevOpsprinciples. Those Ops and Dev teams stillworking over a fence will simply fail”.

Shabe agrees, arguing that while more andmore organisations are embracing the termDevOps, “there is no single prescriptive waythat this can be solved as it depends on theorganisation type, culture and size, so eachmodel will be different. Unlike yourinfrastructure, each organisation will be asnowflake!”

Looking to the future of DevOps, there aresome significant problems to be solved in theface of changing data protection laws, and thecontinued threat of security breaches. Thecommunity is moving in the right direction, butthere is definitely a lot of work to be done. Asthe community matures, we may see the termfade from job titles and team descriptors in lieuof terms such as Site Reliability Engineeringand the like, though as a concept and workingpractice, it seems relatively certain that it willremain an indelible part of softwaredevelopment into the future - and as morepeople come to really understand what DevOpsmeans, the organisations and individuals whodon’t get it will fall behind. DevOps is here tostay, and the future looks extraordinarily bright.

26 DEVOPS JOURNEYS 2.0

CONTRIBUTORS

Aymen El Amri@eon01Currently CTO/Architect at Weblib in Paris, Aymen has a broad technical background from development to operations, diversified work experiences, and a proven understanding of software development, cloud engineering and combining strategic and technical thinking. He helps companies and startups develop modern & scalable applications, build multi-tenant cloud infrastructures, stable production environments, distributed systems and service oriented architectures.

Aymen is author of a number of books including SaltStack For DevOps & Painless Docker. He is also the creator of DevOpsLinks.com, a global community of thousands of IT experts and DevOps enthusiast.

Daniel Bryant@DanielBryantukDaniel Bryant is the Chief Scientist at OpenCredo and CTO at SpectoLabs. He currently specialises in enabling agility within organisations by introducing better requirement gathering and planning techniques, focusing on the relevance of architecture within agile development, and facilitating continuous delivery. Daniel’s current technical expertise focuses on ‘DevOps’ tooling, cloud/container platforms, and microservice implementations. He also contributes to several open source projects, writes for InfoQ, O’Reilly, and Voxxed, and regularly presents at international conferences such as QCon, Devoxx and JavaOne.

David Gildeh@DGildehDavid is a serial entrepreneur. He founded and sold his last startup, SambaStream, to Alfresco in 2011, thus launching their cloud business.

David co-founded and runs Outlyer (previously Dataloop.IO), a SaaS infrastructure monitoring tool, and is the founder of the world's largest DevOps Meetup, DevOps Exchange London, with over 5000 members.

27 DEVOPS JOURNEYS 2.0

CONTRIBUTORS

Richard HaighRichard has extensive experience working in fast paced, agile technology businesses with multinational teams. He most enjoys business change projects and being given problem areas to fix. He draws on experience from his Imperial College London MBA as well as training in SixSigma, Prince2 and Agile methodologies to help deliver them.

He has a sound technical background but can communicate at all levels and is particularly known for his ability to explain the value of technical solutions. Recent projects include a large scale OpenStack build using fully automated infrastructure (IaaS), networking (SDN), platform and software delivery - creating a reference stack for the industry - and continuing to foster the internal DevOps community.

Grant Smith@NextGenDevOpsGrant has created and led high performance Operations teams in some of the largest and fastest growing companies in the UK over the last 18 years and has been at the forefront of the DevOps movement for the last seven years. Grant has driven real collaboration between Operations and Development teams in AOL, Electronic Arts, British Gas and the Department for Work and Pensions by implementing Infrastructure as code and driving application integration from continuous build systems. Grant has delivered game platforms running in the cloud enjoyed by millions of players per day and websites serving a billion page views per month. Most recently he has delivered a high performance, scalable Infrastructure delivery systems at the DWP. Grant is frequently sought out for his cloud and DevOps expertise and can be reached at grant@nextgendevops.com.

Mike DilworthMike has 23 years of experience in the field of IT, where he has held roles within software, systems and network engineering. Mike headed up the infrastructure and operations teams for Thomas Cook's E-commerce Centre of Excellence. At Thomas Cook, as part of his cloud based data centre consolidation and rationalisation strategy, Mike implement key technologies and ways of working that today form the mainstream of DevOps. Following this Mike perfected his work within DevOps at Sainsbury's where he had the opportunity to deliver DevOps at scale as part of a major digital transformation programme. Mike currently works for Capgemini, at the forefront of the DevOps movement, evangelising concepts such as the "Three ways of DevOps", "Technical Value Streams", the "Three Technical Pillars of DevOps" and realising DevOps with a "5 ways to DevOps" process.

28 DEVOPS JOURNEYS 2.0

CONTRIBUTORS

Shabe Razvi@Shabe

From optical network engineering, mobile OS design, build and release engineering, distributed systems development and web operations, Shabe has experienced every side of the fence for over 17 years, applying many of the principles that make up DevOps today. Former Head Of Engineering Services at the Net-A-Porter Group - a leader in DevOps and Continuous Delivery, currently consulting for a number of clients including Digital Catapult and UK Home Office Digital. Part-time adventurer, full-time geek.

Rob Elkin@RobElkin

Rob is the CTO of busuu, the worlds largest social network for language learning, where he leads teams to bring language learning to the world. As well as helping everyone to learn a new language, he is also a founder of AltConf, the Apple Developer community’s biggest independent conference.

Tony Chapman@LinuxRecruiter

Founder and Managing Director of specialist Open Source consultancy LinuxRecruit, who work with organisationsacross the UK, designing their DevOps recruitment strategy and staffing their DevOps Engineering teams. Tony is also co-organiser of the world’s biggest monthly DevOps meetup, the DevOps Exchange in London.

info@linuxrecruit.co.uk @LinuxRecruit020 379 52605