DevOps for IBM Commerce

© 2016 IBM Corporation

DevOps for IBM CommerceMichaelD.ElderIBMDistinguished EngineerChiefDeveloper, UrbanCode

[email protected]@mdelder

2© 2016 IBM Corporation

PleaseNote

2

• IBM’sstatementsregardingitsplans,directions,andintentaresubjecttochangeorwithdrawalwithoutnoticeatIBM’ssolediscretion.

• Informationregardingpotentialfutureproductsisintendedtooutlineourgeneralproductdirectionanditshouldnotbereliedoninmakingapurchasingdecision.

• Theinformationmentionedregardingpotentialfutureproductsisnotacommitment,promise,orlegalobligationtodeliveranymaterial,codeorfunctionality.Informationaboutpotentialfutureproductsmaynotbeincorporatedintoanycontract.

• Thedevelopment, release, andtimingofanyfuturefeaturesorfunctionalitydescribed forourproductsremainsatoursolediscretion.

• Performance isbasedonmeasurementsandprojectionsusingstandardIBMbenchmarksinacontrolledenvironment.The actualthroughputorperformance thatanyuserwillexperience willvarydependinguponmanyfactors,includingconsiderationssuchastheamountofmultiprogrammingintheuser’sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed. Therefore, noassurancecanbegiventhatanindividualuserwillachieveresultssimilartothosestatedhere.


• WhatproblemsdoDevOpsreleasemethodssolve?

• HowcanyoushiftyourculturetoanEnterpriseviewofDevOps?

• PrinciplesandtechniquesofDevOpsfromArchitecturetoAutomatedDeploymentstoVersionControl

Agenda

IBMConfidential


Failures duetoinconsistentdev and

productionenvironments

Bottlenecks tryingtodelivermorefrequentreleases tomeetmarketdemands

Complex,manual,processesforreleaselackrepeatabilityand

speed

Poorvisibilityintodependencies across

releases, resources,andteams

Softwaredeliverychallenges

DailyBuild

MonthlyDelivery

Whodidthis lasttime?

Dave…

Dave’snothereman…

Dev

Prod


HighperformingteamsadoptDevOps

Reference: 2013 State of DevOps Report by PuppetLabs


Aboutphilosophy

Cultureofcontinualexperimentationand

learning

• Productionlikeenvironments• Fullyautomateddeployments• Accelerateddelivery cycles

Build– Measure- Learn

“Successisnotdeliveringafeature; successislearninghowtosolvethecustomer’sproblem.”

7© 2016 IBM Corporation 7

IBMBluemix GarageMethod

PracticesProcesses or actions you & your team perform to achieve an outcome.

TracksA selected set of practices, tools, and toolchains to create a cloud native or cloud enabled solution

ToolchainComplementary & compatible tools that when used together create a flexible solution

Experts & ExperiencesListen to our Experts to learn how we transformed our business

Combines industry bestpracticesforDesignThinking, LeanStartup,AgileDevelopment,DevOps,and Cloudtobuildanddeliverinnovative solutions.

https://www.ibm.com/devops/method

8© 2016 IBM Corporation © IBM Corporation

Newcontentaddedweekly

9© 2016 IBM Corporation © IBM Corporation

Linkage to Bluemix Garage and Lab Services

Method Website – global deploymentUsage analyticsLinkage to Online tools and tutorialsLinkage to Tool Chains for tracksUser Login and profile management

- Tailored views by userMethod personalization & customizationeLearning & certificationsInnovative & new “Method-as-a-Service”

With consultingservices

Self service

Common/Shared:Method Content

Practices & Tracks

Asalargeenterprise, howdoIget2,000developers doingDevOpsonCloud?

GarageMethodforDevOpsTransformation@EnterpriseScale


Accelerate Innovation

ivDeliverCodeThink Run

Systems of engagement

Systems of Record

Enterprise Applications

Production

MostenterprisesneedacombinationofDevOpspatterns

10

API Catalog

Operational Excellence

ivDeliverCodeThink Run Production

Shared Services

Digital Applications

• Coordinatingdifferentteamsworkingatdifferentspeeds

• Balancing between agility,stability, andrisk

• DecoupleacrossAPIboundary,butmanagerelease processwithconsistent approach


ExpectationsofContinuousDeliveryUser Experience Today Emerging

Primary Workload Type Systems of Record (Transactional)

Systems of Engagement (+ Record)Big Data, Analytics, Mobile/Social Channels

Delivery Model Planned Incremental (DevOps)

Development and Operations Team Sizes 100s and costly 10s with built-in DevOps automation

Release Frequency Months to years Days to weeks, based on business opportunity

Integration Frequency Weeks Continuous

Infrastructure Deployment Days Minutes

Time to Value Planned Opportunistic

Operational Model Systems management Built into application, recovery-oriented, continuous availability

Service Sourcing Develop Consume and assemble (public and private)

Feedback Loop Periodic Continuous interaction with clients


Increasetheoperationalawarenessofyoursoftware earlierinthedevelopmentprocess.

WhatdoesDevOpsmeantome?

Howdoyoudothat?§ Architecture

§ AutomatedTesting

§ AutomatedDeployments

§ Production-LikeEnvironments

§ AutomatedReleasePromotionProcess

§ Versioncontrolofallsoftware,automation,andconfigurationWhydothis?

Becausefasterfeedbackloopsenablerapidevolution ofideasandthereforefasteriterationsofyoursoftware

ShiftLeft


“Coolfactor”..seenaspushingtothe“nextbigthing”inourindustryBettercommunication betweenthosewho createandthosewhooperate

(samepeopleinsomecases)Reducedfearofbreakingthebuild/deployment/environment

Failfastbeforeyou“failall”Heavyfocusonexperimentationandlearning

Improvethespeedofyourfeedbacklooptoenablerapidevolutionofideas

WhatarethecharacteristicsofteamswhopracticeDevOps?


§ Architecture

§ AutomatedTesting




§ Versioncontrolofallsoftware,automation,andconfiguration

Aboutournexttopics


Aboutyourarchitecture

• ArchitectureshouldsupportDevOps principlessuchasstagedrollout,operationalinsights,andscriptability

• Eachresourceprovidessomeverypracticaladviceforbuildingsystemswhicharefocusedonreliabilityandfeedbackloops

Experiment!:Websiteconversionrate

optimizationwithA/Bandmultivariatetesting

ReleaseIt!:DesignandDeployProduction-Ready

Software


CommoncomponentsinCommerceconfigurations

Management Center

IBMCommerce

WebSphere ND

Static Content

SwimLane1

SwimLane2www

Management Center

IBMCommerce

WebSphere ND

Static Content

Commerce Data

Content Mgmt System

Commerce Data

Content Mgmt System


CommonmanagedartifactsinCommerceconfigurations

www

Management Center

IBMCommerce

WebSphere ND

Static Content

Commerce Data

Content Mgmt System

.sql.sql

BusinessContent

.war

.earpartial.ear.xml

httpd.conf

iPhone7microsite.war


CommonmanagedartifactsinCommerceconfigurations

www

Management Center

IBMCommerce

WebSphere ND

Static Content

Commerce Data

Content Mgmt System

.sql.sql

BusinessContent

.war

.earpartial.ear.xml

httpd.conf

iPhone7microsite.war

SnapshotofRelatedAssetstobedeliveredasasingleunitofdeployment


Updates&RollbacksinProduction

Management Center

IBMCommerce

WebSphere ND

Static Content

SwimLane1

SwimLane2www

Management Center

IBMCommerce

WebSphere ND

Static Content

Commerce Data

Content Mgmt System

Commerce Data

Content Mgmt System

ApplyUpdates


Automatedtestingiscriticalforautomatedfeedback

• This isTHEHARDESTpartofthewholeexperience

• NOONEisgreatatthis,sodon’tgetdiscouraged

• Facebook’s processforcodereview include statsonhowmanytests areconsistently brokentodecidewhether thetestfailuresarenoteworthy

• Continuously improveyourcoverage, flexibility, anddiscipline

• Consider allaspects oftesting:

• ProgrammaticAPI(JUnit)

• RESTAPI(HTTPUnit, RESTFuse)

• WebUI(Selenium)

• Behaviordriventesting (Cucumber)

• Performancetesting (ApacheJMeter)

• Securitytesting (RationalAppScan &IBMQRadar)


Embeddingunittestsinyourclassesdirectly

• PromotedapproachbyNetflixtoreducefrictionandintroduces limited additionalbytes relativetothirdpartylibraries

• Makes iteasytowritecommands,theUnitTestbecomes thetestharnessandverification

• AlwaystestSuccess() andtestFailure() usecases toensureexpected behavior

importstaticorg.junit.Assert.assertEquals;importstaticorg.junit.Assert.assertNotNull;

publicclassOSDeployEnvironmentCommand extendsOSAbstractOrchestrationCommand<Response>{

publicstaticclassUnitTest {@TestpublicvoidtestSuccess() throwsInterruptedException,

ExecutionException,JSONException {…Future<Response>request=

newOSDeployEnvironmentCommand(…).queue();

Responseresponse =request.get();assertNotNull(response);assertEquals(HttpStatus.SC_OK,

response.getStatus());}

}


TestingYourDeployedApplication– RESTAPI

• WehaveusedRESTFusewhich isanEclipse project

• Captures RESTcallsasannotations, freeingupyourtestmethodtojustverifybehavior

• Requires alittle“hoopjumping”fortesting aseries ofcalls (e.g.POST..PUT..GET)

importstaticcom.eclipsesource.restfuse.Assert.assertCreated;import...restfuse.annotation.Authentication;import...restfuse.annotation.HttpTest;import...restfuse.MediaType;

@HttpTest (method=com.eclipsesource.restfuse.Method.POST, contentMethod="getBlueprintRequestBody",authentications={@Authentication(type=BASIC,user="{keystoneUser}",password="{keystonePassword}")},path="/rest/blueprint",type=MediaType.APPLICATION_JSON

)publicvoidtestCreateBlueprintOK()throwsException{assertCreated(response);}


BehaviorDrivenDevelopment&Testing

• Capturesnatural languagestatements using“Given”,“When”,“Then”clauses

• Eachline ismatched toasnippet ofcodebasedonregularexpression

• Complete codestatementsareexecuted tovalidatebehavior

• Bridgebetween businessgoalsandtechnical details

Scenario:Success– UploadclientdirectoryGivenIprovidevalidcredentialstotheapi clientAndIwanttousetheapi clienttomakerequestsWhenIuploadthedirectory”data/sample-app”withtheclientThenthefilesreturnedmatchestheuploaddirectory”data/sample-app"


TestingYourDeployedApplication– WebUI

• Selenium istheonly“apparent”choicethatwe’vecomeacrossthat’seffective

• Couples automated testtosomedetails ofyourwebUIandmustbekept insync

• We’vedeveloped afewbestpractices aroundlocationelements formaximumflexibility, couldpublishthisifthere’sinterest

publicclassEditorsPage extendsUCDPPage {…@FindBy(id="new-blueprint-button")privateWebElement newButton;…publicvoidcreateFile(String name,Stringid,String[]values){newButton.click();util().waitForElementPresentAndVisible(By.className("new-file-dialog"));WebElement widget=util().findElement(By.className("new-file-dialog"));

Dialog dialog=Widget.init(this, widget,Dialog.class);dialog.setValues(values, true,true);dialog.clickPrimaryButton();

…}…}


Automateddeployments• Visibility andautomatedcontrolofyourapplication deployment process

• Manageapplicationcomponents andversions

• Manageconfigurationsacrossallenvironments

• Offersecure‘self-service’capabilities

• Increasetransparency• Ensuregovernance andcompliancy


IBMUrbanCode forContinuousDelivery

Drive down costReduce amount of manual labor, resource wait-time, and rework by eliminating errors & providing self-service environments

Speed time to marketIncrease frequency of software delivery through automated, repeatable deployment processes across development, test and production

Reduce riskRobust configuration management, coordinated release processes, audits, and traceability

Enabling clients to continuously deliver applications with high quality and low risk

IBM UrbanCode Build is a continuous integration and build management server optimized for the enterprise, designed to make it easy to scale the configuration and management of build infrastructure and seamlessly plug in to development, testing and release tooling.

IBM UrbanCode Deploy enables hybrid cloud by automating the deployment of applications, databases, configurations and containers into development, test and production on prem or in the cloud, helping to drive down cost, speed time to market with reduced risk. With full-stack environment management and deployment automation, users to design, deploy and update full-stack environments for multiple clouds, including Bluemix, Softlayer, AWS, Azure, and virtualized environments.

IBM UrbanCode Release is an intelligent collaboration release management solution that replaces error-prone manual spreadsheets and streamlines release activities for application and infrastructure changes.


Applicationmodel

Reliablycaptureapplicationcontentandconfigurationsettingswithversioningandtraceability


Applicationenvironments

Definewherecomponentsaredeployedandcaptureconfigurationsettingsperdeploymentenvironmentforanapplication


Defineautomatedprocesses foreachapplicationcomponent

Eachcomponentdefinesaseriesofsteps,backedbyplugintasksorcustomscripts,todeployit


Production-LikeEnvironments

PossiblyoneofthemostimportantprinciplesofDevOpsShiftleftanyoperationalconcernsthatleadto

differencesfromDEVtoPROD

https://hub.docker.com/u/ibmcom/#


DeploymentofApplicationsacrossEnvironmentsApplicationsaremanagedfordeployment

Componentsfromtheapp

SpecificversionsoftheComponents’artifactsaredeployed

Compliancykeepstrackofwhethertheactualversionsmatchestheexpected

EachComponentknowsitsrelevantdeploymentprocesses

CompareversionsacrossenvironmentsanddefineSnapshotsofknownworkingsetsofComponents

Viewdeploymentrequestsagainsttheenvironment

https://developer.ibm.com/urbancode/AutomatedReleasePromotionProcess


Inventoryovertime


ConfigurationManagement- SnapshotsCreating a Snapshot

Component Versions

1

1

2

2

3

3

321Web

Mid. Code

DB

Snapshot

3

2

1

Mid. Config 1 2 3 3

Snapshot Deployment

ENVSnapshot

3

2

1

WEB HOST

MID HOST

DB HOST

2

1

1

3

2

X

3 3 1

Contents ofenvironments thatpasstests• containdeployable versionandconfiguration

Deployments arebasedondeltasTheyprovide Automation,Audit,andVisibility


VersionapplicationswithSnapshots

Capturetestedcomponentversionsandconfigurationsettingsasasnapshotthatispromotedtootherenvironmentsforbettervisibilityandcontrol


Updates&RollbacksinProduction

Management Center

IBMCommerce

WebSphere ND

Static Content

SwimLane1

SwimLane2www

Management Center

IBMCommerce

WebSphere ND

Static Content

Schemachangesmaybeharder torollback

App logicchangesjustrollback toaknown good versionorsnapshot

Commerce Data

Content Mgmt System

Commerce Data

Content Mgmt System

SnapshotofAssets


PromotionLifecylces

• Somechanges suchasapplication logicwill followacomplete cyclefromDEVtoPROD

• Othercontent-based changesmayonlyflowfromStagetoPROD

• Allsystem changesshouldflowthroughaconsistent,automated processwithappropriateapprovals inplace

AppLogic&Behavior(Example:Updatesperweekorpermonth)

Content-based changes toDatabase(Example:Nightlyupdates)


Aboutversioncontrol

§ Allapplication sourcecodeandconfigurationshouldbeversioncontrolled

§ Allofyourautomation scripts,configurationfiles, tests,etcshouldalsoversioncontrolled

§ UrbanCodeDeployautomatically understandsversions ofcomponents fromyourbuildprocess,butalsoversions allchanges toconfigurationpropertiesandautomation processes


Codereviewshouldbepartofyourversioningprocess

§ Defineandenforcecodingstandardsearly intheprocess

§ Automatedeployment toproduction-like systems aspartofautomatic verification

§ Eachcommitshould runautomated testingandcodecoverage,andthetoolchainshouldenforcecompliancebeforemergingorpromotingchanges


§ Architecture

§ AutomatedTesting




§ Versioncontrolofallsoftware,automation,andconfiguration

Aboutthisdeck

[email protected] http://linkedin.com/in/mdelder @mdelder

© 2016 IBM Corporation

Q&A

41© 2016 IBM Corporation41

©Copyright IBMCorporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and isprovided AS ISwithout warranty ofanykind, express orimplied. IBMshall notbe responsible for anydamages arising outof theuse of, orotherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have theeffectof, creatinganywarranties or representations from IBMor its suppliers or licensors, oraltering the terms andconditions of theapplicable license agreement governing theuse of IBMsoftware. References in these materialsto IBMproducts, programs, or services donot imply that theywill beavailable inall countries inwhich IBMoperates. Product release dates and/or capabilities referenced in these materials maychange atanytime atIBM’s sole discretion based onmarketopportunities orother factors, and arenot intended tobe acommitment to future product or feature availability inanyway. IBM, the IBM logo, Rational, theRational logo, Telelogic, theTelelogic logo, andother IBMproducts andservices are trademarks of the International Business Machines Corporation, in the United States, other countries or both.Other company,product, or service names maybe trademarks or service marks ofothers.

www.ibm.com/software