Upload
ca-technologies
View
94
Download
0
Embed Size (px)
Citation preview
World®’16
Model-BasedTestingforAchievingMaximumCDPipelineAccelerationAlexMartins- CTOContinuousQuality– CATechnologiesBenJohnson-Ward- ARDJediMaster– CATechnologies
DO5X29S
DEVOPS
2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.
Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.
ForInformationalPurposesOnlyTermsofthisPresentation
3 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Abstract
MostsuccessfulDevOpsteamshavelearnedhowtoworktogetherseamlessly.Then,theygettoapointintheirjourneywheretheylookforawaytoachievemaximumsoftwaredevelopmentlifecycleaccelerationandtheythinkaboutcontinuousdelivery.Oneofthethingsmostorganizationsthatadoptcontinuousdeliveryareoverlookingistheaccelerationoftheentireapplicationlifecycle,notjustthebuild-test-deploy-operatecycle.Whataboutacceleratingtherequirementsgatheringprocess?Thesearestillcommunicatedthroughthewrittenlanguage—thesamewayit’sbeendoneacrossdifferentteamsforthepast30years:WritteninWorddocuments,Excelspreadsheetsorinrequirementsmanagementtools.Usingthiscompletelymanualprocessleadstoambiguityandintroducesapproximately50-percentofdefectsintheapplicationcode.
ThispanelofexpertsfromQAorganizationsandQAproductswillexplorethedifferentwaysthatmodel-basedtestinghasbroughtmaximumaccelerationintothecontinuousdeliverypipelineformanydifferentorganizations.
AlexMartins
CATechnologiesCTOContinuousQuality
4 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Agenda
CHALLENGESOFACHIEVINGCONTINUOUSDELIVERY
HOWOTHERSDESIGNTHEIRPROJECTS
CLOSING
LEVERAGINGDESIGNTECHNIQUESINTOSOFTWAREDESIGN
HOWITWORKS
Q&A
1
2
3
4
5
6
5 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Developers ReleaseTeam Operations
TESTBUILD DEPLOY
Testers
OPERATE
GapinCDPipelineAccelerationInitiatives
PerformanceTesting DeploytoProduction
CustomerExperience
Code|Integrate|Build Functional&UATesting
idea
Extremeaccelerationpossiblethroughbettertoolsandinfrastructurethatsupporttheculturalchangeontheteam
§ Wetypicallyfocusonaccelerating
code-to-deploymentactivities
§ TheIntakeprocessusuallyremainsa
bottleneckandmostlymanual
§ MostlymanualtogofromIdeato
Code(forTDD)andtoTestCasesand
TestAutomationscripts
§ HowtoautomatetheCDPipeline
Intake process?
Requirement
TestCases
TestData
VirtualServicesTest
Automation
6 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CoreChallengestoAchievingContinuousDelivery
Requirements
Development
QA/Testing
Operations
Release
70%ofalltestingisstillmanual.5
63%oftestersadmittheycan’ttest acrossallthedifferentmobiledevicesandOSversions.6
50%oftimespentlookingfortestdata.7
79%ofteamsfaceprohibitiverestrictions,timelimits or accessfeesonneeded3rdpartyservices.3
X X X
64%oftotaldefectcostoriginateintherequirementsanalysisanddesignphase.1
50%ofdeveloperstimeisspentfixingandfindingdefects.3
?
80%ofteamsexperiencedelaysindevelopmentandQAdueto unavailabledependencies.4
?
57%aredissatisfiedwiththetimeittakestodeploynewfeatures7
!
20%ofcompaniesareperforming fullreleaseautomationtoday.8
5PerfectoMobile2014BenchmarkSurvey,WhyMobileAppsFail20146Source:BloorResearch,20147ForresterGlobalModernServiceDeliveryBenchmarkOnlineSurvey20148TheForresterWave™:ApplicationReleaseAutomation20159AnkitOberoi.5ReasonsVisitorsLeaveYourWebsite.WebsiteMagazine201410EMA™WhitePaperAPMEssentials:NavigatingtheMazeofEndUserExperienceSolutionsPreparedforCATechnologies2016
40%ofuserswillabandonyourappifittakesmorethanthreesecondstoload.9
70%of delaysinapplicationdeliveryareattributedtonewandchangingrequirements.2
50%ofcompaniesaremanagingcloudserviceswithhomegrowntoolssuchasscripts,macros,etc.Evenfewerhavemobile-specific monitoringcapabilities.10
? ?
1HyderabadBusinessSchoolGITAMUniversityQualityFlaws:IssuesandChallengesinSoftwareDevelopment20122ForresterBrief:SoftwareRequirementsPracticesAreRipeForDisruption20143StudyconductedbytheJudgeBusinessSchoolatCambridgeUniversity 20133Voke MarketSnapshotServiceVirtualization20154BloorReportAutomatedtestcasegeneration2014
7 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
ClarityandVisionThroughouttheSprint– GoalBUSINESSANALYST PROGRAMMER TESTERUSER
Thecloserthevisionmeanstheusergetsaqualityproduct.Therearefewerbugsandtheproductisdeliveredfaster.
8 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Let’slookathowothersdesigntheirprojects
9 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
DesigningaHouse
§ Customerdescribeshisdreamhousetothearchitect:
§ WeareabouttohaveourfirstbabyandIwanttobuildabeautifulhouseformyfamilywherewecanalllivecomfortably
§ Ineedthehousetohave2bedrooms,1kitchen,1livingspace,1diningroom,1TVroomand1bathroom
§ Thehouseneedstofeelspaciousandwarm
10 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GeneratesLOTSofQuestions
§ HowmanySq ft shouldthehousehave?
§ Howmanyfloors?
§ Howbigeachroomshouldbe?
§ Whereshouldeachroombepositioned?
§ Whatkindoflightfixturesshouldweuse?
§ Whatdoesbeautifulorcomfortablemean?
§ Endlessquestions...
11 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
APictureIsWorthaThousandWords
§ OutsideoftheITworld,Architecturecanmean:– Ageneraltermtodescribe buildings andotherphysicalstructures.[1]
– Theartandscienceofdesigning buildings and(some)non-buildingstructures. [1]
– Thestyleofdesignandmethodofconstruction ofbuildingsandotherphysicalstructures. [1]
§ ThecustomerdrawsaninitialsketchwhichisthenfurtherrefinedwiththeArchitectinmultipleiterationsuntilthecustomerissatisfiedandaccepts it[1]ShorterOxfordEnglishDictionary (1993),Oxford,ISBN0198605757
12 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Persona1– TheHomeOwner
13 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Persona2– TheArchitect
14 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Persona3– TheInteriorDesigner
15 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Persona4– TheElectricalEngineer
16 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
OtherPersonasintheValueStream
Plumbing
HomeAutomationdesign
Sounddesign
Exteriordesign
Landscaping
Andmore…
Questions
§ Couldwereallydesignahousebyjustusingwords?
§ Ifso,woulditbefaster?
§ Howtotracktheimpactacrossthelayersifthehomeownerchangeshismindthroughthedesignprocess?
17 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Contractor1– DesigningaPlantCADSoftware
18 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Contractor2– DesigningaPlantCADSoftware
19 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Contractor3– DesigningaPlantCADSoftware
20 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Computer-AidedDesignSoftware
§ Assistsinacceleratingdesign,butmostimportantly:
§ Enablesmultiplepersonastoworkonthesamedesign
§ Enableseachpersonatoworkonaseparatelayerofthedesign
§ Keepsalllayerstogetherwithfulltraceability
§ Automaticallyidentifiestheimpactofanychangesmadetoonelayerandpromptsthelayerownerforanaction
21 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
CivilEngineeringandSoftwareEngineeringTheBestFromBothWorlds
CivilEngineering
Design
Construction
SoftwareEngineering
Design
Construction
§ MultipleProvenPractices
§ ChangeFriendly
§ SoftwareAssisted
§ ChangeFriendly
§ HistoricallyProvenPractice
§ HardandCostlytoChange
§ ManualDesign§ HardandCostly
toChange
22 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
AligningtoAgileSoftwareDesignBuildingtheArchitecturalRunway
CivilEngineeringDesign
SoftwareEngineeringConstruction
§ MultipleProvenPractices
§ ChangeFriendly
§ SoftwareAssisted
§ ChangeFriendly
§ SWDesignPatternsoriginatedfromCivilEngineeringandArchitecturalDesign,byChristopherAlexanderin1977
§ Model-DrivenVisualizationofRequirements– alsoborrowedfromCivilEngineering
§ ServiceVirtualizationconceptsinspiredbypracticesadoptedinCivilandMilitaryEngineeringwhenworkingwith
multiplecontractors– understandtheinterfacespecsandbuildtothatspec.Flexibilitytochangeimplementation
23 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Howdoweleveragetheseprovendesigntechniquesintosoftwaredesign?
24 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
TheFlightBookingPath
§ As apassengerIwant tosearchfordomesticflightsSothat Icanbuytheonethatmatchesmyneed
ASimpleEpic
§ AcceptanceCriteria– Icancompletethepurchasein
nomorethan6screens– Icanseealistofallflights
availablebasedonmycriteria– Icanchoosemyseats– Iwillnotseeanypromotions
§ DefinitionofDone– Userabletonavigatefrom
searchtopurchasecompletionscreen
25 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
GeneratesLOTSofQuestions
§ Doyouwanttoviewthedepartureandreturnflightsasacombinedpair,ordoyouwanttoselecteachindependently?
§ Whenshouldtheuserselectseats?Afterselectingthedepartureandreturnflightsorjustbeforepayment?
§ Doestheuserneedtobeloggedontodothesearch?Ifnot,willtheuserhavetheopportunitytologonatsomepoint?
§ Endlessquestions…
26 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
APictureIsWorthaThousandWords§ IntheITworld:
– Amodelinsystemsengineeringandsoftwareengineeringisastructuredrepresentationofthefunctions(activities,actions,processes,operations)withinthemodeledsystemorsubjectarea.[1]
– Thepurposesofthemodelaretodescribethefunctionsandprocesses,assistwithdiscoveryofinformationneeds,helpidentifyopportunities,andestablishabasisfordeterminingproductandservicecosts[2]
§ TheProductOwnerdrawsaninitialsketchwhichisthenfurtherrefinedwiththeModelerinmultipleiterationsuntiltheProductOwnerissatisfiedandaccepts it[1]FIPSPublication183releasedofIDEFØDecember1993bytheComputerSystemsLaboratoryoftheNationalInstituteofStandardsandTechnology(NIST).[2]Reader'sGuidetoIDEF0FunctionModels
27 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Persona1– TheProductOwner
28 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Persona2– The(CAD)Modeler
§ Streamlinesunderstandingacrossallteamsinthevaluestreambyremovingallambiguityfromthescope
§ Foundationallayerotherswillbuildontopof(singlesourceoftruth)
29 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Persona3– TheDataEngineer
30 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Persona4– TheServicesEngineer
31 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Persona4– TheTestAutomationEngineer
32 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
OtherPersonasintheValueStream
UXDesigner
PerformanceEngineer
EnvironmentManager
ReleaseManager
OperationsEngineer
Andmore…
Questions
§ Couldwereallydesignanapplicationbasedonwrittenlanguage?– Yes,buthowdidthatwork
forussofar?
§ CanwetakeCDtothenextlevelofSpeedandQuality?
33 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
§ BusinessFlowsbasedonUserStories
§ AlternateScenarios§ NegativeScenarios
§ DataRequirements§ DataParameters§ TestDatamart
connection
TestData
Model
LayeredapproachtoContinuousDelivery
§ ExternalInterfacesRequirements
§ APIDefinitions§ ServiceVirtualization
Services
§ Frameworklinkage§ Keywordlinkage§ Actionsontheapp
TestAutomation
OverlayingallLayers RolesandBenefits
§ PerformanceSLAs§ LoadTesting
Scenarios§ UserProfiles
PerformanceTesting
••Drawsandmaintainsbusinessprocessflows••Baselinesunderstandingacrosstheteam••Removesallambiguities
Modeler(PO/BA/Tester)
••Findandreservedataautomatically••Matchtestdatatoeachtestcase
TestDataEngineer
••Nodependencyonexternalinterfaces••AccelerateCodeDevelopment
ServicesEngineer
••Automaticcreationofscripts••Automaticimpactanalysis
TestAutomationEngineer
••Automaticgenerationofloadscenarios••Automaticcreationofscripts
PerformanceEngineer
34 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Let’stakealookathowitactuallyworks…
35 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Questions?
36 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
RecommendedSessions
SESSION# TITLE DATE/TIME
DO5X47S 'Shift-Right'- RapidEvolutionwithDesignOps 11/16/2016at3:45pm
DO5X27S Whats NewinCAAgileRequirementsDesigner 11/17/2016at12:45pm
DO5X40S CaseStudy:WhyRabobankWasInspired 11/17/2016at3:45pm
37 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
MustSeeDemos
UseModelBasedTesting
CAAgileRequirementsDesigner- 515-DevOpsTheatre5
DevOps5Theatre
AchieveMaxTestCoverage
CAAgileRequirementsDesigner- 516DevOpsTheatre5Theatre
ImproveDataCompliance
CATestDataManager512- DevOpsTheatre5
DeliverTestDataFaster
CATestDataManager511- DevOpsTheatre5
38 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD
Stayconnectedatcommunities.ca.com
Thankyou.
@CAWORLD#CAWORLD ©2016CA.AllRIGHTSRESERVED.39 @CAWORLD#CAWORLD
DevOps– ContinuousDelivery
FormoreinformationonDevOps– ContinuousDelivery,pleasevisit:http://cainc.to/PiTFpu