CA Agile Requirements Designer 101 - an introduction to the general premise of Model-Based Testing

World®’16

Pre-ConLab:ARD101or“SoHowDoesAnyofThisWorkThen?”BenRiley– PrincipalConsultantPresales– CATechnologies

DO5X29L

DEVOPS

ForInformationalPurposesOnly

Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.

TermsofthisPresentation

Abstract

Thislabwillprovideanin-depthoverviewofCAAgileRequirementsDesignerandthefundamentalconceptsthatunderpinitsapproachtosoftwaredesignandtesting.

CAAgileRequirementsDesignerisaflowchart-modellingtoolusedbyBusinessAnalysts,Programmers,TestersandAutomationSpecialiststodefineandautomaticallytestcomplexsystems.

BenRiley

CATechnologiesPrincipalConsultant,Presales

Agenda

2HOURS

SOMELOVELYINFORMATION

HANDSONWITHTHETOOL

LETSSEEWHATYOUGOT….

WhatDoWeWanttoAchieveFromThis?

1. Deepunderstandingofthetoolandwhatitcando

2. Afullerunderstandingofwhyitisimportanttoitsusers!

3. LargerawarenessoftypicalDesign-Dev-TestChallenges

4. Let’sPlay…

AFewValuePropositionSlides…

Nottoomany,don’tpanic.

QuicklyandsimplyintroduceagileprocessesintoyourexistingWaterfallLifecycles

AgileRequirementsDesigner

CommonChallengesPart1

§ 56%ofsoftwaredefectscanbetracedbacktoambiguitiesinrequirements1

§ Detectingandresolvingproductiondefectsisover40xmoreexpensivethaninrequirements2

§ Industrystandardformanualtestcasedesigncoverageisbetween10-20%

Stats:1and2- BenderRBT,2009.Others:basedonCATechnologies’audits

CommonChallengesPart2

§ Over-testingofsomefunctionsbyfactorsof40isnotuncommon

§ Changerequeststoexistingsoftwarerequirementscantakeuptoasmuchas60%oftheSDLC

§ IfImakeachangeIhavenoideatheeffectthiswillhaveonupstreamordownstreamsystems

ContinuousDevelopment

§ ContinuousIntegration

§ IterativeDevelopment

§ DevOps

§ IsmostlyOps

§ Itstartswhendevelopmentandtestingarecomplete

§ Itshouldstartwhentheuserhasanidea!

CAAgileRequirementsDesigner

§ Thereareplentyoftoolstomanagerequirements§ Thereareplentyoftoolstomanagetestcases§ Thesecanbeconsideredlogisticalsupport

CAAgileRequirementsDesignerisanacceleratortoimprovetheinputtothesetools§ IsaTestCaseDesignandTestCaseOptimizationtool§ IsaRequirementsDefinitiontool§ Cancalculatecomplexityveryaccurately§ CanidentifychangeacrosstheentireSDLC

CAAgileRequirementsDesignergivesyoumultipleoutputsforoneinput

WhatisCoverage?

TheBusinessthinksit’s:– CodeCovered– NumberofTestCovered– TestsRun– Percentageofusecases– AllPairedCombinations

Itisactually:– DesigningSufficientTestsToVERIFYThatTheDesignandCode

CorrectlyImplementTheRequirements– Didyougettherightanswerfortherightreason?Twoormoredefects

maysometimescanceleachotherout– Observability

ASimpleForm EasytoTest?

TheUserKnowswhattheywantTheAnalystspecifieswhatthatis

TheProgrammerwritesthecode

TheTesterteststheprogram

Thefurtherthevisionsdiverge Thegreatertheproblems

ClarityandVisionDuringDevelopmentBusinessAnalyst Programmer TesterUser

Therearelessbugsandtheproductisdeliveredfaster

Thecloserthevisionmeanstheusergetsaqualityproduct

Verify&Clarify ClearLogic TestCases

Thescopeforambiguityismuchless

Theproductisofhigherqualityandisdeliveredfaster

CAAgileRequirementsDesigner:§ DesignTests- Minimum- Optimal

§ DesignData- Findit- Makeit

TheResultsareOftenPoor

“Everythingshouldbemadeassimpleaspossible,butnotsimpler.”

–AlbertEinstein

Wedo knowthatwhenoureyesareopen,ourvisionaccountsfortwo-thirdsoftheelectricalactivityofthebrain—afull2billionofthe3billionfiringspersecond—whichwasthefindingofneuroanatomistR.S.Fixot inapaperpublishedin1957.

TesterProjectManager/DevelopmentManager

BusinessAnalyst/RequirementsManager TestManager

TheRequirementsareconceivedandcreatedinCAAgileRequirementsDesigner.

Relevantprojectmanagersfilloutdetail.ThiscanbepassedbackupthechaintoBas,etc.

TestManagersensureimplementationissuesaretackled.Alsocanbepassedupthechainforverification.

Testersreceivecomprehensive,logicaltestplansandtestcases.

Requirements-BasedTestingModelofTestPlanDelivery– FullTraceability

WhoWouldUseCAAgileRequirementsDesigner?

§ Icanseeanyprocessflowandpredicthowlongthedevelopmentandtestingprocesswilltake

§ MyUserscanclearlydefineaprocessastheythinkofit§ MyBusinessAnalystcandefinetheprocess,whichwillbeverifiedby

theuser,createlogicflowsforprogrammersandtestcasesfortesters§ MyProgrammerscanverifytheprocesswiththeBusinessAnalystbefore

codingandTesterswithtestcases§ MyTesterscanquicklydefinefunctionallogicflowsandverifythemwith

theProgrammer,aswelldefinethesmallestsetoftestcasestoverifytheprogram

§ MyOutsourcingPartnerscangetstablerequirementsandclearlydefinedworkpackages,whichresultinlowerongoingcosts

§ Generatetestscriptsthatprovidemaximumfunctionalitydirectlyfromrequirements

§ Quicklyandeasilyrespondtochange,andavoiddelaysandbudgetover-runs

§ Fullytestsoftwareinthesmallestnumberofrunsrequired,deliveringqualitysoftwareontimeandunderbudget

AgileRequirementsDesigner

JustaQuickTouroftheGUI…

Afewthingshavechanged…

BacktoBasics

SettingupSimpleFlows

ProcessBlocks

§ Processblocksrepresent‘doingsomething’i.e.notadecision,butsomethinghappening– forexample‘MakeTea’or‘RejectLoan’–theydon’thavetoberesultsthough,anotherexampleprocesscouldbe‘LookupCandidate’

§ Processblocksmayonlyhaveoneoutput,astheyarenotmakingadecision

§ Processblockscanhaveunlimitedinputsfromotherblocks(twodepictedhere)

§ TestData,Automation,andmanyotherusefulpiecesofsupportingdatacanbeattachedtoaprocessblock,aswe’llseelater.

DecisionBlocks

§ Decisionblocksrepresentachoiceordecisionmadeintheflowchart,forexample‘Havewater?’or‘Amountofloan?’

§ Baseduponthesedecisions,decisionoutputmayendupindifferentpartsofthediagram,buttheydoesn’thaveto

§ Adecisioncanhaveasmanyoutputsasrequired,andcanhaveasmanyinputsasrequired

§ Alldecisionblockoptionsinthetoolbarcanbeeditedafterthefacttoresembletheothers,theyarejustdefaults

§ TestData,Automation,andmanyotherusefulpiecesofsupportingdatacanbeattachedtoadecisionblock,unlikeprocessblocksmuchofthisisdoneperoutputaswe’llseelater.

AddingBlocks

§ Tocreatenewblocks,simplydragthemontothecanvas(righthandside)fromthetoolbox.

§ Youcanalsoaddnewblocksbyclickingonasymbolontheleft,andclickingsomewhereonthediagramforittoappear.

§ Formostsetups,thethreeblocksyouwillbeusingthemostarelistedhereontheright:(Start/End,Process,True/False).

§ Anydecisionblockcanbeeditedtohaveasmanyoutputsasnecessary(thinkYellow,Red,BlueratherthanTrue/False)sodon’tworryaboutthatfornow.

EditingBlocks

§ Toeditanexistingblock,double-clickit(orright-clickà Edit)

§ Thenumberoftabscanbeintimidating.Mostofthemareforadvanceuseandyou’llneverhavetoknowwhattheydo.

§ Differentblocktypeshavedifferentsetsoftabs(process,decision,subflow,etc.)

Exercise1.GototheShop?

§ Aneasyonetostart,openMakeTeafromtheexamplesfolder§ RefamiliarizeyourselfwithmanipulatingtheUI,doubleclickon

blockstoopenthemup,dragtomoveblocksandarrows,dragnewitemsfromthepallettocreatenewblocks

§ Whenready,makethefollowingchange:“WhenIwantmilk,butdonothaveit,Imaygototheshop,theshopmayormaynothavemilk– forwhateverreason,ifIdon’tcomebackwithmilk,Iwillstillnotmaketea,ifIdogetmilkIwillmaketea.”tothediagram,adjustingthediagramtotakeyourchangesintoaccount

Exercise1.GototheShop?– TickTock theclock’srunning…10minutes…

10 5 0

Exercise1.GototheShop?Getsomethinglikethis?

Exercise2.SuperCoffee andSuperHealth

We’regoingtocreateanewflow,baseduponafreedrinksschemeforcustomersofSuperCoffee ifthey’vedoneenoughrunningatSuperHealthHere’showtheschemeworks:1. IfIhaverunover10milesintheweek,Iearnafreedrink2. IfIhaverun>0milesintheweek,andIamaGoldmemberofSuperCoffee,Iearn

afreedrink3. IfIhaverun5-10milesintheweek,andIamaHealthPlus memberof

SuperHealth Iearnafreedrink,unlessitisaweekend.Ofcoursebyrunning5-10milesIalsoqualifyforNo.2potentially.

4. Ihaverun0miles,Igetnothing.5. Youcanonlyqualifyforonefreedrink,ifyouearnoneforanyreason,youcan

neverinvalidateit.Howmanypossiblepathsdoyouget?

Exercise2.SuperCoffee andSuperHealth – TickTock…20minutes…

20 15 10

10 5 0

Exercise2.SuperCoffee andSuperHealthGetanythinglikethis?

Ifnot,whynot?Comparewithyourneighbours!

Generation,Storing,Options,Coverage

§ Pathsaregeneratedtospecificcoveragetargets

§ Theoptimiserwillalwayschoosetheminimumnumberofpathstocoverwhatithasbeenaskedtocover

§ Withtheexception,of‘Allpossiblepaths’

§ Therearemanyoptionsforthepathoptimiser,fromfilters,toconstraints.Typicallythedefaultsarefine.

§ Letshavealookthrough…

Coverage

§ OneofthemainfeaturesofCAAgileRequirementsDesigneristheabilitytomeasurecoverageatalltimes,foranyparticularsetofpathsforanyparticularcoverageoption.

§ Typicallycoverageachievedbyhumansispoor.

§ We’regoingtoletthecomputerplaychessforus.

§ Letshavealookatthecoveragewindows…

StoringPaths,etc.

§ Gobacktomaketea,oroneoftheflowsyouhavecreated.

§ Generatetargetingthe‘allnodes’coverageoption,storethepaths,andopenthecoveragewindow,whatisthe%?Openthe‘more’sectiontoseewhat’sactuallybeingcoveredandwhatisnot.Now,finishyourcoverageforAllin/out,usingthe‘addtoexistingpaths’option.

§ Now,we’llhavealookatmanagingstoredpathtypes…

ChangeImpact,theHeartoftheTool

§ TheprincipleofCAAgileRequirementsDesigneris,“Letsmaintaina40-blockdiagram,ratherthanmaintain1000softests/datapoints/scripts”

§ WhenyoumakeachangetothediagraminCAAgileRequirementsDesigner,theImpactanalyserwillallowyoutomaintainthetestcases,andthusthedataandscripts- itwillcalculatewhatpathsnolongermakesense,andgiveyouoptionsastowhatyou’dliketodowiththem,including,repairthem.Foranyonewhohasexperiencedmanuallymaintainingtests,youmightrecognisehavingthetooldothisisamuchbetteridea!

Exercise4.Let’sMakeaChange

1. Openuptheflow‘SongStreaming’2. Cleardownthetestcasesinthestoredpathtype‘testcases’3. Storetheoptimal‘allpairs’pathsforthisflow.Makesurethe“Trytofindpathsin

previousversion”isUnchecked4. Now,implementthechange“WhenIamnotconnectedto3g,orthe3gisnot

fastenough,Ishouldcheck(beforefailingasbefore)ifIamconnectedto2g,IfIam,Ishouldwaitforthebuffer– else,errorasbefore”

5. Gobacktoviewyourstoredpaths,examinewhatthepathimpactanalysersays,deletetheeffectedpaths

6. Nowgoandlookatthecoverage,you’llnoticeit’sdropped!7. GeneratebothA)thenewoptimalsetofpathsandB)justtheadditionalpaths

requiredtofinishthecoverage.

ARealWorldScenario?

§ Therealworldisn’tsoeasy.

§ Goalistohaveafeasiblemodel,notaperfectone.

§ Presentationmatters.

§ Logicallyimplementwhatitcouldmean,somedefensibleinterpretation.

TestDatainCAAgileRequirementsDesigner

§ Howdoesitwork?

§ Ingeneral,theideaistosetdatabasedonwherewegointhediagram,ratherthanworkingitalloutafterwards.

§ Thereasoningisthis:It’seasiertomake40dataassignmentson20blocks,ratherthanmaintainmillionsofdatapointsmanually!

§ Theotheradvantage,asalludedtobefore,isofcoursethatnow,whenwechangeourdiagram,ourdatacanautomaticallyreacttoo!

§ Letmeshowyou…!

Exercise5.Let’sSetUpSomeData

1. Openup‘Maketea’,ateachdecisionpoint,addthenecessarydataassignmentsforeachoutput!

2. Now,generateandstoresomepaths.3. Gotothetestdatatableandexamineallofthedatathatresults.4. Nowthisisallboring,hardcoded(toanextent)data,butwe’llfix

thatlater.5. Playaround,addmorevariablestosomeofthedecisionoutputs,

changetheirvalues,andhavealookattheeffectinthetable.6. Now,changethediagraminsomeway,andreactaswedidbefore,

nowconsideryournewdata,neat,right?

Exercise6.Let’sSetUpaLoan

1. Soletsmakesomething(mildly)morerealistic.2. Setupaflowwherewearegoingtobegeneratingmanydifferent

typesofloan.3. Weneedadecisiontodecidetheproductcode:CT50,CT33,CT48,CT49,

wemustalsodecidetheamount(fornow,wewillhavetheoptions<1,000,1,000-10,000,10,000-100,000,>100,000)

4. Wewillalsoneedadecisiontosetthetermoftheloan,(1yr,3yr,5yr)5. Wewillneedadecisiontosettheinterestrate(<1%,1-5%,>5%)6. Allofthedecisionsuptonowjustflowintothenext,butfromhere,we

needatruefalsedecision‘backdate’,ifitistrue,wemusthaveadecisiontochooseabackdateTime (1days,2-359days,360days)elsedonot.Nowbothpathsgointoa‘calculateresult’block.

Exercise6.Let’sSetUpaLoan

§ Again,generatesomepaths,andhavealookatyourlovelydata!

§ Butthereareissues…howdowedealwithranges?Howdowemakedatareferentiallyintact?Howdowemakeitmoredynamicandrealistic?

§ …Dataexpressions!

TheDataPainterwindow

§ TheDataPainterwindow

Exercise7.Let’sGenerate:

1. Arandomnumberbetween1and20002. ARandomnumberofdigitsbetween3and7inlength,

paddedtotheleftwith0stoalengthof73. Multiplytworandomnumberstogether4. Addarandomnumberofdaystothe~CDATE~01/01/2009if

cdate misbehaves5. Randomlychooseafirstnamefromaseedlist6. A@percval@ofanytwostrings,or,ifyouwanttobefancy,

twofunctions!7. Anemailaddressbasedonfirst.lastname@gmail.com (useseedlists)

Let’sGenerate

§ Butwhataboutthiscolumn?

Exercise8.ReferentialandRandom?

1. Usingtheknowledgeofthefunctionswe’vejustbeenusing,goandfixoutloanflow,sothateverythingisparameterised (i.e.therangesallhavearandomexpressionforthembetweenthatrangeetc.)

2. Storepaths,resolvethetestdata3. Now,gobacktoourblock‘calculateresult’4. UsingtheformulaPayment =(Interest rate*amount )/(12*100)+(amount /(

term*12))setupthevariableandexpressionfor‘MonthlyPayment’remembertoreferencethecolumnsforamount,term,interestrateetc.

5. Makesurethisresolvesproperly!6. Examineyourdatainthetable7. Wewon’t,butifwemadeachangetothiskindofflow,thechangeswouldreflect

uptoourdatatoo.

Exercise8.ReferentialandRandom?

1. Here’soneImadeearlier…

2. Testerssometimesspenduptohalftheirtimesearchingorcreatingappropriatedatafortheirtests!Nowwecanagainmaintainafewblocks,ratherthan10000sofdatapoints,andrefreshourdataondemand!Neateh?

3. Thiswillbeapartofourkillerdemo(ofcourse,notinasmuchdetail).

Filters

§ Forthemostpart,filtersareusedtoeasilyselectorseepartsofthediagram

§ Intheverylatestversion,wecanalsoapplyfilterspriortoandpostoptimisation

§ Letsconstructsomefilters!

Exercise9.Filters

1. Constructafiltertoonlyseetheblockswhichhavetestdatadefinedforthem

2. Changetheimportanceofthe‘don’tmaketea’blocktocritical,thencombinethefilterabove(settono)withafilterwhichselectsonlycriticallyimportantblocks

3. Storeafilterforlateruse.

Exercise9.Filters

1. Constructafiltertoonlyseetheblockswhichhavetestdatadefinedforthem

2. Changetheimportanceofthe‘don’tmaketea’blocktocritical,thencombinethefilterabove(settono)withafilterwhichselectsonlycriticallyimportantblocks

3. Storeafilterforlateruse.4. Whywouldwebotherwiththese?

ImportandDeduplication

CAAgileRequirementsDesignercanimporttestcasesfrommanydifferentsources!Including…

§ HPEALM

§ JIRA

§ CAAgileCentral

§ VERSIONONE

§ EXCEL

ImportandDeduplication

§ Sowecanimporttestcases?What’sthepoint?

§ Webringineachtestcaseonastepbystepbasis,typicallythiswillmeanthereduplicatestepssharedacrossmanytests.

§ InCAAgileRequirementsDesigner,youcanhavethetoolsuggestduplicateblockstoyou,suggestionsyoucantheneitheracceptorrejectonacasebycasebasis.

§ Onceyou’vemarkedtheduplicates,thetoolcandeduplicate theblocksforyou,butkeepingeverypathintact– thisleavesyouwithaflowthatithasinferredfromthetestcasesyougiveit!

§ Thisisanimportantaccelerator,butnotperfect!

Exercise10.BringinSomeTestsandDedupe!

§ Inanewflow,gototheimporttab,click‘excel/generaltestcases’– selectthe.docx ‘Doc_procedure_testcases’

§ MaptherelevantfieldstofieldsinCAAgileRequirementsDesigner

§ HitImport!

§ Openthededupewindowunderneathtestfactory

§ Haveaplaywiththesensitivitysettings

§ Markalltheduplicates,thenhitde-dupeinthemainAgD window.

§ Re-layouttheresultingdiagram,makeanyfixes/additions– measurethecoverageofthestoredpaths,canwedoitwithlesstestcases?

Constraints

§ Constraintsallowyoutolayer‘rules’ontopofthediagraminordertoachievespecificfunctionalitywithoutanunreasonablylargeflow

§ Imagineadecisionnearthetopofalargediagram,whichcanbetrueorfalse,butmuchlaterinthediagramwemakethesamedecision– iftheyhavetolineuphowwouldwedothiswithoutconstraints?Wouldwedoublethesizeofthediagram?

§ Cananyonethinkofwhythesemaybenecessary§ Letshavealookatthemforreal…§ Ingeneral,youshouldonlyuseconstraintswhenthesamelogiccannot

(easily)beachievedbyflowlogicalone,andthereisusuallyawaytodothat.

Exercise11.BuildaUsefulConstraint

§ Openour‘loan’flow,createaconstraintsuchthatwhenevertheproductisCT50,thebackdatemustbe360

§ Ifyougeneratepathsnow,youwillnoticetherewillbeless

§ Youcanoptionallyhavethepathexplorerignoreconstraints

§ Feelfreetobuildupmorecomplicatedconstraintsinthe‘pathhints’dialogue.

Extras

§ Versioncomparison

§ Richtextimporter

§ Worddocbuilder

§ Clonescopiesandduplicates

§ Customexcelexport

§ Propertieswindow

§ Flowvalidation

§ Diagramexports

§ Accelerators

§ Shapes

§ Loops

Questions?

RecommendedSessions

SESSION# TITLE DATE/TIME

DO5X23EPre-ConEducation:RequirementsareKing– BetterRequirements=BetterSoftware

11/15/2016at11:00am

DO5X27S What'sNewinCAAgileRequirementsDesigner? 11/17/2016at12:45pm

DO5X40S CaseStudy:WhyRabobankWasInspired 11/17/2016at3:45pm

MustSeeDemos

DataComplianceandTesting

CATDMTheater5

DeliverTestDataFaster

CATDMTheater5

UseModelBasedTesting

CAARDTheater5

AchieveMaximumTestCoverage

CAARDTheater5

Stayconnectedatcommunities.ca.com

Thankyou.

DevOps– ContinuousDelivery

FormoreinformationonDevOps– ContinuousDelivery,pleasevisit:http://cainc.to/PiTFpu

CA Agile Requirements Designer 101 - an introduction to the general premise of Model-Based Testing

Technology

E3 The Premise

Premise and Preview

Curriculum Made Relevant with Agile Presented by Marian Willeke, Director of Online Education Scott Marsee, Lead Instructional Designer Ohio Christian

(on-premise), , and

The Way We Are...The Way We Are Agile, Lean and DevOps Self-managing organisations Mark Williams Agile Coach Andrew Holm Organisational Designer Cross-functional teams Both build and

Integrating Agile Studio with Other Applications€¦ · Introduction Agile Studio is integrated with Pega Platform 7.3 to allow teams to change their application’s records in Designer

Agile UX Research Practices Applied · –Agile – You can complete the study in One day! –Collaborative – The team (UX researcher, PM, Designer, Developer) needs to come to

N REMISE On Premise Off Premise Weddings Business Meetings

Moral Premise Sample

DESIGN TIPS FOR THE NON DESIGNER - Agile Australia 19 · DESIGN TIPS FOR THE NON DESIGNER Because anybody can create awesome experiences for users 1. ... USER JOURNEY SYSTEMS PROCESS

Off Premise Catering

premise - bcmj.org

2. THEORETICAL PREMISE

Gensan Buy Premise

On–Premise Monarch Beverage Importance of the On-Premise Sales for off premise start on-premise Focus = Young adults 21-34 What young adults drink today,

From Requirements to Test Automation in Minutes Manual ... · Baseline Agile Designer Test Cases Created 14 17 Time Taken 5 hours 2 hours Test Coverage - % 16% 100% • Agile Designer

Public Premise

Inspection Est Number Premise Name Premise Address 1 Type … · 2020-02-14 · Est Number Premise Name Premise Address 1 Inspection Type Score Inspection Date Violation List Followup

Shopping by Premise

Thesis - Premise