ImprovingTeamThroughputGettingMoreAccomplishedThroughDifferentTestArchitecture

WhoAmI?

DustinWilliamsDirectorofTechnicalOperationsManifestSolutionsdwilliams@manifestcorp.com

WhatAreWeGoingtoTalkAbout?

• Throughput:Therateatwhichasystemachievesitsgoal.

• Goal(value)maybeincontrasttoOutput(inventory)

What’stheGoalforQA?

• MoreTests• FindDefects

AstheCrowFlies

• Moreoutput(miles)doesnotequatetobeingclosertothetarget.

• Goingfastermaynotgetyoutheresooner

So…

What’stheGoalforatest?

MitigateRisk

Atestdeliversvaluebymitigatingriskthroughthepreventionofdefects.

ThroughputinOurContext

Mitigatingthemostriskwiththefewesttestspossible,creatingthosetestsasefficientlyaspossible.

Don’tforget…

Thetestshavetoholdvalueovertime.

Wewillnot bediscussing… Wewillbediscussing…

Sowhatexactlyisarchitecture?

• Decisionsaboutthingsthatarehardtochangelater.

HowCanWeGetBetterThroughput?

• Applyingdifferentpatternsintheautomationcodetoimproveefficiencywhenbuildingnewtests.

• Usingtoolstoidentifycodethatmayshortenthelifeoftests.• Improvingthedivisionoflabortoleverageexpertise.

SignsforOpportunity

• Addingnewtestsbreaksoldtests• Thedecisiontoautomateisbasedonwho’savailable• Moretestsdoesnotequatetofewerdefects• “We’restillautomating”• Whenatestfailswehavetofixatest,nottheappbeingtested• TheUIchanged,sothetestsneedtobere-written• Etc.,etc.

ArchitectureConsiderationsinTestsvsApps

• Testsrunonce,applicationsliveforever• Appsneedtorespondquickly,testscanmovealittleslower• Testsdon’tneedtobeascarefulaboutmemorymanagement• Appsaretested,testsareinherentlytrusted

ANewArchitecture

ToolStuff

ToolStuff

• Hooks• Stepdefinitions• Specs• Scripts

• Codethatisonlyusedtomakethetesttoolfunction

TestLogic

TestLogic

• Themainworkofthetest• Navigatingthroughscreens• Specifyingtestdata• Collectingresultstobeverified

• Thisisthetest

ComplexDetails

ComplexDetails

• Howtogetdataoffofapage• Convertingdatesbetweenformats• Retrievingdatafromthedatabase

• Stuffthatrequiressoftwareengineering

Separatingtestcodefromtoolcode

• ‘test_lib’folderholdsallofthethingsthatmakeupthetests

• Rspec andCucumbercanbothaccesstest_lib withoutduplicatingcode

• Changingoraddingatoolwon’timpactmostothercode

Methodsthatreturndata,notstructure

def search_results #returns Element dataself.results_table_element.tds

end

def search_results #returns Text dataself.results_table_element.tds.collect do |cell|

cell.textend

end

Abstraction

• Thepreviousexampleusessomethingcalledabstraction.• SeparatestheWHATfromtheHOW• Keepsthedetailshidden

• Doyouhavestepdefinitionswithalotof‘_element’

LayersofAbstraction

StepDefinition

Up:Whatbusinesscapabilitiescanbeused(createaclaim)

Down:Whatthingsneedtobedone

Interaction

Up:Whatthingscanbedone(loginasX)

Down:Whatdataisneeded

PageClass

Up:Whatdataisavailable(search

results)

Down:Touchingtheapp

TestYourTests!!!!!!!!!!!!!!

module DateConverterdef self.convert_to_long_format(short_format)time = Time.strptime(short_format, '%m/%d/%y')time.strftime("%B,%e %Y")

endend

describe 'date converter' doit 'converts short format to long format' doactual = DateConverter.convert_to_long_format('2/1/16')expect(actual).to eq 'February, 1 2016'

endend

AssertionsthatDescribetheFailure

Knowingthattherewasafailure Knowingwhatthefailurewas

describe 'list size comparison' dolet(:actual) {['a', 'b', 'c']}it 'tells me there was a failure' doexpect(actual.size == 4).to be true

endend

expected truegot false

describe 'list size comparison' dolet(:actual) {['a', 'b', 'c']}it 'tells me what the failure was' doexpect(actual.size).to eq 4

endend

expected: 4got: 3

Toolsthatcanhelpfindproblems

Reek– findssmells• Smellsmakethecodehardertoextend

Excellent– identifiescomplexcode• Complexcodeismoreexpensivetomaintainovertime

Flay– detectsduplication• Duplicatedcodeincreasesmaintenancecostandraisestheriskoftestdefects

TypesofDevelopers

• Rareneed• InventnewparadigmsScientists

• Occasionalwork• Combiningpatternsinnewwaysforneworcomplexproblems

Engineers

• Mostofthework• Applyingknownpatternstocommonsituations

ApplicationDevelopers

DivisionofLabor(CucumberExample)

Gherkin StepDefinitions Interactions Page

Classes

ComplexCode

Engineering

PlainEnglish

ApplicationDevelopment

BreakingDowntheProblem

Writeoutthescenarios• Getanideaofthefullscope

Writeoutthesteps• Getthestepstructurecorrect,makesureyouknowwhatthestepneedstodo.MakesureyoustartwiththeThens andmovetotheGivens

Stubouttheworkflows•Makethese“work”bypretendingtodothings,likegotothepages.Thishelpsmakesurethestructureofthetest isvalid.Italsoallowsyoutoworkatestfromtoptobottomintheeventthescreensdon’tyetexist.

Implementthestuffyoustubbedout•Makethe“working”testactuallydowhatitneeds.Sometimes thismeansworkingwithanexistingpage,othertimesitmeansworkingwithdeveloperstodrivethecreationofapage.

BIGGEREXAMPLE

WrappingUp

• 3newtoolstoanalyzethecode• Enhanceexistingtools

• Focusonwherethehardworkisandisolateit• Breakdownproblemstoseparatetestsfromengineeringissues

• Keepyourtestcodetestedwherepossible• Failingtestsshouldbecausedbyabrokenapp

FurtherReading

• https://en.wikipedia.org/wiki/Throughput_(business)• https://en.wikipedia.org/wiki/Theory_of_constraints• http://www.rubydoc.info/gems/reek• http://www.rubydoc.info/gems/flay/2.8.1• https://github.com/simplabs/excellent• https://en.wikipedia.org/wiki/Software_architecture