Automating Device Certifications with Robot Framework · Robot Framework Introduction • Open...

Automating Device Certifications with Robot Framework

Pratik Lotia

Pratik Lotia

RobotFrameworkIntroduction•  Opensourcegenerictestautomationframeworkforacceptancetesting

•  KeyworddrivenapproachsupportedwithseverallibrariesinPython&Java

•  Idealimplementationwithhighleveltestspre-writtenandnetworkengineersusingkeywordstodevelopframework

•  Notspecificallymadefornetworkbasedtesting

•  Datadriventestcases•  OSandapplicationindependent

2|

RobotFrameworkMajorComponents

3|

Framework

Definedependencies&pointersTestCases

Ø  KeywordsØ  Variables

PythonCode

Functions

Ø  GenericØ  SingletaskØ  ArgumentsØ  ReturnValue

DataSet

ConfigurationFilesPointers

Server/VMSSH/NETCONF

Router/Firewall/Switch/LoadBalancer

Structure

4|

FrameworkFormat

Ø Extensionbased•  HTML•  TSV–spreadsheet,programmatic•  Plaintext•  reST(HTMLcompiled)

5|

FrameworkStructure•  Modularmodel•  Structurecombines

§  Settings§  Pre-testsetup§  Testcriteria§  Post-testcleanup

•  EachTesthasatrue/falseoutcome•  EachTesthas1ormorefunctions•  Top-downapproachforTestCase

§  Onefail,allfailmodel

6|

FrameworkSample

7|

RobotCommandOptions•  robot/path/to/file.robot•  Optionsto:

§  Setdocumentation§  Setsuite,reportname§  Settags,variables§  Rerunfailedtests§  Run/excludecertaintests§  Setlogginglevel,outputlevel§  Settimestamp§  Errorhandling

8|

StyleConventions

9|

Field Convention Example/CommentsVariable Nameinside${} ${mgt_ip_addr)FunctionName Keyword(s)withorw/

spaceCheckInterfaceConfigcheck_interface_config

PassingVariables

Leave>4whitespacesbetweenfunction&variable

ShowInterface${mgt_ip_addr}${user}${passwd}

Notes Under[Documentation]intestcases

Testcase:checkinterfaceconfig[Documentation]LoadandverifyIP

Variables Defineseparatelyyamlfile

Helpstokeepframeworkgenericanddatadriven

Framework Tabularmodel Equalspacing

CreatingTestCase•  Whitespacesignored*•  Keyword(What?)•  Library+Pythoncode•  Arguments:

– Mandatory–  Default

•  ReturnValue•  SingleTest

10|

Example1–OperationalStatusofDevice–Framework

11|

Example1–Variables

12|*Fakecredentialsonthisandsubsequentslides

Example1–PythonCode

13|

Example1-Results

14|

Results–ExecutiveSummary

15|*Failedtestmaybearesultofmisconfigurationandnotafailureofthedevice

Results–DetailedLogs

16|*Failedtestmaybearesultofmisconfigurationandnotafailureofthedevice

SuiteSetupandTeardown

17|

TroubleshootingErrors•  Defaulterrorsareminimal•  Tedioustolookathtmlforerrors•  LoggingmoduleinPython•  Similartoprintstatements•  Printswhilerunningtests

18|

Libraries•  StandardLibraries

§  Built-inØ  RunwithconditionsØ  EvaluationØ Matchingexpectedbehavior

§  ProcessorientedØ  ControlprocessexecutionØ  FetchprocessattributesØ  Switchprocess

19|

Libraries•  StandardLibraries

§  DateTimeØ  DateandTimeconversionsØ  Addingtime/dateØ  Subtractingtime/date

§  OSlevelfunctionsØ  Directorychanges/verificationØ  Filechanges/verification(copy,size)Ø  EnvironmentvariablesØ Merge/List

20|

Libraries•  StandardLibraries

§  StringfunctionsØ  Lengthcontrol/verificationØ  BehaviormatchingØ  Byteconversion

§  CollectionsØ  ControlLists/DictionariesØ  Behaviormatching

§  Dynamicinput,Telnet§  Screenshots

21|

Libraries•  ExtendedLibraries

§  Selenium,SeleniumwithAngularJS§  Suds(SOAP),MQTT,Faker§  SSH,Ncclient,Django,FTP§  Database,HTTP,Archive

22|

Tagging•  Classifyingtestcases&providing

metadata•  Reportshowsstatisticsbasedon

tags•  Include/Excludeexecutionofspecific•  Tags•  Tagsforcritical,non-critical,trivial•  Types

§  Forcetags§  Defaulttags§  Customizedtags

23|

Editor•  RIDE–Standaloneeditor•  Pluginsforvariouseditors

–  Eclipse–  Sublime–  Vim–  Emacs–  Gedit–  Notepad++

24|*https://www.youtube.com/watch?feature=player_embedded&v=6F_xGKdoN1E

Example2-Framework

25|

Example2-Variables

26|

Example2–PythonCode

27|

Loops•  ‘For’Loop•  Repetitivetasks•  Keyword/Variable

28|

AdditionalTools•  Rebot

§  ProcessXMLoutput§  Generatehtmlreports§  CombineorMergereports

•  Libdoc§  GenerateDocumentation

•  Tidy§  Cleanup/Changeformat

•  DbBot§  ReportstoSQLite§  Unifystorageofreports

29|

AdditionalTools•  RobotCorder

§  RecordGUIactions§  HTMLframeworkgeneration

•  Pabot§  Parallelexecution§  Time

•  Fixml§  Fixingincompletexmlresults

•  Mabot§  Manualtestswithcompatibleoutputs

30|

API•  Runningcodeviacode!•  APIfunctionstoruntestsalongwithoptions•  Includesallbasictoolssuchasrebot,libdoc,tidy•  Retrieveresults•  CustomizereportsHTML/XMLformat•  UsestandardlibrarieswithAPI•  Specifyvariablesandresources

31|

Example3-Framework

32|

Example3-Variables

33|

Example3–PythonCode

34|

DosandDon’ts

35|

•  Dos§  Documentation(options)§  Short&easynaming§  What,nothow

§  Tabularuniformity

§  Genericandsimpleframework

DosandDon’ts

36|

•  Dos§  Logicincode§  Datadriven§  Checks§  Syntax(Given,When,Then)

•  Don’ts

§  Dependencies§  Granulartest§  Hardcodedvariables§  Sleepinginplaceofpolling

Example4-Framework

37|

Example4-Variables

38|

Example4–PythonCode

39|

Demo

40|

Summary•  RobotAutomationFrameworkprovidesseveralusecasescenariosfornetwork

automation•  Keywordbasedacceptancedriventests•  Reusegenerictestlibraries•  Separationofcomponentsallowcustomizationandeaseofunderstanding•  Simplifyautomationofworkflows

41|

Resources•  https://github.com/robotframework/robotframework/blob/master/INSTALL.rst•  http://www.slideshare.net/pekkaklarck/robot-framework-introduction•  https://github.com/robotframework/QuickStartGuide/blob/master/QuickStart.rst•  http://robotframework.org/robotframework/#user-guide•  https://github.com/robotframework/HowToWriteGoodTestCases/blob/master/

HowToWriteGoodTestCases.rst•  http://robotframework.org/robotframework/#standard-libraries•  https://robot-framework.readthedocs.io/en/latest/

42|

pratik.lotia@charter.com @pratiklotia

ThankYouv Questions?

Backupslides

October2,2018|Charter:Confidential 44|