Strategies for agile software test automation

Nokia Technology Institute

Strategies for Agile Software Test Automation: An Industrial Experience

Eliane Collins, Dr. Arilo Dias-Neto, Dr. Vicente F. De Lucena Jr.

STA 2012

Nokia Internal Use Only

Agenda

1. Introduction

2. Process and Tools

3. Software Testing Automation Strategies

4. Lessons Learned

5. Conclusion

1Instituto Nokia de Tecnologia

Nokia Technlogy IntituteIntroduction

INdT – Nokia Technology Institute

Independent and Nonprofit institute

committed to conducting research and

development of technological solutions

through application development, new

technologies and concepts.

Independent and Nonprofit institute

committed to conducting research and

development of technological solutions

through application development, new

technologies and concepts.

Main Areas:

• Product Creation

• Apps and Services

• Hardware and Manufacturing

• Software Open Source and User

Interface

Main Areas:

• Product Creation

• Apps and Services

• Hardware and Manufacturing

• Software Open Source and User

Interface

Manaus

Recife

Brasília

São Paulo

Instituto Nokia de Tecnologia

2


Introduction – Agile Testing

• Testing with a plan to learn about it,let the customer information guidethe testing in line with agile valuesworking software which responding tochange .

• Tests to prevent defects.

• Proactive Testers.

• Test Automation is the key ofsuccessfully agile development andthe core of agile testing.

• [Crispin, L.; Gregory, J. 2009]



Process and Tools

� Scrum agile methodology

� Scrum is an agile development process focusing onteamwork;

� Scrum ceremonies (sprint review, daily, retrospective andplanning meetings);

� Test Process

� Testers should plan and update test cases for sprint stories;

� Automate test scripts when it is possible;

� Execute regression tests (manual and automatic), reportdefects found

� Tools for Unit Testing: GoogleTest, Junit,

� GUI test tools: Sikuli, Selenium RC and IDEand Fitnesse.



Software Testing Automation Strategies

� Strategy 1: Testers Automating Unit and System Tests :

� Development of a location API driver to be embedded in amodem;

� The project schedule was delayed and the developerswanted to focus on developing features instead unittesting;

� Testers decided to perform the unit testing tasks usingGoogleTest tool according to Model V approach;

� Functional tests for the desktop installer and configurationscreen were automated for MS Windows 7-x64 and MSWindows 7-x86 using Sikuli tool ;




� Positive and Negative points from Retrospective meetings:


Positive Negative

• Testers improved programming

skills

• Knowledge sharing on System

Architecture

• Unit testing found important

defects

• Poor knowledge transferring from

testers to developers

• System testing performed after

development

• Separation between testing and

development teams

• Code improvements postponed



� Strategy 2: Developers Automating Unit Tests and TestersAutomating System Tests

� Development of a web system to automate factory tasks oncounting material for production.

� Developers coding new features, unit testing using TDDapproach and Continuous Integration (Hudson).

� Testers automating UI Tests using Selenium, Load Tests andSecurity using JMeter.





Positive

Negative


Positive Negative

• CI strategy improved testing

automation

• High coverage of GUI testing

automation

• Security and load tests executed

early

• Poor knowledge transferring

• High effort to automate tests with record-

and-play approach

• Absence of integration testing

• Separation between testing and

development teams



� Strategy 3: Developers Automating Unit Tests and TestersAutomating System Tests

� Development of a server-side system to receive requestsfrom a mobile device, and send services and applicationsback;

� Testers started to collaborate reviewing test scenarios andcode by pair programming;

� Developers and testers configured tools and programmed inpair funcional test scripts using Fitnesse tool for API Tests;

� Developers coded and wrote unit, integrations, and systemtests, reviewed test cases and executed all automated tests;




� Collaborative Strategy

� Automation tasks are in the intersection and they can be executedby everyone in the project team;






Positive Negative• CI strategy improved testing

automation

• Security and load tests executed early

• Intense team collaboration

• Knowledge transferring regarding the

tools

• Testing responsibilities distribute for all

team

• Teammotivation to learn new solutions

• High initial effort to set the

automation environment

• Rework when customer requires

changes in the system


Lessons Learned

� Collaboration supports the Success of Agile Test Automation;

� Automate Each Layer of Software When It Is Possible and Just forAcceptance Tests in the Sprint;

� Test Automation in the Agile Project Should Be Simple, Reusableand Maintainable for Project Team;

� Automate and Run Security Tests and Stress Tests Early ReducesRisks and Rework;

� Use Test Automation for Documentation and InformationFeedback ;



Conclusion

� It is feasible to adapt agile practices and values to test automation.

� The automation was a resource to document software, reducecost, and allocate tasks in smaller parts.

� Next steps:

� Evaluate the test automation collaborative strategy asframework;

� Evaluate the impact of team collaboration levels in the testautomation.

� To extract the testing effort, number of detected failures preand post-delivery, and schedule adherence.


Nokia Technology Institute

Thank you.

Questions?