Test Automation · 2015-07-17 · y Data-driven testing support y Mobility support y Good and flexible reporting mechanisms y Plug and play assets: offering the flexibility to plug

WHITEPAPER JAnuARy 2015

www.hcltech.com

CodelessTest Automation

AuthOr:

Ravi Kant Sharma and Naveen SaxenaAutomation CoE,HCL Technologies Ltd.

Bangalore, India

[email protected]/[email protected]

BusinessAssurance & Testing

CODELESS TEST AUTOMATION JANUARy 2015

© 2015, HCL TECHnOLOGIES. REPRODuCTIOn PROHIBITED. THIS DOCuMEnT IS PROTECTED unDER COPyRIGHT By THE AuTHOR, ALL RIGHTS RESERVED. 2

TABLE OF CONTENTS

ABSTRACT 3

OVERVIEW 3

NEW BUSINESS DYNAMICS AND TEST AUTOMATION 5

CORE DRIVERS FOR CODELESS AUTOMATION 6

HOW CODELESS TEST AUTOMATION WORKS 7

TOOLS EXPECTATIONS 7

PROSPECTS 8

BENEFITS 8

SUCCESS FACTORS 9

DATA POINTS 10

CONCLUSION 10

ABOUT HCL 11



ABSTRACTThere are heaps of testing tools available in the market today, and many of them are marketed as being simple to use and maintain, having multiple features. As automation SMEs we come across plenty of real-time instances where we need to leverage coding/scripting to bring better coverage for automation and to avoid lines of code from accumulating.

The more code - the more complexity, and the higher the maintenance costs for scripts as well as frameworks. Thus, as practitioners, we need to be able to make this journey with as little coding involved and achieve higher quality and maintainability, setting the foundation for ‘Codeless Automation’.

This whitepaper highlights the need for a Codeless Test Automation Framework that allows us to effectively design automated tests without any programming language and make test design, execution, the tracking of results, and script maintenance, easier.

This paper is for business managers, test leads and software test engineers and the key focus areas of application include system and regression testing.

OVERVIEW

In the traditional Test Automation approach, sophisticated test automation tools were used with the understanding that the tools would do the job. We relied up on the ‘Record and Playback’ feature available in thetool with small augmentations and code changes to perform validations on Application under Test (AUT).

Fig 1: Evolution of Test Automation

Test Automation revolves around the concept that advanced with Microsoft and revolutionized the entire IT industry. Post this, automation tools flooded the market and each vendor boasted of a better tool. The ‘Recordand Playback’ functionality in each of these tools became a selling point, making it easy for testers to use these features for repetitive tasks with parameterization of the test data.

Faster

Cheaper

Simpler

Better

Automation with Record

and Playback

Automation with Reusable

Functions

Automation with

Frameworks

Codeless Automation



This eventually led to an era of sophisticated code development and modularization where many automation frameworks came into being, as listed below:

y Keyword Driven Framework

y Modular Framework

y Hybrid Framework

y This table highlights the key differences between the frameworks mentioned above: Keyword Driven Framework

‘Keyword-driven’/excel driven’ are interchangeable terms that refer to an application-independent automation framework. This framework requires the development of data tables and keywords/functions.

Here, the test script code “drives” the application-under-test and the data. Keyword-driven tests look very similar to manual test cases.

Modular Framework

The modular framework requires the creation of small, independent scripts and functions that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests.

It involves coding/scripts for each of the test cases, and hence is required to be the simplest to grasp for automation engineers.

Hybrid Framework

The hybrid framework is the most commonly implemented framework. It is a combination of both modular and keyword-driven frameworks and leverages the merits of both while mitigating their weaknesses to a certain extent.

Table 1: Key automation frameworks types

These automation frameworks resolved multiple issues in relation to scripts standardization, scripts uniformity, faster development, etc. Most of the time, this would lead to munificent test maintenance efforts that arose due to the complexity involved in these automation frameworks. In some instances, maintenance efforts exceeded half the development efforts due to complexity arising out of debugging the script according to the automation framework that was employed.



The figure above illustrates the different stages of test automation commencing from the development of the automation framework, to test execution and maintenance. Here we spend most of the time in test script creation and maintenance arising due to application changes or data changes. We also expend considerable effort in developing the framework that drives the need for codeless automation where we could create an automated script without any specific automation framework.

NEW BUSINESS DYNAMICS AND TEST AUTOMATIONWith new business dynamics in the software development life cycle and the introduction of the Agile methodology, test automation became more and more challenging. In the traditional Waterfall development approach, the product is built stage-by-stage and testers plan first and then enforce the plans through development. Here, the test automation team predefines a test automation plan within anticipated timelines for a given project scope.

On the other hand, Agile is a non-linear approach where the entire software product is split into small fortnightly or monthly releases when the project scope is unstable. Unlike Waterfall, it has multiple iterations with less time to perform test automation. It is characterized by more time being spent in test automation development and maintenance, creating tests and updating those based on new scope changes and executing them. This leads to the foundation of unconventional script-less test automation that could simplify test automation development and maintenance.

Setting up Automation F/W Structure

Developing the base function libraries for available technological stack

Developing the Result Reporting Mechanism

Framework Integration with HP-ALM

Framework Development

Setting up Automation F/W Structure

Developing the base function libraries for available technological stack

Developing the Result Reporting Mechanism

Framework Integration with HP-ALM

Test AutomationMaintenance

Test Automation Implementation

Analyze Manual Test Cases for Automation

Feasibility and Reusability

Manual Test Case Management

Developing Automated Script for feasible Test

Integrating Test Script with HP-ALM

Reviews and Fixes



A new era commenced with Codeless Test Automation, where we moved away from conventional, time consuming, and code-based processes of writing code. This helped both the test automation team and the manual team create and update tests easily. Codeless Test Automation made it easy to use, with automated assets in ‘English’ and in a format that is easy to create and maintain.

CORE DRIVERS FOR CODELESS AUTOMATIONWith increasing cost pressures, IT companies are focusing more on the cost benefits and on improving work efficiency. The core drivers aggressively encouraging the evolution for Codeless Test Automation are:

y Speed (quick time to market i.e. in Agile) y Maintainability (less maintenance overhead cost) y Configurability (make test automation assets easy to confirm for different

requirements with minimal changes) y Robustness (easy plug and play anywhere) y Work efficiency (with codeless test automation that reduces up to 80% of the test

automation efforts, saving time for test engineers that can be spent on core business areas for results analysis, finding defects, and executing more tests)

HOW CODELESS TEST AUTOMATION WORKSCodeless Test Automation is a concept/methodology supported by a test automation tool. It has a user-friendly GUI with provisions that enable test engineers to create test cases by simply choosing objects and adding operations to them. This is done by selecting the actions from a drop-down lookup list along with the provisioning of test data passed locally or externally, while making use of the codeless test automation

Background

Developing right and quality automated test scripts for better reusability To run test suites in regression testing smoothly Adaptable for changing requirements To have less maintenance overhead

Selection of Right Test Automation Tool To meet Test Environment

Requirements, Application Requirements To meet Test Automation Requirement User friendly To give quick results and reporting abilities

Keep your tests small and lean To help in dealing with complex scenarios To help in debugging To enable easy integration with other tests

Selection of Right Approach and Framework

Agile Projects present their own challenges to the Automation team; unclear project scope (changing requirements),multiple Iterations, minimal documentation availability, speedy deliverables for frequent releases, active involvement of stakeholders and unavailability of application.

Inconsistency in Test Automation Scope

Minimal documentation

Changing Environments i.e. URL, Login Credentials, Workflow Changes, Application look and feel changes

Speedy deliverables for frequent releases

Active stakeholder involvement

Object Maps changes from one sprint to another

Over commitment with limited sprint durations

Selection of “What to Automate and What Not”

Selection of Right Framework

Selection of Approach for speedy deliverables and goal success

Selection of Right Resources and Tools

Resource Management (to have skill set both in manual and automation testing)

Challenges Approach



tool. The end users do not have to program scripts for each test case and test steps in any scripting language. Hence, Codeless Test Automation becomes language agnostic.

TOOLS EXPECTATIONSCodeless Automation is expected to bridge all gaps that code-based test automation carries. It is expected to offer coverage to the following environments:

y Multiple browser support: for cross browser testing

y Multiple technology support: for supporting all existing technological platforms

y Multiple database support

y Multiple file system support

y Data-driven testing support

y Mobility support

y Good and flexible reporting mechanisms

y Plug and play assets: offering the flexibility to plug and play test automation assets and calling already automated assets/components

y E-mail notifications

y API access for integrating with tools such as Test Management, Bug Tracking, etc.

y Provisioning of custom objects/controls in the application, such as Virtual Object Handling

y 12. Provisioning of Batch Runs and Scheduling

The industry currently does not have any tool that covers all the expectations listed above.



PROSPECTSAs described above, Codeless Automation is a futuristic idea to mitigate most of the challenges of Test Automation, but it also raises a concern on whether code-based test automation will eventually vanish. While this is quite subjective, with the present findings and the pace in technological development, it seems evident that script-based test automation may eventually disappear and the void befilled with Codeless Test Automation.

BENEFITSCodeless Test Automation offers several benefits. Some of them include:

y Automating test cases becomes easier and more comprehensive; we can see what is automated and what data has been used

y Time for automating tests/scenarios is reduced considerably through pre-defined sets of methods to be operated on test objects; less time is spent in developing functions or procedures

y Test maintenance becomes easier and less time consuming as code level debugging presence is negligible

y Training non-technical test engineers or business analysts becomes easier, enabling them to contribute to test automation or review automated test scripts

y Creating and managing complex tests becomes easier

y Testers can spend more time analyzing test results, detecting defects, and focusing on business areas/ functional aspects, etc.

y Both automated and manual tests can reside in the same location and provide

MobilitySupport

MultipleBrowserSupport

MultipleTechnology

Support

MultipleDatabaseSupport

Multiple FileSystemSupport

Data DrivenTestingSupport

AutomationSolution



transparency about what is automated and what is not. It also helps eliminate the cost of having another tool to maintain manual tests.

y Promotes greater stakeholder participation and increases testing process productivity

SUCCESS FACTORSThe overall success of Test Automation does not depend on how many tests have been automated or how much better we have coded the tests but on what we have automated, though the numbers may bring up a value and not necessarily the desired success. We see that the decision to opt for automation, the candidates for automation, and the tool used to carry out the automation - all plays a very vital role in the success. Here the candidates (automation requirements i.e. manual tests/scenarios) are the preliminary factors that determine automation success. We therefore emphasize on the following factors:

DATA POINTSBased on our findings across various projects, we have seen that a considerable amount of saving is achieved using Codeless Automation.

Let us consider a typical scenario with minimal functional and technical complexity and the availability of a Test Automation Framework with all processes in place. We find that there is a significant reduction in overall test automation efforts with the use of Codeless Automation, and we save around 56% in automation efforts per script.

Faster

Simpler

Cheaper

Better

Approach

Reusability and Repeatability

Complexity of Tests

Inter Test Dependencies

Data creation

Tests that needs to run on different environments

Business Indispensable tests

Quality

Selection of Right Candidates

Clean up of Test Cases (Redundancy Removal)

Improve overall quality of test cases

Optimize regression test cases

Focus on End to End Testing that mimics business use

Emphasize on testing Needs

Test Strategy

Better Test Coverage

Consistency

Reliability

Robustness

Quality Considerations

Coverage to Application and Environment

Ease of creating and updating tests

Ease of debugging test scripts

Result reporting

Skill availability

License cost and affordability

Assimilation with other Tools

Tool Selection



Traditional Vs Codeless Automation Efforts Vs Cycles

100120140160180

Traditional Vs Codeless Automation

4222

5156

4000

5000

6000

Efforts Vs Cycles

020406080

1413

2400

3289

14391847

22562000

3000

4000

533

1413

233618

10501439

0

1000

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6

Traditional Automation (in Mins) Codeless Automation Traditional Automation (in Hours) Codeless Automation (in Hours)

CONCLUSIONBased on the above findings,we can conclude that Codeless Test Automation has a number of advantages and is also scalable for Agile projects with limited timeframes and unstable scopes. This, however, does not mean that Codeless Test Automation is only suitable for Agile; it can also be plugged in to Waterfall projects. The bigger challenge for these projects include the migration of existing test scripts from existing tools to newer ones supporting Codeless Test Automation. While critics may argue that Codeless Automation poses a threat for the learning curve as well as for test engineers, we believe that it makes the job simple for them and for the business.

Hello there! I am an Ideapreneur. I believe that sustainable business outcomes are driven by relationships nurtured through values like trust, transparency and flexibility. I respect the contract, but believe in going beyond through collaboration, applied innovation and new generation partnership models that put your interest above everything else. Right now 100,000 Ideapreneurs are in a Relationship Beyond the Contract™ with 500 customers in 31 countries. How can I help you?

ABOUT HCL

About HCL Technologies

HCL Technologies is a leading global IT services company working with clients in the areas that impact and redefine the core of their businesses. Since its emergence on the global landscape, and after its IPO in 1999, HCL has focused on ‘transformational outsourcing’, underlined by innovation and value creation, offering an integrated portfolio of services including software-led IT solutions, remote infrastructure management, engineering and R&D services and business services. HCL leverages its extensive global offshore infrastructure and network of offices in 31 countries to provide holistic, multi-service delivery in key industry verticals including Financial Services, Manufacturing, Consumer Services, Public Services and Healthcare & Life sciences. HCL takes pride in its philosophy of ‘Employees First, Customers Second’ which empowers its 95,000 transformers to create real value for customers. HCL Technologies, along with its subsidiaries, had consolidated revenues of US$ 5.5 billion, for the Financial Year ended as on 30th September 2014 (on LTM basis). For more information, please visit www.hcltech.com

About HCL Enterprise

HCL is a $6.7 billion leading global technology and IT enterprise comprising two companies listed in India – HCL Technologies and HCL Infosystems. Founded in 1976, HCL is one of India’s original IT garage start-ups. A pioneer of modern computing, HCL is a global transformational enterprise today. Its range of offerings includes product engineering, custom & package applications, BPO, IT infrastructure services, IT hardware, systems integration, and distribution of information and communications technology (ICT) products across a wide range of focused industry verticals. The HCL team consists of over 100,000 professionals of diverse nationalities, who operate from 31 countries including over 505 points of presence in India. HCL has partnerships with several leading global 1000 firms, including leading IT and technology firms. For more information, please visit www.hcl.com

Documents

Test Automation · 2015-07-17 · y Data-driven testing support y Mobility support y Good and flexible reporting mechanisms y Plug and play assets: offering the flexibility to plug