1 | P a g e

Enhancing Effectiveness of WATIR through Keyword Driven Framework

AVASOFT

2 | P a g e

Table of Contents

1.0 Abstract …………………………………………………………………………………………………..3

2.0 Technology Overview ........................................................................................................................ 4

2.1 Ruby .............................................................................................................................................. 4

2.2 Watir ............................................................................................................................................. 4

3.0 Automation Frameworks .................................................................................................................. 4

3.1 Commonly known automation frameworks ................................................................................ 4

3.2 Keyword Driven Framework ......................................................................................................... 5

3.3 Advantages: ............................................................................................................................... 6

4. Components of WATIR based Test Automation Framework .............................................................. 6

4.1 Driver & Test Script: ..................................................................................................................... 7

4.2 Generic Library: ............................................................................................................................ 7

4.3 Keyword Library:........................................................................................................................... 7

4.4 Test Results ................................................................................................................................... 8

4.5 Drill Down Reports ....................................................................................................................... 9

4.6 Failed Test Cases Snap Shot ......................................................................................................... 9

4.7 Log File ........................................................................................................................................ 10

5.0 Do’s, Don’ts, Advantages & Limitations .......................................................................................... 11

5.1 Do’s ............................................................................................................................................. 11

5.2 Don’ts ......................................................................................................................................... 12

5.3 Advantages ................................................................................................................................. 12

5.4 Limitations .................................................................................................................................. 12

3 | P a g e

1.0 Abstract

There is a myth in the industry that test automation is generally effective when relied on commercial automation tools. To some extent this myth is due to the fact that flexibility offered by commercial tools is often readily not available in open source automation tools. Further, it might be possible that the test teams do not have a clear idea on how to extend the capabilities of the open source tools, hence it is felt, open source test automation tools are not an option worth considering. However, by applying a structured thought process and with a little bit of innovation, it is possible to derive benefits using open source tools as are commonly available in commercial tools. Having said that, it should be remembered that open source tools are a good alternative to commercial tools but should not be considered as a silver bullet for all automation needs.

While there are quite a few areas where open source tools can be enhanced to make them more useful, one such area relates to “Implementing Keyword Driven Framework” while using WATIR

(an open source test automation tool). Using Keyword Driven approach while relying on open source automation tools is a great idea because organization not only have the flexibility of maximizing the benefits offered by open source tools but also in ensuring they eliminate the need to procure expensive commercial licenses. The advantage of this approach is twofold, increase in ROI (due to elimination of licensing cost) and flexibility (as you can extend the capability provided by the open source tool).

The objective of this paper is to clearly demonstrate the approach that can be taken to design and develop a robust keyword driven framework using WATIR. Apart from the approach, the paper elaborates on the common mistakes that are committed when designing automation framework, lists out the do’s and don’ts of Keyword Driven approach and �nally elaborates on the route map following

which, organizations not having automation capabilities can plan, design and develop robust Keyword Driven Automation Framework using WATIR. Additionally, the paper will elaborate on different tools and utilities that can be built on top of WATIR to make the framework relevant to the user needs. Examples of the extensions include: creating customized test reports, automatically capturing images or snapshots of test failures & customized log files that provide context specific information. The approach however is not restricted to WATIR but can also be applicable to other automation tools.

4 | P a g e

To summarize this white paper is a good read for organizations looking at eliminating their commercial licensing costs (for automated tools) and for organizations interested in enhancing the capabilities of their existing automation suites.

2.0 Technology Overview

2.1 Ruby Created by Yukihiro Matsumoto (commonly known as „Matz�) and released in 1995, Ruby

is seen as a flexible language, since it allows its users to freely alter it parts. It is a dynamic programming language with a focus on simplicity and productivity. Apart from the ease its use Ruby is the scripting language used as part of scripting tests for WATIR.

2.2 Watir

Watir, pronounced as “Water” is an open source library for automating web browsers. Watir

is a family of Ruby libraries but it also supports any application testing irrespective of the technology it was developed in. It supports IE on Windows, Firefox on Windows, Mac and Linux, Safari on Mac, Chrome on windows and Flash testing with Firefox. Watir accessed an HTML page using the COM interface of Internet Explorer, which controls external program in Internet Explorer.

3.0 Automation Frameworks

3.1 Commonly known automation frameworks

As defined by Wikipedia “http://en.wikipedia.org/wiki/Test_automation_framework” a “Test

Automation Framework” is a set of assumptions, concepts and tools that provide support for

automated software testing. The main advantage of such a framework is the low cost for maintenance. If there is change to any test case then only the test case file needs to be updated

5 | P a g e

and the Driver Script and Startup script will remain the same. There's no need to update the scripts in case of changes to the application.

While there are many flavors of automation frameworks, some of the most commonly known frameworks include the following:

• Test Script Modularity • Test Library Architecture • Data Driven • Keyword Driven or Table Driven • Hybrid Test Automation • Model Driven

There is significant amount of material available on the internet describing these frameworks

in details, however looking at the scope of the current paper, we will focus on understanding basic of Keyword Driven automation framework.

3.2 Keyword Driven Framework

Keyword-driven testing, also known as table-driven testing or action-word testing is a methodology for automated testing. The advantages for automated tests are the reusability and therefore ease of maintenance of tests that have been created at a high level of abstraction. This requires the development of data tables and keywords, independent of the test automation tool used to execute them and the test script code that "drives" the application-under-test and the data. Keyword-driven tests look very similar to manual test cases. There are three fundamental aspects that get captured as part of this framework. They are the “Object”, its “Action” and relevant “Data”.

For example a text box (let us say a user name) is an object, entering text is the action and the actual text is the data (data is the actual user name that is entered in the text box).

6 | P a g e

3.3 Advantages:

� The data file which is having test data and keywords are independent of the application.

� It enables significant reusability � It is easy to implement

4. Components of WATIR based Test Automation Framework

Now that we have an understanding of the fundamental concepts of Watir, Ruby and Keyword driven framework, the next step will be to come up with a basic design for the overall test automation architecture. While the picture below has been designed to ensure general compatibility with most of the organization’s needs, it might be possible that some modi�cation may be made by di�erent

organizations on a need basis. As can be seen the architecture diagram represents different key components that work together to provide a comprehensive test automation model

7 | P a g e

4.1 Driver & Test Script:

Scripts that execute a set of transactions, by invoking relevant re-usable utilities/components for each test case. Driver scripts can be mapped to a group of test cases related to a scenario/transaction/screen/window.

Scripts that are called by the driver scripts and which internally represent the flow of the Application functionality are called the test scripts.

4.2 Generic Library:

Generic library consist of those functions that are application independent and can be called to perform certain tasks. It can also consist of a group of tools or utilities that will enhance the overall automation framework. In this case the generic library is a collection of utilities that deliver the capabilities to create detailed drill down reports in different formats, help log events and save snapshots of tests that fail (snap shots or screen shots of failures).

4.3 Keyword Library:

In the context of a keyword driven testing, keywords are actions which are nothing but the tasks to be executed in a test. At a most elementary stage a keyword library consists of three key aspects. They are the Object, its Action and the Parameters, where object refers to the object that needs to be called, action refers to its what it needs to do and parameters refers to the test data (example: for an object called Text box, the action is accepting a user name and the test data will be the user name).

8 | P a g e

Action Keywords column describes the keywords which would used in test scripts, Module Name column describes the name of the module in applications, Action Steps column defined the step by step instruction, Object Name and Object Value columns defined the each object on webpage can be identified property name and value and Parameter columns specify to enter the input values.

Initially building the keyword library will take significant time, however once built, it can generally be used across different applications and domain, thus saving significant time in the long run.

4.4 Test Results

Subsequent to the test run, the test scripts consolidate the test details and update the test results. At the start of the test, the Test Suite consists of a list of all test cases that have been marked for execution by selecting a status called “Yes”. You could also assigning a dependency so that a

particular test case will execute only when the dependent test case has been executed. At the end of the test execution, the Test Suite is updated with status of all tests that have passed and failed, thus providing with a single source for planning the test cases that need to get executed and later on showing their pass/fail status.

9 | P a g e

4.5 Drill down Reports

Subsequent to the updation of Test Suite with Pass / Fail Status, the test scripts call the Generic Functions which in turn create test reports in HTML formats. The advantage of this approach results from the fact that these HTML reports offer a neat drill down facility that allows you to drill down and view the specific details of the failures. The report also provides a time stamp that helps team manage versions of test reports. Further, a consolidated status for the group of test steps that get executed gets presented via the “Overall Results”. Another advantage of this approach is that

with a very minimum customization, you can create a host of reports in different formats. This includes .xls, .doc, .pdf, xml and others as needed.

4.6 Failed Test Cases Snap Shot

As discussed earlier, the generic functions are called by the test scripts to take an automatic snap shot of tests that have failed. The generic functions can be modified to provide the user with an option to select between snapshot for all tests (both pass and fail) or to select only failed. With the possibility of adding a time stamp to the snapshot it will be easy for test teams to not only identify the exact cause of failure but also provide evidence of failure in case it is needed.

10 | P a g e

You can also opt between taking a screen of the active window or of the entire desktop.

4.7 Log File

Build in functions in the Generic Library also allow users to generate different types of logs to suite their specific needs. Customization of the logs helps users to view step by step view of the test execution along with time stamp, pass or fail status and detailed of activities performed. This can further be customized so that a user can select to generate logs of only failed tests or can view detailed logs.

AVASOFT Enhancing E�ectiveness of WATIR through Keyword Driven Framework

Page of 13

5.0 Do’s, Don’ts, Advantages & Limitations

5.1 Do’s

As part of you automation journey, ensure the team is first trained on Ruby syntax Before finalizing your framework, understand your needs and then design components that

help you meet your needs

Ensure the relation between the application test scripts and the framework is kept to the minimum. Thus you can design a reusable and truly scalable framework

Create the keyword library and keep updating it with more keywords Unless it is application specific, most of the test data can be reused Use open source tools (like Snap it) to take screen shots Use different folders for reusable function, test scripts, logger files, test results and test data

sheets

Identify object elements using Internet Explorer Development Toolbar. This is easier and provides reliable object information

AVASOFT Enhancing E�ectiveness of WATIR through Keyword Driven Framework

Page of 13

5.2 Don’ts

Don’t be too ambitious with your automation fr amework. Set right expectations There is no Silver bullet. Don’t rely on the framework for ever. Upgrade when needed Do not use the feature of capturing screen shots for all scenarios as it will increase the

memory consumptions and can delay the test execution cycle. Use it only for capturing failure scenarios

5.3 Advantages

The framework is very robust, maintainable and portable Redundancy is eliminated by designing for reusability Easy to debug considering log files and the failed test scenario snap shot Non-programmers develop scripts with minimum efforts Open source tool results in almost zero cost

5.4 Limitations

WATIR cannot automate Flex or Flash based applications. You will have to use couple of other tools (something like FunFX which is an Open Source) to work with Flex / Flash based applications

Finding reference to objects is little tedious work Limited ability to run multiple scripts consistently

Poor exception handling mechanisms, your generic library has to cater for exception handling to ensure consistency in test execution

Limitations when automating dynamic objects and values