Click here to load reader

Conference Paper excerpt From mapping resolver for running in particular machine/domain environments, a resource manager for providing support for running the tests in different locales,

  • View

  • Download

Embed Size (px)

Text of Conference Paper excerpt From mapping resolver for running in particular machine/domain...

  • Permission to copy, without fee, all or part of this material, except copyrighted material as noted, is granted provided that the copies are not made or distributed for commercial use.

    Conference Paper excerpt

    From the

  • Abstract

    This paper describes a test harness that can be used for developing test automation for

    websites. The harness provides a set of classes that a test developer can use to develop

    custom coding solutions for controlling (automating) and verifying websites. The harness

    is completely browser agnostic meaning a single code base is developed that can

    execute on multiple browsers. The current implementation is targeted for the Windows



    Last year, our department began development of a server management website. Our

    team (the UI test team), was given the challenge of developing a test harness that

    would allow for the development of a completely automated regression suite and

    would give the capability of running against multiple browsers using a single code base.

    Our initial idea was to develop a harness compatible with IE7 (Internet Explorer 7) and

    Firefox. However, we wanted to make it easily extendable to other browsers and

    wanted to shield the test developer from needing to write two sets of code

    implementing the same test case.

    We addressed this challenge by developing a set of classes that allows the tester to

    author tests independent of the target browser (IE7 and Firefox in our case). The test

    code implements a single test case that could be targeted towards different browsers.

    The test code is completely browser agnostic. Nowhere would one read anything

    specific to a particular browser. The following example shows our intention:

    Scenario 1

    Test Spec

    1. Open the browser and navigate to a

    2. Enter search criteria “cats”

    3. Click search button

    4. Verify is found

    5. Click on the link if found

    6. Close the browser

    PSNQC Conference Paper

    Presenters: Jagannathan Venkatesan and Craig Merchant

    Contributor: Manuel Tellez

    Microsoft Corp.

    Web UI Automation – A Browser

    Agnostic harness for Web UI Testing

    Excerpt from 2008 PNSQC Proceedings PNSQC.ORG

    Page 2 of 26

  • Implementation (shown in pseudo code)

    //Create browser control object , browser

    browser = new Browser()

    //navigate to


    //try to find the expected search result

    browserInput input = FindInputElement(“”);

    //verify the search result

    If (input == null)






    Existing Solutions

    Microsoft has a significant number of test teams writing website automation. We define

    website automation as the ability to programmatically control a website and verify

    content of the website. We found that most solutions fell into one of three buckets:

    Record and Playback methodologies, JavaScript solutions, and UI Automation based


    The characteristics of these solutions appear below:

    Record and Playback

    1. Advantages

    � Easy to record a script. Just Point and click. A script of all user input

    is created automatically from the recording.

    2. Disadvantages

    � Different scripts need to be recorded for different browsers. This

    type of approach is definitely not browser agnostic.

    � A slight change in the UI requires scripts to be re-recorded. Thus,

    recording tasks must be duplicated for each UI revision.


    1. Advantages

    � Low level DOM(Document Object Model) access.

    � No special development environment required other than cscript.

    Notepad is the editor of choice.

    2. Disadvantages

    � It is generally harder to debug JavaScript than it is to debug higher

    level languages. We need the ability set break points for failure

    investigation instead of relying on print statement debugging.

    � JavaScript is procedural language vs. object-oriented which in our

    experience we have observed naturally caters to UI automation).

    Excerpt from 2008 PNSQC Proceedings PNSQC.ORG

    Page 3 of 26

  • UIAutomation

    1. Advantages

    � Provides extremely low level access to all controls on the window.

    � Fast and reliable.

    � .Net classes available in System. Automation

    2. Disadvantages

    � Different browsers have different automation ids for the same

    control. In fact, the programmer has no control over the

    automation id. Browser agnostic tests would not be possible here.

    We found that no single solution satisfied all our requirements. Thus we decided to

    produce a custom harness. The design of the harness is detailed in the next section.



    Our test team does leverage existing tools when possible. Prior to the proposed web

    based management solution, our team worked on traditional windows software. Our

    team developed DFS (Distributed File System) Management Console, Share and

    Storage Management Console, and some other management applications on

    Windows Server 2008. From our prior work in UI automation, we had a UI automation

    harness. The harness was designed with extendibility in mind. It provided a skeleton on

    which to build the new web automation harness. Building off the existing harness

    allowed our new tests to have an identical structure to our non-web based tests and

    allowed us to reuse code for common functions like logging.

    Our existing harness was designed to allow users to mix and match back end

    automation techniques. Thus the browser agnostic web automation harness was a

    natural extension and did not involve any re-design work. We designed two new plug

    and play components for dealing with Firefox and IE7. At runtime, the harness loads the

    desired plug and play component depending on the browser being tested and from

    there the harness uses the loaded plug and play component to control the browser.

    Additionally, our current harness is .Net based (C#). Test development and debugging

    can be done in MS Visual Studio. Being implemented in a high level object oriented

    language makes it accessible to a wide developer audience.

    Excerpt from 2008 PNSQC Proceedings PNSQC.ORG

    Page 4 of 26

  • The harness has two main layers. They are the Test Case Layer and the External layer.

    (Please see Figure 1 for a block diagram of the layers.)

    The TestCase layer provides several static classes such as a logger, an environment

    mapping resolver for running in particular machine/domain environments, a resource

    manager for providing support for running the tests in different locales, and a

    performance monitor for collecting performance statistics. It also provides several

    dynamic classes for dealing with user interfaces. This is the layer at which we added an

    extension to allow test cases to control the browser.

    The External Layer level provides specific control to particular browsers such as IE7 and

    Firefox. It is at the external layer where we added the plug and play components to

    control IE7 and Firefox. When we say plug and play, we are indicating that each

    component will implement the same interface. In this way, the objects in the test case

    layer can instantiate the objects in the external layer and use them interchangeably to

    achieve browser agnostic behavior. The test developer deals with the objects in the test

    case layer only and is unaware of which plug and play component has been loaded.

    He/she simply calls into the test case layer objects which handle the communication to

    the external layer component.

    Comparison to Existing Approaches

    The following chart shows how our design compares to the existing approaches to web

    automation mentioned above,



    Low level

    access to all






    Special Dev


    Resilient to

    UI changes





    Record &



    Javascript YES YES NO NO NO YES YES






    The harness is structured such that a test case derives from the BaseUITest class which

    derives from the TestCaseBase class. This object oriented approach allows the tester to

    immediately have access to pre-existing functionality such as logging, environment

    management for running in different machine/domain settings, resource management

    for localization testing, performance monitoring for tracking runtime statistics such as

    memory usage and CPU utilization, test data management, and other areas. The tester

    can extend or override existing code in order to

Search related