Test-Driven Development of Graphical User Interfaces:A Pilot Evaluation

Thedore D. Hellmann , Ali Hosseini-Khayat, Frank Maurer

XP 2011

Outline

Capture and Replay Tools

Introduction

Difficulties

A Prototype-Driven Approach to UITDD

Pilot Evaluation

Conclusion

Capture and Replay Tools

Capture and Replay Tools(CRTs) Tools that record users’ interactions with an application.

Can be used to record test scripts.

Introduction

Test-driven development (TDD) has proven benefits for the software engineering process increased developer confidence

increased software quality

In the approach, user interface prototypes are created in such a way as to allow capture/replay tools to record interactions with them.

These recordings can then be replayed on the actual GUI as it is being developed in a test-driven fashion.

TDD of GUI-based applications (UITDD)

Does having tests for a GUI written before the functionality of the GUI is implemented help developers?

Difficulty

What Makes GUI Testing and UITDD Difficult?

1. Complexity

2. Verification

3. Finding Widgets from Test code

Difficulty - Complexity

Modern GUIs are composed of a set of user interface elements, or widgets. Like most classes in object-oriented systems, widgets tend to have a long chain of inheritances.

Even a simple GUI has a huge number of possible user interactions, GUI states, and events.

For Example:

Difficulty - Verification

A test is separated into two parts: test procedure

test oracle

The test procedure interacts with part or all of an application in order to generate a state that is interesting from a testing perspective.

The test oracle verifies that the system behaves as expected in response to this interaction.

The effectiveness of automated tests is directly limited by the difficulty of writing useful test oracles .

Difficulty - Finding Widgets from Test code

In GUI testing, it is necessary to look up widgets when a test is run based on information about the widget that is recorded when a test is created.

This process of look up widgets is in large part responsible for the fragility of GUI tests.

There are two ways of performing this search: testing with object maps

keyword-based testing

A Prototype-Driven Approach to UITDD

ExpenseManager ExpenseManager is a GUI-based

application.

Vies Totals

New Report

Modify Report

A Prototype-Driven Approach to UITDD(2)

After defining the features of ExpenseManager, a prototype of the application was created. This prototype was created using SketchFlow .

SketchFlow Prototypes are created by dragging-and-dropping widgets onto a canvas.

Widgets can also be assigned behaviors such as transitioning to a different state.

we used LEET (LEET Enhances Exploratory Testing) to record events from interactions with prototypes of ExpenseManager.

LEET It is based on the Automation API

Using keyword-based identification to determine which widget to interact with for each step in the test script.

Pilot Evaluation

We designed a pilot evaluation to determine whether our approach is actually useful to developers.

Based on 3 particepants:

1. Researchers completed the development of ExpenseManager, including its GUI and event handlers, based on the tests.

2. The researchers removed all of the code contained in ExpenseManager’s 7 event handlers.

Pilot Evaluation(2)

Interestingly, all three participants entered into the same development cycle during their development of ExpenseManager.

Participants used the provided automated GUI tests as the sole indication of completeness of functionality.

Observations Collected during the Study

Pilot Evaluation(3)

The participants asked to fill out a survey, which perception of the usefulness and usability of this approach.

Observations Collected from Post-Study Surveys

The first and second questions were ranked from A (Very Useful) to E (Useless).

These responses imply that the participants saw the potential of this approach to be of some benefit in their own work.

Pilot Evaluation(4)

Participants were also asked to rank the resources they were provided with in terms of decreasing usefulness for their development task.

Instead of ranking features by frequency of use, participants seem to have ranked resources based on the value they provided to the development effort.

Observations Collected from Post-Study Surveys

This implies that one way to immediately make this approach to UITDD provide higher value to users :

Making technical details of widgets being tested more visible so that users can understand the expected details of the system they are building.

Study Limitations

1. Only three participants took part in this study.

2. The time allotted for each study session was limited to one hour in order to avoid inconveniencing participants.

3. The application used for this study was, based on Table 2, too complicated for a one-hour study.

Conclusion

In this paper we present a pilot evaluation of an approach to test-driven development of GUI-based applications.

we hope to test our approach to UITDD on multi-touch and gesture-based interfaces.

we were able to determine that participants seemed to feel that UITDD had benefits, and would be a useful practice.

Question Time