CS4311 Spring 2011

Unit Testing

Dr. Guoqiang Hu

Department of Computer ScienceUTEP

Announcement

What: CS4390 (Cross-listed with CS5390): “Software Testing”

Testing Only, Nothing Else!

When: This Summer

Prerequisite: CS4311

* Note: The V&V content covered in this course (CS4311) can be considered as an introduction to the above course

Basic Concepts

Unit:Function

Method

Routine

Class (OO)

Unit testing:Testing the unit

Tested in isolation from the rest of the system

Tested in a controlled environment: Use appropriately chosen input data

Unit Testing Procedures

Driver

Unit to be

tested

Stub Stub

Results Test

cases

. . .

Two Fundamental Approaches

Black box:Based on specification

The inner logic structure of the unit is not considered

White box:Based on specification

Based on the inner logic structure of the unit to select test cases

Effectiveness of the two approaches:Black box is similar to white box, but the faults found are different. (Hetzel 1976, Myers 1978)

Black Box Testing

Test set is derived from specification (also known as functional testing)

Goal is to cover the input space with a minimum number of test cases

The normal approaches to describing input space:

• Equivalence Classes

• Boundary Analysis

• Decision Table

• State Transition

• Use Cases

Equivalence Classes MethodEquivalence relation:

Reflexive: x R(x,x)

Symmetric: x,y R(x,y) R(y,x)

Transitive: x,y,z R(x,y) R(y,z) R(x,z)

The rationale: The equivalence partitioning for software testing is based on two

motivations: to have a sense of complete testing, and at the same time, to avoid test cases redundancy.

Through equivalence partitioning, the domain of input can be partitioned into equivalence classes: the union of which is the entire domain, but all the equivalence classes are mutually disjoint with each other. For testing purpose, each value from the same equivalence class is equivalent to each other. In this way, we can use the minimum number of test cases, but at the same time, we can still have a sense that the testing is complete.

The strategy:

Identify the equivalence relation

Partition the input into equivalence classes

Select only one input value from each equivalence class

Equivalence Classes Method: TipsIdentify restrictions for inputs in the specification.

If a restriction on a continuous numerical domain is specified, create one valid and two invalid classes (i.e., above, below).

If a set of values is specified where each may be treated differently, create a class for each element of the set and one more for elements outside the set.

If there is a condition, create two classes, one satisfying and one not satisfying the condition.

Boundary Analysis MethodBoundary analysis focuses on the boundary of the input space to identify test cases. (Used alone, or, as a supplement; Black, Gray, or White)

The rationale behind boundary analysis is that errors tend to occur near the extreme values of input variables:

It is often used to catch “off-by-one” errors.

Also, it is used to catch the extreme compound effects of a group of variables to see if the results have passed some boundaries.

Tips:

If a domain is a restricted set, check the boundaries. e.g., D=[1,10], test 0, 1, 10, 11

For ordered sets, check the first and last elements and their immediate neighbor outside the set.

For dynamic complex data structures, check for its extreme sizes, for example: the empty list, full list, etc.

Extremely large data sets should be tested (limits (e.g., max array size))

The null pointers should be tested

Check for off-by-one errors

Also, check the boundaries of outputs.

The Exhaustive Test Examples

The function fn shall take as input an integer value and return the integer part of the one number look ahead divided by 30,000.

Consider testing the following code with an error (The first line should be: j = j + 1;)

public static int fn (int j)

{

j = j - 1;

j = j / 30000;

return j;

}

Would any of your tests uncover this error?

For 16-bit integers, errors at:

-30,000

-29,999

29,999

30,000

4 cases out of 65,536 possible

Decision Table MethodDecision table method is used for situations in which a number of combination of actions are taken under varying sets of conditions. This structure guarantees that every possible combination of conditions is considered.

To identify test cases with decision tables, we interpret conditions as inputs and actions as outputs.

A simplified version of decision table method works like this:

Identify each rule or condition in the system that depends on some input

For each input to one of these rules, list the combinations of inputs and the expected results

Construct a table using the above information.

State Transition MethodBuild STD for the unit or system, then select test cases that will cover the STD:

Visit each state at least once Trigger each event at least onceExercise each transition at least onceExercise each path (might not be possible)

Use Case MethodUse the use cases/scenarios to specify test cases:

Normal AlternateExceptional

White Box Testing

Test set is derived from the inner logic structure of the unit (also known as glass box testing or structural testing)

Goal is to cover all the inner logic with a minimum number of test cases

Unit inner logic structure is normally represented as Control Flow Graph (CFG)

The normal categories of logic coverages:

• Statement

• Branch

• Condition

• Path

• Data Define-Use

Control Flow Graph

Structure Programming:

Each program and each program construct (called a block) in it should only have a single-entry and single-exit control structure. A structured program progresses in an orderly, disciplined way, rather than jumping around unpredictably. You can read it from top to bottom, and it executes in much the same way.

Structured programming uses only the following three kinds of blocks:

• Sequence

• Selection

• Iteration

CFG: a visual representation of the flow of control of a unit:

• Node represents a sequence of statements with single entry and single exit

• Edge represents transfer of control from one node to another

Control Flow Graph: Notations

n1

Join

n3

n1n1

Join

Sequence If-then-else If-then Iterative

When drawing CFG, ensure that there is only one exit: use a join node if needed.