Reference

• Paulo Alencar, University of Waterloo

• Frank Tsui, Southern Polytechnic State University

2

Overview

Structural TestingIntroduction – General ConceptsFlow Graph Testing

DD-PathsTest Coverage MetricsBasis Path TestingGuidelines and Observations

Data Flow TestingHybrid MethodsRetrospective on Structural Testing

3

Structural Testing

Also known as glass box, structural, clear box and open box testing. A software testing technique

whereby explicit knowledge of the internal workings of the item being tested are used to

select the test data.

Unlike black box testing that is using the program specification to examine outputs, white box

testing is based on specific knowledge of the source code to define the test cases and to

examine outputs. http://www.webopedia.com/TERM/W/White_Box_Testing.html

4

Structural Testing

• Structural testing methods are very amenable to:– Rigorous definitions

• Data flow, control flow, objectives, coverage criteria, relation to programming language semantics

– Mathematical analysis• Graphs, path analysis

– Precise measurement• Metrics, coverage analysis

7

DD-Paths

• More formally a DD-Path is a chain obtained from a program graph such that:– Case1: it consists of a single node with indeg=0.– Case2: it consists of a single node with outdeg=0,– Case3: it consists of a single node with indeg ≥ 2 or

outdeg ≥ 2– Case4: it consists of a single node with indeg =1, and

outdeg = 1– Case5: it is a maximal chain of length ≥ 1

Case1: it consists of a single node with indeg=0.Case2: it consists of a single node with outdeg=0,Case3: it consists of a single node with indeg ≥ 2 or outdeg ≥ 2Case4: it consists of a single node with indeg =1, and outdeg = 1Case5: it is a maximal chain of length ≥ 1

Path Analysis

• Why path analysis for test case design?– Provides a systematic methodology of White Box

testing.• Reproducible• Traceable• Countable

• What is path analysis?– Analyzes the number of paths that exist in the

system– Facilitates the decision process of how many paths

to include in the test

Reasons similar to why we document test cases

Path, as previously defined in Graph Theory, is a sequence of connected nodes

Linearly Independent Path

• A path through the system is Linearly Independent** from other paths only if it includes some segment or some edge that is not covered in the other path.

S1

S3

S2

C1

1

4

2

3

- The statements are representedby the rectangular and diamond blocks.- The segments between the blocks arethe edges, labeled with numbered circles.

Path1 : S1 – C1 – S3Path2 : S1 – C1 – S2 – S3 ORPath1: edges (1,4)Path2: edges (1,2,3)

Path1 and Path2 are linearly independent because each includes some edge that is not included in the other. (note: not necessarily nodes)

** This definition will require more explanation later.

Another Example of Linearly Independent Paths

S1

S2

S3

S4

S5

C1

C2

C3

1

5

6

7

2

4

3

10

9

8

Path1: edges (1,2,8)Path2: edges (1,5,3,9)Path3: edges (1,5,6,4,10)Path4: edges (1,5,6,7)

Note that these are all linearly independent

Statement Coverage Method• Count all the linearly independent paths• Pick the minimum number of linearly independent paths that

will include all the statements (S’s and C’s in the diagram)

S1

S3

S2

C1

1

4

2

3

Path1 : S1 – C1 – S3 : edges (1, 4)Path2 : S1 – C1 – S2 – S3 : edges (1, 2. 3 )

Are both Path1 and Path2 needed to cover all the statements: (S1,C1,S2,S3) ? What would you say?

13

Statement Coverage Testing

• Statement coverage based testing aims to devise test cases that collectively exercise all statements in a program.

foo(1,1)

Another Example of Statement Coverage

S1

S2

S3

S4

S5

C1

C2

C3

1

5

6

7

2

4

3

10

9

8

The 4 Linearly Independent Paths Covers:

Path1: includes S1-C1-S2-S5 : edges (1,2,8)Path2: includes S1-C1-C2-S3-S5 : edges ( 1,5,3,9)Path3: includes S1-C1-C2-C3-S4-S5 : edges (1,5,6,4,10)Path4: includes S1-C1-C2-C3-S5 : edges (1,5,6,7)

For 100% Statement Coverage, all we need are 3 paths : Path1, Path2, and Path3 to cover all the statements (S1,C1,S2,C2,S3,C3,S4,S5) - - - no need for Path4 - - - - !!

Statement Coverage

Now - - -

What do you think of a Software Companywhich boasts that they test every statementin the software before release?

Try this on some students or other professorsand see their reaction.

Branch Coverage Method(Also DD-Paths of our text book)

1. Identify all the decisions

2. Count all the branches from each of the decisions ( out-degree of the decision node)

3. Pick the minimum number of paths that will cover all the branches from the decisions.

Branch Coverage Method

S1

S3

S2

C1

1

4

2

3

One decision C1 :

B1 : Path1 : C1 – S3 B2 : Path2 : C1 – S2

Are both Path1 and Path2 needed to cover Branch1 and Branch2 from C1?

Branch 1 Branch 2

2 linearly independent paths cover :

B1 : Path1 : C1 – S3: B2 : Path2 : C1 – S2 – S3 :

Another Example of Branch Coverage

S1

S2

S3

S4

S5

C1

C2

C3

1

5

6

7

2

4

3

10

9

8

The 3 Decisions and the Branches:

C1: - B1 : C1- S2 - B2 : C1- C2 C2: - B3 : C2 – S3 - B4 : C2 – C3 C3: - B5 : C3 – S4 - B6 ; C3 – S5

We need: Path1 to cover B1, Path2 to cover B2 and B3, Path3 to cover B4 and B5, Path4 to cover B6

The 4 Linearly Independent Paths Cover:

Path1: includes S1-C1-S2-S5Path2: includes S1-C1-C2-S3-S5Path3: includes S1-C1-C2-C3-S4-S5Path4: includes S1-C1-C2-C3-S5

19foo(1,1) and foo(0,1) 

Branch Coverage

How do you feel about a Software Companywho states that they test every branch in theSoftware before release?

Does All Branch Coverage subsume All Statements Coverage?

McCabe’s Cyclomatic Number

• Is there a way to know how many linearly independent paths exist?– McCabe’s Cyclomatic number which we used to

study program complexity may be applied. There are 3 ways to get the Cyclomatic Complexity number from a flow diagram.

• # of binary decisions + 1 • # of edges - # of nodes + 2• # of closed regions + 1

McCabe’s Cyclomatic Complexity NumberEarlier Example

S1

S3

S2

C1

1

4

2

3

We know there are 2 linearly independentpaths from before: Path1 : C1 – S3 Path2 : C1 – S2 – S3

McCabe’s Cyclomatic Number:

a) # of binary decisions +1 = 1 +1 = 2

b) # of edges - # of nodes +2 = 4-4+2 = 2

c) # of closed regions + 1 = 1 + 1 = 2

Closed region

McCabe’s Cyclomatic Complexity NumberAnother Example

S1

S3

S2

C1

1

4

2

3

McCabe’s Cyclomatic Number:

a) # of binary decisions +1 = 2 +1 = 3

b) # of edges - # of nodes +2 = 7-6+2 = 3

c) # of closed regions + 1 = 2 + 1 = 3

Closed Region

C2

S47

5

6

Closed Region

There are 3 Linearly Independent Paths

An example of 2n total path

S1

C1

C2

C3

S2

S3

S4S5

2

1

3

4

5

6

8 9

7

Since for each binary decision, there are 2 paths andthere are 3 in sequence, there are 23 = 8 total “logical” paths path1 : S1-C1-S2-C2-C3-S4 path2 : S1-C1-S2-C2-C3-S5 path3 : S1-C1-S2-C2-S3-C3-S4 path4 : S1-C1-S2-C2-S3-C3-S5

path5 : S1-C1-C2-C3-S4 path6 : S1-C1-C2-C3-S5 path7 : S1-C1-C2-S3-C3-S4 path8 : S1-C1-C2-S3-C3-S5

How many Linearly Independent paths are there?Using Cyclomatic number = 3 decisions +1 = 4

One set would be: path1 : includes edges (1,2,4,6,9) path2 : includes edges (1,2,4,6,8) path3 : includes edges (1,2,4,5,7,8) path5 : includes edges (1,3,5,7,8)

Note 1: with just 2 paths ( Path1 and Path8) all the statements are covered.

Note2: with just 2 paths ( Path1 and Path8) all the branches are covered.

Example with a Loop

S1

S2

S3C1

3

2

1

4

Total number of ‘logical’ paths may be “infinite” (very large) because of the loop

Linearly Independent Paths = 1 decision +1 = 2

path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 )

“One path” will cover all statements: path2 : S1-C1-S2-C1-S3 (S1,C1,S2,S3)

“One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3)

*** Note***: To get one path --- S2 must somehow change C1

Example with a Loop (cont.)

S1

S2

S3C1

3

2

1

4

Total number of ‘logical’ paths may be “infinite” (very large) because of the loop

Linearly Independent Paths = 1 decision +1 = 2

path1 : S1-C1-S3 (segments 1,4 ) path2 : S1-C1-S2-C1-S3 (segments 1,2,3,4 )

“One path” will cover all branches: path2 : S1-C1-S2-C1-S3 branch1 (C1-S2) and branch 2 (C1-S3)

Should we considerthe loop boundaries in C1? & also bring in our boundary value testing thoughts?

More on Linearly Independent Paths

• In discussing dimensionality, we talks about orthogonal vectors (or “basis” of vector space) .

– (e.g.) Two dimensional space has two orthogonal vector from which all the other vectors in two dimension can be obtained via “linear combination” of these 2 vectors:

• [1,0]• [0,1]

[1,0]

[0,1]

[2,4]

e.g. [2,4] = 2[1,0] + 4[0,1]

More on Linearly Independent Paths

• A set of paths is considered to be a Linearly Independent Set if every path may be constructed as a “linear combination” of paths from the linearly independent set. For example:

C1

C2

S1

S2

6

5

4

3

21

We already know: a) there are a total of 22=4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths.

path1

path2

path3

path4

1 2 3 4 5 6

1 1 1

1 1

1 1 1

1 1 1 1

path 4 = path3 + path1 – path2 = (0,1,1,1,0,0)+(1,0,0,0,1,1)- (1,0,0,1,0,0) = (1,1,1,1,1,1) - (1,0,0,1,0,0) = (0,1,1,0,1,1)

We pick: path1,path2 and path3 as the Linearly Independent Set

logical paths

Other Sets of Linearly Independent Paths

-Consider the set of linearly independent paths: 1, 2 & 4 instead.

- Can we get Path3 = (0,1,1,1,0,0)?-Consider path3 = path2 + path4 – path1 = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (1,0,0,0,1,1) = (0,1,1,1,0,0)

path1

path2

path3

path4

1 2 3 4 5 6

1 1 1

1 1

1 1 1

1 1 1 1

-Consider another set of linearly independent paths: 2, 3 & 4 instead.

- Can we get Path 1 = (1,0,0,0,1,1))?-Consider path1 = path2 + path4 – path3 = (1,0,0,1,0,0,)+(0,1,1,0,1,1) – (0,1,1,1,0,0) = (1,0,0,0,1,1)

More on Linearly Independent Paths

C1

C2

S1

S2

6

5

4

3

21

path1

path2

path3

path4

1 2 3 4 5 6

1 1 1

1 1

1 1 1

1 1 1 1

Note : Although path1 and path3 are linearly independent, they do NOT form a Linearly Independent Set because no linear combination of path1 and path3 can get , say, path4.

We already know: a) there are a total of 22=4 logical paths. b) 1 path (path4) will cover all statements c) 2 paths will cover all branches. d) 2 branches +1 = 3 linearly independent paths.

More on Linearly Independent Paths

• Because the Linearly Independent Set of paths display the same characteristics as the mathematical concept of basis in n-dimensional vector space, the testing using the Linearly Independent Set of paths is sometimes called the “basis” testing.

• The main notion is that since the “linear independent set of paths” as a set can span all the paths for the design/code construct, then basis testing covers the “essence” of the whole structure.

Then ------ is there a way to find a Linearly Independent Set ?

An Algorithm to Find the Basis Set1. Select a baseline path, an arbitrary, normal execution path

that contains as many decisions as possible.2. Retrace the baseline path and “flip” each of the decision

encountered; each flip creates a new path. Continue until all the decisions are flipped

3. The basis set is composed of all the paths generated from steps 1 and 2 above

C1

C2

S1

S2

6

4

3

21

1. pick baseline path P1: C1 –S1- C2 – S2: <0,1,1,0,1,1> 2. flip C1 P2 : C1 – C2 – S2 : <1,0,0,0.1,1>3. flip C2 P3: C1 - C2 : <1,0,0,1,0,0>

Cyclomatic # = 2+ 1 = 3;So there are 3 linearly independent paths

5

Can we get the 4th path : C1 – S1 – C2 : <0,1,1,1,0,0> from the above basis set?

How about : (P1 + P3) – P2 ?(P1 + P3) – P2 = (<0,1,1,0,1,1> + <1,0,0,1,0,0>) - <1,0,0,0,1,1> = <1,1,1,1,1,1> - <1,0,0,0,1,1> = <0,1,1,1,0,0> = P4

Total Possible Logical Paths can be Big!

s1

c1

s2 c3

s3

c2

c4

s4

4

1

2

35

There are 5 choices each time we process throughthis loop.

For passing through the loop n times we have 5n

possibilities of logical paths.

If we go through the loop just 3 times, we have (5)3 = 125possible paths!

You also note that for nloops there are possible 3n decisions, and we may have 3*4 = 12 decisions forlooping 4 times.

Paths Analysis Summary

• Interested in total number of all possible combinations of “logical” paths

• Interested in Linearly Independent paths• Interest in Branch coverage or DD-path• Interested in Statement coverage

Which one do you think is the largest set, next largest, - - - , etc.?

Compare this list with page 136 of your text - - - what are we missing?

Definition for DD-path

• DD (decision-decision) path is a path of nodes in a directed graph.

• A chain is a path in which:– Initial and terminal nodes are distinct (not just one node)– All interior nodes have in-degree = 1 and out-degree =1

• A DD-path is a chain in a program graph such that the following are included in the chain:

1. It consists of a single node with in-degree = 0 (initial node)

2. It consists of a single node with out-degree = 0 (terminal node)

3. It consists of a single node with in-deg => 2 or out-deg => 2

4. It consists of a single node with in-deg = 1 and out-deg = 1

5. It is a maximal chain of length => 1.

Condensation of Code to Table then to Graph

1. Program Triangle2. Dim a, b,c As Integer3. Dim IsTriangle As Boolean

4. Output ( “enter a,b, and c integers”)5. Input (a,b,c)6. Output (“side 1 is”, a)7. Output (“side 2 is”, b)8. Output (”side 3 is”, c)

9. If (a<b+c) AND (b<a+c) And (c<b+a)10. then IsTriangle = True11. else IsTriangle = False 12. endif

13. If IsTriangle14. then if (a=b) AND (b=c)15. then Output (“equilateral”)16. else if (a NE b) AND (a NE b) AND (b NE c)17. then Output ( “Scalene”)18. else Output (“Isosceles”)19. endif20. endif21. else Output (“not a triangle”)22. endif23. end Triangle2

Psuedo-code

Def of DD-paths on

Page 140

code statementPath/nodename

DD-pathDef. (prev.page)

Skip 1- 3(or w/4)4 first 15 – 8 A 5

9 B 310 C 411 D 412 E 313 F 314 H 315 I 416 J 317 K 418 L 419 M 320 N 321 G 422 O 323 last 2

Condensation Graph from Table

first

A

B

C D

E

F

G H

I J

K L

MN

O

Last

- Statements coverage - 4 paths- Branch (DD-path) coverage - 4 paths- Cyclomatic # = 4+1 = 5 - 5 lin. Ind paths- All combinations - 8 paths

Closer Look into Path Testing

• Look at 2 paths from A to E. The paths of either it is a triangle or not a triangle.

– But there are many combinations to get “not triangle” ; so we still need to consider utilizing boundary values and equivalence class. Just one test case to cover the path may not be enough

• Look at the path that leads from D to L (D= not triangle and L= Isosceles triangle). Is that path possible - - - can you generate a test case for that path?

• Look at the path from C to G (C = Is a triangle and G = output “not triangle”) - - - is this a possible path?

– There may be logical dependencies that prevent us from generating test case to traverse certain paths. Thus we also need to consider employing decision tables from Black Box testing technique.