Automatic Test Generation for Space

Automatic Test Generation for Space

Ulisses Costa Daniela da Cruz Pedro Rangel Henriques

SLATE’12 - Symposium on Languages, Applications and Technologies

June 21, 2012

Ulisses Costa, Daniela da Cruz, Pedro Rangel Henriques Automatic Test Generation for Space

Context

Problem

VST (Visionspace Technologies) provides services related to testingfor ESA and wants to automate the test generation for theOperational Simulator platform.

This presentation appears in the context of a master’s thesis thataims at:

Generate automatically tests for the Operational Simulator

Generate unit tests for the Operational Simulator language –C++

Parametrize the size of the generated data structures and beable to configure other attributes


Motivation

Extract UML and OCL from the existing code

Extract tests from the existing code


OCL Inference

OCL is a language used to describe logic properties about UMLmodels, typically in the form of invariants.The first step is to extract interesting invariants from the code andmatch them with requirements.

Generate UML diagrams from the existing code (easy)

Infer code invariants (hard)

Relate the discovered invariants with the UML diagrams

Relate the discovered diagrams with requirements


White vs. Black Box Testing

Types of tests regarding the knowledge about the code

White Box , there is knowledge about the code, and thisknowledge is used to perform the test generation.

Black Box , there is only knowledge about the requirements andabout how each component should behave.


Approaches 1/2

Specification-based Generation Testing , aka Model Based Testingconsists in testing a program based on the programspecification or on the program model. Test casescan be generated from the specification, withoutconsider the code.


Approaches 2/2

Constraint-based Generation Testing , can be used to select testcases that meet some variable restrictions. Whencombined with symbolic execution, gathersrestrictions along the different paths in the CFG. It ispossible to solve these restrictions and generate testcases.


Current state

By now, we studied different approaches and tools, the moreimportant to our goal are:

Korat, is a mature framework to automatically constructcomplex structures for JAVA

Pex is a White-box testing framework from Microsoft toolthat tries to give total code coverage


Studied Tools - Pex

1 public class Program {

2 public static int BSearch(int x, int n) {

3 return BinarySearch(x, 0, n);

4 }

5 static int BinarySearch(int x, int lo, int hi) {

6 while (lo < hi) {

7 int mid = (lo+hi)/2;

8 Debug.Assert(mid >= lo && mid < hi);

9 if (x < mid) { hi = mid; } else { lo = mid +1; }

10 }

11 return lo;

12 }

13 }

Result x n result Output/ExceptionX 0 0 0X 0 1 1X 0 3 17 1073741888 1719676992 TraceAssertionExceptionX 1 6 2X 50 96 51


Studied Tools - Korat

1 public class LinkedList <T> {

2 public static class LinkedListElement <T> {

3 public T Data;

4 public LinkedListElement <T> Prev;

5 public LinkedListElement <T> Next;

6 }

7 private LinkedListElement <T> Head;

8 private LinkedListElement <T> Tail;

9 private int size;

10 }

LinkedList class invariants (circular doubly linked list):

〈∀ l : l ∈ LinkedList : Head(l) ≡ null ∨ Tail(l) ≡ null ⇔ size(l) ≡ 0 〉 (1)

〈∀ l : l ∈ LinkedList : Tail(l).Next ≡ null 〉 (2)

〈∀ l : l ∈ LinkedList : Head(l).Prev ≡ null 〉 (3)

〈∀ l : l ∈ LinkedList : size(l) ≡ 1⇔ Head(l) ≡ Tail(l) 〉 (4)

〈∀ l : l ∈ LinkedList : 〈∀ e1, e2 : {e1, e2} ⊆⊆ l : 〈∃ e : e ∈∈ l : e1.Next ≡ e ∧ e2.Prev ≡ e 〉〉〉 (5)

〈∀ l : l ∈ LinkedList : 〈∀ e1, e2 : {e1, e2} ⊆⊆ l : e1 ≡ e2 ⇒ i(e1) ≡ i(e2) 〉〉 (6)


Studied Tools - Pex - LinkedList

(a) LinkedList instancegenerated by Pex to testthe method Remove

(b) LinkedList instancegenerated by Pex to testthe method Find

Figure: Examples of instances generated by Pex to the LinkedList class.


Studied Tools - Korat - LinkedList

(a) LinkedListinstance with 2elements

(b) LinkedListinstance with 5elements


Studied Tools - Summary

Summary

Pex uses static analysis and is very efficient in discovering all thepossible execution paths in C# methods. Pex can also be used togenerate classes testcases, but the generated instances does notkeep the invariants of data structures.On the other hand, Korat is the ideal tool to generate datastructures that meet the invariants.


Conclusion and Future work

Pex has proved to be a powerful tool regarding full coverage.

Korat is a very useful tool to generate complex datastructures.

A mix between the static analysis of Pex with Korat’s capability togenerate useful data structures is the path we will follow.The study of pre- pos conditions inference using static analysis[Moy 2009] will be useful to infer OCL rules.


Korat repOK method for LinkedList

1 public boolean repOK () {

2 if(Head == null || Tail == null)

3 return size == 0;

4 if(size == 1) return Head == Tail;

5 if(Head.Prev != null) return false;

6 if(Tail.Next != null) return false;

7 LinkedListElement <T> last = Head;

8 Set visited = new HashSet ();

9 LinkedList workList = new LinkedList ();

10 visited.add(Head);

11 workList.add(Head);

12 while (! workList.isEmpty ()) {

13 LinkedListElement <T> current = (LinkedListElement <T>) workList.

removeFirst ();

14 if (current.Next != null) {

15 if (! visited.add(current.Next))

16 return false;

17 workList.add(current.Next);

18 if(current.Next.Prev != current) return false;

19 last = current.Next;

20 }

21 }

22 if(last != Tail)

23 return false;

24 return (visited.size() == size);

25 }


Technology

Automatic Test Generation for Space