Test data generation for covering functionality of database applications

Authors: Sergey Zelenov, Vitaly Omelchenko, Evgeny Kostychev

Institute for System Programming at the Russian Academy of Sciences

Moscow, Russian Federation

Database - applications

• extraction source data• changing values of source data• transformation of input data• loading data • changing in the target some data

Extract

Change

TransformLoad

Change

Source Target

Problem

Cartesian product

combinatorial explosion

problem of analyzing test results

Goal

• cover all branches of the functionality• test set → min

Ideally• one test data set per one functionality branch

Hierarchy of combinators

field1 field2 field3 field4

Cartesian product

Cartesian product

All X with any YAll Y with any X

X Y

Comparison of the toolsDTM Data Generator

Turbo Data

DB Monster

Agenda HTDGen Pinery

generation for one field

random

from tables

from libraries

auto incremental

random by a mask

dependent tables

user procedures

generation for many fields

combinators

hierarchy of different combinators

Method description

Model ofthe SUT

Catalogue ofrequirements

Documentation analysis

Formalization of

requirements

Formulation coveragecriteria

Normativedocuments

Generationof test data

CRITERIA

- …- …- …- …

Coveragecriteria

Testset

ExampleBank credit system

• the client type (V, U)• the credit type (A, B) bonus or penalty• the repaid sum

Model ofthe SUT

Catalogue ofrequirements

Documentation analysis

Formalization of

requirements

Formulation coveragecriteria

Normativedocuments

Generationof test data

CRITERIA

- …- …- …- …

Coveragecriteria

Testset

1. Requirements elicitation

Monthly  ≤  Repay  ≤  Debt

VIP USUAL

Repay < Monthly

USUAL VIP

Repay > Debt

Loan ‘A’ Loan ‘B’ Loan ‘A’ Loan ‘B’

A

Penalty0

Penalty3%

Bonus 2%

Bonus 5%

Bonus 1%

Refund

B

e-mail is specified

Notification

Model ofthe SUT

Catalogue ofrequirements

Documentation analysis

Formalization of

requirements

Formulation coveragecriteria

Normativedocuments

Generationof test data

CRITERIA

- …- …- …- …

Coveragecriteria

Testset

MP  ≤  P  ≤  D

“V” “U”

P < MP

“U” “V”

P > D

“A” “B” “A” “B”

A

0 -3% +2%+5% +1% refund

B

e-mail

Notification

MP to be pay

D debt

P repaid

CL client type (V, U)

CR credit type (A, B)

E e-mail

A.I P<MP && CL="V" =>penalty 0;

P<MP && CL="U" =>penalty 3%;

A.II.1 MP≤P≤D && (CL="V" && CR="B" || CL="U" && CR="A")

=> bonus 2%;

A.II.2 MP≤P≤D && CL="V" && CR="A" => bonus 5%;

A.II.3 MP≤P≤D && CL="U" && CR="B" => bonus 1%;

A.III P>D => refund.

B. E≠"" => notification.

2. Formalization of requirements

Model ofthe SUT

Catalogue ofrequirements

Documentation analysis

Formalization of

requirements

Formulation coveragecriteria

Normativedocuments

Generationof test data

CRITERIA

- …- …- …- …

Coveragecriteria

Testset

3. Formulation of coverage criteria The division rules for parameter P

•MP = D

•MP < DMP=D

P<MP P>MPP=MP

MP<P<DP<MP P>D

P=MP P=D

The division rules for parameters CL and CR

•P < MP → 2 subsets (CL)

•MP ≤ P ≤ D → 4 subsets (CL × CR)

•P > D → 1 set MP  ≤  P  ≤  D

“V” “U”

P < MP

“U” “V”

P > D

“A” “B” “A” “B”

A

0 -3% +2%+5% +1% refund

The division rules for the parameter E

•2 subsets (empty, non-empty)

B

e-mail

Notification

Independent aspects

MP  ≤  P  ≤  D

“V” “U”

P < MP

“U” “V”

P > D

“A” “B” “A” “B”

A

0 -3% +2%+5% +1% 0

B

e-mail

Notification

Model ofthe SUT

Catalogue ofrequirements

Documentation analysis

Formalization of

requirements

Formulation coveragecriteria

Normativedocuments

Generationof test data

CRITERIA

- …- …- …- …

Coveragecriteria

Testset

4. Generation of test data

Description ofDB scheme

Configuration

PineryTest set

Test setTest set

Test set

Configuration of Pinery•Constraints on values of one field

– MP = { 6 }; – D = { 6, 30 }; – E = { "", "…@…" }; – P[MP<D] = {MP-1, MP, (MP+D)/2, D, D+1};– P[MP=D] = {MP - 1, D, D + 1};– CL[ P<MP ] = { "V", "U" };– CR[ P<MP ] = { "A" };– ….

• Constraints on combination method for several fields– combinator( CREDITS ) =

Diagonal( Depend( Depend( Product(MP, D), P), Product(CL, CR) ),

E );

MP  ≤  P  ≤  D

“V” “U”

P < MP

“U” “V”

P > D

“A” “B” “A” “B”

A

0 -3% +2%+5% +1% 0

B

e-mail

Notification

Results

• Our method: 22 tuples• Cartesian product: 64 tuples

Economy increases if• possible values of fields ↗• number of independent aspects ↗

Thank you !