An Introduction to Search-Based Software Testing

7/27/2019 An Introduction to Search-Based Software Testing

1/41

Genetic AlgorithmsSearch-based Software Testing

An Introduction to Search-based Software Testing

Raluca Lefticaru

University of Pitesti

05.12.2008

R. Lefticaru An Introduction to Search-based Software Testing
http://find/http://goback/


2/41


Outline

1 Genetic AlgorithmsOverviewSelection Strategies

Recombination (Crossover)Mutation

2 Search-based Software TestingSearch-based Software Engineering

Search-based Software TestingStructural Search-based TestingEvolutionary State-based Testing

http://find/


3/41


OverviewSelection StrategiesRecombination (Crossover)Mutation

Genetic Algorithms

Clasa particulara de algoritmi evolutivi, inspirati din principiulselectiei naturale (C. Darwin).

Sunt algoritmi probabilisti, care combina elemente de cautaredirijata si cautare aleatoare, realizand un echilibru aproapeoptimal ntre explorarea spatiului starilor si exploatarea celormai bune solutii gasite.

Lucreaza cu o populatie de indivizi (solutii potentiale aleproblemei cautare globala) si folosesc tehnici inspirate din

biologie precum selectia, ncrucisarea (reproducerea) simutatia.

Algoritmul simuleaza un proces de evolutie: la fiecaregeneratie, solutiile bune se reproduc, iar cele relativ slabe nusupravietuiesc.

http://find/


4/41



Genetic Algorithms

De regula elementele populatiei sunt codificate n formabinara.

Operatorul de ncrucisare este cel principal, cel de mutatie are

un rol secundar.AG folosesc functii de performanta obtinute prin transformarisimple ale functiei obiectiv.

Nu este necesar ca functia obiectiv sa fie derivabila sau sandeplineasca proprietati speciale de convexitate.

Pot gasi solutii optime sau aproape optime cu o mareprobabilitate.

Sunt folositi n special n rezolvarea problemelor de optimizarecombinatoriala.


O i


5/41



Genetic Algorithms

X spatiul de cautare (spatiul starilor problemei).

Individ (genotip, cromozom, string) al unei populatii = unelement din X, adica o solutie posibila a problemei.

P(t) = {xt1, . . . , xtn} populatia de indivizi de la momentul t.

P(t) reprezinta o generatie. P(t) nu este multime, poatecontine duplicate.

Cromozomul (purtatorul informatiei genetice) este o structura

liniara formata din gene (trasaturi, caractere).Genele se gasesc in cromozom pe anumite pozitii (loci).

Alelele reprezinta valorile genelor.


O i


6/41



Genetic Algorithms

Genotipul reprezinta o solutie potentiala a problemei, iarfenotipul valoarea acestuia.

Evaluarea calitatilor indivizilor se face cu ajutorul unei functii

de fitness (performanta).Operatorul de recombinare R : Xp Xq realizeaza otransformare de tipul (p, q) n care din p parinti iau nastere qdescendenti.

Operatorul de mutatie este o transformare unaram : X X, care realizeaza mici perturbari ale indivizilor.

Supravietuirea determina n ce masura indivizii unei generatiisupravietuiesc n urmatoarea.


Overview
http://find/


7/41



Minimizarea unei functii n Matlab, folosind GADS

(Genetic Algorithms and Direct Search Toolbox)

Exemplu

Functia

Ras(x) = 20 + x21 + x22 10(cos 2x1 + cos 2x2)

are un punct de minim global in (0, 0).Reprezentarea functiei, gasirea minimului folosind interfata graficaa toolbox-ul GADS si apelarea n linia de comanda:

>> plotobjective(@rastriginsfcn,[-5 5; -5 5]);

>> optimtool(ga);

>> [x fval exitflag] = ga(@rastriginsfcn, 2);


Overview
http://find/


8/41



Graficul functiei Rastrigin


Overview
http://find/


9/41



Tipuri de algoritmi genetici

AG standard

codificare binara

lungimea cromozomuluifixa

marimea populatiei fixa

AG hibrizi

codificare reala

lungimea cromozomuluivariabila

marimea populatiei

variabila


Overview
http://find/


10/41



Structura unui algoritm genetic

procedure genetic algorithm

begin

t 0initialize P(t)

while (not termination-condition) do

begin

evaluate P(t)

t t + 1

select P(t) from P(t-1)recombine P(t)

mutate P(t)

end

end


Overview
http://find/


11/41


Selection StrategiesRecombination (Crossover)Mutation

Descriere

P(t) = {xt1, . . . , xtn} populatia de indivizi de la iteratia t.

Fiecare solutie posibila xti este evaluata, valoarea rezultatareprezentand fitness-ul acesteia.

La iteratia t + 1 este formata o noua populatie prin selectareaunor indivizi din populatia anterioara (pasul select).

Cativa membri ai acestei populatii suporta transformari prinintermediul operatorilor genetici (mutatie si recombinare),

rezultand noi indivizi.

Procedeul este repetat pana cand criteriul de oprire estesatisfacut.



12/41

Overview


13/41



Mecanisme de selectie

Algoritmul Monte Carlo (alg. ruletei)

Pentru a selecta noua populatie, se repeta urmatorul procedeu den ori:

Se genereaza aleator un numar real r din intervalul [0..1].

Daca r < q1 atunci se selecteaza primul individ x1, altfel sealege individul xi pentru care qi1 < r qi.

ObservatieUnii indivizi vor avea copii multiple n populatia intermediara, altiinu vor aparea deloc.


GOverviewS S
http://find/


14/41



Alte mecanisme de selectie

Schimbarea de scala a functiei de fitness: liniara sauexponentiala; statica (aceeasi pentru toate generatiile) saudinamica.

Selectia bazata pe ordonare, unde n = nr. de indivizi, rirangul individului i, q [1, 2] presiunea de selectie

pi =1

n

q

(ri 1)2(q 1)

(n 1)

Selectia de tip turnir.

Strategii elitiste.


G i Al i hOverviewS l i S i
http://find/


15/41



Selectie bazata pe ordonare

Figure: Inainte de ordonare

Figure: Dupa de ordonare


G ti Al ith sOverviewS l ti St t i s


16/41



Recombinarea (Incrucisarea)

Combina trasaturile a doi cromozomi parinti rezultand urmasicare mostenesc aceste trasaturi.

Cei doi urmasi rezulta cel mai des prin interschimbarea

anumitor segmente din parinti.Tipuri: ncrucisare cu un punct de taietura; cu mai multepuncte de taietura; ncrucisare adaptiva; segmentata; cuamestec; uniforma.


Genetic AlgorithmsOverviewSelection Strategies


17/41



Incrucisarea cu un punct de taietura

pc probabilitatea ncrucisarii, parametru al algoritmuluigenetic; numarul asteptat de cromozomi care vor participa lancrucisare este pc n

pentru a selecta indivizii care vor fi ncrucisati:pentru fiecare individ, se genereaza un numar aleator r nintervalul [0..1]daca r < pc atunci selecteaza individul respectiv pentruncrucisare

pentru a realiza efectiv ncrucisarea (cu un punct de taiere):alegem cate o pereche de indivizi, selectam aleator un punctde taiere i {1, 2, . . . , n 1}, unde n este lungimeacromozomului, si interschimbam segmentele.


http://find/


18/41



Incrucisarea euristica

Incrucisare pentru codificarea reala: Michalewicz

Foloseste valorile functiei obiectiv n determinarea directieicautarii.

Pentru parintii x = (x1, . . . , xn), y = (y1, ...,yn), x maiadecvat decat y, se genereaza un singur urmasz = (z1, ...,zn), zi = (xi yi) + xi, (0, 1) aleator.

In loc de (0, 1) aleator, se poate folosi si un fixat.

z se va gasi n afara segmentului [x, y], pe dreapta suport aacestuia, la o distanta mai mica de parintele cu fitness maibun.


http://find/


19/41



Mutatia

Altereaza una sau mai multe gene alese arbitrar dintr-uncromozom.

pm probabilitatea mutatiei, parametru al algoritmului genetic;

numarul asteptat de gene mutate este pm m n, mlungimea unui cromozom;

Mutatia unei gene (bit): modificarea bitului din 0 in 1 sauinvers

Pentru fiecare cromozom, si fiecare gena a acestuia:se genereaza un numar aleator r din intervalul [0..1]daca r < pm atunci se modifica valoarea genei

Pastreaza variabilitatea n populatie


http://find/


20/41



Bibliografie AG

D. Dumitrescu.Algoritmi genetici si strategii evolutive - Aplicatii n inteligenta

artificiala si n domenii conexe..Editura Albastra, Cluj-Napoca, 2000.

D. Goldberg.Genetic Algorithms in Search, Optimization and Machine

Learning..Addison-Wesley, 1989.

D. Whitley.A genetic algorithm tutorial.

Statistics and Computing, 1994.

An introduction to genetic algorithms.

http://www.obitko.com/tutorials/genetic-algorithms/R. Lefticaru An Introduction to Search-based Software Testing

http://www.obitko.com/tutorials/genetic-algorithms/http://www.obitko.com/tutorials/genetic-algorithms/http://find/


21/41

gSearch-based Software Testing

gRecombination (Crossover)Mutation

Biblioteci pentru lucrul cu AG

Genetic Algorithms and Direct Search Toolbox (MATLAB):http://www.mathworks.com/products/gads/

JGAP - Java Genetic Algorithms Package:http://jgap.sourceforge.net/

JAGA - Java API for Genetic Algorithms:http://www.jaga.org/

ECJ - A Java-based Evolutionary Computation ResearchSystem: http://cs.gmu.edu/eclab/projects/ecj/

GAlib - A C++ Library of Genetic Algorithm Components:http://lancet.mit.edu/ga/

GAUL - Genetic Algorithm Utility Library (for C):http://gaul.sourceforge.net/


Genetic AlgorithmsSearch-based Software EngineeringSearch-based Software Testing
http://www.mathworks.com/products/gads/http://jgap.sourceforge.net/http://www.jaga.org/http://cs.gmu.edu/~eclab/projects/ecj/http://cs.gmu.edu/~eclab/projects/ecj/http://cs.gmu.edu/~eclab/projects/ecj/http://lancet.mit.edu/ga/http://gaul.sourceforge.net/http://gaul.sourceforge.net/http://lancet.mit.edu/ga/http://cs.gmu.edu/~eclab/projects/ecj/http://www.jaga.org/http://jgap.sourceforge.net/http://www.mathworks.com/products/gads/http://find/


22/41

Search-based Software Testing Structural Search-based TestingEvolutionary State-based Testing

Search-based Software Engineering (SBSE)

Description

Search-based software engineering (SBSE) is a relatively newapproach to transform the software engineering problems intooptimization problems, which can be further solved by applying

metaheuristic search techniques. The term of SBSE was first usedby Harman and Jones, 2001, although there are some previouspapers on this topic.

Applications

Software testing, requirements engineering, automatedmaintenance, service-oriented software engineering, compileroptimization, quality assessment, project planning and costestimation.


Genetic AlgorithmsSearch-based Software EngineeringSearch-based Software Testing
http://find/


23/41


Search-based Software Engineering

Harman and Jones, Information & Software Technology, 2001

This paper claims that a new field of software engineeringresearch and practice is emerging: search-based softwareengineering. The paper argues that software engineering is ideal for

the application of metaheuristic search techniques, such as geneticalgorithms, simulated annealing and tabu search. Suchsearch-based techniques could provide solutions to the difficultproblems of balancing competing (and some times inconsistent)

constraints and may suggest ways of finding acceptable solutions insituations where perfect solutions are either theoretically impossibleor practically infeasible. In order to develop the field ofsearch-based software engineering, a reformulation of classicsoftware engineering problems as search problems is required.


Genetic AlgorithmsS h b d S f T i

Search-based Software EngineeringSearch-based Software TestingS l S h b d T i
http://find/


24/41


Number of publications per year from 1976 to 2008

Source: http://www.sebase.org/sbse/publications/R. Lefticaru An Introduction to Search-based Software Testing

Genetic AlgorithmsS h b d S ft T ti

Search-based Software EngineeringSearch-based Software TestingSt t l S h b d T ti
http://www.sebase.org/sbse/publications/http://www.sebase.org/sbse/publications/http://find/


25/41


The ratio of SE research fields involved in SBSE


Genetic AlgorithmsSearch based Software Testing

Search-based Software EngineeringSearch-based Software TestingStructural Search based Testing
http://www.sebase.org/sbse/publications/http://www.sebase.org/sbse/publications/http://find/http://goback/


26/41


The ratio of publications number in the world countries


Genetic AlgorithmsSearch based Software Testing

Search-based Software EngineeringSearch-based Software TestingStructural Search based Testing
http://www.sebase.org/sbse/publications/http://www.sebase.org/sbse/publications/http://find/


27/41


Relevant publications

M. Harman, B. F. Jones.Search-based software engineering.Information & Software Technology, 2001.

P. McMinn.Search-based software test data generation: a survey.Software Testing, Verification and Reliability, 2004.

M. Harman.The current state and future of search based software

engineering.International Conference on Software Engineering, 2007.

Repository of publications on SBSE.http://www.sebase.org/sbse/publications/repository.html



Search-based Software EngineeringSearch-based Software TestingStructural Search-based Testing
http://www.sebase.org/sbse/publications/repository.htmlhttp://www.sebase.org/sbse/publications/repository.htmlhttp://find/


28/41

Search based Software Testing Structural Search based TestingEvolutionary State-based Testing

Search-based Software Testing (SBST)

Characterized by the usage of guided search techniques fortest generation: hill climbing, alternating variable method,simulated annealing, genetic algorithms, genetic programming,particle swarm optimization, artificial immune systems, tabu

search etc.The test aim (cover all branches, obtain the WCET) is firsttransformed into an optimization problem with respect tosome fitness (cost or objective) function.

Search space = the input domain of the test object (program,function).

The search spaces obtained are usually complex,discontinuous, and non-linear, due to the non-linearity ofsoftware Metaheuristic search methods are recommended.



http://find/


29/41

Search based Software Testing Structural Search based TestingEvolutionary State-based Testing

Aplicatii ale AG n testare

Testare structurala: Control Flow Graph (CFG) ControlDependence Graph (CDG). Scopuri: atingerea nodurilor, cailoretc. Pargas et al., Tonella, Tracey, McMinn etc.

Testare functionala: Functia obiectiv obtinuta dinpreconditie si non postconditie, pornind de la o specificatie Z,Jones et al.

Grey-Box Testing: Violarea assert-urilor, obtinerea

exceptiilor.Testare non-functionala WCET de maximizat, iar BCET deminimizat. Wegener et al.



http://find/


30/41

g gEvolutionary State-based Testing

Exemplu

void example(double x, double y){

1 if (x >= 10 && x = 0 && y


31/41

g gEvolutionary State-based Testing

CFG si CDG

1

2

F

F

T

T

5

43

6

1

FT

6

52

3 4

T F

Control Flow Graph Control Dependence Graph



http://find/


32/41

Evolutionary State-based Testing

Functie de fitness (Pargas)

40

20

0

20

40 4020

0 20

40

0

0.5

1

1.5

2

yx

Functia

de

fitness

fitness = dependent executed



Search-based Software EngineeringSearch-based Software TestingStructural Search-based TestingE l i S b d T i


33/41


Functiile obiectiv ale lui Tracey

Predicat Functia obiectiv objrelational

a = b if abs(a b) = 0 then 0 else abs(a b) + Ka = b if abs(a b) = 0 then 0 else K

a < b if a b< 0 then 0 else (a b) + Ka b if a b 0 then 0 else (a b) + Ka > b if b a < 0 then 0 else (b a) + Ka b if b a 0 then 0 else (b a) + K

Conectiva Functia obiectiv objlogica

Boolean if TRUE then 0 else Ka b obj (a) + obj(b)a b min(obj(a), obj(b))

a xor b obj ((a b) (a b))R. Lefticaru An Introduction to Search-based Software Testing


Search-based Software EngineeringSearch-based Software TestingStructural Search-based TestingE l ti St t b d T ti
http://find/


34/41


Functie de fitness (Tracey, McMinn)

40

20

0

20

40 4020

0 20

40

0

0.5

1

1.5

2

yx

Functia

de

fitness

fitness = approach level + normalized branch distance

http://find/


35/41


Search-based Software EngineeringSearch-based Software TestingStructural Search-based TestingEvolutionary State based Testing


36/41


FSM pentru clasa Book

Reserved

Available Borrowed

borrow

Borrowed&

Reserved

borro

w

return

reserve

return

cancel

reserve

cancel

(a)



Search-based Software EngineeringSearch-based Software TestingStructural Search-based TestingEvolutionary State-based Testing
http://find/


37/41


Statechart pentru clasa Book

R

A B

bor(int x) [ x > 0 ]

BR

bor(int

x)[x

=getR

esCustI

d()]

ret(int x) [ x = getBorCustId() ]

res(intx)[x>0a

nd

xgetBorrCustId()]

ret(int x) [ x = getBorCustId() ]

can(intx)

[

x=getResCustId

()]

res(intx)[x>0

]

can(intx)

[

x=getResCustId(

)]

Book()

(b)



http://find/


38/41

Evolutionary State based Testing

Fitness pt testare bazata pe FSM

Pentru generarea de date de test se poate folosi:

fitness = approach level + normalized branch levelapproach level {0, 1, . . . ,m 1}

normalized branch level [0, 1)

S1

fm [gm]

TARGETMISSED

approach_level=0

branch_level=obj(g

m)

SmSm-1S2

TARGETMISSED

approach_level=m-1

branch_level=obj(g

1)

TARGETMISSED

approach_level=m-2

branch_level=obj(g

2)

S3

f1 [g1] f2 [g2]

Pentru testarea de conformanta se poate folosi functia:

pre-condition post-condition



http://find/


39/41

o ut o a y State based est g

Exemplu

50

0

50

50

0

50

0

0.5

1

1.5

x1x2

Fitness

value

s = Ares(x1)

// Rbor(x2)

// B x1 > 0 si x2 = x1norm : [0, 101] [0, 1), norm(d) = 1 1.05d,K = 1



http://find/


40/41

y g

Numar mediu de generatii

020

406080

100

3.12E-

14

3.11E-

11

6.25

E-10

1.25

E-09

5.16E-08

1.25

E-07

6.09

E-06

1.85

E-05

2.46E-04

3.47E-03

2.04

E-02

3.07

E-01

Generations

Axa Ox: numarul solutiilor / numarul elementelor din spatiulde cautare

Generare valori pentru 35 de cai din FSM cu lungimi ntre 2 si7, folosind JGAP



http://find/


41/41

Numar mediu de esecuri

0%

2%

4%

6%

8%

3.12E-

14

3.11E-

11

6.25

E-10

1.25

E-09

5.16E-08

1.25

E-07

6.09

E-06

1.85

E-05

2.46E-04

3.47E-03

2.04

E-02

3.07

E-01

Failures

Dimensiune populatie = 20 indivizi, codificati real

Nr. maxim de evolutii = 200

Valori din [10p, 10p], p = 1, 2, 3, 4


Documents

An Introduction to Search-Based Software Testing