Erlang Code Evolution Control - unich.it

Erlang Code EvolutionControl

1Universitat Politècnica de València2Universidad Politécnica de Madrid

David Insa1, Sergio Pérez1, Josep Silva1, Salvador Tamarit2

LOPSTR 2017 12/10/2017

Content

• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase

• Recording the trace• SecEr tool• Conclusions&FutureWork

Introduction• Indebugging programmers usebreakpoints toobservethe values ofanexpression during an execution

• Is this feature available intesting?

• It would beuseful tofocus the testcaseson an specific point without codemodifications

OURPROPOSALØ Introducethe ability tospecify Points OfInterest (POI) inthe context of

testing

Ø Atechnique tocomparetwo equivalent POIs indifferent versionsofthe same program for Erlang

Old Version

main(X,Y) ->A = X + Y,

D = X – Y,A * D.

New Version

main(X,Y) ->A = add(X,Y),

D = sub(X,Y),A * D.

add(X,Y)->

X + Y.

sub(X,Y) ->

X – Y.

1.- Identify a POI and a set of input functions2.- A test suite is automatically generated

Each test case contains:• A call to an input function with specific arguments• The sequence of values the POI is evaluated to (trace)

3.- Each test case is passed against the newversion and both traces are compared

4.- A report of the success or failure of the testcases is provided

main(5,4)OldVersionTrace:1NewVersionTrace:1

Success

Introduction

We have implemented our approach for Erlanginatool named SecEr

Old Version

main(X,Y) ->A = X + Y,

D = X – Y,A * D.

New Version

main(X,Y) ->A = add(X,Y),

D = sub(X,Y),A * D.

add(X,Y)->

X + Y.

X – Y.

1.- Identify a POI and a set of input functions2.- A test suite is automatically generated

Each test case contains:• A call to an input function with specific arguments• The sequence of values the POI is evaluated to (trace)

3.- Each test case is passed against the newversion and both traces are compared

4.- A report of the success or failure of the testcases is provided

main(5,4)OldVersionTrace:1NewVersionTrace:1

Success

Introduction

We have implemented our approach for Erlanginatool named SecEr

sub(Y,X) ->

main(5,4)OldVersionTrace:1NewVersionTrace:-1

Failure

Content

• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase

• Recording the trace• SecEr tool• Conclusions&FutureWork

TypeAnalysisPhase

TestCaseGeneration

Phase

ComparisonPhase

Input TestCase

FinalReport

The technique in detail

TypeAnalysisPhase

TestCaseGeneration

Phase

ComparisonPhase

Input TestCase

FinalReport

The technique in detail

TypEr PropEr CutErInput InputTypesModule.erl

TypEr PropEr CutErFunction

Type analysis phase

TypEr.erl

Fun

2.- The length ofthe list is unkwnown

3.- The repeated-variablerestriction is ignored

f(1,[1,2,5,6])

f(1,[2,5])

Solution: Consider each clause independently andrefinethe result totypes perclause

TypEr+newcode PropEr CutEr InputsTypes〈Clause

,Types〉Input

f(A,[A,B]) “-spec f(1|2,[1|2|5|6])->…”

1.- The infered types refer tothe whole function

g(0,0)->…g(1,1)->… “-spec g(0|1, 0|1)->…”

g(0,1)

Type analysis phase

.erl

FunTypEr

+newcode PropEr

Propertytest

prop_identity() ->?FORALL(X, any(), id(X) = X).

id(X) -> X.

genPropErTypes Value

CutEr InputsInput〈Clause,Types〉

gen

PropEr

true|false

Type analysis phase

... ...

PropErgen

PropErtypes Input

.erl

FunTypEr

+ newcode PropErPropEr+newcode CutEr InputsInput〈Clause

,Types〉

TypErtypes

TypEr toPropEr

ClauseParameters

2.- The length ofthe list is unknown

3.- Repeated variablerelation is lostSolution: Traverse the list parameters ofthe clause element by element

Solution: Storethe values ofalready treated variables

f(A,[A,B])

Type analysis phase

〈Clause,Types〉

PropErtypes

TypErtypes ≠

.erl

FunTypEr

+newcodePropEr

+newcodeInput

concolicgen

Input

RuntimeErrors

〈Clause,Types〉

CutEr InputsCutEr

+newcode

CutEr

Inputs...

...

Inputs

Type analysis phase

TypeAnalysisPhase

TestCaseGeneration

Phase

ComparisonPhase

Input TestCase

FinalReport

The technique in detail

.erl1

POI1

codeinstrumentator

.erl1’

Input Input execute Trace1inputselector

randomgen

mutatinggenNew? New

InputT

NewInput

F

add

Continue until alimit is reached

Storeall the generated testcases(Input+Trace)

TestCase

Test case generation phase

TypeAnalysisPhase

TestCaseGeneration

Phase

ComparisonPhase

Input TestCase

FinalReport

The technique in detail

.erl2

POI2

codeinstrumentator

.erl2’

TestCase

testcaseselector

Test case

Trace1

Input

execute Trace2 tracecomparator

ComparisonResult

Result

FinalReport

Resultanalyzer

add

Comparison phase

Content

• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase

• Recording the trace• SecEr tool• Conclusions&FutureWork

Recording the trace

• There areseveral tools for tracing executions inErlang• None ofthem allows us tocollect the traceofpatterns• Debuggers will not provide avalue for aPOIif it is inside anexpression whose evaluation failsOURPROPOSAL

Ø Collect the tracesasaside effect when executing the codeØ Approach based on message passing toatracing server

Ø The code needs tobeinstrumented (4STEPS)

{1,B,3} = {1,2,4}

Recording the trace (1 & 2)

1. Obtain andannotate the Abstract Syntax Tree ofthe program.Annotate each node with two lists ofvariables:

2. Find the selected POIinthe ASTwith atop-down traversal:

§ Variablesbeingbound inits subtree§ Variablesthatwere alreadyboundwhen reaching the node

§ Storethe current traversed pathwith tuples oftheform (Node,ChildIndex)

§ The result is apath that yields directly tothe POI

A

B C

D

POI

GF

(POI,1)

(F,1)

(E,2)

(B,1)

E

3. Analyze the location ofthe POI• Expressions: Add asend command toinform the tracing server• Patterns:Need special treatment

Recording the trace (3)

{1,B,3} = {1,2,4}

Targetexpressions• Pattern-matching• List comprehension• Expressions with clauses:

• if• case• functions• etc.

2

Recording the trace (3)

Dividethe ASTpath into two sub-paths:• PathBefore:Root ->deepest targetexpression• PathAfter:First child ofthe targetexpression ->POI

A

B C

D TE

POI

GF

(POI,1)

(F,1)

(TE,2)

(B,1)PathBefore(B,1),(TE,2)

PathAfter(F,1),(POI,1)

B

TE

F

POI

Recording the trace (4)

4. Perform the actualinstrumentation• Traverse the PathBefore• Transform the code following aruleaccording toPathAfter• TraversePathBefore backwards toupdate the AST

Five exclusiverulestoinstrument expressions• LEFT_PM(pattern-matching)• PAT_GEN_LC (list comprehensions)• CLAUSE_PAT (pattern inexpressions with clauses)• CLAUSE_GUARD (guard inexpressions with clauses)• EXPR(expressions)

A

B C

D TE

GF

B

TE

F

POI

NewSubtree

p = e → p = begin np = e, tracer!{add, npoi}, np end(p = e, _ ) = last(PathBefore)⋀ (_, pos(p)) = hd(PathAfter)

(_, npoi, np) = pfv(p,PathAfter)

(LEFT_PM)if

where

Recording the trace (4)

{1,B,3} = {1,2,4}

{1,B,3} = begin{1,POI,FV} = {1,2,4},tracer ! {add,POI},{1,POI,FV}

end

p = e → p = begin np = e, tracer!{add, npoi}, np end(p = e, _ ) = last(PathBefore)⋀ (_, pos(p)) = hd(PathAfter)

(_, npoi, np) = pfv(p,PathAfter)

p = e

p = begin

np = etracer!{add, npoi}

npend

Content

• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase

• Recording the trace• SecEr tool• Conclusions&FutureWork

SecEr command:$ ./secer -f FILE –li LINE – var VARIABLE [-oc OCCURRENCE]

-f FILE –li LINE – var VARIABLE [-oc OCCURRENCE][-funs INPUT_FUNCTIONS] –to TIMEOUT

SecEr tool

Old_Version_POI

New_Version_POI

SecEr toolhappy0.erl happy1.erl

$ ./secer -f happy0.erl –li 9 – var Happy -oc 1

-f happy1.erl –li 18 – var Happy -oc 1

-funs [main/2] –to 15

SecEr tool

SecEr tool

SecEr tool

Content

• Introduction• The technique indetail• Type Analysis Phase• TestCaseGeneration Phase• Comparison Phase

• Recording the trace• SecEr tool• Conclusions&FutureWork

ConclusionsConclusions

• Combination ofErlang existing tools andmutation toimprove the result• Newapproach toautomatically check the behaviour preservation betweenversions

• Newtracing process that allows for placing aPOIinpatterns,guards andexpressions

Type AnalysisPhase

TestCaseGeneration

Phase

ComparisonPhaseInput Test

Case

FinalReport

{1,B,3} = {1,2,4}

{1,B,3} = begin{1,POI,FV} = {1,2,4},tracer ! POI,{1,POI,FV}

end

FutureWork• Adapt the approach todeal with indeterminism• Increase the information stored intracestoreport non-functional properties such asefficiency

• Allow for the specification ofalist ofPOIs instead ofasinglePOI• Make the tool compatible with tests previously defined by theuser

Future Work

Thank you for your attention!!

Any question?