Testing Model Transformations

PRUEBAS EN EL DESARROLLO DE SOFTWARE DIRIGIDO POR

MODELOS

Juan de Lara

(trabajo conjunto con Esther Guerra)

modelling&software engineeringresearch grouphttp://www.miso.es

PRUEBAS DE TRANSFORMACIONES DE

MODELOS

Juan de Lara

(trabajo conjunto con Esther Guerra)

modelling&software engineeringresearch grouphttp://www.miso.es

Model-Driven Engineering (MDE).• Meta-Modelling and DSMLs.• Model Transformations.

Testing model-to-model transformations

Conclusions and Open lines

AGENDA

3

MODEL DRIVEN ENGINEERINGIncrease the level of abstraction in software development.

Models are actively used to:• Describe the problem…• Simulate/verify/test…• Generate code for the final application.

Move from the solution space (code-centric) to the problem space:

• Higher levels of productivity and quality.• Less accidental details.

4

MODEL DRIVEN ENGINEERINGFor specific domains:

• Avoid coding the same solutions over and over.

• Families of applications.• Domain-Specific Modelling

Languages.

CodeGenerator

• Modelling, validation and automatic generation of telephony services. 5

MODEL DRIVEN ENGINEERINGFor specific domains:

• Avoid coding the same solutions over and over.

• Families of applications.• Domain-Specific Modelling

Languages.

• Code generation from State-Machines for Railway Signaling Systems. 6

Code generator

MetaEdit+

DOMAIN SPECIFIC MODELLING LANGUAGES

Describe the structure of the domain• Relevant primitives and abstractions.• Relations, features.• Explicit expert knowledge.

7

DOMAIN SPECIFIC MODELLING LANGUAGESDSMLs need not be graphical…

xText 8

MODELS AND META-MODELSThe abstract syntax of DSMLs is defined through a meta-model

• Classes, • Attributes, • Relations.

9

Factory meta-model

Machine

Part

Conveyor

Generator Assembler

inps

outs*

*

* parts

Terminator

1..*

1..*

MODELS AND META-MODELSThe abstract syntax of DSMLs is defined through a meta-model

• Classes, • Attributes, • Relations.

10

«conforms to»

c1:Conveyor

g:Generator

a:Assemblerc2:Conveyor

c3:Conveyor t:Terminator

p2: Partp2: Partouts

outs

inps

inps

outs inps

Factory meta-model

Machine

Part

Conveyor

Generator Assembler

inps

outs*

*

* parts

Terminator

1..*

1..*

OCL CONSTRAINTS

11

Object Constraint Language

Well-formedness rules, which every model should satisfy.

Based on First-Order Logic.

g:Generator

«conforms to»

c:Conveyor

Factory meta-model

Machine

Part

Conveyor

Generator Assembler

inps

outs*

*

* parts

Terminator

1..*

1..*

Factory meta-model

Machine

Part

Conveyor

Generator Assembler

inps

outs*

*

* parts

Terminator

1..*

1..*

context Generator inv: self.inps->isEmpty() and self.outs->size()>0

context Generator inv: self.inps->isEmpty() and self.outs->size()>0context Terminator inv: self.outs->isEmpty() and self.inps->size()>0context Assembler inv: self.inps->size()>0 and self.outs.size()>0…

inps

MODELS AND META-MODELSModels are represented using concrete syntax

• Visual • Textual

No need for a 1-1 correspondence between abstract and concrete syntax elements.

12

asse

MODEL TRANSFORMATIONSModels need to be manipulated for:

• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.

in-place transformations

13…



SourceTarget

14

model-to-modeltransformations



15

MMsrc MMtar

Msrc Mtar

Transformationdefinition

from to

«conforms to» «conforms to»

Transformationexecution

Transformationdeveloper

Final user

model-to-modeltransformations



16

Template languages

query andmodel navigation

17

SPECIFYING MODEL TRANSFORMATIONSModel transformations are not normally specified using general purpose languages

Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …

LHS RHS

G H

work

18

SPECIFYING MODEL TRANSFORMATIONSModel transformations are not normally specified using general purpose languages


L R

G H

K

D

l r

m k m*

f d

P.O. P.O.

SPECIFYING MODEL TRANSFORMATIONS

19

Model transformations are not normally specified using general purpose languages


operation main() { // Epsilon Object Language …}operation Machine work(){ for (c in self.inps) { var p : Part := c.parts.random(); c.parts.remove(p); delete p; } for (c in self.outs) c.parts.add(new Part); }http://www.eclipse.org/epsilon/

rule Conveyor2Place { from c : FAC!Conveyor to p : PN!Place (tokens <- c.parts->size())}




20

rule Conveyor2Place { from c : FAC!Conveyor to p : PN!Place (tokens <- c.parts->size())}




21

SourceTarget

Model-Driven Engineering (MDE).

Testing model-to-model transformations• Challenges• PaMoMo: A specification language for MTs• Specification-based MT testing

Conclusions and open lines

AGENDA

22

TRANSFORMATION TESTINGCHALLENGES

23

Urgently needed• we have found 1000+ issues in a repository of ~100 ATL model

transformations (http://www.eclipse.org/atl/atlTransformations/)

Generation of input models• (very) costly if done manually• are we testing interesting cases?

Oracle function• is the result correct?

• syntactically (model conformant to target meta-model)• semantically (produces the intended model)

TRANSFORMATION TESTING APPROACHES

24

Black box• Meta-model based:

• Input models derived covering the input meta-model. • SAT solving/model finding

• Oracle? Random testing: just check for crashes/conformance• Do the models allow testing all properties of interest?

• Specification (contract) based:• Input models derived from a high level specification• The specification serves as oracle• Specifications for model transformations are still rarely built (OCL?)

White box• Look at the transformation logic to generate the tests• Oracle?

A SPECIFICATION LANGUAGE FOR MODEL TRANSFORMATIONSPaMoMo: A formal specification language based on algebraic graph transformation

• Pattern-based, declarative• Category of graphs and morphisms• Compilation into OCL for practical purposes

Pre-conditions: (+/-) conditions on input modelsPost-conditions: (+/-) conditions on output modelsInvariants: relations between input/output models

• Constructive (+)• Constraining (-)

25

CONSTRAINT GRAPHS (1)Encoding of (meta-)models as (typed, attributed) graphs

26

p0: Placename=“inp”tokens=2

t: Transition

in

p1: Placename=“outp”tokens=0

name=“tr”out

p0

t

p1

“inp”

2

“tr”

“outp”

0

in

out

tokens

name

name

name

tokens

Place Transitionin

out

intString

tokensname name

type

CONSTRAINT GRAPHS (2)Encoding of (meta-)models as (typed, attributed) graphs

• G= (V, EV, D, ED,

sV: EV V,

tV: EV V,

sD: ED V,

tD: ED D)• Plus algebras for data types

Typing as graph homomorphism• Four commuting functions

27

p0

t

p1

“inp”

2

“tr”

“outp”

0

in

out

tokens

name

name

name

tokens

Place Transitionin

out

intString

tokensname name

type

CONSTRAINT GRAPHS (3)To express conditions on input/output models, we need:

• Two graphs typed w.r.t source/target meta-models• Attributes of graphs are variables• A formula relating variables of both• C=Gs, Gt,

28

p0: Placename=Y

m: Machinename=X

X=Y

SATISFACTION OF CONSTRAINT GRAPHSSatisfaction of constraint graphs is checked by pattern matching

• Finding graph homomorphisms

29

p: Place


t: Transitionin p1: Placename=“outp”tokens=0

name=“tr”

out

m1={(p, p0)} m2={(p, p1)}

SATISFACTION OF CONSTRAINT GRAPHSHow to relate constraint graphs and graphs?

30

p: Place


t: Transitionin p1: Placename=“outp”tokens=0

name=“tr”

out

m1={(p, p0)}

tokens=XX>0

SATISFACTION OF CONSTRAINT GRAPHS

31

p: Place

p0: Placename=N1tokens=T1

t: Transitionin p1: Placename=N3tokens=T2

name=N2

out

m1={(p, p0)}

tokens=X

1 X>0

2 N1=“inp” T1=2 N2=“tr” N3=“outp” T2=0

A graph can be seen as a constraint graph• Constraint graph homomorphism = graph morphism + (reverse) formula implication (in the

example )2[X/N1]1

Invariants have a bidirectional nature• Forwards/Backwards satisfaction

32

SATISFACTION OF INVARIANTS

p0: Placename=Y

m: Machinename=X

X=Y

Invariants have a bidirectional nature• Forwards satisfaction

33


m: Machinename=X

p0: Placename=Y

m: Machinename=X

X=Y

m: Machinename=“m1”

p0: Placename=“m1”


Invariants have a bidirectional nature• Backwards satisfaction

34


p0: Placename=Y

m: Machinename=X

X=Y

m: Machinename=“m1”



p0: Placename=Y

EXAMPLES

35

EXAMPLES

36

EXAMPLES

37

EXAMPLES

38

WHAT CAN WE DO WITH AN SPECIFICATION?Generate “reference” implementations [OGLE09]

• Based on graph transformation• Terminating, correct, complete• …but not very efficient

Testing: • Generation of oracles based on QVT [GL13], OCL [GS15]• Generation of input models, combining constraints [GS15]

SAT-based refinement checking w.r.t:• Other specifications, implementations in ATL [BEGL13]

39

[BEGL13] Büttner, Egea, Guerra, de Lara. “Checking Model Transformation Refinement”. ICMT 2013: 158-173[GL13] Guerra, de Lara, et al. “Automated verification of model transformations based on visual contracts”. Autom. Softw. Eng. 20(1): 5-46 (2013)[GS15] Guerra, Soeken. “Specification-driven model transformation testing”. SoSyM 14(2): 623-644 (2015)[OGLE09] Orejas, Guerra, de Lara, Ehrig. “Correctness, Completeness and Termination of Pattern-Based Model-to-Model Transformation”. CALCO 2009: 383-397

40

SPECIFICATION-BASED TESTING

report

selectspecification

coverage criterion

mtunit script

oracle(assertions)

input test models

.xmi

SAT solver model transformation

transformationspecification

.pamomo

transformationimplementation

.etl, .atl…

refers totester mtunit

engine

12

3

designer

developer

Given a specification, generation of:•input models.•OCL assertions. •Test script

FROM BPMN TO PETRI NETS

41

OUR SPECIFICATION LANGUAGE

42

Features of Pamomo:

•Formal, pattern-based, declarative

•Positive and negative requirements

•Compilation into OCL

P(OneStartEvent)

e.outgoing.size()=1

e

Preconditions Postconditions Invariants(conditions on (conditions on (relations between the elementsinput models) output models) in the input and output models)

N(UnconnectedPlaces)

pl.inarcs.size()=0 andpl.outarcs.size()=0

pl

N

t2pl2

+

pl1

t1.name = pl1.name and t2.name = pl2.name

N(ParallelGateway3)

t1

g

t2

+g

43

1. Translation of properties into OCL

GENERATION OF INPUT MODELS

N

t2pl2

+

pl1

t1.name = pl1.name and t2.name = pl2.name

N(ParallelGateway3)

t1

g

t2

+g

Task::allInstances()->exists(t1 | Task::allInstances()->exists(t2 | ParallelGateway::allInstances()->exists(g | t1.outgoing->includes(g) and t1<>t2 and not t2.outgoing->includes(g) )))

pre-1 inv-1

1pos-1

ocli-1oclp-1

44


2. Select coverage criteria (e.g. generate models that allow testing one property at a time, two properties, no property, etc.)

=> the selected coverage induces a different set of ocl expressions, e.g., property1 and property2, not property 1 and property2, etc.


pre-1 inv-1

coverage criteria

1

2

pos-1

ocli-1oclp-1

ocli-1 and ocl-j

45


2. Select coverage criteria (e.g. generate models that allow testing one property at a time, two properties, no property, etc.)

3. Use constraint solver to generate models conforming to the meta-model and satisfying the OCL expressions


pre-1 inv-1

coverage criteria

1

2

pos-1

ocli-1oclp-1

ocli-1 and ocl-j

constraint solver3

model-1

sourcemeta-model

46

• A test suite is generated from the specification

• It contains a test case for each invariant and postcondition, defining:

• assertion for the invariant or postcondition• input models to test (all but those generated from

expressions negating the invariant, as they satisfy the invariant vacuously)

GENERATION OF ORACLE

pre-1 inv-1

coverage criteria

1

2

pos-1

ocli-1oclp-1

ocli-1 and ocl-j

constraint solver3

model-1

sourcemeta-model

mtUnittest script(+ oracle)

4

transformation

report

TOOL SUPPORT

47

generation

PAMOMO•EMF-based tool.•Textual/visual syntax for patterns.•Generation of input models and test scripts. •Model generation by using the USE solver.

mtUnit•EMF-based tool.•Automates transformation testing.

Model-Driven Engineering (MDE).

Testing model-to-model transformations

Conclusions

AGENDA

48

CONCLUSIONSMDE: increase the level of abstraction in software development by the use of (domain-specific) models.

V&V of the different MDE artefacts (meta-models, transformations, code generators)

Model transformation testing:• Generation of input models• Oracle function

A specification language like PaMoMo can be used for both

49

CURRENT LINESStatic analysis of (ATL) model transformations

• Type inference• Quick fixes

DSLs for model mutation• Primitives to generate mutants of models conformant to

arbitrary meta-models• Transformations are also models

Model transformation reuse• Transformations used with different meta-models they were

initially built

50

THANKS!

Questions?

51

http://www.miso.es

[email protected]&software engineering

research group

Software

Testing Model Transformations