Model-driven rapid prototyping with programmed graph transformations

Contents lists available at ScienceDirect

Journal of Visual Languages and Computing

Journal of Visual Languages and Computing 24 (2013) 441–462

1045-92http://d

☆ Thisn CorrE-m

saller@ede (I. Retu-darm

1 SuState Going at T

2 SuCollabofor the

journal homepage: www.elsevier.com/locate/jvlc

Model-driven rapid prototyping with programmedgraph transformations$

Anthony Anjorin 1, Karsten Saller n,2, Ingo Reimund, Sebastian Oster,Ivan Zorcic, Andy SchürrReal-Time Systems Lab, Technische Universität Darmstadt, Germany

a r t i c l e i n f o

Available online 30 August 2013

Keywords:Rapid prototypingProgrammed graph transformationsMetamodellingSoftware product linesModel-driven testing

6X/$ - see front matter & 2013 Elsevier Ltd.x.doi.org/10.1016/j.jvlc.2013.08.001

paper has been recommended for acceptanesponding author. Tel.: þ49 6151 16 3776.ail addresses: [email protected] (As.tu-darmstadt.de (K. Saller), [email protected]), [email protected] (S. Oster)stadt.de (I. Zorcic), [email protected] by the ‘Excellence Initiative’ of thevernments and the Graduate School of ComU Darmstadt.pported by the German Research Foundrative Research Center (SFB) 1053 “Multi-MeFuture Internet”.

a b s t r a c t

Modern software systems are constantly increasing in complexity and supporting therapid prototyping of such systems has become crucial to check the feasibility of extensionsand optimizations, thereby reducing risks and, consequently, the cost of development.As modern software systems are also expected to be reused, extended, and adapted over amuch longer lifetime than ever before, ensuring the maintainability of such systems isequally gaining relevance.

In this paper, we present the development, optimization and maintenance of MoSo-PoLiTe, a framework for Software Product Line (SPL) testing, as a novel case study for rapidprototyping via metamodelling and programmed graph transformations.

The first part of the case study evaluates the use of programmed graph transforma-tions for optimizing an existing, hand-written system (MoSo-PoLiTe) via rapid prototypingof various strategies. In the second part, we present a complete re-engineering of thehand-written system with programmed graph transformations and provide a criticalcomparison of both implementations.

Our results and conclusions indicate that metamodelling and programmed graphtransformation are not only suitable techniques for rapid prototyping, but also lead tomore maintainable systems.

& 2013 Elsevier Ltd. All rights reserved.

1. Introduction

Modern software systems are constantly increasing insize and complexity and it has become crucial to develop

All rights reserved.

ce by Shi Kho Chang.

. Anjorin),-darmstadt., [email protected] (A. Schürr).German Federal andputational Engineer-

ation (DFG) in thechanism-Adaptation

appropriate techniques to cope with the challenge of devel-oping such systems at a reasonable cost. Rapid prototypingcan be used to check if an extension or optimization of asystem is feasible and performs as expected. As this isaccomplished at a comparably low cost, the risk of makingwrong decisions is avoided early in the development process,thus reducing the effective cost of software development.

As modern software systems are also expected to bereused, extended and adapted over a much longer lifetimethan ever before, ensuring the maintainability of suchsystems is equally gaining relevance.

In this paper, we present a novel case study formetamodelling and programmed graph transformationsand show with our results that our model-driven approachis not only suitable for rapid prototyping but also leads to amore maintainable system.

www.sciencedirect.com/science/journal/1045926X

www.elsevier.com/locate/jvlc

http://dx.doi.org/10.1016/j.jvlc.2013.08.001



http://crossmark.crossref.org/dialog/?doi=10.1016/j.jvlc.2013.08.001&domain=pdf



mailto:[email protected]









A. Anjorin et al. / Journal of Visual Languages and Computing 24 (2013) 441–462442

For our case study, we investigate MoSo-PoLiTe (Model-Based Software Product Line Testing) [8], a framework forSoftware Product Line (SPL) testing that combines andapplies combinatorial testing and model-based testing toSPL feature models [9].

In the MoSo-PoLiTe approach, a feature model thatdescribes the variability in an SPL as a tree of interrelatedfeatures [9], is converted to a Constraint Satisfaction Prob-lem (CSP) via a series of semantics preserving transforma-tion rules that flatten the feature tree appropriately [10].As the sheer number of possible product configurations isimpossible to test with classical approaches, a representa-tive subset of product configurations is determined using acombinatorial criterion, e.g., that the subset must cover allvalid pairwise combinations of all features.

This task equates to solving the CSP derived from theSPL using well-known approaches such as forward check-ing with some extensions. Details concerning how thechosen subset of products to be tested can be mapped toconcrete test cases using a test model are discussed in [11].

As the flattening transformation that converts thefeature model to a CSP is by no means unique and canbe varied and optimized for a concrete CSP solver, rapidprototyping techniques can be applied to test and evaluatedifferent optimization strategies.

An optimization strategy is, for example, to createredundant constraints that do not change the semanticsof the CSP but lead to a reduction of the search space forvalid combinations. This basically results in a trade-off ofmemory (redundant constraints and annotations in theflattened tree) for efficiency (reducing the search spaceand preventing backtracking).

Our contribution in this paper, which is an extendedversion of our workshop paper [12], is to show, using anovel case study from the domain of SPL testing, thatmetamodelling and Story Driven Modelling (SDM)3 aresuitable techniques for rapid prototyping and supportingmaintainability. To this end, we investigate the followingtwo scenarios:

1.

ma

An existing, hand-written system (in our case theconstraint solver of MoSo-PoLiTe) is regarded as astable black-box and is only to be optimized. We showthat SDMs are well suited for rapid prototyping theflattening (feature model to CSP) transformation, eval-uate various optimization strategies to improve theperformance of the constraint solver, and presentmeasurement results for the MoSo-PoLiTe SPL testingframework. This was the main focus of our workshoppaper [12].

2.
A system (in our case the entire MoSo-PoLiTe frame-work, i.e., constraint solver and flattening transforma-tion) is to be either implemented from scratch orcompletely re-engineered in a model-driven fashionvia rapid prototyping using metamodelling and SDMs.To provide a quantitative and qualitative comparisonof the hand-written system with the re-engineered
3 A concrete language for specifying programmed graph transfor-tions.

implementation, we present measurement results forefficiency, memory consumption and implementationeffort. Finally, to compare the maintainability of bothsystems, we discuss a real-world change request thatwe implemented for both systems.

The paper is structured as follows: In Section 2 weintroduce our running example and define the necessaryconcepts used in the rest of the paper. Section 3 discussesthe transformation rules that flatten a feature model to aCSP and explains how the CSP solver works. In Section 4,various optimization strategies are investigated and weshow how these ideas can be rapid prototyped by translat-ing them almost 1-to-1 in concise graph transformationrules. Corresponding optimization results are presented inSection 5.

For the second scenario, an overview of the re-engineered implementation is provided in Section 6 witha systematic and critical comparison of both systems withrespect to runtime efficiency, memory consumption andimplementation effort in Section 7. The change requestused to evaluate aspects related to the maintainability ofboth systems is motivated in Section 8, while the corre-sponding implementation of the change request for theexisting and re-engineered system is discussed inSection 9. Section 10 gives an overview of related workand Section 11 concludes the paper.

2. Software Product Line (SPL) case study

An SPL architecture provides a systematic means ofderiving different applications from a common architec-ture family, reusing common features (units of function-ality) in the process [1]. SPLs are increasing in relevanceand importance as various domains strive to cope with thechallenges of supporting a high degree of variability. TheSPL paradigm, already applied successfully in variousapplication scenarios, promises increased software quality,reduced development and maintenance costs, and adecreased time-to-market [2].

The systematic testing of SPLs, however, is non-trivialas a high degree of variability implies a vast number ofpossible products. Developing a feasible strategy for test-ing SPLs, which reuse the same software components invery different combinations and contexts, poses quite achallenge [3], as testing every valid product individuallyusing classical approaches quickly becomes infeasible withrespect to time and cost, even for a moderate degree ofvariability [4].

An established strategy is to determine a representativesubset of products which are tested in lieu of the completeproduct line. Determining an optimal subset of productswith respect to a chosen test metric is, however, NP-hardas it can be mapped to the minimum cardinality hitting setproblem [5]. Many approaches, thus, use suitable heuris-tics to guide the choice [5–7].

In the rest of this section, basic SPL concepts areintroduced together with our running example.

A feature f represents a system property that is relevantto some stakeholder [13]. Given the set of all featuresF ¼ ff 1; f 2;…; f ng, a Product Configuration PCAPðFÞ is a

alarmsystem

body comfortsystem

human machineinterface

automaticpower window

manual powerwindow

central lockingsystem

powerwindow

automaticlocking

remotecontrol keystatus LED

security

exclude

LEDcentrallocking system

LED powerwindow

LED alarmsystem

LED exteriormirror

require require

require

alarmsystem

body comfortsystem

human machineinterface

automaticpower window

central lockingsystem

powerwindow

automaticlocking

status LED

security

LED centrallocking system

LED alarmsystem

mandatory optional alternative or require exclude

Fig. 1. A feature model and a valid product configuration in concrete syntax.

A. Anjorin et al. / Journal of Visual Languages and Computing 24 (2013) 441–462 443

combination of features that constitute the correspondingproduct. A Feature Model FMðFÞDPðFÞ restricts featurecombinations to valid product configurations PCAFMðFÞthus defining the variability in an SPL [9].

We employ a FODA-like FM [9] withmandatory, optional,or, and alternative features as well as binary excludes andrequires cross-tree dependencies. Our running example is asample SPL from the automotive domain, a Body ComfortSystem (BCS) [14].

Fig. 1 depicts the feature model for the BCS SPL (left) anda valid product configuration (right) in concrete syntax.

A body comfort system consists of the mandatoryfeatures human machine interface and power window andan optional feature security. The feature model also definescross-tree dependencies, e.g., the remote control key fea-ture requires a central locking system and a manual powerwindow excludes the automatic locking feature. A featuremodel can be transformed to a set of conditions inpropositional logic over the features [15]. A valid productconfiguration is a set of features that fulfills all conditions,or, in terms of the concrete syntax used in Fig. 1, a validsubtree that satisfies all cross-tree dependencies.

As our programmed graph transformation language StoryDriven Modelling (SDM) [16] operates on typed graphs (mod-els) in abstract syntax, we need to represent feature models asgraphs typed according to a type graph (metamodel).

In the following and in the rest of the paper, we shalluse the terms model and metamodel instead of typed graphand type graph, respectively.

With our metamodelling tool eMoflon4 [17], metamodelscan be specified using Ecore/EMF as a modelling language(meta-metamodel). Fig. 2 depicts our metamodel for featuremodels which is of course neither unique nor optimal,but has proven to be suitable for our needs. A FeatureModel

consists of a single root Feature and arbi-trary many cross-tree dependencies Exclude/Require,which connect features in the tree. The tree structure is built

4 www.emoflon.org

via Dependencies that connect a parentFeature witheither a single childFeature (for SingleDependencies)or with multiple childFeatures for GroupDependencies.Note that some references are redundant (childFeature forboth subclasses of Dependency) but lead to more efficienttransformation rules (an iterator does not need to be used toretrieve the childFeature of a SingleDependency as ithas a multiplicity of 1 in contrast to 1‥n for the child-

Feature of a GroupDependency).Testing every valid product configuration of the BCS SPL

individually is already quite challenging as this wouldinvolve testing 152 products. For real-world SPLs that aretypically larger,5 individual product configuration testing isno longer feasible.

The solution provided by the MoSo-PoLiTe frameworkis to determine a representative subset of valid productconfigurations PCUT DFMðFÞ6 according to a combinatorialcriterion on FM(F) [8].

Experience from various industrial cooperations showsthat especially pairwise testing results in a reasonablesubset of products, which can be feasibly tested. Basedon the results of applying MoSo-PoLiTe for real-world use-cases, the basic assumption that most errors can be foundby testing all valid pairwise combinations of featuresappears to hold in practice [6]. For our running examplewe could, for example, require that PCUT comprise all validpairwise combinations of features.

For a small subset of the BCS feature model consistingof the three features security, central locking systemand remote control key, all valid pairwise combinationswould be

(security, central locking system), (security, : centrallocking system),(: security, : central locking system), (security, remotecontrol key),

5 Up to 270 features according to www.splot-research.org.6 UT stands for “Under Test”.

www.emoflon.org

www.splot-research.org

Fig. 2. A metamodel for feature models.


(security, : remote control key), (: security, : remotecontrol key),(central locking system, remote control key), (centrallocking system, : remote control key) and (: centrallocking system, : remote control key).

Where : f indicates that feature f is not a part of theproduct. All invalid pairwise combinations would be

(: security, central locking system), (: security, remotecontrol key) and (: central locking system, remotecontrol key).

Using graph transformations to implement the neces-sary steps is advantageous as feature models typically havenumerous cross-tree dependencies and are, thus, complexgraph-like structures. We argue and show with our case-study that graph transformations provide a declarative,high-level means of manipulating such structures andincrease not only productivity (via rapid prototyping) butalso readability and hence the maintainability of thesystem.

7 The input model is transformed destructively into theoutput model.

8 Transformation between instances of the same metamodel.9 We use Fig. n::m to refer to the annotation m in Fig. n.10 This is also depicted using colours: black (retain), green (create),

red (delete).

3. Model-based software product line testing(MoSo-PoLiTe)

The MoSo-PoLiTe framework for SPL testing combinescombinatorial testing with model-based testing methodsand applies this to SPL feature models [8]. In the following,we concentrate on the subtask of determining a represen-tative subset of valid product configurations according to acombinatorial criterion (e.g., pairwise). For a completeoverview of MoSo-PoLiTe, especially of the aspects relatedto model-based testing, we refer to [8,11]. Fig. 3 gives aschematic representation of the most important stepsinvolved in the subset selection process.

3.1. Flattening transformation

The BCS feature model is transformed in a first step to aConstraint Satisfaction Problem ðCSP ¼ ðP;V ; CÞÞ, which con-sists of a set P of parameters, a set V of values, and a set C ofconstraints. Constraints cAC are of the form ⟨p;R⟩ wherep¼ ðp1; p2;…; pnÞ is an n-tuple of parameters and R is ann-ary relation on V. A valid product configuration PC isdefined by a mapping v : P-V that fulfills all constraints(i.e., for c¼ ⟨ðp1; p2;…; pnÞ;R⟩, ðvðp1Þ; vðp2Þ;…; vðpnÞÞAR holds).

As SDMs work in-place,7 they are especially suitable forendogenous8 model transformations and we exploit thisby interpreting a flattened form of the feature model as aCSP. Via a series of transformations rules, a feature modelis flattened (Fig. 3::19) until it consists of a root and twolayers: the first layer represents the parameters of the CSP,while the leaves are the possible values of the correspond-ing parameters. The constraints of the CSP correspond tothe cross-tree dependencies of the flattened featuremodel. Resulting parameters and values are shown for asmall subset of the BCS feature model in Fig. 4.

Fig. 5 depicts an excerpt of one of the 16 rules used forflattening. In the SDM notation, the left-hand side andright-hand side of a transformation rule r¼ ðL;RÞ aremerged together in a single specification. When the ruleis applied to a match in a model, the elements in R\L andL\R are created (denoted by the stereotype create) anddeleted (destroy), respectively. All other elementsðL \ RÞ are retained and do not have a stereotype.10 SDMsalso have an imperative part for specifying basic controlflow, i.e., the order in which rules should be applied. This ishowever synonymous to an action language with the usual

Solver

3

Fig. 3. The MoSo-PoLiTe framework.

Fig. 4. Parameter and value layer for a BCS-subset.


imperative concepts (if/else, loops) and we focus for nowon the declarative model transformation rules.

The rule depicted in Fig. 5 matches a feature parent

that has an optional child-feature child and flattens thispart of the feature tree by deleting the parent–child

connection and pulling up child by creating an optional

dependency to grandParent, the parent feature ofparent.

We refer to [10] for a complete list of rules and theproof that the flattening transformation is indeed seman-tics preserving (i.e., the equivalent of the feature model infirst order logic is retained).

Fig. 5. A transformation rule used to flatten the feature model. (For interpretation of the references to colour in this figure caption, the reader is referred tothe web version of this article.)


3.2. Optimization

In an optimization step (Fig. 3::2), the flattened featuremodel is annotatedwith additional information to improvethe performance of the CSP solver. In our case, optimiza-tion rules also implemented with SDMs (discussed indetail in Section 4) basically add redundant constraintsto the CSP that help the solver to avoid getting caught in adeadlock (partial configuration that cannot be completedto become a valid configuration) and having to backtrack.

3.3. Solver

In a final step, the optimized CSP is solved, i.e., productconfigurations that fulfil all constraints are determineduntil all valid pairwise combinations of features have beencovered. For the scope of this section, the solver is treatedas a well-tested black-box (Fig. 3::3).

The CSP solver applies the following strategy: (i) a listof all pairwise combinations of features is derived from theflattened feature model, (ii) an initial pair is chosen accord-ing to a greedy component11 and the solver attempts toextend it to a product configuration that fulfills all con-straints in the CSP, backtracking if necessary. If this fails,then the initial pair was invalid and is removed from the listof pairs to be covered. If this was successful, all other pairsthat are contained in the determined product configurationare marked as covered, (iii) this is repeated until all pairsare covered.

The basic backtracking approach in step (ii) is comple-mented with a forward check12 that reduces the searchspace whenever a new parameter value is chosen byremoving all parameter values that contradict the currentchoice. As we shall present in the following section, thisalready quite effective forward checking technique can be

11 Based on weights determined by the frequency of pairs.12 We use the term forward check to indicate a look-ahead of

one step.

further augmented by adding redundant constraints(cross-tree dependencies) to the CSP.

4. Optimization strategies

The basic idea of our optimization strategies is toannotate the flattened feature tree (the CSP) by addingredundant constraints. The goal of this optimization is to(1) help filter out pairs of features that are invalid, and(2) further improve the forward check of the solver so thatwrong decisions and backtracking can be avoided.

The following sections present three optimization stra-tegies, implemented as graph transformations, with shortproofs that the semantics of the entire flattening transfor-mation is preserved.

4.1. Transitive closure for requirements

The first and most intuitive rule involves buildingthe transitive closure of all requirement constraints.As depicted in Fig. 6, a chain of two requirements inducesa direct transitive requirement, which is created if it doesnot exist already. These redundant requirement constraintssimplify other rules, which can now assume the transitiveclosure, i.e., 8 val1; val2; val3 : ðval1 requires val2Þ4 ðval2requires val3Þ ) ðval1 requires val3Þ.

To show preservation of semantics, the correspondingsubtree with the extra transitive requirement can betransformed into a set of expressions in first-order logic,which can be reformulated until it is equivalent to the setof expressions of the subtree without the extra require-ment. This is trivial for transitive require constraints.

4.2. Derived excludes

The second optimization rule concerns deriving extraexclude constraints. As depicted in Fig. 7, if a certainvalue valq of a parameter q requires a value valp1 of afurther parameter p, an exclude constraint between valqand another value valp2 of p can be derived if paq4

Fig. 6. Transitive requires.

Fig. 7. Derived excludes.

Fig. 8. Transformation for propagating exclude constraints.


valqavalp14valqavalp24valp1avalp2 and the excludeconstraint does not already exist, i.e., the parameter p canonly be assigned one of its possible values, and valqrestrains the choice to valp1 excluding all other optionsfor p.

Note that our transformation engine implicitly enforcesinjective matches, i.e., no two object variables in thepattern can be mapped to the same model element. Thismeans we do not have to explicitly demand paq4valqavalp1avalp2 in the pattern. Also note how NACs(Negative Application Conditions) are depicted as negativeelements that are cancelled such as noRequire in Fig. 6and noExclude in Fig. 7.

The NACs in all rules are used to forbid the creation ofmultiple constraints by preventing the application of a rulewhen an appropriate constraint already exists. This alsoguarantees that the optimization process terminates.

As in the case of building the transitive closure for require-ments, deriving excludes according to Fig. 7 is also semanticspreserving (note that � represents an exclusive or)

ðvalq requires valp1Þ4ðvalq excludes valp2Þ4 ðvalp1 � valp2Þ ð1Þ

3ðvalq ) valp1Þ4ð:valq3:valp2Þ4ððvalp14:valp2Þ

3 ð:valp14valp2ÞÞ ð2Þ

3ð:valq3valp1Þ4ð:valq3:valp2Þ4 ðvalp13valp2Þ4 ð:valp13:valp2Þ ð3Þ

3ð:valq4:valp14valp2Þ3 ð:valq4valp14:valp2Þ3 ðvalp14:valp2Þ ð4Þ

3ð:valq3valp1Þ4ð:valp13:valp2Þ4ðvalp13valp2Þ ð5Þ

3ðvalq requires valp1Þ4 ðvalp1 � valp2Þ ð6Þ

4.3. Propagated excludes

A further rule depicted in Fig. 8 also adds redundantexcludes. If a value val1 requires another value val2 whichexcludes a further value val3, then val1 also excludes val3.

This rule is also semantics preserving as shown in thefollowing:

ðval1 requires val2Þ4 ðval2 excludes val3Þ4 ðval1 excludes val3Þ ð7Þ

3ð:val13val2Þ4ð:val23:val3Þ4ð:val13:val3Þ ð8Þ

3ð:val14:val2Þ3ð:val14:val3Þ3ðval24:val3Þ ð9Þ

3ð:val13val2Þ4ð:val23:val3Þ ð10Þ

3ðval1 requires val2Þ4 ðval2 excludes val3Þ ð11ÞAfter deriving the initial CSP via the flattening trans-

formation, the rules depicted in Figs. 6–8 are appliedrepeatedly in sequence until the total number of con-straints of the CSP no longer increases.

5. Evaluation of the optimization

To evaluate our optimization, a series of measurementswere performed on a standard PC with a 2.93 GHz IntelCore2 Duo CPU and 4 GB RAM. The underlying operatingsystem and virtual machine was Windows XP ProfessionalSP3 and Java 1.7, respectively, and the amount of time theCPU spent performing actions for the tests was measured.

0500

10001500200025003000350040004500

0% 20% 40% 60% 80% 100%R

untim

e (m

s)% Require From a Total of 40 Initial Cross-Tree Dependencies

without optimizationwith optimizationwith optimization and time for executing optimization rules

0

1000

2000

3000

4000

5000

6000

0 10 20 30 40 50 60 70

Run

time

(ms)

# Initial Cross-Tree Dependencies

without optimizationwith optimizationwith optimization and time for executing optimization rules

0

200

400

600

800

1000

1200

0 10 20 30 40 50 60 70

# P

rodu

ct C

onfig

urat

ions

without optimization

with optimization

0

5000

10000

15000

20000

25000

30000

0 10 20 30 40 50 60 70

# D

etec

ted

Inva

lid P

airs

with optimization



without optimization

Fig. 9. Effects of optimization strategies. (For interpretation of the references to colour in this figure caption, the reader is referred to the web version ofthis article.)


For all tests, a feature model generator was implemen-ted that can generate random feature models of a specifiedsize and with a given number of cross-tree dependencies.Fig. 9 presents our results13 in 4 plots, which are explainedin detail in the following.

Each test was repeated 10 times with randomly gener-ated test data and the measurements were averaged tocompensate for fluctuations and improve accuracy.Furthermore, 10 different sets of randomly generatedcross-tree dependencies were used. All results aredepicted with a 95% confidence interval in each data pointshowing that concrete values may vary, but our results donot depend on a specific choice of constraints.

Although we were able to verify our results for featuremodels consisting of up to 700 features with 10% initialcross-tree dependencies, we chose to use a feature modelwith about 300 features to present our evaluation. Thisensures that results can be compared to real featuremodels available from www.splot-research.org, whichrange in size from about 9 to 290. With feature modelsconsisting of thousands of features our optimizationbecomes a bottleneck in some cases and an appropriatestrategy to restrain the rules as required is left tofuture work.

Plot (a): The runtime (y-axis) of the whole process isshown for a varying number of initial cross-tree depen-dencies (x-axis) ranging from 0 to 70. The performance ofthe process without our optimization (blue dotted line) iscompared with the performance on the optimized CSP

13 A workspace with our implementation can be downloaded fromwww.emoflon.org.

(magenta dashed line) and with the total time (yellowsolid line), which includes the time for executing theoptimization rules as well. Note that 0 initial cross-treedependencies mean that the feature model was usedwithout initial cross-tree dependencies. As the flatteningtransformation, however, always introduces extra depen-dencies to preserve semantics, the resulting CSP, even for 0initial cross-tree dependencies, still contains a consider-able number of constraints derived from the tree structureof the feature model and from the flattening process. Thisexplains why there is a fairly constant 15% reduction inruntime across the x-axis.

As expected, the runtime for all cases increases steadilywith the number of initial cross-tree dependencies. Theseresults show that our optimizations indeed improveperformance and that the actual time required to executethe optimization rules is negligible compared to theimprovement.

Plot (b): For the same configuration, the sets of gener-ated product configurations are plotted (y-axis) for thesame x-axis as in Plot (a). Our optimization clearly leads toa 15–20% reduction of the generated sets. This result iseven more important than a reduction in runtime, as thisis the actual goal of the whole process.

Plot (c): The reason for the reduction in size of thegenerated sets of product configurations lies in theimproved detection of invalid pairs before the CSP issolved. We obtain best results by filtering as many invalidpairs as possible using heuristics, which become moreeffective with the redundant constraints introduced by theoptimization. The number of invalid pairs that could not befiltered and are later detected by the solver is plotted forthe same x-axis as in Plot (a) and Plot (b) showing a 10%

www.splot-research.org

www.emoflon.org


reduction. Less invalid pairs improve the greedy compo-nent14 of the CSP solver as the used weights become moreaccurate, leading to a reduction in size of the set of productconfigurations (Plot (b)).

Plot (d): Using the same feature model but with aconstant number of 40 initial cross-tree dependencies,runtime (y-axis) is plotted for a varying ratio of requireconstraints to exclude constraints (x-axis). The reduction inruntime is higher for a larger number of exclude con-straints (0% initial cross-tree require dependencies) andreduces slightly as more exclude are replaced with requireconstraints (100% initial cross-tree require dependencies).This is to be expected as the solver can better exploitexclude constraints in the forward check to reduce thesearch space.

6. Re-engineering the existing system

Graph transformations are not only suitable for opti-mizing an existing system, but can also be used in amodel-driven fashion for an actual implementation. In thissection, we share our experience with re-engineering ourCSP solver, which was presented in Section 3 as a black-box implemented in Java. The motivation and reasons forre-engineering or implementing a system from scratchusing graph transformations and metamodelling technol-ogies include improved maintainability, readability, and ahigher quality of software. To understand the detailedcomparison of the hand-written system and the re-engineered SDM-based implementation presented inSections 7 and 8, we discuss the structure of the latter inthe following.

6.1. A suitable metamodel for CSPs

Implementing a CSP solver in a model-driven fashionwith SDMs essentially requires reducing the problem to aseries of model transformations. The data structure (Fig. 2)for representing feature models and used for the flatteningtransformation presented in Section 3 could also be usedto represent and solve the resulting CSP, but this would besuboptimal as (i) the solver requires new concepts such asConstraints and Products which are only relevant for CSPsand not necessarily for feature models, and (ii) manyconcepts such as the tree structure in feature models areno longer required and can be removed. We introduced anew metamodel suitable for CSPs, to be used during thesolving process (Fig. 10), although this comes with theprice of an extra model transformation (flattened featuremodel to CSP instance).

A CSP consists as usual of constraints (CSPConstraint)and Parameters. Parameters each have a set of possibleValues, which can be connected via concrete constraints(CSPRequire, NAryExcludes, and BinaryExclude).15 Inour context, a CSP furthermore contains all Tuples, i.e.,

14 This determines the initial pair of features used to start every newproduct configuration.

15 In contrast to the metamodel for feature models, an explicitrepresentation of binary excludes proved to be advantageous for thesolving process.

combinations of values, which are to be covered by thedetermined Solution, i.e., a subset of all possible Products.

At runtime, during the solving process, products con-sisting of values are created and used to cover the list oftuples. A SolutionSpace is used for each parameter toindicate the current set of possible values, which dependson other values of parameters in the current (incomplete)product.

6.2. Overview of the CSP solving process

After introducing the data structure used by the solver,we can now give an overview of the overall process inFig. 11. Note that the process starts from the flattened andoptimized feature model, which already represents theCSP as explained in Section 3 and depicted in Fig. 3. Thefollowing sections discuss the corresponding steps in thesolving process.

6.2.1. Transformation of the flattened feature model to a CSPAs explained previously, the first and preparatory step

is to transform the flattened three-level feature model intoan explicit CSP conforming to the meta model in Fig. 10.This transformation is fairly straightforward and basicallyinvolves interpreting the leaves of the flattened featuremodel as values and the second level of features ascorresponding parameters. The main advantage here is tohave a dedicated data structure for the ensuing solvingprocess.

6.2.2. Tuple generationThe task of the solver is to generate a subset of valid

products, which covers all tuples and is as small aspossible. Depending on the coverage criteria, the list oftuples to be covered must be generated using the values ofthe CSP. For the case of a pair-wise coverage, the tuple (inthis case pair) generation is depicted in Fig. 12. For allpossible pairs of values of parameters in the CSP, a Tuple

is created and added to the CSP if such a tuple does notalready exist. SDMs use control flow constructs tocombine story patterns in a simple activity diagram.Connected to a start node representing the entry point ofthe activity diagram, the story node [find pair],16 is afor each node (indicated via the double border) andcorresponds to a loop over all possible pairs, as all matchesof the contained story pattern are determined and used inthe story node [does exclude exist], which is con-nected via an each time transition. If one of the valuesexcludes the other (note that true n-ary excludes areexempted via the NAC valueC), then the pair isobviously invalid and is filtered out by moving on directlyto the next pair via the Success transition. If this isnot the case, the Failure transition is taken and atuple containing this pair of values is created and addedto the CSP in the story node [add pair to list

of tuples]. The NAC noTuple is used to avoid redundant(identical) pairs. Although it would be possible to invest a

16 Story nodes are indicated in the following with ½⟨name ofstory node⟩�.

Fig. 10. A metamodel for CSPs.

Fig. 11. Overview of the CSP solving process.

17 Adjusted slightly to simplify the discussion.


greater effort to filter out all invalid pairs, experience withthe solver shows that it is more efficient to weed out theremaining invalid pairs on-the-fly, i.e., when generatingproducts. The list of tuples to be covered generated in thisstep contains, therefore, invalid tuples that can never beextended to valid products of the SPL. The for eachconstruct in Fig. 12 is terminated with an End transitionwhich leads to a stop node representing the end of theactivity diagram. Note that bound object variables(e.g., csp in [find pair]) already have a fixed valueand are depicted with a bold border (this is the case forparameters or object variables bound in previous storynodes).

6.2.3. Product generationThe overall process17 involved in generating a subset of

products that covers all tuples is depicted in Fig. 13. Notethat, to improve the readability of SDM diagrams, our toolsupports the extraction of the story pattern in each storynode to a separate diagram, which can be opened via adouble-click on the node. For example, the story node [is

there still a tuple to be covered] contains a simplestory pattern (not shown for brevity), which checks if thereis still a tuple in the CSP that is not yet covered by a product.

Fig. 12. Tuple generation for pair-wise coverage.


If this pattern fails to match for the CSP then the solution ismarked as being complete and the CSP containing thegenerated subset of products is returned via the stop node.

If a match could be determined for the pattern, i.e., atuple yet to be covered could be found, then a new productis created in [build new product], and each value in thetuple is added to the product via the method addValue

(csp, product, value) which is invoked via a colla-boration call.

If this fails then the tuple was invalid and is removedwith the method deleteAndRecurse(tuple, pro-

duct, csp), invoked via a method call expression,18 whichdeletes the tuple and invokes subsetExtraction

recursively.

18 The keyword this has the same meaning as in Java.

If all values could be added then the product is filledwith further values as necessary in [invoke fill pro-

duct], all further tuples covered by the complete productare removed in [remove tuples covered by product]

and the subset extraction process (i.e., product genera-tion) is invoked recursively in [invoke subset extrac-

tion] before returning the resulting CSP.To give a feeling for the transformation, the SDM used

to add values to a product is depicted in Fig. 14. All otherSDMs, e.g., used to remove covered tuples, are of similarsize and complexity.

If the value to be added is already in the product there isnothing to be done and the SDM returns directly with true.

If not, the value can only be added to the product if itscorresponding parameter (determined via the parameter'ssolution space) does not already have a value in theproduct. This check ensures that every parameter of the

Fig. 13. Product generation and tuple coverage.


CSP only has a single value in a certain product (multiplevalues do not make any sense).

If the check fails then the value cannot be added andfalse is returned. After adding the value in [add value],a simple check removes all direct excludes of the valuefrom their respective solution spaces.

Similarly, all directly required values are added to theproduct immediately via the recursive call addValue

(csp, product, val). If this should fail for some reasonthen the value cannot be added and the SDM terminateswith false. After all required values have been added,true is returned.

Fig. 15 depicts the SDM fillProduct, which isinvoked from the main SDM subsetExtraction in theactivity node [invoke fill product] to complete aproduct. If the product is already complete,i.e., all parameters have already been covered, the check[are some parameters not yet covered in pro-

duct?] fails, the product is marked as being valid, andthe SDM terminates with true. If this is not the case, i.e.,there is still at least one parameter to be covered in theproduct, a forward check is used to check if it makes senseto attempt to complete the current product. This look-ahead is implemented with the SDM forwardCheck

Fig. 14. Adding values from the current tuple to the product.


(explained in detail in the next paragraph), and isused to improve efficiency by avoiding unnecessary qbacktracking. If the forward check is successful, the loop[for each value in solution space] iterates overall possible values in the solution space of thecurrent parameter, and attempts each time to add thevalue for the parameter via addValue. If this already failsthen the product and solution space are updated viaresetProductAndSolutionSpace (not explained indetail) and the next value in the solution space is taken.If the value could be added, however, then fillProduct

is invoked recursively, which can either be successful, i.e.,the product could be completed and the method termi-nates, or can fail, in which case the next value in thesolution space is used. If all values have been attemptedthe SDM finally terminates with false.

This recursive search with backtracking (when theproduct and solution space are reset and the next valueis taken) is costly and is avoided as much as possible withthe forwardCheck depicted in Fig. 16. This pattern is to

be understood as follows: the csp and the current pro-duct are already bound. If a match for this pattern can bedetermined, i.e., the forward check fails, then there exists aparameter of the csp with an empty solution spacesolSpace, which does not already have a value in theproduct. In this case such a parameter can no longer becovered and thus the product can never be completedsuccessfully. Note that the solution spaces of all para-meters are maintained, e.g., in the SDM addValue (Fig. 14)and in resetProductAndSolutionSpace invoked infillProduct (Fig. 15).

7. Evaluation of the re-engineered system

To provide a quantitative comparison, we use the samesetup and feature generator as in Section 5, and performruntime measurements for the old hand-written Javaimplementation and the new re-engineered SDM system.As results for randomly generated feature models varystrongly, we decided to present the comparison based on a

Fig. 15. Completing a product with backtracking and forward check.

parameter:Parameter

solSpace:SolutionSpace

product: Product valueInProduct:Value

alternateValue:Value

csp: CSP

+parameter

+solutionSpace

+parameter

+values

+solutionSpace

+values

+products +values

+csp

+parameters

Fig. 16. Forward check used to avoid unnecessary backtracking.


single feature model with 200 features and with an equalnumber of initial require and exclude cross-tree depen-dencies. Starting with 10% (5% requires, 5% excludes), weincrease the percentage of cross-tree dependencies to 40%(x-axis for all plots) and measure runtime, memory con-sumption and the number of products generated toguarantee pair-wise coverage.

Each data point in the plots is calculated by randomlygenerating the same percentage of constraints 5 times forthe same feature model and depicting the average with a95% confidence interval in the plot. To enable a fair compar-ison, we use the exact same flattening and optimizationtransformation for both implementations as presented andevaluated in Section 5, with an appropriate final transforma-tion to the respective form of the CSP as required for the

different implementations. We focus in all cases on theactual CSP solving process and do not include the times forflattening, optimization and CSP generation.

Fig. 17 shows the runtime for the hand-written and theSDM implementation in seconds (y-axis). Note that con-fidence intervals are not shown in this plot to simplify thediagram for presentation purposes.

The area under the first curve (Java in white) is thetime taken by the hand-written Java implementation.Ranging from about 1–1.5 min, it is relatively efficientand scales well with an increasing number of constraints.

The area under the second curve (SDM-Algorithm) isthe time spent by the main solver method of theSDM implementation (subsetExtraction depicted inFig. 13). Ranging from about 8–15 min, this is already

0

1000

2000

3000

4000

5000

6000

7000

8000

10% 12% 14% 16% 18% 20% 22% 24% 26% 28% 30% 32% 34% 36% 38% 40%

Run

time

(s)

Initial Cross-Tree Dependencies

Java SDM-Algorithm SDM-Datastructures SDM-Checks SDM-Navigation SDM-Util

Fig. 17. Runtime measurements for the hand-written and re-engineered system.


roughly 10 times slower than the hand-written imple-mentation.

This factor is doubled when taking the next curve(SDM-Datastructures) into account. This adds the timefor all auxiliary methods in the SDM implementation(tuple generation, initialization) and all getters and settersfor elements in the metamodel.

The area under the remaining curves comprises(i) SDM-Checks for all null checks, instanceof checksand isomorphic checks, (ii) SDM-Navigation for a utilitywhich enables navigation for non-navigable references viaa standard EMF cross-referencer adapter, and (iii) SDM-

Util for collections (EList, Iterator) and calls toequals. This roughly doubles the factor again resultingin a final 15–90 (mean 44, median 40).

An important limitation of our runtime comparison isthat the hand-written code was implemented and opti-mized during a PhD thesis (i.e., for 4–5 years), while theSDM implementation was part of a Master thesis (less than6 months implementation and optimization time). None-theless, the following conclusions can be drawn from theseresults.

�
SDMs are probably not the best choice if runtimeefficiency is of utmost importance. SDMs are, however,advantageous for rapid prototyping and experimentingwith new ideas as in Section 5, and can also be usedsuccessfully as a test generator and oracle for validating ahighly optimized hand-written implementation in Javaor even C/assembler where readability/testability isclearly compromised for maximal efficiency. The latteris referred to as Story Driven Testing and is discussed indetail in [18].
�

19 For example, a require constraint can be converted to a number ofexclude constraints and this speeds up the algorithm in certain cases.

The application scenario of CSP solving is obviously notthe best choice to showcase the strengths andefficiency-related advantages of pattern matching andsearch plan generation. Especially for model transfor-mations that involve mainly structural changes andrelatively large patterns, an optimized search plangenerator can easily beat naive hand-written

implementations and would probably lead to verydifferent results.

�
It is a very important future work to further reduce theoverhead of EMF collections, reverse navigation andnull/isomorphic checks in our SDM code generator.Ideas include using appropriate data structures suchas red–black trees instead of simple lists and exploitingknowledge of the metamodel (inheritance hierarchy) toreduce, e.g., the number of instanceof checks andtype casts in the generated code.
Fig. 18 shows the memory consumption for bothimplementations in MB. Interestingly, EMF appears to bequite efficient with respect to memory consumption andrequires only about double as much memory as the hand-written Java implementation. It also scales quite well anddoes not explode for an increasing percentage of con-straints. Depending on the application scenario andrequirements, there is, therefore, room for a trade-off ofmemory for runtime efficiency, e.g., via appropriate collec-tions as mentioned previously.

Fig. 19 shows the number of generated product config-urations (y-axis) for both implementations. The SDMimplementation produces better results for fewer con-straints but the difference reduces with an increasingpercentage of constraints. Although we have tried to re-implement all heuristics and optimizations in the SDMvariant of the algorithm, there are still a few tricks19 usedin the Java code to reduce backtracking and cut-down thesearch space drastically. This is probably an additionalreason for the large difference in runtime and a conse-quence is apparently that the heuristics/optimizations donot work so well for a fewer number of constraints andhave a detrimental effect on the size of the generatedsubset of product configurations.

0

10

20

30

40

50

60

70

10% 12% 14% 16% 18% 20% 22% 24% 26% 28% 30% 32% 34% 36% 38% 40%

Mem

ory

(MB

yte)


Java SDM

Fig. 18. Memory consumption for old and re-engineered systems.

0

50

100

150

200

250

10% 12% 14% 16% 18% 20% 22% 24% 26% 28% 30% 32% 34% 36% 38% 40%

# Pr

oduc

t Con

figur

atio

ns


Java SDM

Fig. 19. Number of generated subset of products configurations.


In our experience, using SDMs and metamodellingimproves the readability of the system and has a positiveeffect on maintainability. Before presenting a qualitativeevaluation of the re-engineered system with a focus onmaintainability in Section 9, we now discuss a changerequest that was implemented for both systems in thefollowing section.

8. Change request

MoSoPo-LiTe is designed to derive a minimal set ofconfigurations that covers all pair-wise feature interac-tions [8]. This is a very specific application scenario for aconstraint solver even in the domain of SPL engineering.The usual use case in SPL engineering is to derive oneproduct that contains a pre-defined set of selected featureswithin the feature model [19]. Thus, instead of fulfillinga coverage criteria, like testing for a pair-wise featureinteraction, a simple computation of one configurationthat contains the required features is sufficient in mostapplication scenarios. Therefore, the scope of MoSoPo-LiTes' applicability would be extended if the solver wereextended to fit such an application scenario.

There are several reasons to adapt the existing imple-mentation of MoSoPo-LiTe instead of using an alternativeopen-source or commercial off-the-shelf constraint solver.Our solver has proven to be resource efficient and fast inits computation of suitable configurations [6]. This makesMoSoPo-LiTe the solver of choice if the derivation of a

single product configuration has to be computed onresource limited devices, like mobile devices [20].

Mobile devices can be made sensitive to their contex-tual environment by applying methodologies from thedomain of dynamic software product lines. Assigning aset of selected features to a context the mobile device hasto be reconfigured during runtime as soon as the context ischanged. MoSoPo-LiTe can be used to compute theruntime-configuration using a feature model and a set ofselected contextual-features as input.

Fig. 20 depicts an excerpt from the original MoSo-PoLiTe overview from Fig. 3. To adapt our solver to suchan application scenario, we have to compute a singleconfiguration for one pre-defined tuple of features.

To realize the change request stated above, the constraint-solving process itself must be adapted. MoSoPo-LiTe starts itsderivation process for a pairwise-coverage feature interactionby simply selecting two features that may interact witheach other.

This single feature-pair is extended by adding additionaltuples of features to form a list of feature-pairs. If remainingfeatures cannot be included in this list and, therefore, arenot part of the configuration, the configuration is finishedand added to the resulting set of configurations. In this casethe list is emptied and a new configuration is started toinclude the remaining feature-pairs that are not yet coveredby a configuration. The solving process always tries to findthe maximum of feature-pairs that are not yet included in aconfiguration which yields in a set of configurations. Thus,

Fig. 20. Change request for the MoSo-PoLiTe framework.

Table 1Resulting changes for the implementation.

Type ofRequirement

Old New

Input Derived tuples of featuresaccording to n-wisecoverage criterion

A single user-suppliedtuple of requiredfeatures

Output Set of productconfigurations that coversall tuples (guarantees n-wise coverage)

A single productconfiguration thatcontains all requiredfeatures in the suppliedtuple


MoSoPo-LiTe computes a set of configurations that eachcontains n-tuples. The size of the tuples is determined bythe coverage criteria, e.g., two for pairwise coverage.

The new application scenario simplifies this solvingprocess, as depicted in Fig. 20. Instead of computingseveral configurations until a stated coverage criteria ismet, the solving process can be terminated when the firstconfiguration is found that fulfils the specified constraintsand contains the pre-defined tuple of n selected features.Thus, MoSoPo-LiTe must compute a single configurationthat contains one tuple of arbitrary selected features.

In this case, the tuple size depends on the amount ofselected features. This requires a change in the implemen-ted configuration process as listed in Table 1.

9. Implementing the change request

To compare the re-engineered system with the hand-written implementation regarding maintainability, we rea-lized the change request presented in Section 8 for bothsystems. In the following, we explain the steps that werenecessary and share our experience concerning the effortrequired in each case, before discussing our comparison andimportant details and limitations thereof in Section 9.3.

9.1. Adapting the re-engineered system

The first step is to enable a selection of features that thesingle product configuration should contain. This wasaccomplished by adding a single new reference to themetamodel, representing a collection of features that can

be set in the model. Although this change is almost trivial,we still emphasize it as using an explicit metamodelenforces an update of both the “class diagram” and thecode thus ensuring that the metamodel is always synchro-nized with the generated code. As we argue in thecomparison at the end of this section, this is a majoradvantage compared to a class diagram “only” used asdocumentation.

The second step is to adjust the tuple generation.As depicted in Fig. 12, the default generation of pairs mustbe adjusted. Interestingly, the change request requires asingle tuple that ensures that products containing allvalues in the tuple also cover all selected features. Thismeans that the size of the tuple depends on the choice ofselected features and the flattening process. Determiningthe values in the tuple is non-trivial as a mapping betweenfeatures in the original feature model (on which theselection of features is based) must be derived for featuresin the flattened feature model. Without going into furtherdetails, the flattening process leads either to a one-to-onecorrespondence or one-to-many correspondence betweenvalues and features, i.e., the positive value of a flattenedfeature either corresponds directly to the original featureor to a set of original features that were merged during theflattening process.

Fig. 21 depicts the SDM generateTuplesBoundFea-

tures used to generate the required single tuple. If thefeature model has at least one bound feature in [check

for bound features], a new and empty tuple is gener-ated with [create new tuple]. For every value of everyparameter in the CSP ([for each value]), a correspond-ing bound feature for the value is searched for using themapping implemented by the helper method find-

BoundFeatureForValue (not further explained). If acorresponding bound feature could be identified, the valueis added to the tuple ([add value to tuple]).

When the loop terminates, the process is successfulonly if there are no bound features left ([no bound

features left]). If not, the selection of features wasinvalid and the tuple is deleted ([delete tuple]).

After an appropriate tuple has been generated, thesolver can be used as usual to “cover” all tuples. It startswith the single tuple, uses it to initialize a productconfiguration, and tries to complete the product so thatall constraints are fulfilled. If the process fails, the selection

Fig. 21. Implementing the change request for the re-engineered system.


of features is in conflict with the feature model, if itsucceeds, then the process terminates with a single pro-duct configuration as required according to Section 8.

9.2. Adapting the hand-written system

Adapting the hand-written system involved basicallythe same steps. A new collection was introduced for theselection of features, and the same mapping betweenvalues of flattened features and selected features fromthe original feature model had to be derived. Usingthis mapping, a new subclass was implemented for thetuple generation, and the algorithm could be reused

completely after swapping the subclass responsible fortuple generation.

9.3. Comparison of the hand-written and re-engineeredsystem

In general, one can conclude that both systems weremodular enough to implement the change request withoutchanging the core algorithm. Tuple generation was wellseparated and general enough, i.e., pair-wise coverage wasneither hard-coded nor scattered over many classes.

The effort required for implementing and testing/debugging the change request was nonetheless more thandouble for the old system (about 20 h) than for the

20 EMF Compare Framework: http://www.eclipse.org/emf/compare/.21 Note that, e.g., the SDM interpreter MoTE (www.mdelab.de/mote/)

provides a debugger.


re-engineered system (9 h). In the following we first of allstate obvious limitations of our comparison, then discussfactors in order of importance that, in our opinion, led tothe difference in effort.

9.3.1. Limitations of comparisonThe same developer implemented the change request

for both systems, which is important for any kind ofcomparison. We also ensured that the developer was notthe same developer who implemented either the old orthe re-engineered system, therefore excluding extra sys-tem knowledge that could bias the comparison.

A bias in favour of the re-engineered system is that thechange request was implemented first for the old system,leading to a certain learn effect, i.e., basically the sameproblem was solved twice. We did our best to reduce thisbias by providing a clear outline of the solution strategy inan implementation independent manner, i.e., the architectof both systems explained in an abstract manner what hasto be adjusted and how, and this “knowledge” was thenused for each implementation-specific solution.

An important bias in favour of the hand-written versionis, however, that the developer had years of experiencewith Java and just about a single week with SDMs. Betterresults might have been attained with a developer equallyseasoned in Java and SDMs.

9.3.2. Reasons for improved maintainabilityFor our concrete change request, the difference in effort

was mainly due to missing documentation and the extratime (10 from 20 h) was spent “reading” the hand-writtencode and stepping through examples in the debugger togain an adequate understanding of the static structure anddynamic interactions.

The following factors, in order of importance, enabled afaster implementation of the change request for there-engineered system:

An explicit model of the system: In the re-engineeredsystem, explicit metamodels are used directly for codegeneration. This has the same value as documenting thesystem with class diagrams but with the added value thatthe “documentation” is guaranteed to be up-to-date at alltimes. Metamodels improve readability (and thus main-tainability) as all classes and all connections betweenclasses with multiplicities are represented in concisediagrams. Questions such as “Does a Product consist ofParameters or Values?” or “How are Values, Para-meters and Constraints connected?” can be answeredwith a glance at the metamodel. Professional UML toolsalso support multiple diagrams of the same model, so thatdifferent and incomplete views can be provided by thesystem architect to present selections of classes thatcollaborate in a certain way, further improving readabilityand enabling a clear focus on a certain aspect.

Explicit models of control flow: A related advantage ishaving an overview of the control flow of a method as anactivity diagram. If activity diagrams are kept to a reason-able size, they present an abstract view of the control flowand also help to understand what exactly is going on in amethod. As depicted in Fig. 12, each activity node isdocumented with a brief description (present in the

generated code as a comment) and the actual graphpattern can be extracted to an extra diagram (opened viaa double-click). Doing this consequently results in cleardiagrams such as Fig. 12 that provide a view of the controlflow without further details. If the provided descriptionsare well-chosen this can greatly improve readability asrelevant methods and activity nodes can easily besearched for and found. Again, code generation guaranteesthat the activity diagrams reflect the system behaviourexactly and do not diverge with time.

Level of abstraction: In each activity node, either a graphpattern is used (this is referred to as a StoryNode) or anarbitrary (possibly hand-written) Java method is invokedvia a StatementNode. This provides a higher level ofabstraction as StatementNodes typically represent furtherdetails (and can first of all be ignored when understandingthe current method), while StoryNodes serve as a compact,declarative, rule-based specification of the step. Withsome training, graph patterns consisting of nodes, links,NACs and optional elements can be understood faster thancorresponding multiply nested loops and if/else structuresin code.

EMF persistency and comparison frameworks: Last butnot least, using standardized interfaces (in our case Ecore/EMF) has the advantage that various EMF frameworks andtools can be used. In our case, being able to persist thecomplete model at any step via a single command andviewing this snap-shot in the EMF model editor or as adiagram (after a simple model-to-text transformation togenerate a dot representation from which a well-layouteddiagram can be created) was much more productive thanclicking through complex data structures in the Javadebugger. The EMF compare framework20 was also veryuseful for testing purposes as expected model states canbe compared with actual snap-shots also via a singlecommand.

In sum, maintainability is greatly affected by an explicitdocumentation and a higher level of abstraction (read-ability). This is enforced and kept synchronized whenusing metamodelling and SDMs. Even if the hand-written implementation was perfectly documented (andthis must be maintained!) with class diagrams and activitydiagrams, and if private helper methods were conse-quently used to hide low-level details (adequate functionalabstraction), there would still be the advantages of using astandard modelling framework.

Tool support (Enterprise Architect for modelling andEclipse for Java code) was considered adequate in bothcases although debugging of SDMs can be greatlyimproved21 and currently involves a manual search forthe descriptions of activity nodes in the generated codeand stepping through with the standard Java debugger.Although we expected this to be a major hurdle, ourexperiment showed that the regular structure of thegenerated code was quickly understood and could bedebugged productively.

http://www.eclipse.org/emf/compare/

www.mdelab.de/mote/

Fig. 22. Flattening approaches for an alternative parent with alternative children [6].


10. Related work

We classify related work into three groups: alternativeapproaches to SPL testing in general, and approaches thatare similar to ours, but differ in how feature models aretransformed to a CSP, and other case studies of the addedvalue of using graph transformation and metamodelling.

10.1. Alternative approaches to SPL testing

Approaches to SPL testing can be further categorizedinto three subgroups: contra-SPL philosophy, reuse techniques,and subset heuristics. Contra-SPL philosophy approachesignore the inherent reuse in SPLs and are only appropriatefor small SPLs [5]. Reuse techniques aim to reduce thetesting effort for SPLs by reusing test artifacts (e.g., testcases and data) across products of an SPL. This is achievedby incrementally testing products via regression testingtechniques or by appropriately adapting domain testsduring product testing. Incremental SPL testing, as intro-duced in [4], faces the challenges of identifying a suitableproduct to start the process with and identifying whatneeds to be re-tested. Approaches that adapt domaintests often employ model-based testing techniques forSPLs as models can provide suitable mechanisms todescribe variable test-relevant behaviour. We refer to[21] for an extensive comparison of model-based testingtechniques. Approaches that use subset heuristics aim atgenerating a representative subset of products accordingto a coverage criterion such as combinatorial testing [8,6],or requirements coverage [5]. Although SAT solvers canalso be used to solve the basic problem of identifying asubset of products that satisfies a set of combinatorialconstraints on features [6], feature models mainly employbinary constraints over parameters with non-binarydomains for which CSP solvers seem to be a naturalchoice [22]. We have furthermore compared our concreteimplementation with a SAT approach and our solutionproved to be superior in terms of efficiency and the sizeof determined subsets [6].

10.2. Alternative feature model to CSP transformations

The authors of [23] perform a Cartesian flattening totransform feature models into a knappsack problem whichis solved and used to generate representative subsets ofproducts of the SPL. A major difference to our flatteningapproach is that they choose to prohibit an exponentialexplosion of all possible feature combinations via thetransformation but lose semantic equivalence in the pro-cess. As an example, cardinality/or groups are translatedinto XOR/alternative groups of a bounded size [23].

We prefer to retain semantic equivalence between theoriginal and the flattened feature model and control thenumber of combinations via a coverage criterion that canrange from pairwise to n-wise as required. The flatteningtransformation is, in general, not unique and we also differfrom [23] in the treatment of alternative groups beneath analternative parent as depicted in Fig. 22 for an abstractexample taken from [23].

In the Cartesian flattening approach, the features N andO are merged with its parent feature L. If, however,another feature X requires L, this binary constraint mustbe translated into X requires (L, N xor L, O) which is anon-binary constraint. For this reason we use a differentflattening strategy as depicted in Fig. 22 that results in onlybinary constraints, which are handled better by our subsetextraction algorithm.

10.3. Other case studies on programmed graphtransformations

Based on a case study from the domain of SoftwareConfiguration Management (SCM) systems, the addedvalue of programmed graph transformations (such asSDMs) is discussed in [24] via a quantitative and qualita-tive analysis of an SCM implemented with SDMs. Theirresults were that programmed graph transformations,although more readable, do not really provide a clearadded value, and are at most at a slightly higher-level ofabstraction than conventional programming languagessuch as Java. This might seem to contradict our results atfirst glance, but the case studies and foci differ as follows:

�
The SCM and SPL testing domains are of course differ-ent. The SCM system analyzed in [24] is highly generic,modular and configurable, and this tends to lead tovery small patterns as noted by the authors. In our case,MoSo-PoLiTe is not intended to be a maximally config-urable “framework” and individual steps do not have tobe so fine granular enabling larger patterns. We havealso used graph transformations for preprocessing andour optimization rules do not have any control flow, i.e.,are purely declarative.
�
We do not try to replace Java with SDMs, but ratherregard SDM as a complementary language to be usedfor aspects where a declarative, rule-based specificationmake sense. This means that we do not have any Javacode in our SDM implementations and only supportMethodCallExpressions with which arbitrary methods inthe metamodel can be invoked. This has two advan-tages: (i) the graph transformations remain program-ming language independent, and (ii) readability is

Table 2Quantitative analysis according to [24].

Significant numbers MoSo-PoLiTe MOD2-SCM

Story patterns per story diagram 3.38 1.83Object variables per story pattern 2.9 1.71Link variables per story patterns 1.8 0.73Collaboration calls per pattern 0.43 1.20Statement activities/story act. 0 0.27Negative object variables/object var. 0.047 0.025

calcimp


improved as we do not support activity nodes that justcontain Java code, which is copied 1-1 in the generatedcode. Such helper methods should be implemented inJava and merged together with the generated code (thisis well supported by the EMF code generator). Both(i) and (ii) are major points of criticism in [24] regard-ing Fujaba SDMs.

�
The case study in [24] only analyses the SDM implemen-tation and does not compare it to a corresponding hand-written Java implementation. In contrast, we have bothversions (hand-written and SDM) for the same systemand can thus perform a comparative study with runtimemeasurements. We are thus able to quantify the “cost” ofusing SDMs with respect to efficiency and scalability.
�
The case study in [24] concentrates on SDMs andprogramming in the small and not on the advantagesof EMF and metamodelling in general. We considerthe usage of metamodelling and standard interfacestogether with SDMs as a whole and compare this with astandard Java implementation. Clearly many of ouradvantages actually stem from using a standardizationsuch as EMF and a code generator and are not specificto graph transformations. Using EMF without SDMs orsome other means of modelling behaviour, however,leads to equally important disadvantages!
�
A set of metrics are defined in [24] and used for aquantitative analysis of the investigated SDM implemen-tation MOD2-SCM. Table 2 depicts a comparison of themost important metrics using the same metric tooldeveloped by [24]).22 Although our system, with 44 storypatterns and 128 object variables, is much smaller thanMOD2-SCM, with 988 story patterns and 1688 objectvariables, the measured metrics still indicate that ourSDMs do not have a strong imperative nature (storypatterns per story diagram), and that most patterns arenon-trivial (object variables per story pattern). We alsohave a lower number of collaboration calls per patternand have absolutely no Java code in the SDMs. Last butnot least, a higher percentage of advanced features suchas negative elements was used. For further details, aprecise definition of each metric, and a comparison withother systems implemented with SDMs, we refer to [24].
We are not aware of any other comparable study thatcompares two identical systems: one implemented in Java,

22 With permission from the authors of [24], we offer the metriculation via a simple action in our context-menu for any SDMlementation specified with eMoflon.

and one with SDMs and EMF quantitatively (runtimemeasurements) and qualitatively as we have done inthis paper.

11. Conclusion and future work

In this paper we presented a novel case study for graphtransformations in the domain of SPL testing. We havesuccessfully employed SDMs not only for the involvedflattening transformation that transforms a feature modelinto a CSP, but also for an optimization process that addsredundant constraints to improve the performance of ourunchanged CSP solver. Our results show that the optimiza-tion leads to a reduction in runtime and to an increase inquality (reduction in size) of the generated sets of productconfigurations for pairwise testing. This novel case studyfor the graph transformation community showcases theadvantages of using graph transformations includingimproved readability and maintainability of the transfor-mation and optimization rules, and the possibility of rapidprototyping further optimization rules.

To evaluate a further application scenario of SDMs, were-engineered the complete CSP solver using graph trans-formations and presented not only a qualitative compar-ison of the implementations but also a quantitativecomparison including runtime measurements that showthe tradeoff involved in using graph transformations.

To evaluate maintainability, we defined and implemen-ted the same change request for both systems and dis-cussed the required effort and relevant factors.

As future work, further optimization rules can beinvestigated that involve the current product configurationand must be employed during the CSP resolution processby the solver. This would effectively extend the currentforward check to a larger look-ahead. It is, however,unclear if this will further improve performance. Differentideas for heuristics to speed up the solving process canalso be investigated and easily prototyped using the SDMimplementation.

Last but not least, we shall continue scalability mea-surements on larger feature models. The cost of perform-ing the optimization rules increases for large featuremodels (thousands of features) and leads to a bottleneckas a very large number of constraints is obtained in somecases. Depending on the number of features, an appro-priate handling must be implemented to decide how torestrain the optimization rules.

The SDM implementation can also be further optimizedto handle large feature models. Currently we have investedonly about two weeks in profiling and getting rid of EMF-specific hot-spots as compared to the hand-written imple-mentation which has been optimized over at least twoyears. It would be interesting to see how much faster theSDM implementation can be made without changing thecore strategy of the algorithm.

References

[1] K. Pohl, G. Böckle, F.J.v.d. Linden, Software Product Line Engineering:Foundations, Principles and Techniques, Springer, Berlin, 2005.

http://refhub.elsevier.com/S1045-926X(13)00044-X/sbref1



[2] P. Clements, L. Northrop, Software Product Lines: Practices andPatterns, Addison-Wesley, Boston, 2001.

[3] J.D. McGregor, Testing a Software Product Line, Technical ReportCMU/SEI-2001-TR-022, Software Engineering Institute, CarnegieMellon University, 2001.

[4] E. Engström, Regression test selection and product line systemtesting, in: Proceedings of the Third International Conference onSoftware Testing, Verification and Validation (ICST 2010), IEEE,New York, 2010, pp. 512–515.

[5] K. Scheidemann, Verifying Families of System Configurations, Ph.D.Thesis, Technische Universität München, 2007.

[6] G. Perrouin, S. Oster, S. Sen, J. Klein, B. Baudry, Y. Traon, Pairwisetesting for software product lines: comparison of two approaches,Software Quality Journal—Special Issue on Quality Engineering forSPLs 20 (3–4) (2012) 605–643.

[7] C.H.P. Kim, D.S. Batory, S. Khurshid, Reducing combinatorics intesting product lines, in: P. Borba, S. Chiba (Eds.), Proceedings ofthe Tenth International Conference on Aspect-Oriented SoftwareDevelopment (AOSD 2011), ACM, New York, 2011, pp. 57–68.

[8] S. Oster, F. Markert, P. Ritter, Automated incremental pairwisetesting of software product lines, in: J. Bosch, J. Lee (Eds.), Proceed-ings of the Fourteenth International Software Product Line Con-ference (SPLC 2010), Lecture Notes in Computer Science, vol. 6287,Springer, Berlin, 2010, pp. 196–210.

[9] K.C. Kang, S.G. Cohen, J.A. Hess, W.E. Novak, A.S. Peterson, Feature-Oriented Domain Analysis (FODA) Feasibility Study, Technical ReportCMU/SEI-90-TR-21, Carnegie-Mellon University, Software EngineeringInstitute, 1990.

[10] S. Oster, A Semantic Preserving Feature Model to CSP Transforma-tion, Technical Report 11, Technische Universität Braunschweig,2011.

[11] S. Oster, I. Zorcic, F. Markert, M. Lochau, MoSo-PoLiTe—tool supportfor pairwise and model-based software product line testing, in:K. Czarnecki, U. Eisenecker (Eds.), Proceedings of the Fifth Interna-tional Workshop on Variability Modelling of Software-IntensiveSystems (VaMoS 2011), ACM, New York, 2011, pp. 79–82.

[12] A. Anjorin, S. Oster, I. Zorcic, A. Schürr, Optimizing model-basedsoftware product line testing with graph transformations, in:Proceedings of the Eleventh International Workshop on GraphTransformation and Visual Modeling Techniques (GTVMT 2012),Electronic Communications of the EASST, vol. 47, European Associa-tion of Software Science and Technology (EASST), 2012.

[13] K. Czarnecki, S. Helsen, U. Eisenecker, Staged configuration throughspecialization and multilevel configuration of feature models, Soft-ware Process: Improvement and Practice 10 (2) (2005) 143–169.

[14] M. Lochau, S. Oster, U. Goltz, A. Schürr, Model-based pairwise testingfor feature interaction coverage in software product line engineer-ing, Software Quality Journal—Special Issue on Quality Engineeringfor SPLs 20 (3–4) (2012) 567–604.

[15] K. Czarnecki, A. Wasowski, Feature diagrams and logics: there andback again, in: Proceedings of the Eleventh International Conferenceon Software Product Lines (SPLC 2007), IEEE, New York, 2007,pp. 23–34.

[16] T. Fischer, J. Niere, L. Torunski, A. Zündorf, Story diagrams: a newgraph rewrite language based on the unified modeling language andjava, in: H. Ehrig, G. Engels, H.-J. Kreowski, G. Rozenberg (Eds.),Proceedings of the Sixth International Workshop on Theory andApplication of Graph Transformations (TAGT 1998), Lecture Notes inComputer Science, vol. 1764, Springer, Berlin, 2000, pp. 157–167.

[17] A. Anjorin, M. Lauder, S. Patzina, A. Schürr, eMoflon: Leveraging emfand professional CASE tools, in: 3. Workshop Methodische Entwick-lung von Modellierungswerkzeugen (MEMWe2011), Lecture Notesin Informatics, Gesellschaft für Informatik e.V., Bonn, 2011.

[18] L. Geiger, A. Zündorf, Story driven testing—SDT, Software Engineer-ing Notes 30 (4) (2005) 1–6.

[19] D. Benavides, S. Segura, A. Ruiz-Cortés, Automated analysis offeature models 20 years later: a literature review, InformationSystems 35 (6) (2010) 615–636.

[20] K. Saller, S. Oster, A. Schürr, J. Schroeter, M. Lochau, Reducing featuremodels to improve runtime adaptivity on resource limited devices,in: Proceedings of the 16th International Software Product LineConference. vol. 2 (2012) 135–142, http://dx.doi.org/10.1145/2364412.2364435, isbn: 978-1-4503-1095-6, url, http://dx.doi.org/10.1145/2364412.2364435.

[21] S. Oster, A. Wübbeke, G. Engels, A. Schürr, A survey of model-basedsoftware product lines testing, in: J. Zander, I. Schieferdecker,P. Mosterman (Eds.), Model-Based Testing for Embedded Systems,CRC Press/Taylor&Francis, 2011.

[22] H. Bennaceur, A comparison between SAT and CSP techniques,Constraints 9 (2) (2004) 123–138.

[23] J. White, B. Dougherty, D.C. Schmidt, Selecting highly optimalarchitectural feature sets with filtered cartesian flattening, Journalof Systems and Software 82 (8) (2009) 1268–1284.

[24] T. Buchmann, B. Westfechtel, S. Winetzhammer, The added value ofprogrammed graph transformations—a case study from softwareconfiguration management, in: Proceedings of the Fourth Interna-tional Symposium on Applications of Graph Transformations withIndustrial Relevance (AGTIVE 2011), 2011.



http://refhub.elsevier.com/S1045-926X(13)00044-X/othref0005






































































dx.doi.org/10.1145/2364412.2364435

dx.doi.org/10.1145/2364412.2364435

dx.doi.org/10.1145/2364412.2364435

dx.doi.org/10.1145/2364412.2364435
















Documents

Model-driven rapid prototyping with programmed graph transformations