Delta-oriented Model-based SPL Regression Testing · Sascha Lity j Delta-oriented Model-based SPL Regression Testing j 12 ... Sascha Lity j Delta-oriented Model-based SPL Regression

Institut für Programmierungund Reaktive Systeme

Delta-oriented Model-based SPL Regression Testing

Sascha LityFOSD-Treffen – 22.03.2012

Motivation Delta State Machines Delta-oriented SPL Regression Testing

Outline

Motivation

Delta State Machines

Delta-oriented SPL Regression Testing

Sascha Lity | Delta-oriented Model-based SPL Regression Testing | 2Institut für Programmierung

und Reaktive Systeme


Motivation

Model-based TestingTest model describes system behaviorTest case generationConformance testing

Regression TestingTesting of software versionsReuse of test casesReuse of test results

⇒ Combination for reasoning about reuse potentials




Motivation







Motivation







Variability in Test Models

How to encode variability in test models?Annotative/Selective [WSS08, LOGS11, COLS11]Compositional [UKB10, SV08]Transformational [LSKL12]





Adaption of Delta Modeling [Sch10] approach

TransformationalModeling and Generation of variable state machines

Model-based TestingModel-driven Development

Deltas define transformationsAdd/Remove of basic state machinesAdd/Remove of hierarchyAdd/Remove of statesAdd/Remove of transitionsModification of transition labels





Adaption of Delta Modeling [Sch10] approachTransformational

Modeling and Generation of variable state machinesModel-based TestingModel-driven Development






Adaption of Delta Modeling [Sch10] approachTransformationalModeling and Generation of variable state machines







Adaption of Delta Modeling [Sch10] approachTransformationalModeling and Generation of variable state machines






Transformations (1/2)

s1

s3

Δ Bsm_Root

s2

s1

s3

Bsm_Root

s2

s4

s6

Bsm_Sub

s5

Add a hierarchy and add a basic state machine

s1

s3

Δ Bsm_Root

s2 s1

s3

Bsm_Root

s2

s4

Add a state and a transition





s1

s3

Δ Bsm_Root

s2

s1

s3

Bsm_Root

s2

s4

s6

Bsm_Sub

s5

Add a hierarchy and add a basic state machine

s1

s3

Δ Bsm_Root

s2 s1

s3

Bsm_Root

s2

s4

Add a state and a transition





s1 s2t1:e1/e2;e3

Δ

s1 s2t1:e5/e2;e3

s1 s2t1:e1/e2;e3

Δ

s1 s2t1:e1/e3;e4;e5

Modification of transition labels




Example BCS Delta

FP_on

Name: DRemFPTransManPWAddTransAutoPW Apply to: FP_BSM After: When: Automatic Power Window

FP_off

fp_finger_detected/fp_on = true;

fp_finger_detected/ pw_auto_mv = 0; fp_on = true +

-




Generation of State Machines

Core + Delta 1 +

Delta 2 + Delta 3

Core + Delta 1 +

Delta 2

Core + Delta 1

T6: e3/ T7: e1/

Application Condition

Feature Configuration

Feature Model

S1 S2

S3

T1: e1/e2

T2: e3/ T3: e2/

T5: e4/e6 T4: e5/e6

Core Product

Delta 1 Add

T1: e1/e2

Delta 2 Rem

Delta 3 Mod

T5: e5/e2;e3;e4

+

+

+

S1 S2

S3

T1: e1/e2

T2: e3/ T3: e2/

T5: e4/e6 T4: e5/e6

T6: e3/ T7: e1/ S4

S1 S2

S3

T2: e3/ T3: e2/

T5: e4/e6 T4: e5/e6

T6: e3/ T7: e1/ S4

S1 S2

S3

T2: e3/ T3: e2/

T4: e5/e6

T6: e3/ T7: e1/ S4

T5: e5/e2;e3;e4

Product

S1 S2

S1 S2

S3

+ + +

-

*

+ +

-

* S2

S4




Outline

Motivation


Delta-oriented SPL Regression Testing




Product Test Artifacts

DefinitionThe collection of test artifacts for a product p is a 4-tupletap = (tmp, tgp, tsp, tpp) consisting of a test model tmp, a finite set tgpof test goals in tmp for a criterion C , a test suite tsp, and a test plantpp.




Product Test Model Generation

unlock lock

t0 : key pos lock/cls locked;pw disabled; pw but up

t1 : key pos unlock/cls unlocked;pw enabled

unlock lock



t2 : rck lock/cls locked;pw disabled; pw but up

t3 : rck unlock/cls unlocked;pw enabled

unlock lock



t4 : car drives/cls locked

δtm = {add t2, add t3}δ−1tm = {rem t2, rem t3} δtm′ = {add t4}

δtm,tm′ = {rem t2,rem t3, add t4}

1

δtm,tm′ = (δtm \ δtm′)−1 ∪ (δtm′ \ δtm)




Product Test Model Generation

unlock lock



unlock lock



t2 : rck lock/cls locked;pw disabled; pw but up

t3 : rck unlock/cls unlocked;pw enabled

unlock lock



t4 : car drives/cls locked

δtm = {add t2, add t3}δ−1tm = {rem t2, rem t3} δtm′ = {add t4}

δtm,tm′ = {rem t2,rem t3, add t4}

1

δtm,tm′ = (δtm \ δtm′)−1 ∪ (δtm′ \ δtm)




Delta-oriented SPL Regression Testing (1/2)

s1 s2

s3

t1: e1/e2

t2: e3/ t3: e2/

t5: e4/e6 t4: e5/e6

TM

TG

s1 s2

s3

t2: e3/ t3: e2/

t5: e4/e6 t4: e5/e6

t6: e3/ t7: e1/ s4

t1

t2

t3

t4 t5

t2

t3

t4

t5

t6 t7

C1 C1

t1

TS

s1 s2

s3

t1

t3 t4

t5

s1 s2

s3

t1 t3

s1

s3 t4

t2

t5

s1 s2

s3

t1 t3

tc1

s1

s3 t4

t2

tc2

tc3

tc4

tc1

tc2

tc3

Generation Generation

TP t5

s1 s2

s3

t1 t3

s1

s3 t4

t2

t5 s1

s2

s3

t3

t6 t7 s4 tc2

tc4 tc3

Selection Selection

Δ

Δ

Δ

Δ

t5 s1

s2

s3

t3

t6 t7 s4

s1 s2

s3

t1

t3 t4




Delta-oriented SPL Regression Testing (2/2)

tmi

δtmi,tmi+1 +3 tmi+1

tgiδtgi,tgi+1 +3 tgi+1

tpiδtpi,tpi+1 +3 tpi+1

tsiδtsi,tsi+1 +3 tsi+1

1

δta,ta′ = (δtm,tm′ , δtg,tg′ , δts,ts′ , δtp,tp′)




Body Comfort System Case Study

Finger Protection

Body Comfort System

Power Window

Central Locking System

Automatic Locking

Remote Control

Key

Human Machine Interface

Manual Power

Window

Automatic Power

Window

Control Alarm System

Safety Function

Interior Monitoring

Alarm System

Exterior Mirror

Electric Heatable

require

require

require

Mandatory Or

require

exclude Optional Alternative

Status LED

Security Door

System

LED Central Locking System

LED Power

Window

LED Exterior Mirror

LED Heatable

LED Alarm System

LED Finger Protection

Control Automatic

Power Window

require

exclude

require




Case Study Statistics

Previous results obtained in evaluation of MoSo-PoLiTe [OLZG11]Representative subset of 17 product variants150% state machine test model

105 states107 transitions26 input events33 output events

64 test cases executed in average

Remodelling of test model1 core40 deltas

Testing of 18 variants10 test cases generated in average9 test cases executed in average







64 test cases executed in averageRemodelling of test model

1 core40 deltas








64 test cases executed in averageRemodelling of test model

1 core40 deltas





Case Study Results

30

40

50

60

70

80

New

Selection

MoSo-PoLiTe

0

10

20

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 P16 P17






Literature I

H. Cichos, S. Oster, M. Lochau, and A. Schurr.Model-based Coverage-Driven Test Suite Generation for Software Product Lines.In MODELS’11, 2011.

Malte Lochau, Sebastian Oster, Ursula Goltz, and Andy Schurr.Model-based Pairwise Testing for Feature Interaction Coverage in Software ProductLine Engineering.Software Quality Journal, pages 1–38, 2011.

Malte Lochau, Ina Schaefer, Jochen Kamischke, and Sascha Lity.Incremental model-based testing of delta-oriented software product lines.In 6th International Conference on Tests & Proofs (TAP 2012), Prague, CzechRepublic, 2012.accepted to appear.

S. Oster, M. Lochau, M. Zink, and M. Grechanik.Pairwise Feature-Interaction Testing for SPLs: Potentials and Limitations.In FOSD’11, 2011.




Literature II

Ina Schaefer.Variability modeling for model-driven development of software product lines.Fourth International Workshop on Variability Modelling of Software-intensive Systems(VaMoS 2010), January 2010.

N. Szasz and P. Vilanova.Statecharts and Variabilities.In VAMOS’08, pages 131–140, 2008.

Engin Uzuncaova, Sarfraz Khurshid, and Don S. Batory.Incremental test generation for software product lines.IEEE Trans. Software Eng., 36(3):309–322, 2010.

S. Weißleder, D. Sokenou, and H. Schlingloff.Reusing State Machines for Automatic Test Generation in ProductLines.In MoTiP’2008, 2008.



Documents

Delta-oriented Model-based SPL Regression Testing · Sascha Lity j Delta-oriented Model-based SPL Regression Testing j 12 ... Sascha Lity j Delta-oriented Model-based SPL Regression