Model-based Testing of Hybrid Systems Michiel van Osch IPA Spring Days on Testing 19 April – 21 April 2006

Model-based Testing of Hybrid Systems

Michiel van Osch

IPA Spring Days on Testing19 April – 21 April 2006

2

Motivation• Hybrid Systems

• Testing might be expensive, dangerous, or resources might be limited

• Discrete and real-time model-based testing does not test the continuous aspects of the system

3

Content

• Refresh– Model-based Testing– Discrete Conformance & ioco Tests– Timed Conformance & tioco Tests

• Hybrid Systems• Hybrid Input-output Conformance• Hybrid Input-output Conformance Tests• Conclusions

– Results– Limitations and Future Work

4

Automated Model Based Testing

model

SUT

IUTconforms to

model

pass fail

test

tool

testgeneration

tool

test execution

toolIUT passes tests

IUT conf model

soundexhaustiv

e

5

Input-output Conformance

Impl. ioco Spec. iff for all traces α:

out(Impl. after α) out(Spec. after α)

6

Test-case Generation and Execution

1. Terminate with verdict pass

2. Select an input from the specification and apply it to the implementation

3. Observe an output or a timeout from the implementation and check if it is allowed according to the specification

7

Example

s0s1 s4

s2

s3

?Button2

!Tea

?Button1

?Button2

?Button1

!XLCoffee

!Coffee t0

t1

?Button1

t3

?Button2

Pass

!XLCoffee

t2

!

FailFail Fail

!Coffee !Tea!

FailFail Fail

!Coffee !Tea!XLCoffee

8

Timed Input-output conformances

• Different views: time as observable event versus time as parameter of the (output) actions

s0s3!tick !tick !tick !tick

!XLCoffee

Vs.

s0s3!XLCoffee(4)

9

Timed Input-output Conformance (Krichen)

Impl. tioco Spec. iff for all traces α:

out(Impl. after α) elapse(Impl. after α)

out(Spec. after α) elapse(Spec. after α)

10

Timed Test-case Generation and execution



3. Observe an output or time passing from the implementation and check if it is allowed according to the specification

11

Example Test-case Generation

t0

t1

?Button1

t2

!tick

Fail Fail

t3

?Button2

!tick

Pass

!XLCoffee

Fail Fail

Fail Fail

!tick

!tick

!tick

Fail

!tick

s0s1 s4

s2

s3

?Button2

!Tea

?Button1

?Button2

?Button1

!XLCoffee

!Coffee

!tick !tick !tick

!tick !tick !tick !tick

!tick !tick

!tick !tick

12

Moving forward …

13

Content




14

Hybrid Systems

• Practically:– Discrete behavior plus continuous behavior– Continuous behavior can be input observed through sensors

or output generated by actuators

• Mathematically– Discrete actions plus flow of continuous variables

(trajectories)– Variables can be input variables and output variables

15

Hybrid Systems

Time

Coffee

0

?Button1 ?Button2

s0s1

s2

s3

?Button1

?Button2

?Button1

Coffee’ = 4 cl/sec.

Coffee’ = 3 cl/sec.

Coffee’ = 0 Coffee’ = 0

16

Hybrid Systems

s0s1

s2

s3

?Button1

?Button2

?Button1

Water’ = 4 cl/sec.Coffee’ = Water’

Water’ = 3 cl/sec.Coffee’ = Water’

Water’ = 0Coffee’ = 0


Water

17

Content




18

The Idea

• Besides for the output actions, also for every reachable state the set of trajectories allowed by the implementation is a subset of the set of trajectories allowed by the specification

• No quiescence because continuous output always occurs

19

Hybrid Conformance (continuous output only)

Impl. is input-output conform a Spec. iff for all traces α:

out(Impl. after α) out(Spec. after α) and

traj(Impl. after α) traj(Spec. after α)

s0s1

s2

s3

?Button1

?Button2

?Button1

Coffee’ = 4

Coffee’ = 3

Coffee’ = 0 Coffee’ = 0

20

Hybrid Conformance Issues

• The continuous output directly depends on the continuous input, therefore we have to define our relation on both input and output trajectories.

• The implementation is input enabled (for both discrete behavior and continuous behavior), but we do not want the specification to be input complete.

21

Hybrid Conformance (continuous input plus output)

s0s1

s2

s3

?Button1

?Button2

?Button1

Water’ = 4Coffee’ = Water’




infilter(traj(Impl. after α), traj(Spec. after α)) is the set of trajectories allowed by the implementation, with respect to the input trajectories allowed by the specification

Water’ = 3

Coffee’ = 3

Water’ = 2

Coffee’ = 2

22

Hybrid Input-output Conformance

Impl. hioco Spec. iff for all traces α:


infilter(traj(Impl. after α), traj(Spec. after α)) traj(Spec. after α)

23

Hybrid Conformance (continuous input plus output)

s0s1

s2

s3

?Button1

?Button2

?Button1





s4 s5!”NoWater”

Water’ =0Coffee’ = Water’


Coffee’ = 0Water = 0


infilter(traj(Impl. after α), traj(Spec. after α)) traj(Spec. after α) {if the implementation cannot do output action “!NoWater”}

{“NoWater”} and

In state s4:

24

Hybrid Conformance(continuous input plus output)

If there is a trajectory from state s then

out(s) = {o in O| s →} {ξ}else

out(s) = {o in O| s →}

o

o


infilter(traj(Impl. after α), traj(Spec. after α)) traj(Spec. after α) {if the implementation cannot do output action !”noWater”}

{ξ} {“NoWater”} and

In state s4:

25

Hybrid Input-output Conformance

Impl. hioco Spec. iff for all traces α:


infilter(traj(Impl. after α), traj(Spec. after α)) traj(Spec. after α)

26

Content




27

Hybrid Test-case

• Tree like structure• Two terminal states: pass and fail• Deterministic for actions• Strongly time deterministic for trajectories

28

Hybrid Test-case Generation



s0s1?Button1



Specification

t0

t1

?Button1

Test-Case

29


3. Select an input trajectory from the specification, apply it to the implementation and observe the output trajectory simultaneous, possibly interrupted by an output action.

Specification

s1Water’ = 0Coffee’ = 0

Test-Case

t1

t2


Fail Fail

Fail Fail

!”Boilerlow”

30


4. If an output action has to happen immediately according to the specification then observe an output action and check if it is allowed according to the specification or let time pass by selecting and applying and observing a trajectory

fail fail

s4 s5!”NoWater”

Coffee’ = 0Water = 0

Specification

t4

Pass

!”NoWater”

Test-Case

31

Example Test-case Generation

t0

t1

?Button1

t3

?Button2

Fail FailWater’ = 0Coffee’ = Water’

t4

t2 Fail Fail


!”Boilerlow”

Fail Fail

Pass

!”NoWater”

fail fail

32

Content



– Results – Limitations and Future Work

33

Results

• A hybrid conformance theory• Sound and exhaustive• Natural extensions to other theories• Still, open issues …

34

Limitations and Future Work

• Continuous input depending on continuous output• Uncountable many states and trajectories• Taking into Practice

– Sampling– Test-selection– Inaccuracy / robustness– …

35

Questions?

Documents

Model-based Testing of Hybrid Systems Michiel van Osch IPA Spring Days on Testing 19 April – 21 April 2006