1

Presented by:Hagit Cohen

April 2006

Tree-Like Counterexamples in Model Checking

Edmund Clarke Somesh Jha

Yuan Lu Helmut Veith

2

Outline

Introduction and linear counterexamples. Tree-like Kripke structures. Tree-like counterexamples for ACTL. Tree-like counterexamples for AΩ. Applications.

3

Counterexamples - motivation

A tool for detecting bugs.

Major importance in verification of large systems.

Automatic generation.

Abstraction refinement methodology for model checking.

4

What are counterexamples?

Given a property φ claimed to hold for each element of a given set S:

φ can be disproved by choosing a single element s ∈ S such that φ does not hold for s.

φS:

φS:

5

What are counterexamples? (cont.)

Existential properties can not be disproved by counterexamples.

For temporal logics counterexamples are expected for universal fragments.

6

Universal logics

A logic L is universal if the simulation theorem holds for L:

Let ψ be an ACTL formula. If K ≽ C and K ⊨ ψ then C ⊨ ψ.

ACTL(*)

AΩ

7

Linear counterexamples

Simple non-branching structures. Finite or infinite paths. Limitation of most model checkers.

Example: AF¬x

x

x

x

x

8

Linear counterexamples (cont.)

Insufficient for ACTL – only properties in ACTL ∩ LTL have linear counterexamples.

AFAXp – a counterexample has to show that there exists an infinite path π such that from every state of π, a state with property ¬p is reachable in one step. ⇒ Branching by definition.

Recognizing ACTL formulas with linear counterexamples is PSPACE-hard.

9

From Linear to Tree-Like

Desired attributes of a counterexample class:

Completeness

Effectiveness

Intelligibility

Viability

10

Counterexamples for ACTL

K ⊭ φ, C - a counterexample. What do we expect of C ?

C violates φ:C ⊭ φ ,or:C ⊨ ¬φ.Where ¬φ is an ECTL formula.C is a witness of ¬φ.

11

Counterexamples for ACTL (cont.)

What do we expect of C ?

Violation on C “explains” the Violation on K:By the relation K ≽ C.

C is viable:Demand that C is tree-like.

12

Tree-like graphs and Kripke structures

A graph is tree-like, if:(i) All SCCs are cycles.(ii) The component graph is a directed

tree.

A Kripke structure K = (S, R, L, {sinit}) is tree-like if the graph (S, R) is a finite tree-like graph whose root is the initial state sinit of K.

13

Tree-like Kripke structure - example

S1

S3

S2

S4

S6

S5

S7

S1

S3

S2

S4

S6

S5

S7

14

Tree-like CE for ACTL - Example φ = AG¬x ⋁ AF¬y

A counterexample for φ shows existence of:

(i) A finite path leading to a state satisfying x.

AND(i) An infinite path along which y is

always true.

15

Tree-like CE for ACTL - Example (cont.) Counterexample for the ACTL formula

φ = AG¬x ⋁ AF¬y is a model of the ECTL formula φ’ = EFx ⋀ EGy

y

y

y

xy

16

Duality of ACTL and ECTL

Counterexamples for ACTL are closely related to finite models for ECTL.

ECTL has the tree-like model property.

¬φ - an ECTL formula

A tree-like model of ¬φ

A possible counterexample of the formula φ

Duality of ACTL & ECTL Tree-like model

property of ECTL

One of all possible counterexamples over all different Kripke structures

φ - an ACTL formula

17

Weakness of ACTL(*)

Weakness of the path formulas. Example: no ACTL formula to express

the property “φ holds at all even time points“:

2 3 4 5 6 …1

18

Monotonicity of linear time operators

Example: φ = Fp π ⊨ Fp

……

For every ϭ such that π ⊆p ϭ, ϭ ⊨ Fp:

……

Results from the monotonicity of the operator F.

19

From ACTL(*) to AΩ (cont.)

AΩ – an extension of ACTL byω-regular linear time operators.

More expression power.

Retains the monotonicity of the linear time operators.

20

LTL operators as patterns

View LTL operators as patterns on the time line.

Can be observed on paths.

Example: Fφ describes the following path patterns:M1, ⊥M1, ⊥ ⊥ M1, ⊥ ⊥ ⊥ M1, …

M1- marker - the position where φ holds.

⊥ - “don’t care”.

21

LTL operators as regular expressions

F (⊥)*M1

X ⊥M1

G (M1(ω

U (M1)*M2

22

Temporal operators as regular expressions – formal definition

A temporal operator O with n input formulas is defined over the set of words over the alphabet Σ = P({M1....Mn}).

Abbreviations: ⊥ for Φ M1 for the singleton {M1}

23

Temporal operators as regular expressions- terminology

If O is defined by an ω-regular expression, we say that O is:

Buchi operator regular computable

24

Semantics of regular temporal operators

Let O – a regular temporal operator. π = s0, s1,… a path in in a Kripke

structure K. φ1,…, φn – formulas.

Then K,π ⊨ O(φ1,…, φn) if there exists a pattern o ∈ O such that for all positions i < |o|, and for all Mk ∈ o(i), it holds that K,πi ⊨ φk.

25

Regular temporal operators – example 1

Define a new operator – Oeven(φ):

φ holds at all even time points.

An ω-regular expression for Oeven:

(⊥ M1)ω

2 3 4 5 6 …1

26

Regular temporal operators – example 1 (cont.)

K,π ⊨ Oeven(φ) ? The marker M1 denotes that φ1 holds. A single possible pattern o ∈ O:

o = ⊥ M1⊥ M1⊥ M1⊥ M1⊥ M1 …

For a path π such that K,π ⊨ Oeven(φ):

∀i: even(i) πi ⊨ φ1, since M1 ∈ o(i).

⇒ φ1 holds at all even time points.

27

Regular temporal operators – example 1 (cont.)

¬even(i) ?

πi ⊨ φ1 √

…2 3 4 5 61

πi ⊭ φ1 √

…2 3 4 5 61

Monotonicity of Oeven.

28

Regular temporal operators – example 2

Define a new operator – Omax4gap(φ):

There should be no more than four time units between two occurrences of φ.

……

An ω-regular expression for Omax4gap:

(M1| ⊥M1 | ⊥⊥M1 | ⊥⊥⊥M1 | ⊥⊥⊥⊥M1)ω

29

Regular temporal operators – example 2 (cont.)

K,π ⊨ Omax4gap(φ) ? The marker M1 denotes that φ1 holds. Many (infinity) possible patterns o ∈ O,

constructed of the 5 building blocks.

Any path π with more than four time units between two occurrences of φ1 will not match any of the patterns, and thus will not satisfy O(φ1).

30

Preservation of monotonicity

No enforcement of negation of a sub-formula as a marker.

Therefore all operators we define are monotonic.

31

Monotonicity – formal definition

Given a path π and a formula φ, φπ denotes the set of states in π where φ holds.

For a sequence of formulas {φ1,.., φn}, we define π ⊆φ1,.., φn ϭ iff ⋀i=1 φi

π ⊆ φi ϭ.

Lemma - Monotonicity:If K,π ⊨ O(φ1,.., φn) and π ⊆φ1,.., φn ϭ, then K,ϭ ⊨ O(φ1,.., φn).

n

32

Monotonicity and counterexamples

We conclude that if K,π ⊭ O(φ1,.., φn) and π ⊆φ1,.., φn ϭ, then K,ϭ ⊭ O(φ1,.., φn).

The refutation of O(φ1,.., φn) on π does not depend on satisfied sub-formulas, but only on violated sub-formulas.

33

Monotonicity and counterexamples (cont.)

Example: if K,π ⊭ Oevenφ, then

K,π ⊨ Oevenφ can be disproved by finding an even position j such that K, πj ⊭ φ.

In general: disprove O(φ1,.., φn) by identifying all violations of φ1,.., φn on π.

Counterexample

For O(φ1,.., φn)

CE For φnCE For φ1 …

34

The logic AΩ

Given: Ω - a set of temporal regular operators. AP – a set of atomic proposition.

AΩ consists of the following formulas:

I. Every p ∈ AP is in AΩ.

II. For each p ∈ AP, ¬p is in AΩ.

III. If O ∈ Ω is an n-ary operator, and φ1,.., φn ∈ AΩ, then AO(φ1,.., φn) ∈ AΩ.

35

The logic AΩ (cont.)

IV. If φ1, φ2 are in AΩ, then φ1⋀φ2 ∈ AΩ and φ1⋁φ2 ∈ AΩ.

V. If φ1,φ2… ∈ AΩ, then ⋀i≥1φi ∈ AΩ.

36

Semantics of AΩ K,s ⊨ φ ?

I. If φ is atomic then K,s ⊨ φ iff φ ∈ L(s).

II. K,s ⊨ ¬φ iff K,s ⊭ φ.

III. K,s ⊨ AO(φ1,.., φn) iff for all paths π starting at s it holds that K,s ⊨ O(φ1,.., φn).

37

Semantics of AΩ (cont.)

IV. K,s ⊨ φ1⋁φ2 iff K,s ⊨ φ1 or K,s ⊨ φ2.

V. K,s ⊨ φ1⋀φ2 iff K,s ⊨ φ1 and K,s ⊨ φ2.

VI. K,s ⊨ ⋀i≥1φi iff K,s ⊨ φi for all i≥1.

38

The logic AΩ (cont.)

AΩ is universal.

ACTL and ACTL* can be definedas subsets of AΩ with finite conjunction.

Any prove of the tree-like counterexample property for AΩ is also valid for ACTL(*).

EΩ is defined similarly by replacing: A ⇒ E ⋀i≥1φi ⇒ ⋁i≥1φi

39

Counterexample theorem

Let Ω be a set of temporal Buchi operators. Then AΩ has tree-like counterexamples.

Furthermore, the tree-like counterexamples are effectively computable.

Corollary: EΩ has the tree-like model property.

40

Constructing counterexamples

Lemma:Let O be a Buchi operator, K a Kripke structure, and s0 a state such that

K, s0 ⊭ AO(Ψ1,..Ψk).

Then there exists a path ϭ=s0,... such that:

I. K,ϭ ⊭ O(Ψ1,..Ψk).

II. ϭ has the form: s0,…,sN, sN,…,sN+M, sN,… sN+M,…

41

Proof sketch for the lemma

The idea:Construct a Buchi automaton for the patterns of ¬O, and use an accepting run of the automaton to obtain a path ϭ with the required property.

42

Proof sketch for the lemma (cont.)

O – a set of patterns over the alphabet Σk = P({M1,…,Mk}).

Patterns for ¬O cannot be obtained by using the set-theoretic complement of O, Example:The pattern (⊥)*M1 for the operator F. it’s complement contains the pattern M1M1, although a path where the constraint M1M1 holds will satisfy F.

43

Proof sketch for the lemma (cont.)

Therefore, the complement should be calculated for the set O’, where O’ is the “monotonic hull” of O.

Denoting: R - the regular expression for O. R’ - the regular expression for O’. ϭ’ - the set of all symbols of the alphabet

which are supersets of ϭ ∊ Σk.

R’ is obtained from R by replacing all occurrences of ϭ by ϭ’.

44

Proof sketch for the lemma (cont.)

What is O’ = L(R’)?If a pattern o is in O, then all patterns obtained from o by adding zero or more additional markers are in O’.

⇒ ¬O’ is the set of all patterns which violate the operator.

Due to monotonicity, in the context of AΩ the operators O and O’ are identical!

45

Proof sketch for the lemma (cont.)

Let A be the Buchi automata accepting ¬O’, and π a path such that K,π ⊭ O(Ψ1,..Ψk).

Construct a word sπ such that sπ is accepted by A: sπ(i) = {Mj : K,πi ⊭ Ψj} for all i≥0.

Let q be an accepting state of A which appears infinitely in an accepting run of A for sπ, for the indices a1<a2<…, and the corresponding states in K: π(a1), π(a2)...

46

Proof sketch for the lemma (cont.)

K has a finite number of states.⇒ There are 2 indices J<J’ such that π(J)=π(J’).

Choosing the minimal such J<J’, the path ϭ given by:π(0),…, π(aJ), π(aJ),…, π(aJ’-1), π(aJ),… π(aJ’-1),…

matches a word excepted by A.

⇒ A path as stated by the lemma.

47

The tree-like property of the path

ϭ = s0,…,sN, sN,…,sN+M, sN,… sN+M,…

If all states are different, then ϭ describes a simple tree-like substructure of K containing the path s0,…,sN, leading to the loop sN,…,sN+M.

Otherwise, a tree-like structure is obtained by un-raveling the path using the indexed Kripke structure.

48

The indexed Kripke structure - Kω

K=(S,I,R,L) ⇒ Kω=(Sω,Iω,Rω,Lω): Sω=S x ℕ Iω=I x ℕ (s1

i,s2j) ∈ Rω ⇔ (s1,s2) ∈ R

Lω(si)=L(si)

π = s0,s1,s2… a path on K.

⇒ unravel (C,π) = s0C,s1

C+1,s2C+2…

49

The algorithm CEX

Given K, s, φ such that K,s ⊭ φ, CEX(K,s0,φ) computes a tree-like counterexample for K,s ⊨ φ.

The tree-like counterexample is constructed as a substructure of Kω (the index of states is denoted by a global constant C, initialized to 0).

Assumptions: K,s ⊭ φ A model checking procedure for AΩ.

50

Algorithm CEX - output format

Description, constructed of:I. Path descriptors <s0,…,sn>

II. Loop descriptors<s0,…,sn,s0 > or <s0,…, sn >ω

S1

S3

S2

S4

S5

S6

<s1,s2>

<s1,s5,s6>

<s6,s6>

<s2,s3,s4>ω

51

Algorithm CEX

CEX(K,s0i,φ)

case φ of φ1 ⋁ φ2: CEX(K,s0

i,φ1)

CEX(K,s0i,φ2)

⋀i≥1 φi:

φ1 ⋀ φ2: select j such that K,s ⊭ φj,

CEX(K,s0i,φj)

52

Algorithm CEX (cont.)

AO(Ψ1,..Ψk):

determine s0,……,sN,……,sN+M

desc1=<s0i, unravel (C, s1,…sN)>

desc2=<unravel (C+N, sN,…sN+M)>ω

output desc1, desc2.

for all states p in {desc1,desc2}for j ∈ {1,…,k}

if K,p ⊭ Ψj then CEX(K,p,Ψj)

53

CEX – correctness

I. Monotonicity – enables reduction of counterexample computation for a formula φ to counterexample computation for the sub-formulas of φ.

II. The indexed Kripke structure allows us to refer to logically independent parts of the counterexample separately.

Monotonicity Kω ω-regularity

54

CEX – correctness (cont.)

III. The ω-regularity of the Buchi operators allows us to build the global counterexample from small counterexamples of the form<s0,…,X>,<X,…>ω, using the lemma.

Monotonicity Kω ω-regularity

55

Application I :Symbolic counterexample generation

By constructing witnesses for the dual logic ECTL.

Based on: SAT(φ). Symbolic fixpoint computations.

EX EF,EU EG

Singlesymbolic

step

Greatestfixpoint

Leastfixpoint

56

Constructing witnesses for ECTL

Algorithm print witness(φ, si)case φ of:

EXΨ: print_witnessEX (Ψ, si)

EFΨ: print_witnessEF (Ψ, si)

EGΨ: print_witnessEG (Ψ, si)

EUΨ: print_witnessEU (Ψ, si)

Ψ1⋀Ψ2: print_witness (Ψ1, si) print_witness (Ψ2, si)

Ψ1⋁Ψ2 : if si ∈ SAT(Ψ1) then print_witness (Ψ1, si) else print witness (Ψ2, si)

57

Constructing a witness for EF

The model checker computes the least fixpoint of the operator τ:τ(X) = SAT(φ) ⋃ Img-1(X).

The sets S1 ⊆ S2 ⊆,… ,⊆ Sn, the stages of the fixpoint computation, are then used for computation of a witness.

Obtain a finite path s0,…,sj ,j<n, and unravel it in the same way as in CEX.

58

Constructing a witness for EF (cont.)

Algorithm print_witnessEF(φ, s0i)

Determine the stages (S1,…,Sn) of computing EFφ.j := 0repeat

j := j + 1S := Img(sj-1) ⋂ Sn-j

choose sj ∈ Suntil sj ∈ S1

desc := <s0i; unravel(C, s1,…,sj)>

output descC := C + jprint witness(φ, sj

C)

59

Constructing a witness for EG

Uses only the last stage, Sn, of the greatest fixpoint computation.

Algorithm print_witnessEG(φ, s0i)

Sn = SAT(EGφ) T = {s0}j = 0repeat

j = j + 1S = Img(sj-1) ⋂ Sn

choose sj ∈ ST = T ∪ {sj}Q = Img(sj) ⋂ T

until Q ≠ Ø…

60

Constructing a witness for EG (cont.)

…choose N where sN ∈ Qdesc1 = <s0

i, unravel (C, s1,…,sN)>desc2 = <unravel (C+N, sN,…,sj)>ω

output desc1 and desc2

C = C + j +1for all states p in ⋃{desc1,desc2}

if K,p ⊨ φ then print witness(φ, p)

61

Application II : Counterexample-guided refinement

A method for handling the state explosion problem.

When a model is too big for direct model checking, the model checking is applied to an abstraction of the original model.

Given a counterexample for the abstraction, if it is spurious a refinement of the abstraction is done.

62

Counterexample-guided refinement – implementation in the system aSMV

Implementation for the ACTL fragment with linear counterexamples.

Given a descriptor of a linear counterexample, symbolic procedures – CheckPATH/CheckLOOP check if the counterexample is spurious.

63

Completeness for ACTL

Generalization of spurious check for a descriptor to spurious check of a description.

Finding a set of concrete states from which a linear counterexample

consistent with the descriptor exists.

Finding a set of concrete states from which tree-like counterexamples

consistent with the description exist.

64

Completeness for ACTL (cont.)

Algorithm CheckRefine(a0,Q)T = h-1( a0) for each q ∈ Q

if q(0) = a0 thenl = |q|

S1 = h-1(a0)for (i=2 to l)

Si = CheckRefine(q(i-1), Q)if q is a path descriptor then

T = T ⋂ CheckPATH(q, S1,S2,…,Sl)if q is a loop descriptor then

T = T ⋂ CheckLOOP(q, S1,S2,…,Sl)return T

65

Summery

Linear counterexamples. Tree-like Kripke structures. Tree-like counterexamples for AΩ (⇒

ACTL). Algorithm CEX for AΩ. Efficient ECTL witnesses construction. Generalization of spurious check for tree-

like counterexamples in aSMV.