: A Finite State Verification

Approach for Software Systems

Lori A. Clarke, George S. Avrunin, Jamieson M. Cobleigh,

Heather M. Conboy,Matthew B. Dwyer,Gleb Naumovich, and Leon J. Osterweil

Laboratory for Advanced Software Engineering Research

University of Massachusetts, Amhersthttp://laser.cs.umass.edu/

Finite State Verification (FSV)

• Verifies properties about system behavior

• Seeks a middle ground between testing and theorem provingo Testing cannot provide definitive resultso Theorem proving requires mathematical

sophistication and considerable human insight

• Works on a finite model of the system• Has been successfully used to prove a

variety of properties of systems

State Explosion Problem

• FSV techniques can quickly become intractable due to the state explosion problemoThe cost can be exponential in the size of the system being analyzed

• Analysts typically have to figure out how to create a small, abstract model of the system

Our Approach

• Automatically create an abstract modelo Imprecise, but conservativeoHelps control state explosion

• If needed, add precision incrementally until model is precise enough tooprove the property of interest or o reveal a fault (in the system or property)

Outline

• Introduction• FLAVERS Model• Checking Properties• Improving Precision• Experimental Results• Conclusion

FLAVERS

• FLow Analysis for VERification of Systems• Verifies properties about concurrent and

sequential systems• Automatically creates an abstract, event-

based graph model of the systemo Imprecise, but conservative

• Represents properties as Finite-State Automatao Sequences of syntactically recognizable events

• Uses an efficient data-flow analysis algorithmo Precision can be improved incrementallyo Can be used for both safety and liveness properties

Modeling Concurrent Systems• Common model for a concurrent

system is a reachability graph• Represents all the states a concurrent

system may reacho <l1,l2, …ln>, where each element of the

tuple is a location in one of the taskso Often includes the full state of execution

location and values of all variables for each task

o Interleaved model of execution

Reachability Graph

task body t1 is begin

u; t2.synch; v; w;end t1;

task t2 body is begin

x; accept synch; y; z;end t2;

bb,,bb

u,u,bb

u,xu,x

bb,x,x

ss,,ss

ss,y,yv,v,ss

w,w,ss v,yv,y

w,yw,y

ee,,ee

ss,z,z

v,zv,z

w,zw,z

Reachability Graph

task body t1 is begin

u; t2.synch; v; w;end t1;

task t2 body is begin

x; accept synch; y; z;end t2;

bb,,bb

u,u,bb

u,xu,x

bb,x,x

ss,,ss

ss,y,yv,v,ss

w,w,ss v,yv,y

w,yw,y

ee,,ee

ss,z,z

v,zv,z

w,zw,z

<0,0><0,0> <0,1><0,1> <1,1><1,1><1,0><1,0>

<0,0><0,0> <1,0><1,0>

<0,0><0,0> <0,1><0,1> <1,1><1,1><1,0><1,0>

<0,0><0,0> <0,1><0,1> <1,1><1,1><1,0><1,0>

Trace Flow Graph (TFG)

• A TFG represents how events flow through a systemo Built from Control Flow Graphs for the

tasks in the systemo For a concurrent systems, nodes and edges

may be added to represent concurrency

`̀`̀

TFG Construction

xx

yy

uu

vv

ww

synchsynch accept synchaccept syncht2.syncht2.synch

task body t1 is begin

u; t2.synch; v; w;end t1;

task t2 body is begin

x; accept synch; y; z;end t2; zz

xx

yy

uu

vv

ww

synchsynch

zz

bb,,bb

u,u,bb

u,xu,x

bb,x,x

ss,,ss

ss,y,yv,v,ss

w,w,ss v,yv,y

w,yw,y

ee,,ee

ss,z,z

v,zv,z

w,zw,z

u

b,b

u,b

synchs,s

vv,s

w

w,s

z

w,z

e,e

u,xx

w,y

y

Feasible Paths

xx

yy

uu

vv

ww

synchsynch

zz

bb,,bb

u,u,bb

u,xu,x

bb,x,x

ss,,ss

ss,y,yv,v,ss

w,w,ss v,yv,y

w,yw,y

ee,,ee

ss,z,z

v,zv,z

w,zw,z

Infeasible Paths

synch

u

b,b

u,b

Outline

• Introduction• FLAVERS Model• Checking Properties• Improving Precision• Experimental Results• Conclusions

Elevator Property

The elevator does not movewhile its doors are open.

L(P) is the set of all stringsaccepted by property P

11

22

33

closeclose openopen

movemove

closeclosemovemoveopenopen

openopen

closeclosemovemove

Annotated TFG

• A TFG G is <N, ninitial, nfinal, E> • Associate events with nodes

G is the alphabet of Go Events must be indivisible wrt other events

in the property

• L(G) is the language of Go The set of all strings in (G) that occur on

paths from the initial node to the final node• CFG is alphabet refined

o Remove nodes that do not affect the property being verified

Alphabet Refinement

• p is the alphabet of the property Po E.g., p = {open, close,

move}

• For alphabet refinement, remove all nodes in the TFG that do not have events in p OR do not control the flow of execution for nodes with such events

11

22

33

closeclose openopen

movemove

closeclosemovemoveopenopen

openopen

closeclosemovemove

Simple Sequential Example

……1:1: if (stopped) thenif (stopped) then2:2: open;open;

end if;end if;……

3:3: if (stopped) thenif (stopped) then4:4: close;close;

end if;end if;……

5:5: move;move;……

1: if1: if

2: open2: open

3: if3: if

4: close4: close

5: move5: move

Proving Properties

• Given a CFG G and a property Po Alphabet refine G with respect to P

o Need to show L(G) L(P)

• Use data-flow analysis to propagate states of P to the nodes of Go In(n) = Upred(n)Out(n)o Out(n) = UtIn(n)(t)

where is the transition function

• Worst-case cost is O((NG)2 SP)

State Propagation

2: open2: open

4: close4: close

5: move5: move

Worklist: 2, 3Worklist: 2, 3, 4, 5, 4, 5

11

22

33

closeclose openopen

movemove

closeclosemovemoveopenopen

openopen

closeclosemovemove

3: if3: if

1: if1: if{1}{1}

{1}{1}

{2}{2}

{1}{1}

{1,2}{1,2}

{1,2}{1,2}

{1}{1}

{1,2}{1,2}

{1,3}{1,3}

{1,2}{1,2}

State Propagation

1

22

3

closeclose openopen

movemove

closeclosemovemoveopenopen

openopen

closeclosemovemove

Worklist: 2, 3Worklist: 2, 3, 4, 5, 4, 5

2: open2: open

4: close4: close

5: move5: move

3: if3: if

1: if1: if{1}{1}

{1}{1}

{2}{2}

{1}{1}

{1,2}{1,2}

{1,2}{1,2}

{1}{1}

{1,2}{1,2}

{{1,31,3}}

{1,2}{1,2}

State Propagation

1: if

2: open

3: if

4: close4: close

5: move

{1}{1}

{2}{2}

{1}{1}

{1,2}{1,2}

{1,{1,33}}

11

22

3

closeclose openopen

movemove

closeclosemovemoveopenopen

openopen

closeclosemovemove

State Propagation

1: if

2: open

3: if

4: close4: close

5: move

……1:1: if (stopped) thenif (stopped) then2:2: open;open;

end if;end if;……

3:3: if (stopped) thenif (stopped) then4:4: close;close;

end if;end if;……

5:5: move;move;……

Boolean Variable Constraint

== is a predicate== is a predicate= is assignment= is assignment

S==tS==tS=tS=t

S==tS==tS=tS=t

S==tS==t

S==fS==fS=fS=f

S==fS==f

S==tS==tS=tS=t

S==fS==fS=fS=f

S==fS==fS=fS=f

S=fS=f

S=tS=t

uu

fftt

vv

Boolean Variable Constraint

== is a predicate== is a predicate= is assignment= is assignment

S==tS==tS=tS=t

S==tS==tS=tS=t

S==tS==t

S==fS==fS=fS=f

S==fS==f

S==tS==tS=tS=t

S==fS==fS=fS=f

S==fS==fS=fS=f

S=fS=f

S=tS=t

uu

fft

v

Improving Precision

• Use constraints to improve precisiono Represented as FSAs

• Given a CFG G, a property P, and constraints C1,…,Cno Alphabet refine G with respect to

(P C1 … Cn)o Want (L(G) L(C1) … L(Cn)) L(P)

• Worst-case cost is O(NG2 SP SC1 …

SCn)

Elevator Revisited

1: if

2: S==t

5: if5: if

9: move9: move

4: S==f

3: open3: open

6: S==t6: S==t 8: S==f8: S==f

7: close7: close

……1,2,4:1,2,4: if (stopped) thenif (stopped) then3:3: open; open;

end if;end if;……

5,6,8:5,6,8: if (stopped) thenif (stopped) then7:7: close; close;

end if;end if;……

9:9: move;move;……

, 6, 8, 6, 8, 5, 5, 3, 3

State Propagation

2: S==t2: S==t

1: if1: if

5: if5: if

9: move9: move

4: S==f4: S==f

3: open3: open

6: S==t6: S==t 8: S==f8: S==f

7: close7: close

11

22

33

closeclose openopen

movemovecloseclosemovemoveopenopen

openopen

closeclosemovemove

tt

vv

S==tS==t

ff

uuS==fS==f

S==fS==f S==tS==t

S==tS==t S==fS==f

S==fS==fS==tS==t

Worklist: 2, 4Worklist: 2, 4

<1,t><1,t>

<1,u><1,u>

<1,u><1,u>

<1,u><1,u>

<1,u><1,u>

<1,f><1,f>

<1,u><1,u>

<2,t>,<1,f><2,t>,<1,f>

<2,t>,<1,f><2,t>,<1,f>

<2,t><2,t>

<1,t><1,t>

<2,t>,<1,v><2,t>,<1,v>

<2,t>,<1,f><2,t>,<1,f>

1: if

5: if

, 6, 8, 6, 8, 5, 5, 3, 3

State Propagation

2: S==t2: S==t

9: move9: move

4: S==f

3: open3: open

6: S==t 8: S==f8: S==f

7: close7: close

<2,t>,<2,t>,<1,v><1,v>

Worklist: 2, 4Worklist: 2, 4

<1,u><1,u>

<1,t><1,t>

<2,t><2,t>

<1,f><1,f>

<2,t>,<1,f><2,t>,<1,f>

11

22

33

closeclose openopen

movemovecloseclosemovemoveopenopen

openopen

closeclosemovemove

tt

vv

S==tS==t

ff

uuS==fS==f

S==fS==f S==tS==t

S==tS==t S==fS==f

S==fS==fS==tS==t

<1,u><1,u>

<1,u><1,u><1,u><1,u>

<1,t><1,t>

<2,t>,<1,f><2,t>,<1,f>

<2,t>,<1,f><2,t>,<1,f>

1: if1: if

5: if5: if

, 6, 8, 6, 8, 5, 5, 3, 3

State Propagation

2: S==t2: S==t

9: move9: move

4: S==f4: S==f

3: open3: open

6: S==t6: S==t 8: S==f8: S==f

7: close7: close

<2,t>,<1,v><2,t>,<1,v>

Worklist: 2, 4Worklist: 2, 4

<1,u><1,u>

<1,t><1,t>

<2,t><2,t>

<1,f><1,f>

<2,t>,<1,f><2,t>,<1,f>

11

22

33

closeclose openopen

movemovecloseclosemovemoveopenopen

openopen

closeclosemovemove

tt

vv

S==tS==t

ff

uuS==fS==f

S==fS==f S==tS==t

S==tS==t S==fS==f

S==fS==fS==tS==t

, 7, 9, 7, 9<1,u><1,u>

<1,u><1,u><1,u><1,u>

<1,t><1,t>

<2,t>,<1,f><2,t>,<1,f>

<2,t>,<1,f><2,t>,<1,f> <2,t>,<1,f><2,t>,<1,f>

<2,t>,<1,f><2,t>,<1,f><1,t><1,t>

<2,v>,<1,f><2,v>,<1,f>

<1,t>,<1,f><1,t>,<1,f><1,t>,<1,f><1,t>,<1,f>

, 6, 8, 6, 8, 5, 5, 3, 3

State Propagation

Worklist: 2, 4Worklist: 2, 4

1

22

33

closeclose openopen

movemovecloseclosemovemoveopenopen

openopen

closeclosemovemove

t

vv

S==tS==t

f

uuS==fS==f

S==fS==f S==tS==t

S==tS==t S==fS==f

S==fS==fS==tS==t

, 7, 9, 7, 9

1: if1: if

5: if5: if

2: S==t2: S==t

9: move9: move

4: S==f4: S==f

3: open3: open

6: S==t6: S==t 8: S==f8: S==f

7: close7: close

<2,t>,<1,v><2,t>,<1,v>

<1,u><1,u>

<1,t><1,t>

<2,t><2,t>

<1,f><1,f>

<2,t>,<1,f><2,t>,<1,f>

<1,u><1,u>

<1,u><1,u><1,u><1,u>

<1,t><1,t>

<2,t>,<1,f><2,t>,<1,f>

<2,t>,<1,f><2,t>,<1,f> <2,t>,<1,f><2,t>,<1,f>

<2,t>,<1,f><2,t>,<1,f><1,t><1,t>

<2,v>,<1,f><2,v>,<1,f>

<1,t>,<1,f><1,t>,<1,f><1,t>,<1,f><1,t>,<1,f>

Outline

• Introduction• FLAVERS Model• Checking Properties• Improving Precision• Experimental Results• Conclusion

Experimental Goals

• Evaluate how FLAVERS performance scales as program size increaseso Timeo Memoryo Number of constraints

Chiron

• User interface systemo Developed at UC Irvine

• Uses event-based notificationo Similar to Listeners in Java

• Proved several properties about Chirono Avrunin, Corbett, Dwyer, Pasareanu, Siegel

Example Properties

• p07 - If listener1 registers for event1 before listener2, then listener1 will be notified of event1 before listener2

• p09 - The program never terminates while a listener is listening for an event

Chiron

• The Chiron system was scaled by increasing the number of events that can be listened for

• Lines of codeo 2 events 259o 53 events 3,557

• Constraints Neededo For every property, the constraints needed

to verify a property in the 2 event system are sufficient to verify the property for any system with more events

o Never needed more than 4 constraints

FLAVERS Times

1

10

100

1000

10000

0 5 10 15 20 25 30 35 40 45 50 55

Events

Time (s)

p01p02p03p04p05p06p07p08p09

Comparison to Other Approaches

• SMV [McMillan, 1993]o Symbolic model checking

• SPIN [Holzmann, 1991]o Optimized reachability analysis

• INCA [Corbett and Avrunin, 1995]o Integer linear programming

p07 Comparison (Original)

0.1

1

10

100

1000

10000

0 5 10 15 20 25 30 35 40 45 50 55

Events

Time (s)

INCA

Spin

SMV

NuSMV

Native Spin

FLAVERS

p07 Comparison (Decomposed)

0.1

1

10

100

1000

10000

100000

0 10 20 30 40 50 60 70 80 90 100

Events

Time (s)

INCA

Spin

SMV

NuSMV

Native Spin

FLAVERS

p09 Comparison (Original)

0.1

1

10

100

1000

10000

0 5 10 15 20 25 30 35 40 45 50 55

Events

Time (s)

INCA

Spin

SMV

NuSMV

Native Spin

FLAVERS

Experimental Results

• FLAVERS usually demonstrated subcubic performance in the size of the systemo The few examples that were not subcubic could be

solved by restating the properties

• Only took a few iterations to determine the constraints that should be selected

• Once a set of constraints was found for a small system configuration, the same set of constraints was sufficient for larger configurations

• Often did not have the *best* performance compared to other approaches, but consistently performed well

FLAVERS Times

1

10

100

1000

10000

0 5 10 15 20 25 30 35 40 45 50 55

Events

Time (s)

p01p02p03p04p05p06p07p08p09

Related Work

• Data-flow analysis o DAVE [Osterweil and Fosdick, 1976]o CESAR/CECIL [Olender and Osterweil, 1990 & 1992]o FLAVERS [Dwyer and Clarke, 1994]

• Other FSV Toolso SMV, NuSMVo SPINo Java PathFindero SLAMo INCAo …

Recent and Future Work• Support for Java• Specifying properties (Propel) (ICSE2002)• Heuristics for constraint selection (FSE2003)• Heuristics for counterexample selection

(FSE2004)• Compositional techniques• Design-time verification applied to

architectural description languages• Verification of process descriptions

o Scientific processeso Medical processes

To improve safety and efficiencyo ecommerce and egovernment processes

To assure security and privacy

Conclusions

• Finite state verification approaches are improving

• Being used in industry for hardware systems

• With the increasing interest in software security and quality, may become widely used for software systems

• FLAVERS provides a demonstration of its potential effectiveness