A quick look at three approaches to FSV
• Model Checking• Flow Equations• Data Flow Analysis
• FLAVERS
Property
System
Property Translator
SystemTranslator
ReasoningEngine
System ModelProperty Verified
Property Representation
Counter Examples for Model
High-Level Architecture of FSV Systems
Data Flow Based Verification: some history
• Mid-70’s: Originally proposed for def-ref anomalies in FORTRAN (Osterweil and Fosdick)
• Early 80’s: Extended to general properties (Olender and Osterweil) & concurrency (Taylor and Osterweil)
• 90’s: Deadlock detection (Masticola and Ryder); Efficient representation of concurrency & incremental precision improvement (Dwyer and Clarke)
• Recent: Optimizations, Java (Avrunin, Clarke, Cobleigh, Naumovich, and Osterweil)
Data Flow Analysis: FLAVERS
• FLow Analysis for VERification of Systems
• Represents property as a finite state automaton
• System model is collection of annotated control flow graphs
• intertask communication and interleavings are represented with additional nodes & edges
• does not enumerate all reachable states• over-approximates relevant executable
behaviors
• Reasoning engine based on data flow analysis
PropertySpecification
System
Property Translator
SystemTranslator
StatePropagation
Trace Flow Graph(TFG)
Property Verified
Property FSA
Counter Example Trace through TFG
High-Level Architecture of FLAVERS
Representing Properties
• Properties are represented as Finite State Automata (FSAs)
• Properties are either all or none• An all property is a behavior that must
always happen• A none property is a behavior that must
never happen
Elevator Property
The elevator does not movewhile its doors are open
P is the alphabet of the property
L(P) is the set of all stringsaccepted by Property P
11
22
33
closeclose openopen
movemove
closeclosemovemoveopenopen
openopen
closeclosemovemove
Control Flow Graph (CFG)
• A CFG G is <N, E , ninitial, nfinal > • Associate events with nodes
o G is the alphabet of G
• L(G) is the language of G• The set of all strings in (G) that occur on
paths from the initial node to the final node
• CFG is alphabet refined• Remove nodes that do not affect the
property being verified
Simple Sequential Example
……1:1: if (stopped) thenif (stopped) then2:2: open;open;
end if;end if;……
3:3: if (stopped) thenif (stopped) then4:4: close;close;
end if;end if;……
5:5: move;move;……
1: if1: if
2: open2: open
3: if3: if
4: close4: close
5: move5: move
Proving Properties
• Given a CFG G and a property P• Alphabet refine G with respect to P
• Want to show L(G) L(P)
• Use data-flow analysis to propagate states of P to the nodes of G
• Worst-case cost is O((NG)2 SP)
FLAVERS (similar to Cesar)
• Forward Flow, Any Path Problem• IN and OUT are sets of FSA states• IN(n) = OUT(m)
• OUT(n) = (s, n)
• is the FSA transition function
• Result: Let f be the final node of the TFG• All property: Want OUT(f) Accept(P)• None property: Want OUT(f) Accept(P) = • Accept(P) is the set of accepting states of a
property, P
m in pred(n)
s in IN(n)
State Propagation
2: open2: open
4: close4: close
5: move5: move
Worklist: 2, 3Worklist: 2, 3
{1}{1}
{2}{2}
{1}{1}
{1,2}{1,2}
{1,3}{1,3}
, 4, 5, 4, 5
11
22
33
closeclose openopen
movemove
closeclosemovemoveopenopen
openopen
closeclosemovemove
3: if3: if
1: if1: if{1}{1}
{1}{1}
{1,2}{1,2}
{1,2}{1,2}
{1,2}{1,2}
State Propagation
1
22
3
closeclose openopen
movemove
closeclosemovemoveopenopen
openopen
closeclosemovemove
Worklist: 2, 3Worklist: 2, 3
{1}{1}
{2}{2}
{1}{1}
{1,2}{1,2}
{{11,,33}}
, 4, 5, 4, 5
2: open2: open
4: close4: close
5: move5: move
3: if3: if
1: if1: if{1}{1}
{1}{1}
{1,2}{1,2}
{1,2}{1,2}
{1,2}{1,2}
State Propagation
1: if
2: open
3: if
4: close4: close
5: move
{1}{1}
{2}{2}
{1}{1}
{1,2}{1,2}
{1,{1,33}}
11
22
3
closeclose openopen
movemove
closeclosemovemoveopenopen
openopen
closeclosemovemove
State Propagation
1: if
2: open
3: if
4: close4: close
5: move
……1:1: if (stopped) thenif (stopped) then2:2: open;open;
end if;end if;……
3:3: if (stopped) thenif (stopped) then4:4: close;close;
end if;end if;……
5:5: move;move;……
PropertySpecification
System
Property Translator
SystemTranslator
StatePropagation
Trace Flow GraphProperty Verified
Property FSA
Counter Example Trace through TFG
Incrementally Improving Precision
...
Constraints
Boolean Variable Constraint
== is a predicate== is a predicate= is assignment= is assignment
S==tS==tS=tS=t
S==tS==tS=tS=t
S==tS==t
S==fS==fS=fS=f
S==fS==f
S==tS==tS=tS=t
S==fS==fS=fS=f
S==fS==fS=fS=f
S=fS=f
S=tS=t
uu
fftt
vv
Boolean Variable Constraint
== is a predicate== is a predicate= is assignment= is assignment
S==tS==tS=tS=t
S==tS==tS=tS=t
S==tS==t
S==fS==fS=fS=f
S==fS==f
S==tS==tS=tS=t
S==fS==fS=fS=f
S==fS==fS=fS=f
S=fS=f
S=tS=t
uu
fft
v
Constraints
• Constraints are represented as FSAs• Constraints describe conditions
necessary for feasible execution• Constraints have a special violation
state that is entered when an infeasible path is detected
• Violation is a trap state; once it is entered, never leave that state
Improving Precision
• Use constraints to improve precision
• Given a CFG G, a property P, and constraints C1,…,Cn
• Alphabet refine G wrt (P C1 … Cn)
• Want (L(G) L(C1) … L(Cn)) L(P)
• Worst-case cost is O(NG
2 SP SC1 … SCn)
How do constraints affect the data flow equations
• IN and OUT are now sets of tuples of FSA states
• Merge is still union• Transfer function now has to look at
each FSA state in the in-tuple when computing the out-tuple
• Result looks at paths that are feasible with respect to the constraints• The property state is the same as before• Every constraint must be in an accepting
state
Elevator Revisited
1: if
2: S==t
5: if5: if
9: move9: move
4: S==f
3: open3: open
6: S==t6: S==t 8: S==f8: S==f
7: close7: close
……1,2,4:1,2,4: if (stopped) thenif (stopped) then3:3: open; open;
end if;end if;……
5,6,8:5,6,8: if (stopped) thenif (stopped) then7:7: close; close;
end if;end if;……
9:9: move;move;……
, 6, 8, 6, 8, 5, 5, 3, 3
State Propagation
2: S==t2: S==t
1: if1: if
5: if5: if
9: move9: move
4: S==f4: S==f
3: open3: open
6: S==t6: S==t 8: S==f8: S==f
7: close7: close
11
22
33
closeclose openopen
movemovecloseclosemovemoveopenopen
openopen
closeclosemovemove
tt
vv
S==tS==t
ff
uuS==fS==f
S==fS==f S==tS==t
S==tS==t S==fS==f
S==fS==fS==tS==t
<2,t>,<1,v><2,t>,<1,v>
Worklist: 2, 4Worklist: 2, 4
<1,u><1,u>
<1,t><1,t>
<2,t><2,t>
<1,f><1,f>
<2,t>,<1,f><2,t>,<1,f>
<1,u><1,u>
<1,u><1,u> <1,u><1,u>
<1,t><1,t>
<2,t>,<1,f><2,t>,<1,f>
<2,t>,<1,f><2,t>,<1,f>
1: if
5: if
, 6, 8, 6, 8, 5, 5, 3, 3
State Propagation
2: S==t2: S==t
9: move9: move
4: S==f
3: open3: open
6: S==t 8: S==f8: S==f
7: close7: close
<2,t>,<2,t>,<1,v><1,v>
Worklist: 2, 4Worklist: 2, 4
<1,u><1,u>
<1,t><1,t>
<2,t><2,t>
<1,f><1,f>
<2,t>,<1,f><2,t>,<1,f>
11
22
33
closeclose openopen
movemovecloseclosemovemoveopenopen
openopen
closeclosemovemove
tt
vv
S==tS==t
ff
uuS==fS==f
S==fS==f S==tS==t
S==tS==t S==fS==f
S==fS==fS==tS==t
<1,u><1,u>
<1,u><1,u><1,u><1,u>
<1,t><1,t>
<2,t>,<1,f><2,t>,<1,f>
<2,t>,<1,f><2,t>,<1,f>
State Propagation 1
22
33
closeclose openopen
movemovecloseclosemovemoveopenopen
openopen
closeclosemovemove
t
vv
S==tS==t
f
uuS==fS==f
S==fS==f S==tS==t
S==tS==t S==fS==f
S==fS==fS==tS==t
1: if1: if
5: if5: if
, 6, 8, 6, 8, 5, 5, 3, 3
2: S==t2: S==t
9: move9: move
4: S==f4: S==f
3: open3: open
6: S==t6: S==t 8: S==f8: S==f
7: close7: close
<2,t>,<1,v><2,t>,<1,v>
Worklist: 2, 4Worklist: 2, 4
<1,u><1,u>
<1,t><1,t>
<2,t><2,t>
<1,f><1,f>
<2,t>,<1,f><2,t>,<1,f>
, 7, 9, 7, 9<1,u><1,u>
<1,u><1,u><1,u><1,u>
<1,t><1,t>
<2,t>,<1,f><2,t>,<1,f>
<2,t>,<1,f><2,t>,<1,f> <2,t>,<1,f><2,t>,<1,f>
<1,t><1,t>
<2,v>,<1,f><2,v>,<1,f>
<1,t>,<1,f><1,t>,<1,f>
<2,t><2,t>
<1,t>,<1,f><1,t>,<1,f>
Versatility of Constraints
• Can be used to model:• Values of variables• Control flow within a specific thread of
control• Behavior of hardware components• Possible behaviors of the environment• Intended behaviors of unimplemented
modules
Support for Concurrency:
• Different concurrency constructs• Rendezvous-based in Ada• Shared variable-based, monitor style in Java• FLAVERS/Ada and FLAVERS/Java• Hence, different program models
• Still use the same state propagation algorithm
Reachability based model
3
Ra
T1 T2 1,6
2,6
5,10
3,6
Ra
8
1,71 6
2,7 1,8
3,7 2,8
b_3,b_8
•State explosion
2 7
5 10
Can build the reachability graph
b_3,7 2,b_8
b_3,6 1,b_8
b_3,8
3,8
3,b_8
Trace Flow Graph Model
3
Ra
2
10
T1 T2
5
7
8
1 6
• FLAVERS model is a Trace Flow Graph (TFG)
• Automatically created from the source code
• Instead of the state space, explicitly represents interleaved execution May Immediately Precede (MIP) edges
• Smaller model• Less precise
Trace Flow Graph
3
Ra
2
10
T1 T2
5
7
8
1 6
• This model has many infeasible paths because of the MIP edges
• Can use constraints to keep track of the flow through a task
Using Constraints
3
Ra
2
10
T1 T2
5
7
8
1 6
Task constraint for T1
1
2
3
Ra
5
v
2,3,Ra,5
1,3,Ra,5
1,2,Ra,5
1,2,3,5
1,2,3,Ra
1,2,3,Ra,5
Experimental Goals
• Evaluate how FLAVERS performance scales as program size increases
• Time• Memory• Number of constraints
Chiron
• User interface system• Developed at UC Irvine
• Uses event-based notification• Similar to Listeners in Java
• Proved several properties about Chiron
• Avrunin, Corbett, Dwyer, Pasareanu, Siegel
Example Properties
• p07 - If listener1 registers for event1 before listener2, then listener1 will be notified of event1 before listener2
• p09 - The program never terminates while a listener is listening for an event
Chiron
• The Chiron system was scaled by increasing the number of events that can be listened for
• Lines of code• 2 events 259• 53 events 3,557
• Constraints Needed• For every property, the constraints needed to verify a
property in the 2 event system are sufficient to verify the property for any system with more events
• Never needed more than 4 constraints
38
FLAVERS Times
1
10
100
1000
10000
0 5 10 15 20 25 30 35 40 45 50 55
Events
Tim
e (s
)
p01p02p03p04p05p06p07p08p09
Comparison to Other Approaches
• SMV [McMillan, 1993]• Symbolic model checking
• SPIN [Holzmann, 1991]• Optimized reachability analysis
• INCA [Corbett and Avrunin, 1995]• Integer linear programming
40
p07 Comparison (Original)
0.1
1
10
100
1000
10000
0 5 10 15 20 25 30 35 40 45 50 55
Events
Tim
e (s
)
INCASpinSMVNuSMVNative SpinFLAVERS
41
p07 Comparison (Decomposed)
0.1
1
10
100
1000
10000
100000
0 10 20 30 40 50 60 70 80 90 100
Events
Tim
e (s
)
INCASpinSMVNuSMVNative SpinFLAVERS
42
p09 Comparison (Original)
0.1
1
10
100
1000
10000
0 5 10 15 20 25 30 35 40 45 50 55
Events
Tim
e (s
)
INCASpinSMVNuSMVNative SpinFLAVERS
Related Work
• Data-flow analysis • DAVE [Osterweil and Fosdick, 1976]• CESAR/CECIL [Olender and Osterweil, 1990 & 1992]• FLAVERS [Dwyer and Clarke, 1994]
• Other FSV Tools• SMV, NuSMV• SPIN• Java PathFinder• SLAM• INCA• Blast• …
Some Observations: Data Flow Analysis
• Overall complexity is O(N2S)• N is the # nodes in the model • S is the number of states:
property x constraints• In our experience, many important
properties can be proven with a small number of constraints
• Experimentally: performance sub-cubic• Usually requires several iterations to
determine needed constraints• Constraints
• Many automatically generated on request