Towards a Compositional SPIN

Corina Păsăreanu QSS, NASA Ames Research Center

Dimitra GiannakopoulouRIACS/USRA, NASA Ames Research Center

Main objective:An integrated environment that supports software development and verification/validation throughout the lifecycle; detect integration problems early, prior to coding

Approach: Compositional (“divide and conquer”) verification, for increased scalability, at design levelUse design level artifacts to improve/aid coding, testing and fault containment

CompositionalVerification

TestingDesign CodingRequirements Deployment

C1 C2

C1

C2

M1

M2

models implementations

Cost of detecting/fixing defects increases

Integration issues handled early

Project Overview

Towards a Compositional SPIN

Learning based compositional analysis for increased scalability [TACAS’03] – LTSA tool

Contributions– Generic tool architecture for learning based assume guarantee

reasoning• Handles multiple components

• Uses SPIN to answer model checking questions

• Other tools can be used (e.g. Java PathFinder)

– Heuristic for automated generation of interface specifications – Application to realistic resource arbiter for a spacecraft

• Significant memory gains over “traditional” non-compositional model checking

Outline

Introduction

Background: assume-guarantee reasoning and learning

Tool architecture

Implementation– Using SPIN for answering learning questions– Promela subset

Case study: MER Arbiter model – Description, results, discussion

Conclusions and future work

Compositional Verification

M2

M1

A

satisfies P?

Check P on entire system: too many states!

Use the natural decomposition of the system into its components to break-up the verification task

Check components in isolation:

Does M1 satisfy P?

– Typically a component is designed to satisfy its requirements in specific contexts / environments

Assume-guarantee reasoning: introduces assumption A representing M1’s “context”

Does system made up of M1 and M2 satisfy property P?

Assume-Guarantee Rules

M2

M1

A

satisfies P?

“discharge” the assumption

1. A M1 P2. true M2 A

3. true M1 || M2 P

How do we come up with the assumption?(usually a difficult manual process)

Reason about triples:

A M PThe formula is true if whenever M is part of a system

that satisfies A, then the system must also guarantee P

Approaches

Infer assumptions automatically

Two solutions developed1. Algorithmic generation of assumption (controller);

knowledge of environment is not required [ASE’02]

2. Incremental assumption computation based on counterexamples, learning and knowledge of environment [TACAS’03, SAVCBS’03 (symmetric rules)]

Formalisms

Components modeled as finite state machines (FSM)– FSMs assembled with parallel composition operator “||”

• Synchronizes shared actions, interleaves remaining actions

A property P is a FSM– P describes all legal behaviors

– Perr – determinize & complete P; bad behaviors lead to “error”

– Component M satisfies P iff error state unreachable in (M || Perr)

Assume-guarantee reasoning– Assumptions and guarantees are FSMs A M P holds iff error state unreachable in (A || M || Perr)

– The alphabet of A, A, contains all environment actions that appear in P

Example

Input

in send

ack

Output

outsend

ack

Ordererr

in

out

in out

||

Computing the Weakest Assumption

Given component M, property P, and the interface of M with its environment, generate the weakest environment assumption A such that: assuming A, M ╞ P

Weakest means that for all environments E:

(E || M ╞ P) IFF E╞ A

Learning for Assume-guarantee Reasoning

Use an off-the-shelf learning algorithm to build appropriate assumption for the rule

Process is iterativeAssumptions are generated by querying the system, and are gradually refined Queries are answered by model checkingRefinement is based on counterexamples obtained by model checkingTermination is guaranteed

Extended with symmetric rules

1. A M1 P2. true M2 A

3. true M1 || M2 P

Learning with L*

L* algorithm by Angluin, improved by Rivest & Schapire

Learns an unknown regular language U (over alphabet ) and produces a DFA A such that L(A) = U

Uses a teacher to answer two types of questionsUnknown regular language U

L*

conjecture: Ai

query: string s

true

false

remove string t

add string t

output DFA A such that L(A) = U

true

false

false

is s in U?

is L(Ai)=U?

Learning Assumptions

Use L* to generate candidate assumptions

A = (M1 P) M2

L*

query: string s

true

false

s M1 P

conjecture: AiAi M1 P

true M2 Ai

true error?

true

false

true

yes

false

no

remove counterex.

add counterex.

P holds in

M1 || M2

P violated

1. A M1 P2. true M2 A

3. true M1 || M2 P

Model Checking

Characteristics

Terminates with minimal automaton A for U

Generates DFA candidates Ai: |A1| < | A2| < … < |A|

Produces at most n candidates, where n = |A|

# queries: (kn2 + n logm),– m is size of largest counterexample, k is size of alphabet

For n components (M1 || M2 || … || Mn) apply algorithm recursively: Ai M1 P as before

true M2 || … || MnAi invokes the framework

true P Ai

Learning Interface Specifications

Compute an assumption (as weak as possible) for a component M1 to guarantee some property P, when environment is not available

A = P

L*

query: string s

true

false

s M1 P

conjecture: AiAi M1 P

true

counterexample

true

false

return Ai

counterexample

not accurate with respect to !P || !M1

Implementation in SPIN

Stand-alone application– Invokes SPIN for queries and conjectures

Handles only a Promela subset– Components are processes – Communication through rendezvous channels

Safety properties – SPIN trace assertions

Checking assume-guarantee triples– Encode assumptions into environment processes that run in

parallel with components

Case Study

Model derived from MER Resource Arbiter– Local management of resource contention between resource

consumers (e.g. science instruments, communication systems)– Avoid simultaneous conflicts (e.g. driving while capturing a

camera image are mutually incompatible)– Enforce orderly activity in accordance with predefined priorities

encoded in a lookup table

5 resources– Comm, Drive, PanCam, Arm, Rat

5 User threads– Non-deterministically decide to use any of the 5 resources

~3000 LOCChecked several safety properties

Arbiter Architecture

ARB

U5

U4

Request, CancelU3

U2

U1Grant, Deny

Rescind

Property PMutual exclusion between resources

Comm and Drive can not be used at the same time

Incremental Property Checking

Compute A1 … A5 s.t.

A1 U1 P

A2 U2 A1

A3 U3 A2

A4 U4 A3

A5 U5 A4

true ARB A5

ResultP holds on U1 || .. || U5 || ARB

Two techniques– Recursive invocation– Interface specification

Comparison with non-compositional analysis

U1

U2

P

A2 A1

ARB

A1

U3

U4

A2

A4 A3

A3

U5A5 A4

A5

Analysis Memory State Space Time

tmodel tcompile trun

Assumption Size

Monolithic 544 MB 3.9e+06 0.02s 0.8s 33.7s N/A

Recursive 2.6 MB 1002 0.03s 1.1s 0.03s 6 .. 12

Interface 2.6 MB 2941 0.04s 1.3s 0.02s 12

Analysis Results

Results do not reflect the cost of generating the assumptions

Cost of Assumption Generation

Analysis queries oracle1 oracle2 tSPIN+tLearn MemLearn tLTSA MemLTSA

Recursive 4884 48 1 5646.3s 1743K 42.8s 20400K

Interface

(A1)

852 12 1 818.2s 508K 3.07s 4845K

LTSA tool

Discussion

Significant memory savings

Serious time overhead

Experimented with SPIN’s feature for separate compilation– Encode assumptions as never claims– Restrict search of the verifier

New encoding for queries– Significant improvement– Cost of interface generation reduced from 818.213s to 185.185s

Conclusion and Future Work

Tool architecture for compositional verification– Uses L* for automatic derivation of assumptions– Automated derivation of interface specification

“Loose” integration with SPIN– Model checking for queries and conjectures

Application to Arbiter case study– Significant memory gains– Serious time overhead ; techniques to address this issue

Future work– Tighter integration with SPIN– Parallel checks for queries– Investigate buffered message passing communication– Liveness (learning for infinitary regular sets)– Distinguish between read and write operations