Application of Design for Verification with Concurrency Controllers to

Air traffic Control Software

Aysu Betin-Can, Tevfik Bultan

Department of Computer Science

University of California, Santa Barbara

Mikael Lindvall, Benjamin Lux, Stefan Topp

Fraunhofer Center for Experimental Software Engineering, Maryland

Concurrent Programming in Java

• Java uses a variant of monitor programming

• Synchronization using locks– Each object has a locksynchronized(o) { ... }

• Coordination using condition variables– Objects can be used as condition variablessynchronized (condVar){ while (!condExp) wait(condVar); ... notifyAll(condVar); }

Dangers in Java Concurrency

• Nested locks

synchronized m(other) { other.m(); }• Missed notification

notify(condVar);• Forgotten condition check

if(!condExp) wait(condVar);• Dependency among multiple condition variables can be

very complicated– Conservative notification and condition check:

Inefficient– Optimizing the notification and condition checks:

Error prone

What will I talk about today?

• A design for verification approach for eliminating concurrency errors in Java programs

• Application of this design for verification approach to a safety critical air traffic control software component

• Experiments demonstrating the effectiveness of the presented design for verification approach

Model Checking Software

• Scalability of software model checking depends on– Extracting compact models from programs

• This typically requires a reverse engineering step based on user guidance and/or static analysis techniques to – Rediscover some information about the software that

may be known at design time

• Alternative approach: Design for verification – Structure software in ways that facilitate verification– Document the design decisions that can be useful for

verification– Improve the scalability of verification using this

information

A Design for Verification Approach

We have been investigating a design for verification approach based on the following principles:

1. Using design patterns that facilitate automated verification

2. Use of stateful, behavioral interfaces which isolate the behavior and enable modular verification

3. An assume-guarantee style modular verification strategy which separates verification of the behavior from the verification of the conformance to the interface specifications

4. A general model checking technique for interface verification

5. Domain specific and specialized verification techniques for behavior verification

• A verifiable behavioral design pattern for concurrent programs

– Defines customized synchronization policies

– Avoids usage of error-prone Java synchronization primitives: synchronize, wait, notify, notifyAll

– Separates controller behavior from the threads that use the controller

• Supports modular verification and model extraction

Concurrency Controller Pattern

ControllerShared

Concurrency Controller Pattern

ThreadA

ThreadB

StateMachine

Controller

-var1-var2

+action1()+action2()

Action

+blocking()+nonblocking()-GuardedExecute

SharedStub

+a()+b()

Shared

+a()+b()

GuardedCommand

+guard()+update()

int GuardedCommand

ControllerStateMachine

+action1()+action2()

used at runtimeused at interface verificationused both times

Helperclasses

class RWController implements RWInterface{int nR; boolean busy;final Action act_r_enter, act_r_exit;final Action act_w_enter, act_w_exit;RWController() {... gcs = new Vector(); gcs.add(new GuardedCommand() { public boolean guard(){ return (nR == 0 && !busy );} public void update(){busy = true;}} ); act_w_enter = new Action(this,gcs);}public void w_enter(){ act_w_enter.blocking();}public boolean w_exit(){ return act_w_exit.nonblocking();}public void r_enter(){ act_r_enter.blocking();}public boolean r_exit(){ return act_r_exit.nonblocking();}}

Reader-Writer Controller

Action Class: Used As Is

class Action{protected final Object owner;. . .private boolean GuardedExecute(){ boolean result=false; for(int i=0; i<gcV.size(); i++) try{

if(((GuardedCommand)gcV.get(i)).guard()){ ((GuardedCommand)gcV.get(i)).update(); result=true; break; }

}catch(Exception e){} return result; }

public void blocking(){ synchronized(owner) { while(!GuardedExecute()) { try{owner.wait();}catch

(Exception e){} } owner.notifyAll(); }}public boolean nonblocking(){ synchronized(owner) { boolean

result=GuardedExecute(); if (result) owner.notifyAll(); return result; }}

Controller Interfaces

• A controller interface defines the acceptable call sequences for the threads that use the controller

• Interfaces are specified using finite state machines

public class RWStateMachine implements RWInterface{ StateTable stateTable; final static int idle=0,reading=1, writing=2; public RWStateMachine(){ ...

stateTable.insert("w_enter",idle,writing); } public void w_enter(){ stateTable.transition("w_enter"); } ...}

writing

reading

idle

r_enter

r_exit

w_exit

w_enter

Verification Framework

Action LanguageVerifier

Java PathFinder

Behavior Specification in Action Language

Optimized Java Code

Program with Stubs

Controllers andInterfaces

Behavior Translator

Notification-OptimizerData Stubs

Rest of the Program

Thread Isolation

Error Trace Verified

VerifiedError Trace

Behavior Verification

Interface Verification

Counting Abstractor

• Utilizes behavior and interface decoupling in the pattern

• Behavior verification– Verify the controller properties (e.g. safety, liveness)– Assume that the user threads adhere to the controller

interface

• Interface verification – Check that each user thread obeys the interface:

• A thread is correct with respect to an interface if all the call sequences generated by the thread can also be generated by the finite state machine defining the interface.

Modular Verification

Behavior Verification

• Analyzing properties (specified in CTL) of the synchronization policy encapsulated with a concurrency controller and its interface– Assume threads obey the controller interfaces

• Behavior verification with Action Language Verifier– Infinite state symbolic model checker– Suitable for specifications with unbounded variables and

parameterized constants– We wrote a translator which translates controller classes

to Action Language

Behavior Verification for Arbitrary Number of Threads

• Counting abstraction– Create an integer variable for each interface state– Each variable counts the number of threads in a

particular interface state– Automatically generate updates and guards for these

variables based on the interface specification

• Counting abstraction is automated

Three Types of Controller Properties

• Properties that only refer to controller variables

AG(busy => nR=0)

AG(busy => AF(!busy))

• Properties that refer to interface states

AG(pc=WRITING => AF(pc=IDLE)

AG(pc=READING => nR > 0)

• Properties for arbitrary number of threads

AG(WRITING > 0 => AF(IDLE > 0)

AG(READING = nR)

Interface Verification

• Checks if all the threads invoke controller methods in the order specified in the interfaces

• Checks if the threads access shared data only at the correct interface states

• Interface verification with Java PathFinder– Verify Java implementations of threads– Correctness criteria are specified as assertions

• Look for assertion violations• Assertions are in the StateMachine and SharedStub

– Performance improvement with thread Isolation

Thread Isolation: Part 1

• Interaction among threads

• Threads can interact with each other in only two ways:– invoking controller actions– Invoking shared data methods

• To isolate the threads– Replace concurrency controllers with controller interface

state machines– Replace shared data with shared stubs

Thread Isolation: Part 2

• Interaction among a thread and its environment

• Modeling thread’s call to its environment with stubs– File I/O, updating GUI components, socket operations,

RMI call to another program• Replace with pre-written or generated stubs

• Modeling the environment’s influence on threads with drivers– Thread initialization, RMI events, GUI events

• Enclose with drivers that generate all possible events that influence controller access

Automated Airspace Concept

• Automated Airspace Concept by NASA researchers automates the decision making in air traffic control

• The most important challenge is achieving high dependability

• Automated Airspace Concept includes a failsafe short term conflict detection component– Dependability of this component is even more important

than the dependability of the rest of the system– It should be a smaller, isolated component compared to

the rest of the system so that it can be verified

Tactical Separation Assisted Flight Environment (TSAFE)

• TSAFE is an implementation of this failsafe short term conflict detection component– It is developed at MIT based on the design of the NASA

researchers

• It is responsible for detecting conflicts in flight plans of the aircraft within 1 minute from the current time

• Functionality:

1. Display aircraft position

2. Display aircraft planned route

3. Display aircraft future projected route trajectory

4. Show conformance problems

Server

Computation

FlightDatabase

GraphicalClient

Client

<<RMI>>

21,057 lines of code with 87 classes

Radar feed<<TCP/IP>>

User

EventThread

Feed Parser

Timer

Tactical Separation Assisted Flight Environment (TSAFE)

Reengineering TSAFE

• Found all the synchronization statements in the code

(synchronize, wait, notify, notifyAll)

• Identified 6 shared objects protected by these synchronization statements

• Used 2 instances of a reader-writer controller and 3 instances of a mutex controller for synchronization

• In the reengineered TSAFE code the synchronization statements appear only in the Action helper class provided by the concurrency controller pattern

Behavior Verification Performance

RW 0.17 1.03

Mutex 0.01 0.23

Barrier 0.01 0.64

BB-RW 0.13 6.76

BB-Mutex 0.63 1.99

Controller Time(sec) Memory (MB) P-Time (sec) P-Memory (MB)

8.10 12.05

0.98 0.03

0.01 0.50

0.63 10.80

2.05 6.47

P denotes parameterized verification for arbitrary number of threads

Interface Verification Performance

Thread Time (sec) Memory (MB)

TServer-Main 67.72 17.08

TServer-RMI 91.79 20.31

TServer-Event 6.57 10.95

TServer-Feed 123.12 83.49

TClient-Main 2.00 2.32

TClient-RMI 17.06 40.96

TClient-Event 663.21 33.09

Fault Categories

• Concurrency controller faults– initialization faults (2) – guard faults (2)– update faults (6)– blocking/nonblocking faults (4)

• Interface faults– modified-call faults (8)– conditional-call faults

• conditions based on existing program variables (13)• conditions on new variables declared during fault

seeding (5)

Effectiveness in Finding Faults

• Created 40 faulty versions of TSAFE

• Each version had at most one interface fault and at most one behavior fault – 14 behavior and 26 interface faults

• Among 14 behavior faults ALV identified 12 of them– 2 uncaught faults were spurious

• Among 26 interface faults JPF identified 21 of them– 2 of the uncaught faults were spurious– 3 of the uncaught faults were real faults that were not

caught by JPF

Falsification Performance

TServer-RMI 29.43 24.74

TServer-Event 6.88 9.56

TServer-Feed 18.51 94.72

TClient-RMI 10.12 42.64

TClient-Event 15.63 12.20

Thread Time (sec) Memory (MB)

RW-8 0.34 3.26

RW-16 1.61 10.04

RW-P 1.51 5.03

Mutex-8 0.02 0.19

Mutex-16 0.04 0.54

Mutex-p 0.12 0.70

Concurrency Controller Time (sec) Memory (MB)

Conclusions

• ALV performance– Cost of parameterized verification was somewhere

between concrete instances with 8 and 16 threads– Falsification performance was better than verification

• Completeness of the controller properties– Effectiveness of behavior verification by ALV critically

depends on the completeness of the specified properties

• Concrete vs. parameterized behavior verification– When no faults are found, the result obtained with

parameterized verification is much stronger – However for falsification we observed that concrete

instances were as effective as parameterized instances

Conclusions

• JPF performance– Typically falsification performance is better than

verification performance– In some cases faults caused execution of new code

causing the falsification performance to be worse than verification performance

• Thread isolation– Automatic environment generation for threads result in

too much non-determinism and JPF runs out of memory– Dependency analysis was crucial for mitigating this

• Deep faults were difficult to catch using JPF– Three uncaught faults were created to test this

Conclusions

• Unknown shared objects– The presented approach does not handle this problem– Using escape analysis may help

• We could not find a scalable and precise escape analysis tool

• Environment generation – This is the crucial problem in scalability of the interface

verification– Using a design for verification approach for environment

generation may help

Related Work

• Design for Verification– [MP03] Mehlitz et al. promote using design patterns and

exploiting their properties for automated verification– [SBK01] Sharygina et al. focus on verification of UML

models– Design for verification has been used in hardware design

and embedded systems [SF03], [GSB02], [SBBCM01]

• Assume-guarantee style modular verification for software– [PDH99], [CCGJV03]

Related Work

• Design Patterns for multi-threaded systems– [SSRB00], [Lea99], [Gra02],[SPM96]

• Verification and synthesizing monitors– [DDHM02], [YKB02], [MK99]

• Environment extraction/generation– [PDH99],[TD03], [TDP03]

Related Work

• Behavioral and stateful interfaces– Interface compatibility checking [CAHJM02]– Extended type systems with stateful interfaces and

interface checking as a part of type checking [DF01,DF04]

– Interface discovery and synthesis by analyzing existing code [WML02],[ACMN05]