Jpf model checking

1

Model Checking Programswith

Java PathFinder

Willem Visser<wvisser@email.arc.nasa.gov>

Peter Mehlitz<pcmehlitz@email.arc.nasa.gov>

NASA Ames Research Center

2

Motivation

✦ 1997 – Deep Space 1 Remote Agent

✦ 1999 – Honeywell DEOS IMA O/S

✦ Model extraction by hand doesn’t scale

✦ Automated Program Model Checking

✦ Translation-based model checking is only has good as the target model checker• Java to PROMELA translation for SPIN

3

Program Model Checking

✦ 10 years ago• Almost no one did it

✦ 5 years ago• Just a handful of tools existed

✦ Now• BANDERA, BLAST, CBMC, dSPIN, JPF, MAGIC, SLAM, SPIN 4,

Verisoft, CMC, ZING, etc.

4

Overview

✦ What is JPF? • A model checker for Java Bytecode

✦ Getting Started with JPF• Downloading, Installation and Running

✦ Examples Part 1• Deadlock in the Remote Agent• K9 Rover

✦ Internal workings of JPF• Model Java Interface• Listeners• Dynamic Partial-order reductions• Choice Generators• Extensions• Regression tests (JUnit Integration)

✦ Examples Part 2• Stoned Hippies in three variations• Race Detection• Numeric Exceptions• Partial Trace Analysis

✦ Symbolic Execution✦ How we spent the summer!

• Abstractions• Decision Procedure Extensions • User Interface Analysis• Java Static Analysis

9h00 – 10h30

11h00 – 13h00

5

What is Java PathFinder (1)

✦ explicit state model checker for Java bytecode

✦ focus is on finding bugs in Java programs• concurrency related: deadlocks, (races), missed signals etc.• Java runtime related: unhandled exceptions, heap usage, (cycle budgets)• but also: complex application specific assertions

6

What is JPF (2)

✦ goal is to avoid modeling effort (check the real program), or at least use a real programming language for complex models

✦ implies that the main challenge is scalability

✦ JPF uses a variety of scalability enhancing mechanisms• user extensible state abstraction & matching• on-the-fly partial order reduction• configurable search strategies: "find the bug before you run out of memory"• user definable heuristics (searches, choice generators)

✦ key issue is configurable extensibility: overcome scalability constraints with suitable customization (using heuristics)

7

Key Points

✦ Models can be infinite state• Unbounded objects, threads,…• Depth-first state generation (explicit-state)• Verification requires abstraction

✦ Handle full Java language • but only for closed systems• cannot directly handle native code

‣ no Input/output through GUIs, files, Networks, …‣ Must be modeled by java code instead

✦ Allows Nondeterministic Environments• JPF traps special nondeterministic methods

✦ Checks for User-defined assertions, deadlock and user-specified properties

8

JPF Status

✦ developed at the Robust Software Engineering Group at NASA Ames Research Center

✦ currently in it’s fourth development cycle• v1: Spin/Promela translator - 1999• v2: backtrackable, state matching JVM - 2000• v3: extension infrastructure (listeners, MJI) - 2004• v4: symbolic execution, choice generators - 4Q 2005

✦ open sourced since 04/2005 under NOSA 1.3 license:<javapathfinder.sourceforge.net>

✦ it’s a first: no NASA system development hosted on public site before

✦ 11100 downloads since publication 04/2005

9

Getting and Installing JPF

✦ Getting JPF• svn co https://svn.sourceforge.net/svnroot/javapathfinder javapathfinder

✦ Compiling JPF• Go to the JPF root directory• Type build-tools/bin/ant

‣ Compiles JPF‣ Adding “run-tests” will also run the regression tests

✦ The Eclipse Route• Get Eclipse 3.2

‣ www.eclipse.org• Install Subclipse

‣ Goto: Help – Software Updates – Find and Install – Select New Features‣ Type the following in the url field: http://subclipse.tigris.org/update_1.2.x‣ Follow the instructions

• Create a new SVN project by selecting “Checkout Projects from SVN”• Create a new repository for https://svn.sourceforge.net/svnroot/javapathfinder • Select trunk• It will automatically download and compile in Eclipse• Hint

‣ If you ever run JPF from outside the JPF project then put a copy of default.properties in the build/jpf/gov/nasa/jpf directory – otherwise you’ll get a vm.class error at initialization

10

How To Run JPF

✦ generally speaking: like a VM (“java” replacement): > bin/jpf <jpf-options> <test-app main class>

✦ BUT: lots of configuration (classes) and parameterization (booleans, integers etc.)

✦ JPF is an open system

✦ need for flexible, extensible configuration system

✦ quite powerful, but can be somewhat confusing

11

JPF Configuration

12

Some Examples

✦ Remote Agent

✦ K9 Rover (‘real’ model, size)

13

Remote Agent

✦ “oldclassic.java” • Simplified version of the deadlock encountered in the Remote Agent

✦ Fixing oldclassic!• Or rather trying to fix…

T1 T2

signal

signalnotify

notify

if (no_action) wait();signal();

14

K9 Rover

✦ Executes flexible plans for autonomy • branching on state / temporal conditions

✦ Multi-threaded system• communication through shared variables• synchronization through mutexes and condition variables

✦ Main functionality: 8KLOC, C++/JAVA

commands

plans

Exec

Executive

CondCheckerDatabase

ActionExec

TempChecker

15

Directory Structure

16

Under the Hood - Toplevel Structure

✦ two major concepts: Search and VM

✦ Search is the VM driver and Property evaluator

✦ VM is the state generator

17

Under the Hood - Search

18

Extending JPF - Listeners

✦ preferred way of extending JPF: ‘Listener’ variant of the Observer pattern - keep extensions out of the core classes

✦ listeners can subscribe to Search and VM events

19

Extending JPF - SearchListener

public interface SearchListener { /* got the next state */  void stateAdvanced (Search search); /* state was backtracked one step */ void stateBacktracked (Search search);  /* a previously generated state was restored    (can be on a completely different path) */ void stateRestored (Search search);  /* JPF encountered a property violation */ void propertyViolated (Search search);  /* we get this after we enter the search loop, but BEFORE the first forward */  void searchStarted (Search search);  /* there was some contraint hit in the search, we back out could have been turned into a property, but usually is an attribute of the search, not the application */ void searchConstraintHit (Search search);  /* we're done, either with or without a preceeding error */ void searchFinished (Search search);}

20

Extending JPF - VMListener

public interface VMListener {  void instructionExecuted (JVM vm); // VM has executed next instruction  void threadStarted (JVM vm); // new Thread entered run() method   void threadTerminated (JVM vm); // Thread exited run() method   void classLoaded (JVM vm); // new class was loaded   void objectCreated (JVM vm); // new object was created   void objectReleased (JVM vm); // object was garbage collected   void gcBegin (JVM vm); // garbage collection mark phase started   void gcEnd (JVM vm); // garbage collection sweep phase terminated   void exceptionThrown (JVM vm); // exception was thrown  void nextChoice (JVM vm); // choice generator returned new value}

21

Extending JPF - Listener Example

public class HeapTracker extends GenericProperty implements VMListener, SearchListener { class PathStat { .. int heapSize = 0; .. } // helper to store additional state info

PathStat stat = new PathStat(); Stack pathStats = new Stack();

public boolean check (JVM vm, Object arg) { // GenericProperty return (stat.heapSize <= maxHeapSizeLimit); } public void stateAdvanced (Search search) { // SearchListener if (search.isNewState()) {.. pathStats.push(stat); stat = (PathStat)stat.clone(); .. } public void stateBacktracked (Search search) { // SearchListener .. if (!pathStats.isEmpty()) stat = (PathStat) pathStats.pop(); } public void objectCreated (JVM vm) {.. // VMListener ElementInfo ei = vm.getLastElementInfo(); ..stat.heapSize += ei.getHeapSize(); .. } public void objectReleased (JVM vm) { // VMListener ElementInfo ei = vm.getLastElementInfo(); ..stat.heapSize -= ei.getHeapSize(); .. } ...}

22

Extending JPF - Listener Configuration

✦ listeners are usually configured, not hard coded

✦ per configuration file: search.listener = MySearchListener vm.listener = MyVMListener jpf.listener = MyCombinedListener:MySecondListener...

✦ per command line: jpf ... +jpf.listener=MyCombinedListener ...

✦ hard coded: MyListener listener= new MyListener(..); .. Config config = JPF.createConfig( args); JPF jpf = new JPF( config); jpf. addSearchListener (listener); jpf. addVMListener ( listener); jpf.run(); ..

23

Going Native - Model Java Interface

✦ JPF is a state-tracking JVM, running on top of a general JVM

✦ Java Native Interface (JNI) analogy: “execute one level lower”

✦ Model Java Interface (MJI): execute in the host VM that runs JPF itself

24

MJI - Why?

✦ one obvious reason: running native Java methods in JPF(otherwise we couldn’t run apps using standard libraries, which have lots of native methods)

✦ specific use of native methods: interface library methods to JPF runtime system (e.g. java.lang.Thread -> ThreadInfo)

✦ enables usage of specialized verification API in app, interfacing to JPF functionality: int input = gov.nasa.jpf.jvm.Verify.randomInt(10);

✦ but also useful for scalability reasons• native methods can save state space• native methods are executed atomically• native methods execute much faster

✦ example: java.io.RandomAccessFile

25

MJI - Components

✦ Model class: has native method declaration, executed by JPF

✦ NativePeer class: native method implementation, executed by JVM

✦ MJIEnv: native method calling context (to get back to JPF)

26

MJI - How

27

MJI - Example

✦ application calls method to intercept .. System.out.println(“a message”);

✦ model class declares the method we want to intercept (doesn’t have to be native), is executed by JPF public class PrintStream .. { .. public void println (String s) {..} // usually native method }

✦ native peer has the method implementation that gets executed by host VM (not simulated by JPF) class JPF_java_io_PrintStream { .. public static void println__Ljava_lang_String_2 (MJIEnv env,int objRef, int strRef) { env.getVM().println(env.getStringObject(strRef)); } }

0: getstatic #23: ldc #35: invokevirtual #4

28

Scalability - Partial Order Reduction

✦ concurrency is major contributor to state space explosion

✦ reduction of thread interleavings is paramount for scalability

✦ JPF employs on-the-fly Partial Order Reduction mechanism

✦ leveled approach that makes use of JVM instruction set and infrastructure (memory management)

✦ completely at runtime (on-the-fly)

29

POR - Scheduling Relevance

30

POR - Shared Objects

✦ to detect races, we have to identify read/write access to objects that are visible from different threads

✦ expensive operation, BUT: can piggyback on garbage collection

✦ two phase approach:• mark root set with thread id (statics are shared by default)• traverse marked objects - if another thread id is reached, mark as shared

✦ problem: GC based on reachability, not accessibility -> need to break on certain fields (Thread.group->ThreadGroup.threads)

31

Choice Generator Motivation

32

JPF Perspective

State consists of 2 main components, the state of the JVM and the current and next choice Generator (i.e. the objects encapsulatingthe choice enumeration that produces new transitions)

Transition is the sequence of instructions that leads from one state. There is no context within a transition, it's all in the same thread. There can be multiple transitions leading out of one state Choice is what starts a new transition. This can be a different thread, i.e. scheduling choice, or different “random” data value.

33

Role of Choices

In other words, possible existence of Choices is what terminates the last Transition, and selection of a Choice value precedes the next Transition.

The first condition corresponds to creating a new ChoiceGenerator, and letting the SystemState know about it.

The second condition means to query the next choice value from this ChoiceGenerator (either internally within the JVM, or in an instruction or native method).

34

Extensions

✦ JPF was built to be extended✦ Architecture is such that the core classes can be left alone

when doing an extension• Add code to the “extensions” directory that follows the same layout as the core classes

Numeric Extension to check for errors such as Overflow

Most of this extension was provided by Aleksandar Milicevic and Sasa Misailovic from UIUC

35

Regression Tests with JUnit

36

JUnit Example

package gov.nasa.jpf.mc;import org.junit.Test;import org.junit.runner.JUnitCore;import gov.nasa.jpf.jvm.TestJPF;

public class TestOldClassicJPF extends TestJPF { static final String TEST_CLASS = "gov.nasa.jpf.mc.oldclassic";

public static void main (String[] args) { JUnitCore.main("gov.nasa.jpf.mc.TestOldClassicJPF"); }@Test public void testDFSearch () { String[] args = { TEST_CLASS }; runJPFDeadlock(args); }@Test public void testBFSHeuristic () { String[] args = { "+search.class=gov.nasa.jpf.search.heuristic.HeuristicSearch", "+search.heuristic.class=gov.nasa.jpf.search.heuristic.BFSHeuristic", TEST_CLASS }; runJPFDeadlock(args); }}

37

TestJPF

✦ Extends junit.Assert

✦ Interface methods• runJPFDeadlock(args)

‣ Expects a deadlock

• runJPFassertionError(args)‣ Expects an assertion violation

• runJPFnoAssertionError(args)‣ Don’t want to see an assertion violation

• runJPFException(args)‣ Expects an exception

• runJPFnoException(args)‣ Don‘t want to see an exception

38

More Examples

✦ Stoned Hippies• Regular version• Using MJI• Using a Listener

✦ Race Detection• A more sophisticated use of a Listener

✦ Numeric Extensions• With Junit integration

✦ Partial Trace Analysis• Recording a path and then re-analyze from the end of the path

39

Stoned Hippies

Germany Netherlands

5 10

21

40

Stoned Hippies

Germany Netherlands

5 10

1 22

41

Stoned Hippies

Germany Netherlands

5 10

1 23

42

Stoned Hippies

Germany Netherlands

10

1 2

5

8

43

Stoned Hippies

Germany Netherlands

1 2

5 10

19

44

Symbolic Execution

✦ Explicit-state model checking cannot handle large data domains

✦ Want to generate test cases for systems that manipulate complex data structures

Collaborators

Corina PasareanuSarfraz Khurshid

Saswat Anand

45

Concrete Execution Path (example)

x = 1, y = 0

1 >? 0

x = 1 + 0 = 1

y = 1 – 0 = 1

x = 1 – 1 = 0

0 – 1 >? 0

int x, y;

if (x > y) {

x = x + y;

y = x – y;

x = x – y;

if (x – y > 0)

assert(false);

}

46

Symbolic Execution Tree (example)

x = X, y = Y

X >?

Y

[ X > Y ] y = X + Y – Y = X

[ X > Y ] x = X + Y – X = Y

[ X > Y ] Y - X >? 0

[ X <= Y ] END [ X > Y ] x = X + Y

[ X > Y, Y – X <= 0 ] END [ X > Y, Y – X > 0 ] END

int x, y;

if (x > y) {

x = x + y;

y = x – y;

x = x – y;

if (x – y > 0)

assert(false);

}

47

Example

class Node {int elem;Node next;

Node swapNode() { if (next != null) if (elem > next.elem) { Node t = next; next = t.next; t.next = this; return t; } return this;}

}

? null

E0 E1

E0

E0 E1 null

E0 E1 ?

E0 E1

E0 E1

Input list + Constraint Output list

E0 > E1

none

E0 <= E1

none

E0 > E1

E0 > E1

E0 > E1

E1 E0 ?

E1 E0

E1 E0

E1 E0 null

E0 E1

E0

? null

NullPointerException

48

Challenges in Generalizing Symbolic Execution

✦ how to handle fields in dynamic structures?

✦ how to handle aliasing?

✦ how to generate tests?• satisfy criteria• satisfy precondition• are inequivalent

49

Generalized Symbolic Execution

✦ model checker generates and explores “symbolic” execution tree• non-determinism handles aliasing

‣ explore different heap configurations explicitly• concurrency• off-the-shelf decision procedures check path conditions

✦ lazy initialization• initializes program’s inputs on an “as-needed” basis• no a priori bound on input sizes

✦ preconditions to initialize inputs only with valid values

50

Algorithm (lazy initialization)

✦ to symbolically execute a method• create input objects with uninitialized fields• execute!

‣ follow mainly Java semantics‣ initialize fields “as-required”‣ add constraints to path condition

51

Algorithm (aliasing)

✦ when method execution accesses field fif (f is uninitialized) { if (f is reference field of type T) { non-deterministically initialize f to null a new object of class T (with uninitialized fields) an object created during prior field initialization (alias) } if (f is numeric/string field) initialize f to a new symbolic value }

52

Algorithm (illustration)

E0next

E1next

tnull

tE0

nextE1

next?

nextE0

nextE1

t next E0 nextE1

next

t

E0next

E1next

t

consider executingnext = t.next;

Precondition: acyclic list

E0 E1next

tnull

next

tE0 E1

next?

nextnext

53

program instrumentation

counterexample(s)/test suite[heap+constraint+thread scheduling]

Implementation via Instrumentation

model checking

decision procedure

instrumented program

correctness specification

continue/backtrack

path condition (data)heap configurationthread scheduling

state:

original

program

54

Testing with Symbolic Execution

✦ Focus on programs that manipulate complex data• Java container classes, e.g. java.util.TreeMap

✦ Black box test input generation• Using structural invariants• Using API calls

‣ With symbolic data, i.e. kind-of gray-box

✦ White box• Completely symbolic execution over structures

55

Red-Black Trees

(1) The root is BLACK

(2) Red nodes can only have black children

(3) All paths from a node to its leaves contain the same number of black nodes.

Self-balancing Binary Search TreesJava TreeMap Implementation

(4) Acyclic

(5) Consistent Parents

repOk(): conditions (1)-(5)

56

repOk() Fragment

boolean repOk(Entry e) { // root has no parent, root is black,… // RedHasOnlyBlackChildren workList = new LinkedList(); workList.add(e); while (!workList.isEmpty()) { Entry current=(Entry)workList.removeFirst(); Entry cl = current.left; Entry cr = current.right; if (current.color == RED) { if(cl != null && cl.color == RED) return false; if(cr != null && cr.color == RED) return false; } if (cl != null) workList.add(cl); if (cr != null) workList.add(cr); } // equal number of black nodes on left and right sub-tree… return true;}

57

Black-box TIG Symbolic Execution

✦ Symbolic execution of repOk()• Generate new structures only when repOk() returns true• Limit the size of the structures generated

✦ Only correct structures will be generated• repOk() returns true after all nodes in the tree have been visited, hence they must all

be concrete• symbolic (partial) structures can fail repOk()

✦ Similar to Korat• Not based on symbolic execution• Cannot deal with constraints on primitive data • Korat uses custom algorithms and is much faster

58

Symbolic Execution of repOk() Example

public static boolean repOk() {

if (root == null) return true;

if (root.color == RED) return false;

…

Size 1

59

White-box TIG Symbolic Execution

✦ Consider code coverage criterion when generating test inputs

✦ Use repOk() as a precondition during symbolic execution of source code

60

repOk() x 2abstract and concrete

Symbolic Execution of Code During Lazy Initializationcheck Abstract repOk()

When coverage is achieved, solve the symbolic constraints

to create concrete inputs

Concretize inputs by symbolic execution of

Concrete repOk()over symbolic structures

- as with Black-box TIG -

Abstract repOk() : Symbolic Structure {true,false,don’t know}

Concrete repOk() : Symbolic Structure Concrete Structure

61

Abstract repOk()

✦ Eliminate symbolic structures that cannot be converted to a concrete structure that satisfy repOk()• Can accept symbolic structures that could lead to illegal concrete structures, i.e. it is

conservative

✦ Abstract RepOk() can return TRUE, FALSE or Don’t Know• if FALSE, eliminate structure• if TRUE or Don’t Know, continue ...

✦ Example: (2) Red nodes have only black children.

FALSE TRUE Don’t Know

62

White-box TIG: cover branches in deleteEntry(Entry p)

/* precondition: p.repOk() */

private void deleteEntry(Entry p) {

if (p.left != null && p.right != null) {

Entry s = successor(p);

swapPosition(s, p);

}

Entry replacement = (p.left != null ? p.left : p.right);

if (replacement != null) {

replacement.parent = p.parent;

if (p.parent == null)

root = replacement;

else if (p == p.parent.left) {

p.parent.left = replacement;

}

else

p.parent.right = replacement;

p.left = p.right = p.parent = null;

if (p.color == BLACK)

fixAfterDeletion(replacement); ...

63

Symbolic Execution for white-box TIG

if (p.left != null && p.right != null) { ...

Symbolic structurebefore executingbranch

Concretize

Concrete structurethat will cover the code

The symbolic structure is used as input to repOk() and lazily executed to obtain the concrete structure

Symbolic structure(s)that coverthe branch

This structure “passes”the abstract repOk()

64

API Based Testing

SUT

ENV (m,n)

m is the seq. length of API calls &

n is the number of values used in the parameters of the calls

API…

put(v)del(v)

Evaluate different techniques for selecting

test-cases from ENV(m,n)to obtain maximum coverage

65

Framework

SUTwith

minor instrumentationENV

TestListener

Abstraction Mapping+

State StorageCoverage Manager

JPF

66

Environment Skeleton

M : sequence lengthN : parameter valuesA : abstraction used

for (int i = 0; i < M; i++) { int x = Verify.random(N - 1); switch (Verify.random(1)) { case 0: put(x); break; case 1: remove(x); break;} }Verify.ignoreIf(checkStateMatch());

67

Symbolic Environment Skeleton

M : sequence lengthA : abstraction used

for (int i = 0; i < M; i++) {

SymbolicInteger x = new SymbolicInteger(“X“+i); switch (Verify.random(1)) { case 0: put(x); break; case 1: remove(x); break;} }Verify.ignoreIf(checkStateMatch());

68

Sample Output

Test case number 77 for '15,L+R+P-REDroot': put(0);put(4);put(5);put(1);put(2);put(3);remove(4);

Unique ID for the testBranch Number Predicate Values

Test-case to achieve above coverage

Test case number 7 for '32,L-R-P+RED':X2(0) == X1(0) && X2(0) < X0(1) && X1(0) < X0(1)put(X0);put(X1);remove(X2);

Test case number 7 for '32,L-R-P+RED': put(1);put(0);remove(0);

Concrete

Symbolic

Path Conditionwith solutions

Symbolic TC

69

Subsumption Checking

x1

x2

x3 x4

x5 +x1 > x2 & x2 > x3 & x2 < x4 & x5 > x1

x1

x2

x3 x4

x5 +x1 > x2 & x2 > x3 & x2 < x4 & x5 > x1

If only it was this simple!

70

Existential Elimination

x1

x2

x3 x4

x5

PCs1 < s2 & s4 > s3 & s4 < s1 & s4 < s5 &s7 < s2 & s7 > s1

s1

s4 s2

s3 s5

+

s1,s2,s3,s4,s5 such that x1 = s1 & x2 = s4 & x3 = s3 & x4 = s5 & x5 = s2 & PC

x1 > x2 & x2 > x3 & x2 < x4 & x5 > x1

71

Results from ISSTA 2006 Paper

✦ We compared the following techniques (1st 3 are exhaustive; last 2 lossy)• Traditional Model Checking• Model Checking using Symmetry Reductions• Symbolic Execution• Model Checking using Abstract Matching on the shape of the containers• Random Testing

✦ Using the following container classes• Binary Tree• Fibonacci Heap• Binomial Heap• Tree Map

✦ In how well they achieve• Statement coverage• Predicate coverage

✦ As we increase the length of the sequence of API calls• For the concrete cases we also increase the number of parameters to be the same as the API calls

✦ We found the following general result• Traditional Model Checking failed on all but the most trivial tasks (i.e. statement coverage)• Model Checking with Symmetry reductions worked much better here (scaled to much larger sequence

lengths)• Symbolic Execution worked the best of the Exhaustive techniques (got optimal coverage in a number

of cases) – in one case one needed a sequence length of 14 calls to obtain branch coverage and symbolic execution could do it

• Lossy techniques could obtain optimal Predicate coverage• Shape abstractions worked better than random selection but only just

See examples/issta2006 folderunder the SVN repository to reproduce the experiments

72

Symbolic Execution Demo

✦ Start by Running Saswat Anand’s symbolic instrumenter on the SwapValues.java File

✦ Then run it through JPF using symbolic execution• We will use CVC-Lite as a decision since it is the only decision procedure that currently

compiles under Windows!

JPF Symbolic Execution - BEFORE

Omega Interface

Formula satisfiable/unsatisfiable

OmegaJava Version

JPF

JPF Symbolic Execution - NOW

Generic Decision Procedure Interface

Formula satisfiable/unsatisfiable

OmegaMaryland

JPF

CVCLiteStanford

YicesSRI

STPStanford

Collaborator

Saswat Anand

75

Communication Methods

✦ Issue • JPF and the Interface code is in Java• Decision procedures are not in Java, mainly C/C++ code

✦ Various different ways of communication ✦ Native

• Using JNI to call the code directly

✦ Pipe• Start a process and pipe the formulas and results back and forth

✦ Files• Same as Pipe but now use files as communication method

76

Optimization using Tables

JPF State

Path Condition:X > Y & Z > X & …

JPF State

Path Condition:pc100

Outside JPF-Table of PCs –

… X > YZ > X

X >Y & Z > X & …

4546

100

…

…

77

Optimization – Run DPs incrementally

✦ Some decision procedures support running in a incremental mode where you do not have to send the whole formula at a time but just what was added and/or removed.•CVCLite•Yices

78

Decision Procedure Options

✦ +symbolic.dp=• omega.file• omega.pipe• omega.native• omega.native.inc

‣ …inc - with table optimization• yices.native• yices.native.inc• yices.native.incsolve

‣ …incsolve - Table optimization and incremental solving• cvcl.file• cvcl.pipe• cvcl.native• cvcl.native.inc• cvcl.native.incsolve• stp.native

✦ If using File or Pipe one must also set• Symbolic.<name>.exe to the executable binary for the DP

✦ For the rest one must set LD_LIBRARY_PATH to where the DP libraries are stored• Extensions/symbolic/CSRC

✦ Currently everything works under Linux and only CVCLite under Windows• Symbolic.cvclite.exe = cvclite.exe must be set with CVClite.exe in the Path

79

Results TCAS

0

5

10

15

20

25

30

35

TCAS (2694 quesries)

omega.pipe

omega.file

cvcl.pipe

cvcl.file

omega.native

omega.native.inc

cvcl.native

cvcl.native.inc

cvcl.native.incsolve

yices.native

yices.native.inc

yices.native.incsolve

stp.native

80

Results TreeMap

0

100

200

300

400

500

600

700

800

TreeMap size 6 (83592 queries)

omega.pipe

omega.file

cvcl.pipe

cvcl.file

omega.native

omega.native.inc

cvcl.native

cvcl.native.inc

cvcl.native.incsolve

yices.native

yices.native.inc

yices.native.incsolve

STP took > 1 hour

81

State Matching in JPF

VM State

Matchable+

Restorable

Stored State(hashed)

Matchable

compression

Abstract Stateerase some parts

of the statecould be lossy

Matchable

Collaborator

Peter Dillinger

82

Old Architecture

VM/Search

• Integrated collapse compression of raw VM state • integrated backtracker

StateSet

int[]

bool

83

New Architecture

VM/SearchRestorer

Serializer

DefaultBacktracker

Serialized StateSet

int[] set

VM

VM

bool

int[]

84

Old Scheme in the New Architecture

VM/Search

DefaultBacktracker

FullStateSet

int[] set

VM

bool

int[]

Collapsing(de)Serializer

int[]

85

New Architecture

VM/SearchCollapsingRestorer

FilteringSerializer

DefaultBacktracker

JenkinsStateSet

int[] set

VM

VM

bool

int[]

objects

This is the default setting for JPF at the moment:vm.backtracker.class = gov.nasa.jpf.jvm.DefaultBacktrackervm.restorer.class = gov.nasa.jpf.jvm.CollapsingRestorervm.serializer.class = gov.nasa.jpf.filter.FilteringSerializervm.storage.class = gov.nasa.jpf.jvm.JenkinsStateSet

86

FilteringSerializer

✦ By default captures the fundamental VM state• i.e. with all known unnecessary information removed

✦ Can also ignore fields based on annotations• @FilterField(condition="foo") static int mods = 0;• Filters the field if the condition is true, the condition statement is optional

✦ Does garbage collection after erasing/filtering fields

✦ Does heap canonicalization to give (heap) symmetry reductions

✦ By default it eliminates fields like “modCount” from the java.util classes• Otherwise even the most trivial example will have an infinite state space• See the ModCount.java demo

✦ Can easily specify shape abstractions by filtering out all non-reference fields from an object• This will allow automatically doing the ISSTA 2006 shape abstraction

87

New Architecture Revisited - Abstraction

VM/SearchCollapsingRestorer

AbstractingSerializer

DefaultBacktracker

JenkinsStateSet

int[] set

VM

VM

bool

int[]

objects

This is the setting for the above configuration:vm.backtracker.class = gov.nasa.jpf.jvm.DefaultBacktrackervm.restorer.class = gov.nasa.jpf.jvm.CollapsingRestorervm.serializer.class = gov.nasa.jpf.abstraction.abstractingSerializervm.storage.class = gov.nasa.jpf.jvm.JenkinsStateSet

88

AbstractingSerializer

✦ First does FilteringSerializer

✦ Also gives the ability to specify abstractions that are not just deleting part of the state (what FilteringSerializer does)• The parts of the VM state is converted to an abstract state graph node

‣ Graph based abstractions can then be performed• Other examples include

‣ Rounding floating point to the nearest 0.001‣ Include only difference of two variables‣ Ignore ordering of values

✦ Adds support for sets modulo permutations, which is currently only used for specifying the set of threads• i.e. this now gives thread symmetry reductions

89

Examples

✦ K9 revisited• Filtering visits many less states• Abstraction doesn’t give anything here since there is no thread symmetry to exploit

✦ ModCount example• Infinite state without filtering

✦ Dining Philosophers• Regular setup cannot go beyond 5 philosophers• Abstraction is better than Filtering here, but takes longer

‣ There is a lot of thread symmetries here to exploit

✦ Filtering Fields• Show the effect of removing fields through annotations

90

User-Interface Model Checking

✦ Extension to JPF to deal with UIs directly• See extensions/ui directory• Deals with swing and awt

✦ New JPF front-end for querying a UI to find its structure

✦ Allows the scripting of nondeterministic executions using the structure of the UI• Very simple language right now but will be extended in the future

✦ This is work in progress, but it has been able to find a serious problem in a prototype launch sequencer to be used in the launch vehicle for the ARES project (going back to the Moon and on to Mars)

✦ TestMe demo• Run TestMe by itself and see if you can find a bug!• Then run UIInspector on it:

‣ JPF finds the bug‣ Replays it on the real UI for TestMe

Collaborator

Peter Mehlitz

91

Something Completely Different

✦ Well, maybe not completely…

✦ We developed a new static analysis tool for Java over the summer• Aaron Tomb (USC) did all the hard work

✦ Problem we were addressing• Static analyzers for finding defects are now popular tools, even commercial successes

‣ Coverity and KlockWork to name two• But once they find a possible how do you know it is a real bug

✦ When we find a possible bug we want an input that will lead us to the bug

✦ Existing tools didn’t give us a hook into the internals to allow enough information to create such an input

✦ We developed a new tool based on symbolic execution to do it• Hooked it into SOOT• Used CVCLite as a decision procedure• Once a constraint on the inputs is found that leads to a possible bug, we solve the constraints with POOC and run

the code with those inputs

✦ Highly customizable • Allows intra- or inter-procedural analysis• Heuristics for finding array bounds exceptions• Etc.

✦ Again work in progress, but has found real NASA bugs!

Collaborator

Aaron Tomb