Avoiding Exponential Explosion: Generating Compact Verification Conditions Cormac Flanagan and James...

Avoiding Exponential Explosion:

Generating Compact Verification Conditions

Cormac Flanagan and James B. Saxe

Compaq Systems Research Center

With help from our ESC colleagues:

Rustan Leino, Mark Lillibridge, Greg Nelson, Shaz Qadeer, Raymie Stata

Software QA via Testing

Useful (the dominant methodology), but .. Costly

half of development cost is testing finds errors late in development cycle

Incomplete often fails to ensure needed reliability hard to test all configurations

Software QA via Static Checking

Statically verify many correctness properties

Type systems catch many errors e.g. “Cannot multiply a number and a string”

Would like to catch additional errors e.g. “Array index out of bounds at line 10”

And verify other correctness properties assertions object invariants lightweight method specifications

Extended Static Checker ArchitectureJava method + annotations Java method + annotations

CounterexamplesCounterexamples

x.x.y.(x > y ==> … )y.(x > y ==> … )Verification ConditionVerification Condition

VC GeneratorVC Generator

Decision ProcedureDecision Procedure

Index out of bounds on line 218Index out of bounds on line 218Method does not preserve object invariant on line 223Method does not preserve object invariant on line 223

Extended Static Checker ArchitectureJava method + annotations Java method + annotations

Guarded Command Guarded Command

CounterexamplesCounterexamples

Front EndFront End

Verification ConditionVerification Condition

VC GeneratorVC Generator

Decision ProcedureDecision Procedure

Index out of bounds on line 218Index out of bounds on line 218Method does not preserve object invariant on line 223Method does not preserve object invariant on line 223

Intermediate representationIntermediate representation assume preconditionsassume preconditions assume object invariantsassume object invariants ... translated body ...... translated body ... assert postconditionsassert postconditions assert object invariantsassert object invariants

private int scanPunctuation(int nextchr) { try { boolean possibleFloatingPointNumber = (nextchr == '.'); text[0] = (char)nextchr; textlen = 1; m_in.mark(); // All paths out of the try must unmark the stream!! PunctuationPrefixTree prefix = punctuationTable; PunctuationPrefixTree lastPunctuation = prefix; int lastPunctuationLength = 0; int index = nextchr - '!'; if (index < 0 || PunctuationPrefixTree.CHILDLEN <= index) prefix = null; else prefix = prefix.children[nextchr - '!']; nextchr = m_in.read(); if (possibleFloatingPointNumber && Character.isDigit((char)nextchr)) { m_in.clearMark(); return finishFloatingPointLiteral(nextchr); } this.append(nextchr); if (prefix != null && prefix.code != TagConstants.NULL) { lastPunctuation = prefix; lastPunctuationLength = textlen - 1; m_in.mark(); } while(prefix != null) { index = nextchr - '!'; if (index < 0 || PunctuationPrefixTree.CHILDLEN <= index) prefix = null; else prefix = prefix.children[nextchr - '!']; nextchr = m_in.read(); this.append(nextchr); if (prefix != null && prefix.code != TagConstants.NULL) { lastPunctuation = prefix;

lastPunctuationLength = textlen - 1; m_in.mark();

} } m_in.reset(); textlen = lastPunctuationLength; endingLoc = m_in.getLocation(); ttype = lastPunctuation.code; if (ttype != TagConstants.C_COMMENT&& ttype != TagConstants.EOL_COMMENT) nextchr = m_in.read(); return ttype; } catch (IOException e) { m_in.clearMark(); ErrorSet.fatal(m_in.getLocation(), e.toString()); return TagConstants.NULL; // Dummy }

}

Extended Static Checker ArchitectureJava method + annotations Java method + annotations

Guarded Command Guarded Command

CounterexamplesCounterexamples

Exponential in size of GCExponential in size of GC

Front EndFront End

Verification ConditionVerification Condition

VC GeneratorVC Generator

Decision ProcedureDecision Procedure

Index out of bounds on line 218Index out of bounds on line 218Method does not preserve object invariant on line 223Method does not preserve object invariant on line 223

Weakest preconditionsWeakest preconditionsStrongest postconditionsStrongest postconditionsSymbolic forward executionSymbolic forward execution

Statement S

x := e

A ; B

assert e

assume e

A B�

while {I} e do S end

{exceptions}

Guarded Command Language

Variables have arbitrary values in program’s initial state

if e then A else B end

(assume e ; A)

(� assume e ; B)

Weakest Precondition Semantics

wp.S.Q

Q(x e)

wp.A.(wp.B.Q)

e Q

e Q

wp.A.Q wp.B.Q

Statement S

x := e

A ; B

assert e

assume e

A B�

Blow-up from assignment rule

wp.(x := e).Q = Q(x e)

Q(x e) may contain many copies of e

Sequential composition of assignment statements

may yield exponentially large VC, e.g.

wp.( b=a+a ; c=b+b ; ... ; z=y+y).(z>0)

Blow-up from Choice Statements

wp.(A B).Q = wp.A.Q � wp.B.Q

The postcondition Q of a choice statement occurs

twice in weakest precondition

Copies of Q modified due to assignment statements

in A and B

Sequential composition of choice statements may

yield exponentially large VC

Key Insight

Assignment statements are the culprit! They cause problems both by themselves

And through their interaction with choice statements

Let’s get rid of them!

VC GeneratorVC Generatorfor Passive Formfor Passive Form

Two-Stage VC Generation Alg.

Passive Form Passive Form

CompactCompactVerification ConditionVerification Condition

Guarded Command Guarded Command

PassifyPassify Translation TranslationRemove AssignmentsRemove Assignments

Basic Passify Translation

To passify an assignment statement

x := e Introduce a fresh variable, say x’ Replace assignment statement by

assume x’ = e Subsequently use x’ instead of x

Passify for Choice Statements To passify a choice statement

A B� Let A’ and B’ be passive forms of A and B Suppose x resides in xa after A’

and x resides in xb after B’ Introduce a fresh variable, say x’ Replace the choice statement by (A’; assume x’=xa) (B’; � assume x’=xb)

Subsequently use x’ instead of x

Introduce a fresh variable, say x’ Replace assignment statement byassume x’ = e

Subsequently use x’ instead of x

VC GeneratorVC Generatorfor Passive Formfor Passive Form

Two-Stage VC Gen. Results (I)

Passive Form Passive Form

CompactCompactVerification ConditionVerification Condition

Guarded Command Guarded Command

PassifyPassify Translation TranslationRemove AssignmentsRemove Assignments

At most quadraticAt most quadraticin size of GCin size of GC

Generating VCs for Passive Form

Execution of a passive statement Cannot affect the program state (!) Can only choose among the two possible outcomes

• Normal termination

• Going wrong

Semantics of a passive statement S can be completely captured by two outcome predicates N.S - initial states from which S may terminate normally W.S - initial states from which S may go wrong

Outcome Predicates Semantics

N.S

e

e

N.A N.B

N.A N.B

Statement S

assume e

assert e

A B�

A ; B

W.S

false

e

W.A W.B

W.A (N.A W.B)

Normal outcome

Wrong outcome

Size of Outcome Predicates

The size of N.S is linear in the size of S The size of W.S is quadratic in the size of S W.(A;B;C)

= W.A (N.A W.B) (N.A N.B W.C)

= let t = N.A in W.A (t W.B) (t N.B W.C)

VC GeneratorVC Generatorfor Passive Formfor Passive Form

Two-Stage VC Gen. Results (II)

Passive Form Passive Form

CompactCompactVerification ConditionVerification Condition

(W.P)

Guarded Command Guarded Command

PassifyPassify Translation TranslationRemove AssignmentsRemove Assignments

At most quadraticAt most quadraticin size of GCin size of GC

At most At most n^4 (without lets)n^4 (without lets) quadratic (with lets)quadratic (with lets)in size of GCin size of GC

Results in Practice

Benchmark: ESC/Java front-end, 20 KLOC Passify increases code size by ~30% on average For “simple” methods

VC size 60% - 70% of original proof time roughly the same

For “complex” methods VC size 0.1% - 10% of original proof time 2% - 50% of original

Can now verify all methods in benchmark

private int scanPunctuation(int nextchr) { try { boolean possibleFloatingPointNumber = (nextchr == '.'); text[0] = (char)nextchr; textlen = 1; m_in.mark(); // All paths out of the try must unmark the stream!! PunctuationPrefixTree prefix = punctuationTable; PunctuationPrefixTree lastPunctuation = prefix; int lastPunctuationLength = 0; int index = nextchr - '!'; if (index < 0 || PunctuationPrefixTree.CHILDLEN <= index) prefix = null; else prefix = prefix.children[nextchr - '!']; nextchr = m_in.read(); if (possibleFloatingPointNumber && Character.isDigit((char)nextchr)) { m_in.clearMark(); return finishFloatingPointLiteral(nextchr); } this.append(nextchr); if (prefix != null && prefix.code != TagConstants.NULL) { lastPunctuation = prefix; lastPunctuationLength = textlen - 1; m_in.mark(); } while(prefix != null) { index = nextchr - '!'; if (index < 0 || PunctuationPrefixTree.CHILDLEN <= index) prefix = null; else prefix = prefix.children[nextchr - '!']; nextchr = m_in.read(); this.append(nextchr); if (prefix != null && prefix.code != TagConstants.NULL) { lastPunctuation = prefix;

lastPunctuationLength = textlen - 1; m_in.mark();

} } m_in.reset(); textlen = lastPunctuationLength; endingLoc = m_in.getLocation(); ttype = lastPunctuation.code; if (ttype != TagConstants.C_COMMENT&& ttype != TagConstants.EOL_COMMENT) nextchr = m_in.read(); return ttype; } catch (IOException e) { m_in.clearMark(); ErrorSet.fatal(m_in.getLocation(), e.toString()); return TagConstants.NULL; // Dummy }

}

VC GenerationAlgorithm

VC size(nodes)

Prooftime

Weakest Preconditions

4.7 M >5 min

Two-Stage Translation

18 K 7 sec

Current Status of ESC

Scales well to complex methods

Ready for educational/research use? Yes! http://research.compaq.com/SRC/esc/

Ready for commercial use? Not really. annotation overhead significant

annotations increase program size by 10%

requires 1 programmer-hour to annotate 300 lines of code

Future Directions

Annotation Inference Houdini annotation inference system

• Infer annotations via whole-program analysis (up to 40 KLOC)• “Generate and test” strategy leverages ESC

Loop invariant inference via predicate abstraction• Infers universally-quantified loop invariants, e.g. (\forall int j;

spot == MAXDIRENTRY && 0 <= j && j < i

==> bdisk[addr].dirEntries[j].inum != UNUSED )

Full verification of systems software Frangipani distributed file system