1

Regression Verification: Proving the equivalence of similar programs

Benny Godlin Ofer Strichman

Technion, Haifa, Israel

Recently joined: Yossi Levhari

Looking for an additional PhD student for this project.

2

Functional Verification

The main pillar of the grand challenge [H’03].

Suppose we ignore completeness.

Still, there are two major problems: Specification Complexity

3

Words of wisdom

“For every problem that you cannot solve, there is an easier problem that you cannot solve” *

* George Polya, in How To Solve It

4

A more modest challenge: Regression Verification

Develop a method for formally verifying the equivalence of two similar programs.

Pros: Default specification = earlier version. Computationally easier than functional verification.

Ideally, the complexity should depend on the semantic difference between the programs, and not on their size.

Cons: Defines a weaker notion of correctness.

5

Regression-based technologies

In Testing: Regression testing is the most popular automatic testing method

In hardware verification: Equivalence checking is the most used verification method in the industry

6

Hoare’s

1969

paper

has it all.

. . .

…and 6 pages later:

7

Previous work

In the theorem-proving world (mostly @ ACL2 community): Not industrial programming languages Not utilizing the similarity between the two programs

Industrial / realistic programs: Code free of: loops, recursion, dynamic-memory allocation

microcode @ Intel [AEFMMSSTVZ-05], embedded code @ Feng & Hu [FH-05], symbolic simulation @ Matsumoto et al. [TSF-06]

8

Goals

Develop notions of equivalence

Develop corresponding proof rules

Present a prototype for verifying equivalence of C programs, that incorporates the proof rules sensitive to the magnitude of change rather than the magnitude of

the programs

9

Notions of equivalence (1 / 6)

Partial equivalence

Executions of P1 and P2 on equal inputs …which terminate, result in equal outputs.

Undecidable

10

Notions of equivalence (2 / 6)

Mutual termination

Given equal inputs, P1 terminates , P2 terminates

Undecidable

11

Notions of equivalence (3 / 6)

Reactive equivalence

Let P1 and P2 be reactive programs. Executions of P1 and P2

…which read the same input sequence, emit the same output sequence.

Undecidable

12

Notions of equivalence (4 / 6)

k-equivalence

Executions of P1 and P2 on equal inputs …where loops and recursions are restricted to k iterations, result in equal output.

Decidable

13

Notions of equivalence (5 / 6)

Full equivalence

P1 and P2 are partially equivalent and mutually terminate

Undecidable

14

Notions of equivalence (6 / 6)

Total equivalence

P1 and P2 are partially equivalent and both terminate

Undecidable

15

Notions of equivalence: summary

1. Partial equivalence

2. Mutual termination

3. Reactive equivalence

4. k-equivalence

5. Full equivalence* = Partial equivalence + mutual termination

6. Total equivalence** = partial equivalence + both terminate

* Appeared in Luckham, Park, and M. Paterson [LPP-70] / Pratt [P-71]

** Appeared in Bouge and D. Cachera [BC-97]

16

Partial equivalence

Consider the call graphs:

… where A,B have: same prototype no loops

Prove partial equivalence of A, B How shall we handle the recursion ?

A B

Side 1 Side 2

17

Hoare’s Rule for Recursion

Let A be a recursive function.

“… The solution... is simple and dramatic: to permit the use of the desired conclusion as a hypothesis in the proof of the body itself.” [H’71]

18

Hoare’s Rule for Recursion

// {p}

A( . . . )

{

. . .

// {p}

call A(. . .);

// {q}

. . .

}

// {q}

19

//in[A]

A( . . . )

{

. . .

//in[call A]

call A(. . .);

//out[call A]

. . .

}

//out[A]

Rule 1: Proving partial equivalence

A B//in[B]

B( . . . )

{

. . .

// in[call B]

call B(. . .);

//out[call B]

. . .

}

//out[B]

20

Rule 1: Proving partial equivalence

Q: How can a verification condition for the premise look like? A: Replace the recursive calls with calls to functions that

over-approximate A, B, and are partially equivalent by construction

Natural candidates: Uninterpreted Functions

21

Proving partial equivalence

Let AUF , BUF be A,B, after replacing the recursive call with a call to (the same) uninterpreted function.

We can now rewrite the rule:

The premise is decidable

22

unsigned gcd1UF

(unsigned a, unsigned b)

{ unsigned g;

if (b == 0)

g = a;

else {

a = a % b;

g = gcd1(b, a);

}

return g;

}

unsigned gcd2UF

(unsigned x, unsigned y)

{ unsigned z;

z = x;

if (y > 0)

z = gcd2(y, z % y);

}

return z;

}

Using (PART-EQ-1): example

?=

U

U

a, b) x, y)

g;

z;

Transition functions

Inputs

Outputs

Tgcd1 Tgcd2

a,b x,y

g z

23

Rule 1: example

Transition functions Tgcd1 Tgcd2

Inputs a,b x,y

Outputs g z

Equalinputs

Equaloutputs

24

Partial equivalence: Generalization

Assume: no loops; 1-1 mapping map between the recursive functions of both sides

Mapped functions have the same prototype

Define: For a function f, UF(f) is an uninterpreted function such that

f and UF(f) have the same prototype

(f,g) 2 map , UF(f) = UF(g).

25

Partial equivalence: Generalization

Definition: is called in A]

26

Partial equivalence: Example (1 / 3)

Side 1 Side 2

f ’g g’f

{(g,g’),(f,f’)} 2 map

Need to prove:

f ’UFf UF

UFg g’UF

=

=

27

Partial equivalence: Example (2 / 3)

An improvement:

Find a map that intersects all cycles, e.g., (g,g’) Only when calling functions in this map replace with

uninterpreted functions

Side 1 Side 2

f ’

UF

g g’f

UF

28

Partial equivalence: Example (3 / 3)

Connected SCCs… Prove bottom-up Abstract partially-equivalent functions Inline

Side 1 Side 2

f ’g g’f

h h’UF UF

29

RVT: Decomposition algorithm

A: B:

f1()

f2()f5()

f3() f4() f6()

f1’()

f2’()

f3’() f4’()

f5’()

Equivalent pair

Syntactically equivalent pair

Equivalence undecided yetCould not prove equivalent

Legend:

check

check Unpaired function

f7’()U UU U

U U

30

RVT: Decomposition algorithm (with SCCs)

A: B:

f1()

f2() f5()

f3() f4() f6()

f1’()

f3’() f4’()

f5’()

f6’()

Equivalent pair

Syntactically equivalent pair

Equivalence undecided yetCould not prove equivalent

Legend:

Equivalent if MSCC

U UU U

U U

check

U UU U f2’()

31

PART-EQ: Soundness

Proved soundness for a simple programming language (LPL) Covers most features of modern imperative languages …but does not allow

call by reference, and address manipulation.

32

PART-EQ: Completeness

We show a sufficient condition for completeness.

33

PART-EQ: Completeness

Definition: A,B are reach-equivalent if… (by example)

A() {

…

if (cond1)

f(a1,b1)

else …

…

}

B() {

…

if (cond2)

g(a2,b2)

else …

..

}

34

PART-EQ: Completeness

Definition: A,B are reach-equivalent if… For equal inputs:

For callees F,G s.t. (F,G) 2 map: F is called , G is called F and G are called with the same arguments.

Completeness: (PART-EQ) is complete for reach-equivalent functions.*

* under some “reasonable” assumptions (e.g. no dead-code)

35

What (PART-EQ) cannot prove...

Not reach-equivalent: calling with different arguments.

returns n + nondet() returns n + n -1 + nondet()

36

What (PART-EQ) cannot prove...

Not reach-equivalent: calling under different conditions:

returns 1 returns 1 + nondet()

when n == 1 :

(news)

41

The Regression Verification Tool (RVT) Given two C programs:

loops recursive functions.

Map functions, globals, etc.

After that: Decompose to the granularity of pairs of functions Use a C verification engine (CBMC) to discharge

42

The Regression Verification Tool (RVT)

CBMC: a C bounded model checker by Daniel Kroening

Our use: No loops or recursion to unroll... Use “assume(…)” construct to enforce equal inputs. Use assert() to demand equal outputs.

Uninterpreted functions are implemented as C functions: Return consistent nondeterminisitic values.

43

The Regression Verification Tool (RVT) The premise of (PART-EQ) requires comparing arguments.

What if these arguments are pointers ?

What our system does: Dynamic structures: creates an unrolled nondeterministic

structure Arrays: attempts to find references to cells in the array.

44

RVT: User-defined equivalence specification The user can define pairs of ‘checkpoints’:

side 1: <label1, cond1, exp1>side 2: <label2, cond2, exp2>

In each side: update an array with the value of exp each time it reaches label and

condition holds.

Assert that when executed on the same input…, … these arrays are equivalent.

exp1 exp2 . . .P1:

exp’1 exp’2 . . .P2:

= = = =

45

RVT

Version A Version B

CBMC

rename identical globals enforce equality of inputs. assert equality of outputs add checkpoints Supports:

Decomposition Abstraction some static analysis …

feedback

result counterexample

C program

RVT

46

47

RVT: Experiments

Automatically generated sizable programs with complex recursive structures and loops.

up-to thousands of lines of code

Limited-size industrial programs: Parts of TCAS - Traffic Alert and Collision Avoidance

System. Core of MicroC/OS - real-time kernel for embedded

systems. Matlab examples: parts of engine-fuel-injection simulation.

We tested the Regression Verification Tool (RVT) with:

48

Testing RVT on programs: Conclusions For equivalent programs, partial-equivalence checks were very

fast: proving equivalence in minutes.

For non-equivalent programs: RVT attempts to prove partial-equivalence but fails

then RVT tries to prove k-equivalence

49

Summary

Regression verification is an important problem

A solution to this problem has a better chance to succeed in the industry than functional verification

A grand challenge by its own right…

Lots of future research...

50

More Challenges

Q1: How can we generalize counterexamples ?

Q2: What is the ideal gap between two versions of the same program, that makes Regression Verification most effective ?

Q3: How can functional verification and equivalence verification benefit from each other ?

51

The end…

Thank you

64

Long list of future work

Future directions: Strengthen the rules with invariants. Support larger parts of C

65

Slides to add(?)

show the direct reduction. this is called ‘self composition’. see c:\temp\equivalence_fmics09_submission.pdf

discuss check points discuss how the verified programs look like

66

Outline

Inference rules for proving equivalence Hoare’s rule of recursion Rule 1: partial equivalence Rule 2: mutual termination Rule 3: reactive equivalence

Regression verification for C programs CBMC – the back engine Architecture Decomposition Handling dynamic data structures

67

Advantages of Regression Verification

Specification by reference to an earlier version.

Can be applied from early development stages.

Equivalence proofs are potentially easier than functional verification.

Ideally, the complexity should depend on the semantic difference between the programs, and not on their size.

68

C normal form...

Every two C programs can be brought to the form:

... but this will hardly ever work contradict our goal to decompose the problem

mainmain

69

Checkpoints with channels

To allow more flexibility in comparison, allow channels

exp1 exp2 . . .P1,C2:

exp’1 exp’2 . . .P2,C2:

= = = =

exp1 exp2 . . .P1,C1:

exp’1 exp’2 . . .P2,C1:

= = = =