A Hybrid SAT-based Decision Procedure for Separation Logic with Uninterpreted Functions

A Hybrid SAT-based Decision Procedure for Separation Logic with Uninterpreted Functions

Sanjit A. SeshiaSanjit A. Seshia

Joint work withJoint work with

Shuvendu K. Lahiri & Randal E. BryantShuvendu K. Lahiri & Randal E. Bryant

Carnegie Mellon University, USACarnegie Mellon University, USA

June 2003June 2003

– 2 –

Decision Procedures in Formal VerificationDecision Procedures in Formal Verification

RTL/ Sourc

e Code

+Specif

i-cation

Abstraction Verification OK

Error

Formal

Model+

Specifi-

cation

Decision Procedure for Decidable Fragment of First-Order Logic

Decision Procedure for Decidable Fragment of First-Order Logic

Satisfiable/Unsatisfiable

Decision Procedure for Decidable Fragment of First-Order Logic

Formula

Applications: Out-of-order, Pipelined Microprocessors; Cache Coherence Protocols; Device Drivers; Compiler

Validation; …

– 3 –

x0x1

x2

xn-1

Data and Function AbstractionData and Function Abstraction

ALU

x

f

Bit-vectors to (unbounded) Integers

Functional units to Uninterpreted Functions a = x Æ b = y ) f(a,b) = f(x,y)

Common Operations

1

0

x

y

p

ITE(p, x, y)

If-then-else

x

y x = y=

Test for equality

x

y x < y<

Test for orderingx

1 x +1+

Counters

– 4 –

Sufficiently expressive for afore-mentioned Sufficiently expressive for afore-mentioned applications applications

System property expressed as SUF formula System property expressed as SUF formula FF– Efficiently decided via translation to SATEfficiently decided via translation to SAT

Terms (Terms (T T )) Integer Expressions

ITEITE((FF, , TT11, , TT22)) If-then-elseIf-then-else

Fun Fun ((TT11, …, , …, TTkk)) Function applicationFunction applicationT T + 1+ 1 IncrementIncrementT T - 1- 1 DecrementDecrement

Formulas (Formulas (F F )) Boolean Expressions

FF, , FF11 FF22, , FF11 FF22 Boolean connectivesBoolean connectives

TT11 = = TT22 EquationEquation

TT11 < < TT22 InequalityInequality

PredPred((TT11, …, , …, TTkk)) Predicate applicationPredicate application

Separation Logic with Uninterpreted Functions (SUF)Separation Logic with Uninterpreted Functions (SUF)

– 5 –

SAT-based Decision ProceduresSAT-based Decision Procedures

Input Formula

Boolean Formula

satisfiable unsatisfiable

Satisfiability-preserving Boolean

Encoder

SAT Solver

EAGER ENCODING

Input Formula

Boolean Formula

satisfiable

unsatisfiable

Approximate Boolean Encoder

SAT Solver satisfying assignment

satisfiable

First-order Conjunctions SAT Checker

unsatisfiable

additional clause

LAZY ENCODING

– 6 –

Talk Outline Talk Outline

SUF SUF Separation Logic Separation Logic SAT SAT– Two eager encoding techniquesTwo eager encoding techniques– Pros and cons of each techniquePros and cons of each technique

Combining eager encoding techniquesCombining eager encoding techniques– The The HybridHybrid eager encoding technique eager encoding technique

Experimental resultsExperimental results– Superior performance to lazy encoding methods Superior performance to lazy encoding methods

and non-SAT-based decision proceduresand non-SAT-based decision procedures ConclusionsConclusions

– 7 –

Eliminate function and predicate applications using Eliminate function and predicate applications using fresh variables and ITE expressions fresh variables and ITE expressions [Bryant, German, [Bryant, German, Velev, CAV’99]Velev, CAV’99]

– f(x)f(x) vv11 andand f(y) f(y) ITE(x = y, vITE(x = y, v11, v, v22))

SUF Separation LogicSUF Separation Logic

Separation Predicate

Terms (Terms (T T )) Integer Expressions

ITEITE((FF, , TT11, , TT22)) If-then-elseIf-then-else

Fun Fun ((TT11, …, , …, TTkk)) Function applicationFunction applicationTT + 1 + 1 IncrementIncrementT T - 1- 1 DecrementDecrement

v Integer variable

Formulas (Formulas (F F )) Boolean Expressions

FF, , FF11 FF22, , FF11 FF22 Boolean connectivesBoolean connectives

TT11 = = TT22 EquationEquation

TT11 < < TT22 InequalityInequality

PredPred((TT11, …, , …, TTkk)) Predicate applicationPredicate applicationbBoolean variable

– 8 –

Separation Logic Formula

Per-Constraint Encoding (EIJ)

Small Domain Encoding (SD)

Eager Boolean Encoding Methods for Separation LogicEager Boolean Encoding Methods for Separation Logic

Boolean Formula

SAT Solver

satisfiable/unsatisfiable

– 9 –

Small Domain Encoding (SD)Small Domain Encoding (SD)

x ¸ y Æ y ¸ z Æ z ¸ x+1

Can use Boolean encoding of finite range of valuesCan use Boolean encoding of finite range of values– 4 values in this case, so 2-bit encoding4 values in this case, so 2-bit encoding

Observation: Observation: To check satisfiability, need to consider all possible To check satisfiability, need to consider all possible relativerelative orderings of orderings of finitely-manyfinitely-many expressions expressions

h0x1x0i ¸ h0y1y0i Æ h0y1y0i ¸ h0z1z0i Æ h0z1z0i ¸ h0x1x0i + 1

x x+1y

z

x x+1 y z

Values increase

[Bryant, Lahiri, Seshia, CAV’02]

– 10 –

Per-Constraint Encoding (EIJ) Per-Constraint Encoding (EIJ)

x ¸ y Æ y ¸ z Æ z ¸ x+1

e1 Æ e2 ) e4

e4 x ¸ z

New Separation Predicate

e4 ) : e3 Æ

ÆOverall Boolean

Encoding

Transitivity Constraints

e1

y ¸ z

z ¸ x+1

x ¸ y

e2

e3

e1 Æ e2 Æ e3

[Strichman, Seshia, Bryant, CAV’02]

– 11 –

Comparing Eager Encoding MethodsComparing Eager Encoding Methods

Of SD and EIJ encoding methods, which one is Of SD and EIJ encoding methods, which one is better?better?

Comparison with respect toComparison with respect to– Size of resulting Boolean formulaSize of resulting Boolean formula– Performance of SAT solverPerformance of SAT solver

– 12 –

Size of Boolean Encoding: SD better than EIJSize of Boolean Encoding: SD better than EIJ Let Let NN be size of original separation logic formula be size of original separation logic formula

– Size of a directed acyclic graph representation Size of a directed acyclic graph representation SDSD encoding size is worst-case encoding size is worst-case OO((NN22)) EIJEIJ encoding size is worst-case encoding size is worst-case OO((22NN))

– Can generate Can generate OO((22NN) transitivity constraints ) transitivity constraints

> 1000000> 1000000EIJEIJ

5446554465SDSD

Boolean Encoding SizeBoolean Encoding SizeMethodMethodExample: Example: N = N = 68136813

– 13 –

Impact on SAT problem: SD vs EIJ Impact on SAT problem: SD vs EIJ

Experimentally compared zChaff performance on SD and EIJ encodings of several unsatisfiable formulas

Sample result:

EIJ better than SD for zChaff

MethodMethod # Boolean # Boolean variablesvariables

# CNF # CNF Clauses Clauses

# Conflict # Conflict ClausesClauses

zChaff zChaff Time Time (sec)(sec)

EIJEIJ 5721157211 169387169387 150150 0.560.56

SDSD 2311223112 6769967699 1581115811 21.6321.63

– 14 –

Impact on SAT: Why is EIJ better than SD? Impact on SAT: Why is EIJ better than SD? Conjecture: For SD, SAT solver has to Conjecture: For SD, SAT solver has to

“discover” transitivity constraints as conflict “discover” transitivity constraints as conflict clausesclauses– Violation of transitivity constraint might be discovered only Violation of transitivity constraint might be discovered only

after assigning bits of several bit-vectorsafter assigning bits of several bit-vectors

EIJ adds all such constraints a prioriEIJ adds all such constraints a priori– Less learning and backtracking required by the SAT solver Less learning and backtracking required by the SAT solver

– 15 –

Eager Encoding TradeoffsEager Encoding Tradeoffs

SD encodingSD encoding+ Polynomial size encodingPolynomial size encoding– Worse for SAT solversWorse for SAT solvers

EIJ encodingEIJ encoding– Worst-case exponential size encodingWorst-case exponential size encoding+ Better for SAT solvers Better for SAT solvers

Can we automatically select between SD and EIJ Can we automatically select between SD and EIJ based on the input formula?based on the input formula?

– 16 –

Selection StrategySelection Strategy

Problem:Problem:– Computationally hard Computationally hard

to estimate number of to estimate number of transitivity constraintstransitivity constraints

Can we use a different Can we use a different metric?metric?– IdeaIdea: Identify feature of : Identify feature of

the input formula that the input formula that varies monotonically varies monotonically with run-time of EIJ with run-time of EIJ (but not with run-time (but not with run-time of SD) of SD)

Estimate number of transitivity constraints,

C

C > T ?YES NO

Use SD encoding

Use EIJ encoding

– 17 –

A Good Formula Feature: Number of Separation PredicatesA Good Formula Feature: Number of Separation Predicates

– 18 –

A Good Formula Feature: Number of Separation PredicatesA Good Formula Feature: Number of Separation Predicates

– 19 –

Revised Selection StrategyRevised Selection Strategy

+ Easy to count number Easy to count number of separation predicatesof separation predicates

– Very approximate Very approximate measure of # of measure of # of transitivity constraintstransitivity constraints– Constraints only relate Constraints only relate

predicates that share predicates that share variablesvariables

Also need to automate Also need to automate setting of threshold Tsetting of threshold T– Statistically estimate Statistically estimate

from “training” set of from “training” set of benchmarksbenchmarks

Count number of separation predicates,

m

m > T ?YES NO

Use SD encoding

Use EIJ encoding

– 20 –

Identifying Variable ClassesIdentifying Variable Classes

x ¸ y y ¸ z

z ¸ x+1u ¸ v

u = v-2

{x,y,z} shared {u,v} shared

Assignments to {u,v} are independent of those to {x,y,z}

Ç

Æ

Æ

Ç

– 21 –

Hybrid Encoding TechniqueHybrid Encoding Technique

Compute 1. Variable classes based on predicates 2. Number of separation predicates for each class

{x,y,z}, m1

{u,v}, mk

Encode each class using SD or EIJ based on local decision

Encoded Boolean Formula

Separation Logic Formula

m1 > T ? mk > T ?YESNO YESNO

SD EIJEIJ SD

– 22 –

Automatically Selecting a Threshold Value: IntuitionAutomatically Selecting a Threshold Value: Intuition

EIJ run time increases drastically beyond a certain number of separation predicates

– 23 –

Automatically Selecting a Threshold Value using ClusteringAutomatically Selecting a Threshold Value using Clustering

Cluster total time (Y-axis) values, minimizing variance of each cluster

– 24 –

Experimental Evaluation SetupExperimental Evaluation Setup

Compared Hybrid against Compared Hybrid against – SD and EIJ encodingsSD and EIJ encodings– Cooperating Validity Checker (CVC) based on lazy encoding Cooperating Validity Checker (CVC) based on lazy encoding

method [Stump et al.’02]method [Stump et al.’02]– Stanford Validity Checker (SVC) – non SAT-based [Barrett et Stanford Validity Checker (SVC) – non SAT-based [Barrett et

al. ’96]al. ’96]– CVC & SVC can handle more expressive logics than SUFCVC & SVC can handle more expressive logics than SUF

BenchmarksBenchmarks– 49 unsatisfiable SUF formulas49 unsatisfiable SUF formulas– Load-store unit, out-of-order unit, device driver code, Load-store unit, out-of-order unit, device driver code,

compiler validation, DLX pipelinecompiler validation, DLX pipeline– Threshold value calculated from subset of 16 benchmarksThreshold value calculated from subset of 16 benchmarks

Worked well for 39 out of the 49 benchmarksWorked well for 39 out of the 49 benchmarks

SetupSetup– Used zChaff SAT solverUsed zChaff SAT solver– Imposed timeout of 1800 sec. on total time (Encoding+SAT)Imposed timeout of 1800 sec. on total time (Encoding+SAT)

– 25 –

Hybrid vs. SD (39/49 benchmarks)Hybrid vs. SD (39/49 benchmarks)

Hybrid better

SD better

– 26 –

Hybrid vs. EIJ (39/49 benchmarks)Hybrid vs. EIJ (39/49 benchmarks)

Hybrid better

EIJ better

– 27 –

Hybrid vs. Lazy Encoding (CVC) (39/49 benchmarks)Hybrid vs. Lazy Encoding (CVC) (39/49 benchmarks)

CVC better

Hybrid better

– 28 –

Hybrid vs. Non-SAT-based Procedure (SVC) (39/49 benchmarks)Hybrid vs. Non-SAT-based Procedure (SVC) (39/49 benchmarks)

SVC better

Hybrid better

– 29 –

SD outperforms Hybrid on 10/49 benchmarksSD outperforms Hybrid on 10/49 benchmarks

SD better

Hybrid better

– 30 –

Conclusions & Ongoing WorkConclusions & Ongoing Work

Hybrid combination of EIJ and SD encodings Hybrid combination of EIJ and SD encodings – is robust to formula variationsis robust to formula variations– outperforms lazy encoding methods (CVC)outperforms lazy encoding methods (CVC)– outperforms non-SAT-based methods (SVC)outperforms non-SAT-based methods (SVC)

Ongoing & Future workOngoing & Future work– Alternate estimators for number of transitivity Alternate estimators for number of transitivity

constraintsconstraints– Threshold setting technique based on clustering Threshold setting technique based on clustering

applies to other CAD problems tooapplies to other CAD problems too– Combination of lazy and eager encoding Combination of lazy and eager encoding

techniques might perform well on satisfiable techniques might perform well on satisfiable formulas?formulas?

More on UCLID project webpage More on UCLID project webpage http://www.cs.cmu.edu/~uclidhttp://www.cs.cmu.edu/~uclid