Decision Procedures for Linear Arithmetic

1

Decision Procedures for Linear Arithmetic

Presented By Omer Katz01/04/14

Based on slides by Ofer Strichman

3

Agenda

DPLL-T (a very short reminder) What is Linear Arithmetic and why is it needed Decision procedures for Decision procedures for A few preprocessing improvement steps

Given enough time: Difference logic Delayed Theory Combination

Improvement on Nelson-Oppen Not related to Linear Arithmetic

4

Reminder: DPLL-T

Our goal is to solve SMT problems with formulas from a theory T

DPLL-T is the most common approach Based on the DPLL algorithm for SAT problems Combines a decision procedure for the theory T

5

Reminder: DPLL-T

6

Linear Arithmetic

Theory grammar:

Can be defined over Rational () or Integers ()

Example:

7

Why is it needed?

Given the following C code:

The following assembly can be generated:

8

Why is it needed?

A possible optimization: Read the value of a[j] only once

Need to verify that loop won’t change a[j] Can be encoded as Linear Arithmetic problem

If no solution is found, optimization is safe

9

Decision Procedures for

Gaussian’s elimination Fourier-Motzkin Simplex

10

Gaussian’s elimination

A simple method for solving a set of equalities Less suitable for inequalities

Given a linear system Ax = b

A x = b

11

Gaussian’s elimination

Manipulate A|b to an upper-triangular form

Then, solve backwards from the k’s row according to:

12

Gaussian elimination - example

And now… x3 = -1, x2 = 3, x1 = 1 problem solved.

13

Fourier-Motzkin Elimination

Earliest method for solving linear inequalities

Given linear non-strict inequalities:

Pick a variable and eliminate it Continue until all variables but one are eliminated

If problem included equalities, eliminate by assignment

14


nnmnm

n

b

bb

x

xx

aa

aaaaa

::

::

......:::::

....

2

1

2

1

1

2221

11211

bIA A system of conjoined linear inequalities

m constraints

n variables

15


1. When eliminating xn, partition the constraints according to the coefficient ain: ain > 0: upper bound ain < 0: lower bound

assume ai,n >0

16


Example:(1) x1 – x2 ≤ 0(2) x1 – x3 ≤ 0(3) -x1 + x2 + 2x3 ≤ 0(4) -x3 ≤ -1

Assume we eliminate x1.Upper bound

Upper bound

Lower bound

Category?

17


2. For each pair of a lower bound aln<0 andupper bound aun>0, we have

3. For each such pair, add a constraint

18



(5) 2x3 ≤ 0 (from 1 and 3)(6) x2 + x3 ≤ 0 (from 2 and 3)

We eliminate x1.

19



(5) 2x3 ≤ 0 (from 1 and 3)(6) x2 + x3 ≤ 0 (from 2 and 3)

(7) 0 ≤ -1 (from 4 and 5)Contradiction (the system is unsatisfiable)!

We eliminate x3.

20

Complexity of Fourier-Motzkin

Worst-case complexity: m2n

Popular in compilers Because of simplicity

Popular when the problems are small – then it can be the fastest. Not suitable for large problems

Need another solution for big problems

21

Simplex

Simplex was originally designed for solving optimization problems: Given coefficient matrix and bound vector find a

satisfying assignment that maximizes a goal function described by

Also known as Linear Programming

We are only interested in the feasibility problem If problem is feasible than it is satisfiable

22

General form

Given same input as Fourier-Motzkin, convert input to general form

General form:

A combination of: Linear equalities of the form Lower and upper bounds on variables.

23

Converting to General Form

Replace (where )

with

and

s1,..., sm are called the additional variables.

24

Example 1

Convert

to:

25

Example 2

Convert

to:

26

Same problem, geometrically

27

Matrix form

x y s1 s2 s3

Due to the additional variables: now A is an m x (n + m) matrix.

28

The tableau

The diagonal part is inherent to the general form

Marked section will always be there

We can instead write:

x y s1 s2 s3

x ys1s2s3

29

The tableau

The tableau changes throughout the algorithm, but maintains its m x n structure

Distinguish between basic and nonbasic variables Initially, basic variables = the additional variables.

x ys1s2s3

30

The tableau

Denote by B – Basic variables N – Nonbasic variables

The tableau is simply a rewrite of the system:

The basic variables are also called the dependent variables. Their value is determined by the values of the other variables

31

The general simplex algorithm

Simplex maintains and updates: The tableau, an assignment to all variables The bounds

Two invariants:

All nonbasic variables satisfy their bounds

32

Invariants

Initially, B = additional variables N = problem variables (xi) = 0 for i {1,...,n+m}

Trivial to see that initial state satisfies invariants

33

The simplex algorithm

The initial assignment satisfies If the bounds of all basic variables are satisfied by ,

return `Satisfiable’

Otherwise... choose a variable and pivot Pivoting is the basic step of the algorithm

34

Pivoting

Find a basic variable xi that violates its bounds. Suppose that (xi) < li We need to fix the value of xi

Find a nonbasic variable xj such that aij > 0 and (xj) < uj, or aij < 0 and (xj) > lj

Such a variable xj is called suitable. If there is no suitable variable – return ‘Unsatisfiable’

35

Pivoting xi with xj

Solve equation i for xj:

From:

To:

Swap xi and xj, and update the i-th row accordingly.

From

To:

ai1 ... aij ... ain

-ai1

aij

... 1 aij

... -ain

aij

36

Pivoting xi with xj

Update all other rows: Replace xj with its equivalent obtained from row i:

37

Pivoting

Update as follows: Increase (xj) by

Now xj is a basic variable: it can violate its bounds.

Update (xi) to li

Update for all other basic (dependent) variables.

38

Example

Recall the tableau and constraints in our example:

Initially assigns 0 to all variables Bounds of s1 and s3 are violated

39

Example


We will pivot on s1

x is a suitable nonbasic variable for pivoting It has no upper bound

So now we pivot s1 with x

40

Example


Solve 1st row for x: Replace x with s1 in other rows:

41

Example

The new state:

Solve 1st row for x: Replace x with s1 in other rows:

42

Example

The new state:

We should increase x by Hence, (x) = 0 + 2 = 2 Now s1 is equal to its lower bound: (s1) = 2 Update all the others

43

Example

The new state:

Now s3 violates its lower bound y is a suitable nonbasic variable for pivoting

44

Example

The new state:

We should increase y by

45

Example

The final state:

All constraints are now satisfied

46

A few observations

The additional variables: Only additional variables have bounds. These bounds are permanent. Additional variables exit the base only on extreme points

(their lower or upper bounds). When entering the base, they shift towards the other bound

and possibly cross it (violate it).

47

A few observations

Can it be that we pivot(xi,xj) and then pivot(xj,xi) and enter a (local) cycle ? No.

Is termination guaranteed ? No. Perhaps there are bigger cycles.

48

A few observations

In order to avoid circles, we use Bland’s rule: determine a total order on the variables. Choose the first basic variable that violates its bounds, and

first nonbasic suitable variable for pivoting. It can be proven that this guarantees that no base is

repeated, which implies termination. We won’t prove this

49

A few observations

Simplex is exponential in the worst case However, considered very efficient on most real

practical problems Need for exponential number of steps is rare

50


Decision problem for is NP-hard Unlike

Branch & Bound Cuts

Omega test Based on Fourier-Motzkin elimination Will not be discussed today

51


Both Branch & Bound and Cuts rely on a solver to provide a (possibly non-integer) solution For example: Simplex

All variables in the final solution must be integers In the case of simplex, this does not includes the additional

variables The additional variables do not have to be a part of the outputted

solution

52

x1

x2

Non-Integral Solution

Is there a difference?

52

Previous methods found non-integral solutions

Rounding will not suffice

53

Is there a difference?

Previous methods found non-integral solutions

Rounding will not suffice No guarantee that integral

solution exists

53

x1

x2

54

Assume that all variables are finite. Enumerate all solutions with a tree

Guaranteed to find a feasible solution if it exists But, exponential growth in the size of the tree /

computation time

A naïve solution

x1=0

x2=0 x2=2x2=1

x1=1 x1=2

x2=0 x2=2x2=1x2=0 x2=2x2=1

55

Branch and Bound

The main idea: Solve the ‘relaxed’ problem, i.e. no integrality constraints. If the relaxed problem is infeasible – backtrack (there is no integer

solution in this branch) If the solution is integral – terminate (‘feasible’). Otherwise split on a variable for which the assignment is non-integral,

and repeat for each case.

Branch & Bound is guaranteed to find an integer solution if one exists

Branch & Bound can extended to handle the case where some of the variables are integers and some remain rational

56

x2≤1

Splitting on non-integral solutions.

x1

x2

x1

x2

Solve Relaxation to get Non-Integral solutions If relaxation is infeasible, prune branch

Create two sub-branches by adding constraints

x2≥2

Non-integral solution

57

Example

Suppose our system A has variables x1… x4, and that the solver returned a solution (1, 0.7, 2.5, 3).

Choose one of x2, x3. Suppose we choose x2. Solve two new problems:

A1 = A {x2 0} A2 = A {x2 1}

Clearly A1 or A2 are satisfiable iff A is.

58

The branch and bound tree

A

A2 A1

A12 A11

(1,0 .7,2.5,3)

(1,-1.5,1.5,4.1)

x2 · 0 x2 ¸ 1

x3 ¸ 1

(1,3,0.5,2)

(1,3,0.5,2)

x3 · 0

(1,3,4,1) x

Each leaf is a feasible solution.

Pruned due to infeasibility

59

Termination

x

y

Does B & B guaranteed to terminate ? No.

For example: Consider a constraint like This constraint won’t terminate

60

Termination

Does B & B guaranteed to terminate ? No.

For example: Consider a constraint like This constraint won’t terminate

It is always possible to find tighter bounds on variable values Will not be discussed

62

Improvement - Cutting Planes

Eliminate non-integer solutions by adding constraints (i.e. improve tightness of relaxation).

All feasible integer solutions remain feasible

Last non-integer solution is not feasible

x1

x2

Added Cut

63

Cutting Planes

Cutting planes might never find a solution Will get exponentially closer to a solution

Usually applied in conjunction with Branch & Bound

x1

x2

Added Cut

64

Cutting planes

Adding valid inequalities

Examples: From 2x 11

we can conclude x 5 What can be learned from the bounds , the constraint and

the assignment ?

65

Cutting planes

Examples: What can be learned from the bounds , the constraint and

the assignment ?

We know that the following currently holds:

We need to make both sides of the equation integers

66

Cutting planes

Examples: We need to make both sides of the equation integers

1, 0.5 are lower bounds therefore right side of equation is positive

Therefore:

In case of upper bounds replace “” with “”

New constraint eliminates current assignment

67

Gomory cuts

If basic variable as non-integer value and all nonbasic variables are at bounds => we can apply cut

Given constraint of the formand assignment

Split nonbasic variables into 2 groups:

68

Gomory cuts

Substract bounds from constraint

And make left side integer

69

Gomory cuts

Further split J and K:

70

Gomory cuts

If right side is positive:

71

Gomory cuts

If right side is negative:

72

New constraint:

Gomory cuts

Could probably be 2

73

Preprocessing

Constraints can be removed Example:

x1 + x2 2, x1 1, x2 1 First constraint is redundant.

In general, for a set:

is redundant if

74

Preprocessing

Bounds can be tightened Example:

2x1 + x2 2, x2 4, x1 3 From 1st and 2nd constraints: x1 -1

In general, if a0 > 0

And, if a0 < 0

75

Preprocessing (Only for integers)

Convert strict inequalities to non-strict inequalities Example:

Given inequality

Rewrite as Or

77

Questions?

Documents

Decision Procedures for Linear Arithmetic