Embed Size (px)
Citation preview
1
Decision Procedures for Linear Arithmetic
Presented By Omer Katz
01/04/14
Based on slides by Ofer Strichman
3
Agenda
DPLL-T (a very short reminder) What is Linear Arithmetic and why is it needed Decision procedures for Decision procedures for A few preprocessing improvement steps
Given enough time: Difference logic Delayed Theory Combination
Improvement on Nelson-Oppen Not related to Linear Arithmetic
4
Reminder: DPLL-T
Our goal is to solve SMT problems with formulas from a theory T
DPLL-T is the most common approach Based on the DPLL algorithm for SAT problems Combines a decision procedure for the theory T
5
Reminder: DPLL-T
6
Linear Arithmetic
Theory grammar:
Can be defined over Rational () or Integers ()
Example:
7
Why is it needed?
Given the following C code:
The following assembly can be generated:
8
Why is it needed?
A possible optimization: Read the value of a[j] only once
Need to verify that loop won’t change a[j] Can be encoded as Linear Arithmetic problem
If no solution is found, optimization is safe
9
Decision Procedures for
Gaussian’s elimination Fourier-Motzkin Simplex
10
Gaussian’s elimination
A simple method for solving a set of equalities Less suitable for inequalities
Given a linear system Ax = b
A x = b
11
Gaussian’s elimination
Manipulate A|b to an upper-triangular form
Then, solve backwards from the k’s row according to:
12
Gaussian elimination - example
And now… x3 = -1, x2 = 3, x1 = 1 problem solved.
13
Fourier-Motzkin Elimination
Earliest method for solving linear inequalities
Given linear non-strict inequalities:
Pick a variable and eliminate it Continue until all variables but one are eliminated
If problem included equalities, eliminate by assignment
14
Fourier-Motzkin Elimination
nnmnm
n
b
b
b
x
x
x
aa
aa
aaa
:
:
:
:
......
::
::
:
....
2
1
2
1
1
2221
11211
bIA A system of conjoined linear inequalities
m constraints
n variables
15
Fourier-Motzkin Elimination
1. When eliminating xn, partition the constraints according to the coefficient ain: ain > 0: upper bound
ain < 0: lower bound
assume ai,n >0
16
Fourier-Motzkin Elimination
Example:(1) x1 – x2 ≤ 0
(2) x1 – x3 ≤ 0
(3) -x1 + x2 + 2x3 ≤ 0
(4) -x3 ≤ -1
Assume we eliminate x1.Upper bound
Upper bound
Lower bound
Category?
17
Fourier-Motzkin Elimination
2. For each pair of a lower bound aln<0 andupper bound aun>0, we have
3. For each such pair, add a constraint
18
Fourier-Motzkin Elimination
Example:(1) x1 – x2 ≤ 0
(2) x1 – x3 ≤ 0
(3) -x1 + x2 + 2x3 ≤ 0
(4) -x3 ≤ -1
(5) 2x3 ≤ 0 (from 1 and 3)
(6) x2 + x3 ≤ 0 (from 2 and 3)
We eliminate x1.
19
Fourier-Motzkin Elimination
Example:(1) x1 – x2 ≤ 0
(2) x1 – x3 ≤ 0
(3) -x1 + x2 + 2x3 ≤ 0
(4) -x3 ≤ -1
(5) 2x3 ≤ 0 (from 1 and 3)
(6) x2 + x3 ≤ 0 (from 2 and 3)
(7) 0 ≤ -1 (from 4 and 5)Contradiction (the system is unsatisfiable)!
We eliminate x3.
20
Complexity of Fourier-Motzkin
Worst-case complexity: m2n
Popular in compilers Because of simplicity
Popular when the problems are small – then it can be the fastest. Not suitable for large problems
Need another solution for big problems
21
Simplex
Simplex was originally designed for solving optimization problems: Given coefficient matrix and bound vector find a
satisfying assignment that maximizes a goal function described by
Also known as Linear Programming
We are only interested in the feasibility problem If problem is feasible than it is satisfiable
22
General form
Given same input as Fourier-Motzkin, convert input to general form
General form:
A combination of: Linear equalities of the form Lower and upper bounds on variables.
23
Converting to General Form
Replace (where )
with
and
s1,..., sm are called the additional variables.
24
Example 1
Convert
to:
25
Example 2
Convert
to:
26
Same problem, geometrically
27
Matrix form
x y s1 s2 s3
Due to the additional variables: now A is an m x (n + m) matrix.
28
The tableau
The diagonal part is inherent to the general form
Marked section will always be there
We can instead write:
x y s1 s2 s3
x y
s1
s2
s3
29
The tableau
The tableau changes throughout the algorithm, but maintains its m x n structure
Distinguish between basic and nonbasic variables Initially, basic variables = the additional variables.
x y
s1
s2
s3
30
The tableau
Denote by B – Basic variables N – Nonbasic variables
The tableau is simply a rewrite of the system:
The basic variables are also called the dependent variables. Their value is determined by the values of the other variables
31
The general simplex algorithm
Simplex maintains and updates: The tableau, an assignment to all variables The bounds
Two invariants:
All nonbasic variables satisfy their bounds
32
Invariants
Initially, B = additional variables N = problem variables (xi) = 0 for i {1,...,n+m}
Trivial to see that initial state satisfies invariants
33
The simplex algorithm
The initial assignment satisfies If the bounds of all basic variables are satisfied by ,
return `Satisfiable’
Otherwise... choose a variable and pivot Pivoting is the basic step of the algorithm
34
Pivoting
Find a basic variable xi that violates its bounds.
Suppose that (xi) < li We need to fix the value of xi
Find a nonbasic variable xj such that aij > 0 and (xj) < uj, or
aij < 0 and (xj) > lj
Such a variable xj is called suitable.
If there is no suitable variable – return ‘Unsatisfiable’
35
Pivoting xi with xj
Solve equation i for xj:
From:
To:
Swap xi and xj, and update the i-th row accordingly.
From
To:
ai1 ... aij ... ain
-ai1
aij
... 1
aij
... -ain
aij
36
Pivoting xi with xj
Update all other rows: Replace xj with its equivalent obtained from row i:
37
Pivoting
Update as follows: Increase (xj) by
Now xj is a basic variable: it can violate its bounds.
Update (xi) to li
Update for all other basic (dependent) variables.
38
Example
Recall the tableau and constraints in our example:
Initially assigns 0 to all variables Bounds of s1 and s3 are violated
39
Example
Recall the tableau and constraints in our example:
We will pivot on s1
x is a suitable nonbasic variable for pivoting It has no upper bound
So now we pivot s1 with x
40
Example
Recall the tableau and constraints in our example:
Solve 1st row for x: Replace x with s1 in other rows:
41
Example
The new state:
Solve 1st row for x: Replace x with s1 in other rows:
42
Example
The new state:
We should increase x by Hence, (x) = 0 + 2 = 2
Now s1 is equal to its lower bound: (s1) = 2 Update all the others
43
Example
The new state:
Now s3 violates its lower bound
y is a suitable nonbasic variable for pivoting
44
Example
The new state:
We should increase y by
45
Example
The final state:
All constraints are now satisfied
46
A few observations
The additional variables: Only additional variables have bounds. These bounds are permanent. Additional variables exit the base only on extreme points
(their lower or upper bounds). When entering the base, they shift towards the other bound
and possibly cross it (violate it).
47
A few observations
Can it be that we pivot(xi,xj) and then pivot(xj,xi) and enter a (local) cycle ? No.
Is termination guaranteed ? No. Perhaps there are bigger cycles.
48
A few observations
In order to avoid circles, we use Bland’s rule: determine a total order on the variables. Choose the first basic variable that violates its bounds, and
first nonbasic suitable variable for pivoting. It can be proven that this guarantees that no base is
repeated, which implies termination. We won’t prove this
49
A few observations
Simplex is exponential in the worst case However, considered very efficient on most real
practical problems Need for exponential number of steps is rare
50
Decision Procedures for
Decision problem for is NP-hard Unlike
Branch & Bound Cuts
Omega test Based on Fourier-Motzkin elimination Will not be discussed today
51
Decision Procedures for
Both Branch & Bound and Cuts rely on a solver to provide a (possibly non-integer) solution For example: Simplex
All variables in the final solution must be integers In the case of simplex, this does not includes the additional
variables The additional variables do not have to be a part of the outputted
solution
52
x1
x2
Non-Integral Solution
Is there a difference?
52
Previous methods found non-integral solutions
Rounding will not suffice
53
Is there a difference?
Previous methods found non-integral solutions
Rounding will not suffice No guarantee that integral
solution exists
53
x1
x2
54
Assume that all variables are finite. Enumerate all solutions with a tree
Guaranteed to find a feasible solution if it exists But, exponential growth in the size of the tree /
computation time
A naïve solution
x1=0
x2=0 x2=2x2=1
x1=1 x1=2
x2=0 x2=2x2=1x2=0 x2=2x2=1
55
Branch and Bound
The main idea: Solve the ‘relaxed’ problem, i.e. no integrality constraints. If the relaxed problem is infeasible – backtrack (there is no integer
solution in this branch) If the solution is integral – terminate (‘feasible’). Otherwise split on a variable for which the assignment is non-integral,
and repeat for each case.
Branch & Bound is guaranteed to find an integer solution if one exists
Branch & Bound can extended to handle the case where some of the variables are integers and some remain rational
56
x2≤1
Splitting on non-integral solutions.
x1
x2
x1
x2
Solve Relaxation to get Non-Integral solutions If relaxation is infeasible, prune branch
Create two sub-branches by adding constraints
x2≥2
Non-integral solution
57
Example
Suppose our system A has variables x1… x4, and that the solver returned a solution (1, 0.7, 2.5, 3).
Choose one of x2, x3. Suppose we choose x2.
Solve two new problems: A1 = A {x2 0}
A2 = A {x2 1}
Clearly A1 or A2 are satisfiable iff A is.
58
The branch and bound tree
A
A2 A1
A12 A11
(1,0 .7,2.5,3)
(1,-1.5,1.5,4.1)
x2 · 0 x2 ¸ 1
x3 ¸ 1
(1,3,0.5,2)
(1,3,0.5,2)
x3 · 0
(1,3,4,1) x
Each leaf is a feasible solution.
Pruned due to infeasibility
59
Termination
x
y
Does B & B guaranteed to terminate ? No.
For example: Consider a constraint like This constraint won’t terminate
60
Termination
Does B & B guaranteed to terminate ? No.
For example: Consider a constraint like This constraint won’t terminate
It is always possible to find tighter bounds on variable values Will not be discussed
62
Improvement - Cutting Planes
Eliminate non-integer solutions by adding constraints (i.e. improve tightness of relaxation).
All feasible integer solutions remain feasible
Last non-integer solution is not feasible
x1
x2
Added Cut
63
Cutting Planes
Cutting planes might never find a solution Will get exponentially closer to a solution
Usually applied in conjunction with Branch & Bound
x1
x2
Added Cut
64
Cutting planes
Adding valid inequalities
Examples: From 2x 11
we can conclude x 5
What can be learned from the bounds , the constraint and
the assignment ?
65
Cutting planes
Examples: What can be learned from the bounds , the constraint and
the assignment ?
We know that the following currently holds:
We need to make both sides of the equation integers
66
Cutting planes
Examples: We need to make both sides of the equation integers
1, 0.5 are lower bounds therefore right side of equation is positive
Therefore:
In case of upper bounds replace “” with “”
New constraint eliminates current assignment
67
Gomory cuts
If basic variable as non-integer value and all nonbasic variables are at bounds => we can apply cut
Given constraint of the formand assignment
Split nonbasic variables into 2 groups:
68
Gomory cuts
Substract bounds from constraint
And make left side integer
69
Gomory cuts
Further split J and K:
70
Gomory cuts
If right side is positive:
71
Gomory cuts
If right side is negative:
72
New constraint:
Gomory cuts
Could probably be 2
73
Preprocessing
Constraints can be removed Example:
x1 + x2 2, x1 1, x2 1 First constraint is redundant.
In general, for a set:
is redundant if
74
Preprocessing
Bounds can be tightened Example:
2x1 + x2 2, x2 4, x1 3 From 1st and 2nd constraints: x1 -1
In general, if a0 > 0
And, if a0 < 0
75
Preprocessing (Only for integers)
Convert strict inequalities to non-strict inequalities Example:
Given inequality
Rewrite as Or
77
Questions?
