Boolean Satisfiability

The most fundamental NP-complete problem, and now a powerful technology

for solving many real world problems

Overview

• CNF, SAT, 3SAT and 2SAT• Resolution and Unit Propagation• DPLL search• Conflict-driven Nogood Learning• Activity-based Search• Modelling for SAT

Conjunctive Normal Form

• SAT solvers solve problems in

–Conjunctive Normal Form (CNF)• Boolean variable: b (true/false) or (0/1)• Literal: l variable b or its negation –b

– negating a literal: -l = -b if l = b, and

-l = b if l = -b

• Clause: C disjunction or set of literals• CNF: theory T set of clauses

• Assignment: set of literals A with {b,-b} not subset: e.g. {-b1,b2} Assign b1=false, b2=true

Boolean Satisfiability (SAT)

• This is the most basic NP-complete problem.

• SAT: Given a set of clauses T, find an assignment A such that

for each C in T

• Each clause C in T is satisfied by A

€

A ∩ C ≠ ∅

3SAT

• 3SAT: SAT with the restriction that each clause has length at most 3.

• SAT -> 3SAT– {l1, l2, l3, l4, l5, l6} l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6

becomes the set of clauses– {l1, l2, b1}, {-b1, l3, b2}, {-b2, l4, b3}, {-b3, l5, l6}

where b1, b2, b3 are new Boolean variables

• 3SAT is NP-complete (by the above reduction)

2SAT

• 2SAT: SAT with the restriction that each clause has length at most 2.

• 2SAT is decidable in polynomial time (n3)

• 2SAT is NL-complete– which is a crazy complexity class– Nondeterministic Turing machine with log writeable

memory!

Resolution

• The fundamental inference for CNF– {l1, l2, l3, b} {-b, l4, l5, l6} implies– {l1, l2, l3, l4, l5, l6}

• Or– (l1 ∨ l2 ∨ l3 ∨ b) ∧ (-b ∨ l4 ∨ l5 ∨ l6) ->

l1 ∨ l2 ∨ l3 ∨ l4 ∨ l5 ∨ l6

• One can prove unsatisfiability of a CNF formula by repeatedly applying all possible resolutions– if this generates an empty clause then UNSAT– otherwise SAT

• Exponential process!

Unit Propagation

• Resolution = too expensive• Unit Propagation (=unit resolution)

– Restricted form of resolution = finds unit clauses

• {l1, l2, …, ln, l} where –li in A forall 1 ≤ i ≤ n• Add l to assignment A

• {-b1, b2, b} A = {b1, -b2}– {-b1, b2, b} A := {b1, -b2, b}

• (-b1 ∨ b2 ∨ b) ∧ b1 ∧ -b2 -> b

Unit Propagation

• Repeatedly apply unit propagation, – until no new unit consequences can be found– or failure detected

• A = {b1, -b2, b3, -b4} C = {-b1, -b3, b4}• {-b1, -b3, b4}

• Failure detected

Unit Propagation Example• T = {{-e11,-e21}, {-e11,-e31},{-e11,-e41},

{e21,-b21}, {e31,-b31},{e41,-b41},

{b21,-b51},{b51,-b52,e52},{b41,-e52},• A = {e11,b52}

{-e11,-e21}A ={e11,b52,-e21}

{-e11,-e31}A ={e11,b52,-e21,-e31}

{-e11,-e41}A ={e11,b52,-e21,-e31,-e41}

{-e21,-b21}A ={e11,b52,-e21,-e31,-e41,-b21}

{-e31,-b31}A ={e11,b52,-e21,-e31,-e41,-b21,-b31}

{-e41,-b41}A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41}

{b21,-b51}A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51}

{b51,-b52,e52}A ={e11,b52,-e21,-e31,-e41,-b21,-b31,-b41,-b51,e52}

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51

b52

e52

fail

{b41,-e52}A = false

Implication Graph

• Records the reason why each Boolean literal became true– Decision– Propagation(and why)

• Used for nogood reasoning!

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51

b52

e52

fail

Unit Propagation Exercise• T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5},

{b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,-b8,b10},

{-b5,-b10},{b5,b6,-b10} ,{-b8,b11}}• A = {b1,-b3,b8}• Perform unit propagation and draw the implication

graph

DPLL search

• Davis-Putnam-Logemann-Loveland algorithm– interleave decisions + unit propagation

• dpll(A)A' = unitprop(A)

if A' == false return false

else if exists Boolean variable b not appearing in A

if dpll(A' union {b}) return true

else if dpll(A' union {-b}) return true

else return false

else return true

DPLL search with nogood learning

• dpll(A)A' = unitprop(A)

if A' == false

Add a clause C explaining the failure to T

Backjump to the first place C can give new information

else if exists Boolean variable b not appearing in A

if dpll(A' union {b}) return true

else if dpll(A' union {-b}) return true

else return false

else return true

Nogood Learning

• The implication graph shows a reason for failure• Any cut separating the fail node from decisions

– explains the failure

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51

b52

e52

fail

e52 ∧ -b41 false {b41,-e52}b52 ∧ -b51 ∧-b41 false {b41,b51,-b52}b52 ∧ -b51 ∧-e41 false {e41,b51,-b52}b52 ∧ e11 false {-e11,-b52}

Which Nogood?

• SAT solvers almost universally use the– First Unique Implication Point (1UIP) Nogood

• Closest nogood to failure– only one literal from the last decision level

• Asserting: on backjump it will unit propagate• General: more general than many other choices• Fast: doesn’t require too much computation

– replace last literal in nogood until only one at last level

1UIP Nogood Creation

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51 b52 e52

b22 e22

-e32

-e42

-b32

-b42

b33

b43

e33

e43

fail

e33 ∧ e43 false {-e33,-e43}

e43

e33

-b42 ∧ b43 ∧ e33 false {b42,-b43,-e33}

b43

{-b32,-b42,b33,b43} false {b32,b42,-b33,-b43}

b33

{-b21,-b32,-b42,b33} false {b21,b32,b42,-b33}

-b32

-b21

-b42

{-b21,-b32,-b42} false {b21,b32,b42} {-b21,-b41,-e42,-b32} false {b21,b41,e42,b32}

-e42-b41

{-b21,-b31,-b41,-e32,-e42} false {b21,b31,b41,e32,e42}

-e32-b31

{-b21,-b31,-b41,e22,-e32} false {b21,b31,b41,-e22,e32}

e22

{-b21,-b31,-b41,e22} false {b21,b31,b41,-e22}

1 UIP Nogood

Backjumping

• Backtrack to second last level in nogood

• Nogood will propagate• e.g. {b21,b31,b41,-e22}

e11

-e21

-e31

-e41

-b21

-b31

-b41

-b51

-e22

{b21,b31,b41,-e22}

Continue unit propagationthen make next choice

-b22

-b52

Why Add Nogoods

• We will not make the same choices again– {e11,b52} leads to failure– After choosing e11 we infer –b52– Better yet, any choice that leads to b21,b31,b41

prevents the choice b52

• Drastic reduction in search space• Faster solving

1UIP Exercise• T = {{-b1,-b2,b3},{-b4,b5},{b2,-b5},

{b2,b7}, {b5,-b11,b9},{b4,-b9},{-b1,-b8,b10},

{-b5,-b10},{b5,b6,-b10},{-b8,b11}}• A = {b1,-b3,b8}• Assume the order of the decisions was

b8, -b3, b1

Determine the 1UIP nogood

Where does execution backjump to?

Activity

• Each time a Boolean variable is seen during nogood creation, i.e.– appears in final nogood, or– is eliminated in the reverse propagation

• Increase activity by one• These variables are helping cause failure• Periodically divide all activities by some amount

– activity reflects helping cause recent failure

Activity-based Search

• Select the unfixed variable with highest activity• MiniSat: set to false• RSAT: set to the last value it took

– works with backjumping to recreate the same path– e.g. -b1, b3, -b11, b4, -b5, b7, fail– backjump to -b1, b3, -b11 – If b4 is now highest activity variable set it true [b4]– If b5 is next highest activity variable set it false [-b5]

• Activity-based search– concentrates on the variables causing failure– learns shorter nogoods by failing earlier

Activity-based Search

• Works well with restart• On restart we concentrate on the now most active

variables– a new part of the search space– learn new nogoods about this

Modern SAT Solvers

• Modern SAT solvers can handled problems with– (low) millions of clauses– millions of variables

assuming the input has structure• Random CNF is much harder (but uninteresting)• Before the advent of nogood learning

– (low) thousands of clauses– hundreds of variables

Modelling for SAT

• Boolean constructs can all be converted to clauses• b1 = b2∨b3

– {-b2,b1}, {-b3,b1}, {b2,b3,-b1}

• b1 = b2∧b3– {-b1,b2}, {-b1,b3}, {b1,-b2,-b3}

• b1 = (b2b3)– {-b2,b3,b1}, {b2,-b3,b1}, {b2,b3,-b1}, {-b2,-b3,-b1}

• etc.

Modelling Integers in SAT

• Two main representations: i in 0..m (m = 2k-1 – 1)• Binary: i = 2k-1ik-1 + 2k-2ik-2 + … + 2i1 + i0

• Unary: i = im + im-1 + … + i1

– where ik ik-1

• Example representations

Integer Binary Unary

11 1011 0000011111111111

0 0000 0000000000000000

6 0110 0000000000111111

Pseudo-Boolean Constraints

• Psuedo-Boolean constraints: bi Boolean

• Cardinality constraints:

• Note that is b1 ∨ … ∨ bm

• Straight clausal definitions are exponential• Encode: Binary Decision Diagram, Binary or

Unary arithmetic• Examine cardinality constraint encodings

€

bii=1

m

∑ ≤ k

€

bii=1

m

∑ = k

€

bii=1

m

∑ ≥ k

€

bii=1

m

∑ ≥ 1

€

aibii=1

m

∑ ≤ k

€

aibii=1

m

∑ = k

€

aibii=1

m

∑ ≥ k

Cardinality Constraints (BDD)

5

1

3i

ix

5

2

2i

ix

5

2

3i

ix1x

a if b then c else d

{-a,-b,c}, {-a,b,d}, {a,-b,-c}, {a,b,-d}

if then else

Cardinality Constraints (BDD)

5

1

3i

ix if then else

5

2

2i

ix

5

2

3i

ix1x

5

2

3i

ix if then else

5

3

2i

ix

5

3

3i

ix2x

5

2

2i

ix if then else

5

3

1i

ix

5

3

2i

ix2x

5

3

1i

ix if then else

5

4

0i

ix

5

4

2i

ix3x∨

∨

∨

∨

Cardinality Constraints (BDD)

5

1

3i

ix

5

3

2i

ix

5

2

2i

ix

true

x1 = 0 x1 = 1

€

xi ≤ 1i=3

5

∑

x2

x3

5

2

3i

ix

€

x4 + x5 ≤ 0

€

x5 ≤ 0

€

x4 + x5 ≤ 1

• Dashed arc: xi = 0

• Full arc: xi = 1• Number of intermediatestates is (n – k) ×k

x4

x5

x1

BDD Exercise

• Draw the BDD for the cardinality constraint

€

xi = 2i=1

4

∑

Cardinality Constraints (binary)

bi

1 2 3 bi =(sk-1,…,s0)

s2

s1

s0

Sum(n)

Cardinality Constraints (binary)

Sum(n)

Sum(n/2)

1

1 2 3 bi =(sk-1,…,s0)

Sum(n/2)

2

s2

s1

s0B

inary adder 3B

inary adder 3bi

Binary Adder

FAFA

a2 b2 a1 b1

C2 C1 Cin

y3 y2 y1

{y2,-a2,-b2,-c1}, {y2,-a2,b2,c1},{y2,a2,-b2,c1}, {y2,a2,b2,-c1},  {-y2,a2,b2,c1}, {-y2,a2,-b2,-c1}, {-y2,-a2,b2,-c1}, {-y2,-a2,-b2,c1},{y3,-a2,-b2}, {y3,-a2,-c1}, {y3,-b2,-c1},                              {-y3,a2,b2}, {-y3,a2,c1}, {-y3,b2,c1}

Cardinality Constraints (binary)

bi

s2

s1

s0

Sum(n)

bi ≤4

s2 s1 ∧s0

(s2,s1,s0) ≤4

{-s2,-s1}, {-s2,-s0}How do you encode bi ≤5

Sorting Networks

A Comparator is a device with two inputs, a and b, and two outputs, c and d

comparatorab

c=max(a,b)

d=min(a,b)

A Sorting network is a composition of wires and comparators that sorts its inputs

b1a1

a2 b2

b3a3

a4 b4

CNF of comparator{-a,c}, {-b,c}, {-a,-b,d}, {a,-d}, {b,-d}, {a,b,-c}

Sorting Networks

There are “handcrafted” sorting networks

And there are sorting networks based on your favorite sorting algorithm

x1x2x3x4

x8

y1y2y3

x5x6x7

y5y6y7y8

x9 y9

Custom sorting network, n=9

x1x2x3x4

y1y2y3

x5x6

y5y6

y4

y4

“Bubble sort” network

163245

613245

631245

632145

632415

654321

Odd-Even (OE) Sorting Networks

x1

x2

x3

x4

x5

x6

x7

x8

y1

y2

y3

y4

y5

y6

y7

y8

Sorter(n)

Sorter(n/2)

Sorter(n/2) OE

-Mer

ger(

n)

Sorter(n/4)

OE

-Mer

ger(

n/2)}

),...,1(

),...,12

(

)2

,...,1(

{ )(

nmerge

nn

sort

nsort

nsort

OE

-Mer

ger(

n/2)

Sorter(n/4)

Sorter(n/4)

Sorter(n/4)

Odd-Even Merger

OE

-Mer

ger(

4)

x1

x2

x3

x4

x5

x6

x7

x8

y1

y2

y3

y5

y6

y7

y8

y4

Given two sorted input sequences

sorted

sorted

Merge the odd input sequences

Merge the even input sequences

OEMerger(8)

OE

-Mer

ger(

4)

Odd-Even Merger

x1

x2

x3

x4

x5

x6

x7

x8

y1

y2

y3

y5

y6

y7

y8

y4

Given two sorted input sequences

sorted

sorted

Merge the odd input sequences

Merge the even input sequences

Combine the outputs into a sorted output

OE

-Mer

ger(

4)

OE

-Mer

ger(

4)

Sorting Networks

• Draw the best sorting network you can for 5 inputs! Minimize levels and/or comparators

Cardinality Constraints (unary)

• Sorting Network encodes unary sum

• Cardinality by setting output bits

0110

1100

1010

1010

1010

x1

x2

x3

x4

x5

x6

x7

x8

y1

y2

y3

y4 Sorter(8)0 0 0 0

€

y ii=1

8

∑ ≤ 4

€

x ii=1

8

∑ ≤ 4

Cardinality Constraints

• BDD– quadratic size construction– Unit propagation on CNF enforces domain consistency

• Binary– Linear size construction– Unit Propagation on CNF does not enforce domain

consistency

• Unary– n log2 n size construction– unit propagation enforces domain consistency

Psuedo-Boolean Constraints

• Similar to Cardinality Constraints• BDD

– exponential in size, strong propagation

• Binary– O(n log n) in size, weak propagation

• Unary– O(n log2 n) in size, medium propagation– interesting problem to find base for representation

Integer Constraints

• BDD– can handle arbitrary formula but exponential

• Unary– addition, subtraction, multiplication by constant– multiplication is impractical

• Binary– addition, subtraction– multiplication is quadratic

Boolean Solving in MiniZinc

• mzn –b satinvokes a SAT solver

• all constraints and variables must be Boolean• Without recursion it’s a bit hard to model

interesting problems with just MiniZinc

Example: Latin Squaresint: n; % sizearray[1..n,1..n,1..n] of var bool: b;

predicate atmostone(array[int] of var bool:x) = forall(i,j in index_set(x) where i < j)( (not x[i]) \/ (not x[j]));

predicate exactlyone(array[int] of var bool:x) = atmostone(x) /\ exists(x);

constraint forall(i,j in 1..n)( exactlyone([ b[i,j,k] | k in 1..n]) /\ exactlyone([ b[i,k,j] | k in 1..n]) /\ exactlyone([ b[k,i,j] | k in 1..n]));

solve satisfy;

output [ if fix(b[i,j,k]) then show(k) ++ if j == n then "\n" else " " endif else "" endif | i,j,k in 1..n];

5 7 4 6 8 2 1 3 9 12 10 11 138 2 1 7 6 4 11 10 3 9 5 13 126 8 7 1 2 3 10 9 4 5 13 12 117 6 5 3 4 1 9 11 2 13 12 10 83 5 8 4 7 13 2 1 12 6 11 9 101 3 2 8 5 12 13 4 11 10 6 7 92 1 6 5 3 10 4 12 13 11 9 8 74 9 3 2 1 11 12 13 10 7 8 5 612 4 10 11 13 9 3 2 1 8 7 6 510 11 9 13 12 5 6 7 8 1 2 3 49 10 13 12 11 6 5 8 7 2 1 4 311 13 12 9 10 7 8 5 6 3 4 1 213 12 11 10 9 8 7 6 5 4 3 2 1

size 13

Summary

• Modern SAT solvers are powerful solvers– nogood learning– activity based search– restart

• State of the art solutions to Boolean problems• Many CP problems which focus on alldifferent

and ≠ can be mapped to Boolean problems– quasi-group completion (latin squares)– graph coloring

Exercise 1: atmostone

• There is a linear encoding of atmostone(x)– introduce a new array of sum variables s

encoding s[i] = exists(j in 1..i)(x[i])– s[1] = x[1] – if s[i] is true then x[i+1] must be false– s[i+1] can be defined in terms of s[i] and x[i+1]– its actually the sorting network circuit simplified

• Write a new predicate defn of atmostone(x)• Compare the execution speed on latin squares