Constraint Satisfaction and the

Davis-Putnam-Logeman-Loveland Procedure

Henry Kautz

Guessing versus Reasoning

• A central feature of all search algorithms is making guesses

–Which node to expand?–Which child to explore?

• We can reduce the amount of guesswork by doing more reasoning about the consequences of past decisions

–Reasoning about past choices can prune away many future choices

Discrete Constraint Satisfaction Problem

• X is a set of variables x1, x2, …, xn• D is a set of finite domains D1, D2, …, Dn• C is a set of constraints C1, C2, …, Cm

Each constraint restricts the joint values of a subset of the variables

• Example: 3-ColoringXi = countries

Di = { Red, Blue, Green}

For each adjacent xi, xj there is a constraint Ck(xi,xj) { (R,G), (R,B), (G,R), (G,B), (B,R), (B,G) }

Kinds of Problems

• Find a solution that satisfies all constraints

• Find all solutions

• Find a “tightest form” for each constraint(x1,x2) { (R,G), (R,B), (G,R), (G,B), (B,R), (B,G) }

(x1,x2) { (R,G), (R,B) }

• Find a solution that minimizes some additional objective function

Chinese Dinner Constraint Network

Soup

Total Cost< $30

ChickenDish

Vegetable

RiceSeafood

Pork Dish

Appetizer

Must beHot&Sour

No Peanuts

No Peanuts

NotEggplant

Not BothSpicy

What is the arity of each constraint?

Solving CSP by Depth-First Search

• Interleave inference and guessing

• At each internal node:–Select unassigned variable–Select a value in domain–Backtracking: try another value

• At each node:–Propagate Constraints

Where are the guesses?

4 Queens

Q4 {1,2,3,4}

Q3 {1,2,3,4}

Q2 {1,2,3,4}

Q1 {1,2,3,4}

Q

Q

Q

Q

Q1 Q2

1 3

1 4

2 4

3 1

4 1

4 2

Q1 Q3

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Q1 Q4

1 2

1 3

2 1

2 3

2 4

3 1

3 2

3 4

4 2

4 3

1 2 3 4

4

3

2

1

Q2 Q3

1 3

1 4

2 4

3 1

4 1

4 2

Q3 Q4

1 3

1 4

2 4

3 1

4 1

4 2

Q2 Q4

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Constraint Checking

Q

Q

x

Q x

x

Q x

x x

Q x x

Q

x

x

Q x

Q

x

x Q

Q x x

Q x

x x

x Q x

Q x x x

Takes 5 guesses to determine first guess was wrong

Forward Checking

x

x

x

Q x x x

x x

Q x x

x x x

Q x x x

Q x x

x x

x x

Q x x x

Q x x

x x x

x Q x

Q x x x

Takes 3 guesses to determine first guess was wrong

When variable is set, immediately remove inconsistent values from domains of other variables

Arc Consistency• Iterated forward

checking Q

Q1 Q2

1 3

1 4

2 4

3 1

4 1

4 2

Q1 Q3

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Q1 Q4

1 2

1 3

2 1

2 3

2 4

3 1

3 2

3 4

4 2

4 3

1 2 3 4

4

3

2

1

Q2 Q3

1 3

1 4

2 4

3 1

4 1

4 2

Q3 Q4

1 3

1 4

2 4

3 1

4 1

4 2

Q2 Q4

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Arc Consistency• Reduce domain of Q1

x

x

x

Q x x x

Q1 Q2

1 3

1 4

2 4

3 1

4 1

4 2

Q1 Q3

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Q1 Q4

1 2

1 3

2 1

2 3

2 4

3 1

3 2

3 4

4 2

4 3

1 2 3 4

4

3

2

1

Q2 Q3

1 3

1 4

2 4

3 1

4 1

4 2

Q3 Q4

1 3

1 4

2 4

3 1

4 1

4 2

Q2 Q4

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Arc Consistency• Reduce domain of Q2

x

x

x

Q x x x

Q1 Q2

1 3

1 4

2 4

3 1

4 1

4 2

Q1 Q3

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Q1 Q4

1 2

1 3

2 1

2 3

2 4

3 1

3 2

3 4

4 2

4 3

1 2 3 4

4

3

2

1

Q2 Q3

1 3

1 4

2 4

3 1

4 1

4 2

Q3 Q4

1 3

1 4

2 4

3 1

4 1

4 2

Q2 Q4

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Arc Consistency• Reduce domain of Q3

x

x

x

Q x x x

Q1 Q2

1 3

1 4

2 4

3 1

4 1

4 2

Q1 Q3

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Q1 Q4

1 2

1 3

2 1

2 3

2 4

3 1

3 2

3 4

4 2

4 3

1 2 3 4

4

3

2

1

Q2 Q3

1 3

1 4

2 4

3 1

4 1

4 2

Q3 Q4

1 3

1 4

2 4

3 1

4 1

4 2

Q2 Q4

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Arc Consistency• Reduce domain of Q4

x

x

x

Q x x x

Q1 Q2

1 3

1 4

2 4

3 1

4 1

4 2

Q1 Q3

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Q1 Q4

1 2

1 3

2 1

2 3

2 4

3 1

3 2

3 4

4 2

4 3

1 2 3 4

4

3

2

1

Q2 Q3

1 3

1 4

2 4

3 1

4 1

4 2

Q3 Q4

1 3

1 4

2 4

3 1

4 1

4 2

Q2 Q4

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Arc Consistency• Note that Q3=2 is

determined by (Q2,Q3)

x x

x

x

Q x x x

Q1 Q2

1 3

1 4

2 4

3 1

4 1

4 2

Q1 Q3

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Q1 Q4

1 2

1 3

2 1

2 3

2 4

3 1

3 2

3 4

4 2

4 3

1 2 3 4

4

3

2

1

Q2 Q3

1 3

1 4

2 4

3 1

4 1

4 2

Q3 Q4

1 3

1 4

2 4

3 1

4 1

4 2

Q2 Q4

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Arc Consistency• Propagating Q3=2

eliminates all rows in (Q3,Q4)

x x

x

x

Q x x x

Q1 Q2

1 3

1 4

2 4

3 1

4 1

4 2

Q1 Q3

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Q1 Q4

1 2

1 3

2 1

2 3

2 4

3 1

3 2

3 4

4 2

4 3

1 2 3 4

4

3

2

1

Q2 Q3

1 3

1 4

2 4

3 1

4 1

4 2

Q3 Q4

1 3

1 4

2 4

3 1

4 1

4 2

Q2 Q4

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3 violatedconstraint!

Arc Consistency• First choice Q1=1 shown

bad with no further guessing!

x x

x

x

Q x x x

Q1 Q2

1 3

1 4

2 4

3 1

4 1

4 2

Q1 Q3

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3

Q1 Q4

1 2

1 3

2 1

2 3

2 4

3 1

3 2

3 4

4 2

4 3

1 2 3 4

4

3

2

1

Q2 Q3

1 3

1 4

2 4

3 1

4 1

4 2

Q3 Q4

1 3

1 4

2 4

3 1

4 1

4 2

Q2 Q4

1 2

1 4

2 1

2 3

3 2

3 4

4 1

4 3 violatedconstraint!

Huffman-ClowesLabelingHuffman-ClowesLabeling

++++

++-

Waltz’s Filtering: Arc-Consistency

Waltz’s Filtering: Arc-Consistency

•Lines: variables

•Conjunctions: constraints

•Initially Di = {+,-, , )

•Repeat until no changes:

Choose edge (variable)

Delete labels on edge not consistent with both endpoints

No labeling!

                                                       

Path Consistency• Path consistency (3-consistency):

–Iterated check of every triple of variables• K-consistency:

• N-consistency: backtrack-free search

1

| | k-tuples to check

Worst case: each iteration eliminates 1 choice

| || | iterations

| || | steps! (But usually not this bad)

k

k

V

D V

D V

Variable and Value Selection

• Select variable with smallest domain– Minimize branching factor– Most likely to propagate, reduce guessing– Most constrained variable heuristic

• Which values to try first?– Most likely value for solution– Forward checking reduces fewest constraints– Least constrained value heuristic

CSP Applications

• Scheduling – NASA: Shuttle repair, Hubble telescope

scheduling, ...– College basketball scheduling

• Configuration– #5ESS Switching System: >200 options,

highly complex interactions– Reduced configuration time from

2 months 2 hours

(Barry and Humblet 93, Cheung et al. 90, Green 92, Kumar et al. 99)

CONFLICT FREELATIN ROUTER

Inp

ut

po

rts

Output ports

3

1

2

4

Input Port Output Port

1

2

43

Router Design

• Dynamic wavelength router design:– each channel cannot be repeated in the same

input port (row constraints);– each channel cannot be repeated in the same

output port (column constraints);

SAT as a CSP

• Domain of all variables is {0, 1}

• Clauses are constraints – specify illegal combinations of values

(A v B v ~C) rules out (A=0,B=0,C=1)A B C

1 0 0

1 0 1

1 1 0

1 1 1

0 1 0

0 1 1

0 0 0

Basic Backtrack Search for SAT

Solve( F ): return Search(F, { });

Search( F, assigned ):if vars(F)=vars(assigned) then

if evaluate(F, assigned) then return assigned;else return FALSE;

choose unassigned variable x;return Search(F, assigned U {x=0}) ||

Search(F, assigned U {x=1});end;

Propagating Constraints

• Suppose formula contains(A v B v ~C)

and we set A=0.

• What is the resulting constraint on the remaining variables B and C?

(B v ~C)

• Suppose instead we set A=1. What is the resulting constraint on B and C?

No constraint

Empty Clauses and Formulas

• Suppose a clause in F is shortened until it become empty. What does this mean about F and the partial assignment?

F cannot be satisfied by any way of completing the assignment; must backtrack

• Suppose all the clauses in F disappear. What does this mean?

F is satisfied by any completion of the partial assignment

Arc Consistency for SAT

• Suppose a clause in F is shortened to contain a single literal, such as

(A)

What should you do?Immediately add the literal to assigned, and repeat constraint propagation

• Arc consistency for SAT is called unit propagation

DPLL

DPLL( F, assigned ):while F has a unit clause (c) do

assigned = assigned U {c};shorten clauses containing ~c;delete clauses containing c;

endif F is empty then return assigned;if F contains an empty clause then return FALSE;choose an unassigned literal c; // variable and initial valuereturn Search(F U { (c) }, assigned) ||

Search(F U { (~c) }, assigned);end;

I.e., ((not x_1) or x_7) ((not x_1) or x_6)

etc.

What is BIG?

x_1, x_2, x_3, etc. our Boolean variables(set to True or False)

Set x_1 to False ??

Consider a real world Boolean Satisfiability (SAT) problem

I.e., (x_177 or x_169 or x_161 or x_153 …x_33 or x_25 or x_17 or x_9 or x_1 or (not x_185))

clauses / constraints are getting more interesting…

10 pages later:

…

Note x_1 …

4000 pages later:

…

Finally, 15,000 pages later:

Current SAT solvers solve this instance in approx. 1 minute!

Search space of truth assignments: HOW?

Progress in SAT Solvers

Instance Posit' 94

ssa2670-136 40,66s

bf1355-638 1805,21s

pret150_25 >3000s

dubois100 >3000s

aim200-2_0-no-1 >3000s

2dlx_..._bug005 >3000s

c6288 >3000s

Grasp' 96

1,2s

0,11s

0,21s

11,85s

0,01s

>3000s

>3000s

Sato' 98

0,95s

0,04s

0,09s

0,08s

0s

>3000s

>3000s

Chaff' 01

0,02s

0,01s

0,01s

0,01s

0s

2,9s

>3000s

Source: Marques Silva 2002

Questions

• How?–Clause learning

• Why?–Phase transition phenomena–Small backdoors

How: Clause Learning

• Idea: backtrack search can repeatedly reach an empty clause (backtrack point) for the same reason

A

B B

C C

Example: Propagation from B=0 and C=0 leads to empty clause

How: Clause Learning

• If reason was remembered, then could avoid having to rediscover it

A

B B

C D

Example: Propagation from B=0 and C=0 leads to empty clause

I had better set C=1

immediately!

How: Clause Learning

• The reason can be remembered by adding a new learned clause to the formula

A

B B

C D

learn (~B V

~C)Example: Propagation from B=0 and C=0 leads to empty clause

Set C=1 by unit

propagation

How: Clause Learning

• Savings can be huge

A

B B

C

Example: Setting B=0 is the root cause of reaching an empty clause

D E

C

D E

How: Clause Learning

• Savings can be huge

A

B C

C

Example: Setting B=0 is the root cause of reaching an empty clause

D E

Have learned unit clause (B)

Determining Reasons

• The reason for a backtrack can be found by (incrementally) constructing a conflict graph of the implications between assumed or derived literals (unit clauses)

• Analyzing the graph yields learned clauses

43

Conflict Graphs

Decision scheme(p q b)

1-UIP schemet

p

q

b

a

x1

x2

x3

y

yfalset

Known Clauses(p q a)

( a b t)(t x1)(t x2)(t x3)

(x1 x2 x3 y)(x2 y)

Current decisionsp falseq falseb true

Scaling Up Clause Learning

• Clause learning greatly enhances the power of unit propagation

• Tradeoff: memory needed for the learned clauses, time needed to check if they cause propagations

• Clever data structures enable modern SAT solvers to manage millions of learned clauses efficiently