SAT 2009 Ashish Sabharwal

Backdoors in the Context of Learning(short paper)

Bistra Dilkina, Carla P. Gomes, Ashish Sabharwal

Cornell University

SAT-09 Conference

Swansea, U.K., June 30, 2009

1

SAT 2009 Ashish Sabharwal

• Boolean Satisfiability or SAT :– Given a Boolean formula F in conjunctive normal form

e.g. F = (a or b) and (¬a or ¬c or d) and (b or c)determine whether F is satisfiable

– NP-complete [note: “worst-case” notion]

– widely used in practice, e.g. in hardware & software verification, design automation, AI planning, …

• Large industrial benchmarks (10K+ vars) are solved within seconds by state-of-the-art complete/systematic SAT solvers

• Even 100K or 1M not completely out of question• Good scaling behavior seems to defy “NP-completeness”!

Real-world problems have tractable sub-structure

“Backdoors” help explain how solvers canget “smart” and solve very large instances

SAT: Gap between theory & practice

2

not quite Horn-SATor 2-SAT…

SAT 2009 Ashish Sabharwal

(~500 vars)

Informally:

A backdoor to a given problem is a subset of its variables such that, once assigned values, the remaining instance simplifies to a tractable class.

Formally:define a notion of a poly-time “sub-solver” handles tractable substructure of problem instance e.g. unit prop., pure literal elimination, CP filtering, LP solver, …

• Weak backdoors for finding feasible solutions

• Strong backdoors for finding feasible solutions or proving unsatisfiability

Backdoors to TractabilityBackdoors to Tractability

3

A notion to capture “hidden structure”

SAT 2009 Ashish Sabharwal

The notion of backdoors has provided powerful insights, leading totechniques like randomization, restarts, and algorithm portfolios for SAT

Domain Instance Vars Clause %Vars in Bgraph coloring gcp 1500 187556 0.43planning map_50_97 38364 438840 0.25game theory pne 5000 98930.79 0.64car configuration C210_FS_RZ 1755 5764.333 0.70car configuration C210_FW_UT 2024 9720 0.74verification ssa0432-003 435 1027 3.91verification bf2670-001 1393 3434 2.80verification bf1355-638 2177 6768 10.66

Are backdoors small in practice?

4

Enough to branch on backdoor variables to “solve” the formula heuristics need to be good on only a few vars

SAT 2009 Ashish Sabharwal

• “Traditional” backdoors are defined for a basic tree-search procedure, such as pure DPLL– Oblivious to the now-standard (and essential) feature of

learning during search, i.e, clause learning for DPLL

• Note: state-of-the-art SAT solvers rely heavily on clause learning, especially for industrial and crafted instances– provably leads to shorter proofs for many unsatisfiable formulas

– significant speed-up on satisfiable formulas as well

Does clause learning allow for smaller backdoorswhen capturing hidden structure in SAT instances?

This Talk: Motivation

5

SAT 2009 Ashish Sabharwal

Affirmative answer:

1. First, must extend the notion of backdoors to clause learning SAT solvers: take ‘order-sensitivity’ into account

2. Theoretically, learning-sensitive backdoors for SAT solvers with clause learning (“CDCL solvers”) can be exponentially smaller than traditional strong backdoors

3. Initial empirical results suggesting that in practice,– More learning-sensitive backdoors than traditional (of a given size)– SAT solvers often find much smaller learning-sensitive backdoors

than traditional ones

This Talk: Contribution

6

SAT 2009 Ashish Sabharwal

Input: CNF formula FAt every search node:

– branch by setting a variable to True or False;current partial variable assignment:

– consider simplified sub-formula F|

– apply a poly-time inference procedure to F|(e.g. unit prop., pure literal test, failed literal test / “probing”) Contradiction learn a conflict clause Solution declare satisfiable and exit

Not solved continue branching

“sub

-sol

ver”

fo

r S

AT

DPLL Search with Clause Learning

7

SAT 2009 Ashish Sabharwal

Traditional Backdoor

Bac

kdoo

r

{Sub-solver

infers solution

x

y

z

w

=1

=0

=1

=1

{B

ackd

oor?

Search Tree to Solution

Contradiction:Conflict clause

learnedEarly contradictiondue to previouslylearned clause

Sub-solver infers solutionwith help from

learned clauses

x

y y

=0 =1

=1=0 =0

Search order matters!Search order matters!

Backdoors and Search with Learning

8

SAT 2009 Ashish Sabharwal

Definition [Williams, Gomes, Selman ’03]:

A subset B of variables is a strong backdoor(for F w.r.t. a sub-solver S) if for every truth assignment to variables in B,

S “solves” F|.

Issue: oblivious to “previously” learned clauses; sub-solver must infer contradiction on F| for every from scratch.

“Traditional” Backdoors

9

either finds a satisfying assignment for For proves that F is unsatisfiable

SAT 2009 Ashish Sabharwal

Definition:

A subset B of variables is a learning-sensitive backdoor(for F w.r.t. a sub-solver S) if there exists a search order s.t. a clause learning solver

– branching only on the variables in B– in this search order– with S as the sub-solver at each leaf

“solves” F.

New: Learning-Sensitive Backdoors

10

either finds a satisfying assignment for For proves that F is unsatisfiable

Theoretical Results

SAT 2009 Ashish Sabharwal

Setup:

• Sub-solver: unit propagation

• Clause learning scheme: 1-UIP

• Comparison w.r.t. traditional strong backdoors

Theorem 1: There are unsatisfiable SAT instances for which learning-sensitive backdoors are exponentially smaller than the smallest traditional strong backdoors.

Theorem 2: There are satisfiable SAT instances for which learning-sensitive backdoors are smaller than the smallest traditional strong backdoors.

Learning-Sensitive Backdoors Can Provably be Much Smaller

12

used Rsat for experiments

SAT 2009 Ashish Sabharwal

Proof Idea: Simple Example

13

{x} is a learning-sensitive backdoor (of size 1) :

x=0

p1

p2

qa b

contradiction

Learn 1-UIP clause:(q)

x=1a b

contradictionq

r

With clause learning, branching on xin the right order suffices to prove unsatisfiability(x appears only

in a “long” clause)

SAT 2009 Ashish Sabharwal

Proof Idea: Simple Example

14

In contrast, without clause learning, must branch onat least 2 variables in every proof of unsatisfiability! every “traditional” strong backdoor is of size ≥ 2

Why?•every variable, in at least one polarity, only in “long” clausese.g., p1, q, r, a do not appear in any 2-clauses

•therefore, no unit prop. or empty clause generation by fixing this variable to this value•therefore, this variable by itself cannot be a strong backdoor

SAT 2009 Ashish Sabharwal

Construct an unsatisfiable formula F on n vars. such that

1. certain long clauses must be used in every refutation(i.e., removing a long clause makes F satisfiable)

2. many variables in at least one polarity appear only in such long clauses with (n) variables Controlled unit propagation / empty clause generation Must branch on essentially all variables of the long clauses to

derive a contradiction Such variables must be part of every traditional backdoor set

3. With learning: conflict clauses from previous branches on O(log n) “key variables” enable unit prop. in long clauses

Proof Idea: Exponential Separation

15

SAT 2009 Ashish Sabharwal

Corollary (follows from the proof of Theorem 1) :

There are unsatisfiable SAT instances for which learning-sensitive backdoors w.r.t. one value ordering are exponentially smaller than the smallest learning-sensitive backdoors w.r.t. another value ordering.

Order-Sensitivity of Backdoors

16

Experimental evaluation

SAT 2009 Ashish Sabharwal

Learning-Sensitive Backdoors in Practice

18

Preliminary evaluation of smallest backdoor size Reporting “best found” backdoors over 5000 runs of Rsat (with clause learning) or Satz-rand (no learning) :

•up to 10x smaller than traditional on satisfiable instances•often 2x or less smaller than traditional on unsatisfiable instances

SAT 2009 Ashish Sabharwal

• Considering only the size of the smallest backdoor does not provide much insight into this question

• One way to assess this difficulty:– How many backdoors are there of a given cardinality?

• Experimental setup:– For each possible backdoor size k, sample uniformly at random

subsets of cardinality k from the (discrete) variables of the problem

– For each subset, evaluate whether it is a backdoor or not

How hard is it to find small backdoor sets with learning?

19

Recently reported in a paper at CPAIOR-09(backdoors in the context of optimization problems)

SAT 2009 Ashish Sabharwal

Backdoor Size Distribution

20

E.g., for a Mixed Integer Programming (MIP)optimization instance:

SAT 2009 Ashish Sabharwal

Added Power of Learning

21

E.g., for a Mixed Integer Programming (MIP)optimization instance:

SAT 2009 Ashish Sabharwal

• Defined backdoors in the context of learning during search (in particular, clause learning for SAT solvers)

• Proved that learning-sensitive backdoors can be smaller than traditional strong backdoors– Exponentially smaller on unsatisfiable instances– Somewhat smaller on satisfiable instances (open?)

• Branching order affects backdoor size as well

Future work: stronger separation for satisfiable instances; detailed empirical study

Summary

22