Quality of LP-based Approximations for Highly Combinatorial Problems

Lucian Leahu and Carla Gomes

Computer Science DepartmentCornell University

Motivation Increasing interest in combining Constraint Satisfaction Problem

(CSP) formulations and Linear Programming (LP) based techniques for solving hard computational problems.

Successful results for solving problems that are a mixture of linear constraints – where LP excels – and combinatorial constraints – where CSP excels.

In a purely combinatorial setting,

surprisingly difficult to effectively

integrate LP- and CSP-based techniques

Goal

Study and characterize the quality of LP based heuristics

for highly combinatorial problems.

Research Questions

Is the quality of LP-based Approximations related to the structure of the problem? (Typical case, rather than worst case)

How is the quality of LP-based Approximations influenced by different formulations of the problem?

Does the LP relaxation provide a global perspective of the search space? Is the LP relaxation good as a heuristic to guide complete solvers?

Outline

A highly combinatorial search problem --- quasigroup completion problem (QCP)

LP-based formulations for QCP Assignment based formulation Packing formulation

Quality of LP based approximations LP as a global search heuristic Conclusions

Latin Squares or Quasigroups Given an N X N matrix, and given N colors, a

quasigroup of order N is a a colored matrix, such that:

all cells are colored. each color occurs exactly once in each row. each color occurs exactly once in each column.

Quasigroup or Latin Square(Order 4)

Latin Squares/Quasigroups Completion Problem

Given a partial assignment of colors (10 colors in this case), can the partial Latin Square be completed so we obtain a full square?

Latin Squares/Quasigroups Completion Problem

Given a partial assignment of colors (10 colors in this case), can the partial Latin Square be completed so we obtain a full square?

Example:

Structure of this problem characterizes several real-world applications: e.g., Timetabling, sports scheduling, rostering, routing, etc.

Quasigroup with Holes (QWH)Quasigroup with Holes (QWH)

Given a full quasigroup, “punch” holes into it

QWH is NP-Hard.

32% holes

Advantage: we know the optimal value.

LP-based formulations for QCP

Assignment Formulation

Variables -

}1,0{ijkx

....,,2,1,,;, nkjikcolorhasjicellijkx

3n

kijPLStskjikji

x ..,,

1,,

Max number of colored cells

s.t. at most one color per cell:

a color appears at most once per row

a color appears at most once per column

Assignment Formulation

New Phase Transition Phenomenon:Integrality of LP

Note: standard phase transition curves are w.r.t existence of solution)

Sudden phase Transition in solution integrality of LP relaxation

and it coincides with the hardest area

holes/n^1.55

No

of

ba

ck

tra

ck

sM

ax

va

lue

of

LP

Re

lax

ati

on

Packing formulation

Max number of colored cells in the selected patterns

s.t. one pattern per family

a cell is covered at most by one pattern

Families of patterns

(partial patterns are not shown)

Packing formulation

Previous Results 0.5 approximation based on Assignment formulation –

Kumar et al. – 1999

(1-1/e ≈ 0.63) approximation based on Packing formulation – Gomes, Regis, Shmoys – 2003

Use of LP to select variables and values and to prune search trees – Refalo et al. – 1999, 2000

No typical case results on the quality of LP based approximation

Quality of LP-based Approximations

Approximation Schemes

LP Formulations: Assignment formulation; Packing formulation;

Approximation scheme: solve the LP relaxation and interpret the resulting solution as a

probability distribution;

Order for Variable SettingUniformly at RandomGreedy RandomGreedy Deterministic

Increasing greediness

Uniformly at Random

Uniformly at Random

Uniformly at Random

holes/n^1.55 holes/n^1.55

% o

f c

olo

red

ho

les

% o

f c

olo

red

ho

les

Uniformly Random - Comparison

Drop in quality of approximation as we enter the critically constrained area

The quality stabilizes in the under constrained area

Random LP Packing does better, since the corresponding LP relaxation is stronger

Random LP Packing is a 1 – 1/e≈0.63 approximation, while LP assignment ½ approximation.

% o

f c

olo

red

ho

les

holes/n^1.55

Greedy Random

Greedy Random

Greedy Random%

of

co

lore

d h

ole

s

% o

f c

olo

red

ho

les

holes/n^1.55 holes/n^1.55

Greedy Random - Comparison

Drop in quality of approximation as we enter the critically constrained area

The quality increases in the under constrained area --- info provided by LP is used in a more greedy way (more valuable); forward checking also improves quality.

Random LP Packing does slightly worse, since it optimizes an entire matching

% o

f c

olo

red

ho

les

holes/n^1.55

Greedy Deterministic

Greedy Deterministic

Greedy Deterministic - Comparison

Drop in quality of approximation as we enter the critically constrained area

The quality increases in the under constrained area --- info provided by LP is used in a more greedy way and deterministically (more valuable); forward checking also improves quality.

Random LP Packing does slightly worse, since it is less greedy (sets an entire matching), doesn’t use as much lookahead

% o

f c

olo

red

ho

les

holes/n^1.55

Comparison with Pure Random Strategy

% o

f c

olo

red

ho

les

holes/n^1.55

LP as a Global Search Heuristic

Can LP guide complete solvers?

Use an LP relaxation to set a certain percent of variables (the highest values)

Run a complete solver on the resulting instances and check if it is still completable (we start with a PLS that is completable)

LP as a Global Search Heuristic - Results

1 hole

% o

f s

ati

sfi

ab

le i

ns

tan

ce

s

holes/n^1.55

5%

holes/n^1.55

% o

f s

ati

sfi

ab

le i

ns

tan

ce

s

Conclusions

Quality of approximation is directly correlated with phase transition phenomenon – closely related to constrainedness regions of the problem (sharp decrease in the critical region)

New phase transition in the integrality of the LP relaxation solution

Typical case analysis – although theoretical bounds for LP packing are stronger, the empirical results for enhanced versions of approximations (with forward-checking) seem to indicate that LP approximations based on the assignment formulation are better (but difficult to analyze theoretically)

LP can provide useful high level guidance + should be combined with random restart strategies to recover from potential mistakes made at the top of the tree

Quality of LP-based Approximations for Highly Combinatorial Problems

Lucian Leahu and Carla Gomes

Computer Science DepartmentCornell University