1

On the Integration of Constraint Propagation

and Local Search

Pedro Barahona

Department of Computer ScienceFaculty of Science and Technology

New University of Lisbon

2

Summary

• Constraint Solvers for CSP and CSOP

– Pure Methods (Propagation, Local Search, ...)

– Hybrid Methods

• Integration of Propagation & Local Search

– Digital Circuits Testing

– Handling Continuous Domains

– Determination of Protein Structure

• Conclusions

3

CSP and CSOP

• CSP: Contraint Satisfaction Problems

Given variables V1 to Vn, find values in their respective domains D1 ... Dn that satisfy Constraints C1 to Cp.

Constraints Ci D1 x D2 x ... x Dn

• CSOP: Contraint Sat & Optimisation Problems

Given variables V1 to Vn, find values in their respective domains D1 ... Dn that satisfy Constraints C1 to Cp and, additionally, optimise some objective function F

Objective Function F: D1 x D2 x ... X Dn R

4

Constraint Solving Methods

• Algebraic Approach

– Available in some specific domains

– Examples: Boolean, Rational Numbers

• Search Methods

– General Purpose

– Finite Domains, Integers, Real Numbers, Intervals, Sets, Strings, ...

• Variety of Search Techniques

– Constructive vs. Repairing

5

Constructive Methods

• General Idea

– Iterative narrowing of the possible values of the variables within their domains, until a completely characterised solution is found

• Different Techniques for Narrowing– Assign Values to Variables (Finite Domains / Satisf.)

– Split Current Domains (Continuous Domains / Optim.)

• Key Issues– Early detection of dead ends

– Efficient Backtracking

6

Repairing Methods

• General Idea

– Starting with a completely characterised solution, somehow unsatisfactory, search for a better solution in its neighbourhood.

• Key issues

– Selection of neighbour

– Escaping from local optima

– Search the whole solution space

– Stopping criteria

7

Comparison of Methods

• Constructive Methods– Complete

– Inherently satisfaction methods• Optimisation considered in heuristics and deadends

(BB)

• More adequate for few and sparse feasible solutions

• Repairing Methods– Incomplete

– Adaptation of solutions from similar problems

– Inherently optimisation methods• Satisfaction encoded in the optimising function

• More adequate for many feasible solutions

8

• Complete Search: Backtracking

– Efficient Backtracking: dependency directed

Constructive Methods

1 3 4 2 5 4 5 3 5 1 2 3

•Analyse dependencies

•Backtrack in non-chronological order

•“greatest of the least”

– Inneficiency: late detection of deadends

9

Constructive Methods

– What kind of propagation?

1 1

1

1

1

1

11

1

1

1

1

1

12

2

2

2

2

2

2

2

2

2

2

3

3

3

3

3

3

4

3

4

4

44

•Propagate “Conflicts”

•Avoid irrelevant backtracking

• Complete Search: Backtracking + Propagation

– Early detection of deadends - propagation

10

Constructive Methods

What kind of propagation

– Trade-of: pruning vs. complexity 

Backtracks Prunings RunTime (ms)N Queens 32 64 96 32 64 96 32 64 96

Node Consistency

11 382 503 609 9951 13594 30 200 370

Arc Consistency

8 221 255 588 7729 10345 701 10875 52707

Controlled Arc-Consistency

8 221 255 588 7729 10345 50 531 1252

11

Constructive Methods

• Difficulty with Backtracking

– Typically Chronological

– Costly backtracking of early errors

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

0 1 0 1 0 1 0 1

0 1 0 1

0 1X1

X2

X3

X4

X

12

Repairing Methods

• Even treatment of all variables

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

0 1 0 1 0 1 0 1

0 1 0 1

0 1X1

X2

X3

X4

0010

X1

1010x

X2

1110x

13

Repairing Methods

• Satisfaction via Optimization

• Modeling issue: Definition of Neighbourhood

Objective Function = 3

# Violations (attacks)

Neighbourhood:

Permutations

14

Repairing Methods• Difficulty - Escaping from Local Optima

• Restarts / Stochastic methods

2 3 1 3 0

15

Pure vs. Hybrid Approaches• Compete or Cooperate

• Many Examples of Cooperation

– Operations Research

– Constraint Programming

• Examples from Operations Research

– Integer Programming with Linear Programming

• LP provides Bounds for IP

– Nonlinear and Linear Programming

• Stepwise Linearization

16

Pure vs. Hybrid Approaches• Constraint Programming

– Global Constraints improve Propagation

• Graph Theory / OR Methods with Propagation

– Integer Programming and Propagation

• Cutting Planes (CPLEX and ECLiPse)

• Benders Decomposition

– Continuous Domains

• Interval Arithmetic (Newton)

• Quadratic Optimization

– Propagation and Local Search

17

Experiments with CP and LS

• Optimisation of Test Patterns (Digital Circuits)

– Francisco Azevedo

• Global-Hull Consistency (Continuous Domains)

– Jorge Cruz

• Determination of Protein Structure (NMR Data)

– Ludwig Krippahl

Interaction of Constraint Programming

and Local Searchfor Optimisation Problems

Francisco Azevedo & Pedro Barahona

19

Introduction

• Scope: Optimisation problems in the field of Automatic Test Pattern Generation (ATPG)

• Goal: For a given stuck-at fault, find a test (an input vector) with the maximum number of unspecified bits.

• Approach: Integration of branch-and-bound CLP with Local Search (in a cycle CP <-> LS) with multi-valued logics

• Key feature - Different encodings in CP and LS.

20

CP Modeling: 5-valued Logic

• 5 valued logic to encode dependency/ignorance– Values D/ D encode dependency on a

fault– Value X encodes “don’t know” situations

G s-a-011

D (1/0)

G s-a-010

0

Input 0 1 D D x S-buffer/0 output 0 D D 0 x S-buffer/1 output D 1 1 D x

• Encoding “stuck-at” faults (S-buffers)

G s-a-1

010

D (0/1)

21

CP Modeling: 5-valued Logic

_• Goal: An output D/D that maximises # X inputs

• Encoding normal behaviour in 5-valued Logic

D D

NOT0 11 0D D

D Dx x

AND 0 1 D D x

0 0 0 0 0 01 0 1 D D xD 0 D D 0 xD 0 D 0 D xx 0 x x x x

X0

0

DX

X

CombinationalCircuit

DPIs POs

D

maximise #{i PI: i = x}

x

22

Incompleteness of CP Modeling

• 5 Valued Logic does not detect the fault

• Test ab=x0 detects b s-a-1

x

0/1

_D

x x

x

x

x

0

0/1 0/1

0/0 1/1 0/1

0/0

0/1

x

1

0/1 0/1

1/1 0/0 0/0

0/1

0/1

x

23

Improving Completeness (1)

• Naming unspecified values, including inversion parity, improves detection of faults

NOT AND 0 1 id-0 id-1 XOR 0 1 id-0 id-10 1 0 0 0 0 0 0 0 1 id-0 id-11 0 1 0 1 id-0 id-1 1 1 0 id-1 id-0

id-0 id-1 id-0 0 id-0 id-0 0 id-0 id-0 id-1 0 1id-1 id-0 id-1 0 id-1 0 id-1 id-1 id-1 id-0 1 0

a-0

0 0 0

a-1 00

a-0

0 /1 1 a-0

a-1 a-11

(a) (b)

x

24

Improving Completeness (2)

• Modelling still incomplete• final output should have value 1

• Multiple Sources of Unspecified Values

Id G-0Id A -p A

Id B -p B

Id G

A B Z=A.B0 Arg 01 Arg Arg

Arg Arg Argid-0 id-1 0idA-pA idB-pB idZ-0

a-0

0 /1 1b-0

a-1 a-1w-0

x-0

y-0a

a-0

0 00

a-1 00

x-0

y-0a

b b-0 b-0b

w

x

25

Integration CP and LS

• Constraint Propagation (weaker then full arc-consistency) finds solutions efficiently

• (Still) Incomplete CP Model

• Maximisation in usual Branch and Bound would still be incomplete

• CP solutions as seeds for LS optimisation

• LS uses a complete logic

• Interesting neighbours heuristically detected with such logic

26

Annotations in Logic for LS

• Complete LS Model

• Heuristic for “good” neighbours

• Dependency on (single) unspecified values

{}:000

(a)

xy

z{x/ x-0}:00

1

(b)

z {x/ x-0, y/ y-0}:111

(c)

z

000

(a)

xy

z0

01

(b)

z1

11

(c)

z

27

Annotations in Logic for LS

• Simulation

/1

0a= x a-0

1

b=0

a-0

00

0

0

a-0

a-1a-1

1 0

a-1

/1

{b/b-0}:0a=0 {a/a-0}:0

{}:1

b=0

{a/a-0}:0

{b/b-0}:0{b/b-0}:0

{b/b-0}:0

{b/b-0}:0

{a/a-0}:0

{a/a-1}:1{a/a-1}:1

{}:1{}:0

{a/a-1}:1

0/1 output does not depend on inputa , which may flip to x

• Example

• Local Search (over original test ab=00)

28

Experimental Results (1)

• Evaluation of Maxx on ISCAS circuitsc17 c432 c499 c880 c1355 c1908 c2670 c3540 c5315 c6288 c7552

PI 5 36 41 60 41 33 233 50 178 32 207G 6 160 202 383 546 880 1193 1669 2307 2416 3512F 22 524 758 942 1574 1878 2746 3425 5350 7744 7550

Atalanta MTP MAXX%X %X Gain T/f %X Gain T/f Diff

c432 56.2 60.8 10.5 3.21 71.4 34.7 7.16 24.2c499 17.1 18.7 1.9 4.35 25.6 10.3 7.95 8.3c880 82.2 83.8 9.0 2.54 85.2 16.9 6.08 7.9c1355 13.3 13.7 0.5 9.12 20.0 7.7 7.93 7.3c1908 44.7 48.4 6.7 9.61 52.8 14.6 7.51 8.0c2670 92.0 92.4 5.0 10.99 94.1 26.2 5.90 21.2c3540 74.6 77.3 10.6 16.81 79.2 18.1 7.07 7.5c5315 92.6 92.9 4.1 9.34 93.6 13.5 7.53 9.5c6288 22.2 22.2 0.0 36.65 28.4 8.0 12.51 8.0c7552 86.9 86.9 0.0 17.46 90.9 30.5 10.17 30.5

• MTP100 and Maxx improvements over Atalanta

29

Experimental Results (2)

• Breakdown of Maxx PerformanceMTP LS1 CP1 LS2 CP2 LS3 CP3 LS4 CP4 LS5

c6288 52,874 10,749 2,126 303c7552 1,334,794 56,052 5,630 628 22 29 26 8 2 2

• MTP1000 and Maxx improvements over MTP100

MTP100 MTP1000 MAXX %X %X Gain T/f %X Gain T/f Diff

c432 60.8 64.1 8.4 23.83 71.3 26.7 7.03 18.2 c499 18.7 19.5 1.0 29.36 26.3 9.3 7.08 8.3 c880 83.8 85.6 11.1 19.80 86.1 14.3 6.00 3.1 c1355 13.7 15.2 1.7 55.74 16.5 3.2 7.44 1.5 c1908 48.4 51.0 5.0 63.83 57.5 17.6 7.50 12.6 c2670 92.4 93.0 7.9 72.47 94.2 23.3 5.57 15.4 c3540 77.3 n/a n/a n/a 80.6 14.5 6.95 n/a c5315 92.9 n/a n/a n/a 93.8 13.1 7.49 n/a c6288 21.4 n/a n/a n/a 26.8 6.8 12.34 n/a c7552 86.9 n/a n/a n/a 91.0 31.1 9.61 n/a

30

Maintaining Global Hull Consistency with

Local Search for Continuous CSPs

Jorge Cruz & Pedro Barahona

31

Introduction

• Scope: Continuous Constraint Satisfaction Problems for Decision Support

• Goal: Find an enclosure of the solution space where a decisor may look for “good” solutions.

• Approach: Maintaining Global Hull Consistency as the basic Constraint Propagation Method

• Key Feature: Local Search in Enforcement Algorithms

32

Under-Constrained CCSPs

• Representation of Continuous Domains

F-interval R

F

[r1..r2]

[f1 .. f2]

r

[r..r]

F-box

Canonical solution

33

Solving CCSPs

• Branch and Prune algorithms

• Prune: Constraint Propagation

• Branch: Isolate Solutions

34

• Based on Interval Arithmetic

• Hull-consistency

• Decomposes constraints into sets of primitive constraints

x3+xy+y5=3 { Z1= X3 ; Z2= XY ; Z3= Y5 ; Z1+Z2+Z3 =3 }

• Enforces consistency on the bounds of each domain

• Box-consistency

• Does not decompose constraints

• Enforces consistency on domain bounds combining the Interval Newton method with spliting

F(x,y) = x3+xy+y5-3=0 Xn = xm - F(xm, Y) Xn-

1

F’ (Xn-

1,Y)

Basic CP Methods for CCSPs

35

Higher Order Consistency

• Basic Methods are often Insufficient

• Higher Order Consistency Criteria

• 3B- consistency, nB consistency (Hull)• if n-2 bounds are fixed then the problem is Hull-

consistent

• Bound-consistency, nBound consistency (Box)

• if n-2 bounds are fixed then the problem is Box-consistent

• However, these higher order methods may still be insufficient

• Our Approach: Global Hull Consistency (GH)

36

Global Hull Consistency

Each bound of the domain of each variable is justified by a canonical solution for the global set of constraints.

• Decision Support Context (“many” solutions)

• Requirement: search for extreme solutions

37

Hull Consistency Enforcing Algorithms

• Several Algorithms were tried

• OS1, OS2: Independent search for each bound of each variable

• OS3: Alternate search for the bounds of all variables

• TSA: Alternate search keeping some tree search structure of the domains

• Common goal: search for canonical solutions

• Alternatives: Pure splitting or local search

38

Local Search in GH Enforcing

• Find a canonical solution within a box

canonical solution ?

x

y

upper bound of x

minimization

initial point

Newton’s vector

new point

Newton’s vector

minimization

new point

Stops when:convergesfind a solution

39

Hull Consistency Enforcing Algorithms

Newton’s Vector

f(x,y)=0

g(x,y)=0

h(x,y)=0

Constraints:

X

F

Current Point : X0 F(X0)0

goal: X F(X0+ X)=0

Newton’s vector

By the Taylor expansion (ignoring higher order terms):

F(X0+ X)= F(X0)+J·X

Jacobian0

X= - J-1 F(X0)

Singular Value Decomposition

40

Experimental Results (1)

• The need for Global Hull (toy examples):

x2 + y2 1

x2 + y2 2

Box-consistency

3B-consistency

Global Hull-consistencyØ

x2 + y2 + z2 2

x2 + y2 + z2 3Global Hull-consistency

4B-consistencyØ

Box-consistencyr=1.415

3B-consistencyr=1

41

Experimental Results (2)

• Determination of Protein Structure (6 atoms) 2B 3B GH

x4 -1.001 .. 1.415 -0.056 .. 0.049 -0.004 .. 0.004

y4 0.585 .. 3.001 1.942 .. 2.047 1.996 .. 2.004

z4 -1.415 .. 1.416 -1.415 .. 1.415 -1.415 .. 1.415

x5 -0.415 .. 2.001 0.998 .. 1.002 0.999 .. 1.001

y5 -0.001 .. 2.001 0.999 .. 1.001 0.999 .. 1.001

z5 -1.415 .. 1.416 -1.415 .. 1.415 -1.415 .. 1.415

x6 -2.001 .. 2.001 -1.110 .. 1.053 -1.008 .. -0.992

y6 -0.001 .. 2.001 -0.894 .. 1.169 0.999 .. 1.001

z6 -2.001 .. 2.001 -1.483 .. 1.483 -1.420 .. 1.402

t (ms) 10 7 380 62 540

1 in << 7 secs

42

Experimental Results (2)

• How useful is Local Search ?

Time (s) Max Storage (F-boxes)

k LS =10-1 =10-2 =10-3 =10-1 =10-2 =10-3

2 n 2.99 11.37 600+ 30 47 60

OS2 y 14.97 38.46 600+ 14 16 24

3 n 41.01 150.21 359.69 22 37 53

y 19.14 75.98 234.65 3 8 25

2 n 10.16 60.96 600+ 332 6786 76506

TSA y 44.79 47.59 600+ 221 621 16764

3 n 42.52 184.38 600+ 52 207 843

y 38.63 97.88 275.85 99 185 392

43

PSICO

Combining Constraint Programming and Optimisation to Solve Macromolecular

Structures

Ludwig Krippahl & Pedro Barahona

44

Introduction

• Scope: Fast Interpretation of data collected in Nuclear Magnetic Resonance (NMR) experiments.

• Goal: Determine Protein Structure from inter-atomic distances provided by NMR data.

• Approach: Combining CP with local search optimisation (in continuous domains)

• Key Feature: CP and Local Search acting on different models

45

Motivation (1)

• Protein (amino acid) sequence, coded in the genes, determines its structure (shape) which, in turn, determines much of its functionality

Catalysis Signaling

RegulationMotion

Function

Genes

Structure

46

Motivation (2)

• NMR avoids crystalisation and other costly procedures, but requires fast data post-processing (to correct interpretation errors)

What we want

What NMR provides

C

Glycin Leucin

Tyrosine

N

OH

W Y L L H W S T

Basic Knowledge

47

Overall Method

• Phase 1: Establish Distance Constraints

• From Basic Knowledge

• From NMR Data

In: | Xi - Xj | Uij Out: | Xi - Xj | Lij

• Phase 2: Approximate Solution

• Constraint Propagation

• Phase 3: Repair Solution

• Local Search

48

Phase 1: Domain Modeling

Domains modeled as 3-D boxes

Good

NoGoodDistance

Out

In

x1 , y1

x

y x2 , y2

a b c

Modeling allows “arc-consistency”

stronger than “bounds-

consistency”

49

Phase 2: Constraint Propagation

Good d

Max

Min

Intersection

In Constraint Out Constraint

d

d

Exclusion Zone

Good

GoodNoGood

Exclusion Zone

NoGood

50

• Change representation to torsion angles (i.e the chemical bond allow some rotation)

Phase 3: Local Search (Optimisation)

• Fit approximate solution from CP to the torsion angle model

• Optimise overall constraint violation (Conjugate Gradient Method)

51

• Fit CP solution to the torsion angle model

Phase 3: Local Search (Optimisation)

n

jjj ppF

1

*)(min

p* are CP positions and p() those determined by torsion angles

1

1 1,

1

)(NA

j

NA

jiij

NC

jj ACGmin

Cj() are penalties for violation of distance constraints, and Ajj() penalties associated to implicit atomic “collisions”.

• Optimise overall constraint violation (via Conjugate Gradient Method)

52

•Experimental Data (Desulforedoxin dimer, …)

Experimental Results

DYANA

Time: 10 hours

Solutions: 1Å

PSICO•Time: 9 min

•1 min (CP) •8 min (15 LS solutions)

•Solutions RMSD: 2.3 Å•Constraints Violated:

15%

• 520 atoms 8000 constraints where

•> 800 provided from NMR

•> 7000 from amino acid knowledge

53

Conclusions• Several Methods to solve CSP and CSOP

•Constraint Propagation

•Local Search

• Opportunities for Cooperation but...

•Careful Modeling

•Seek advantages from complementary nature

• Adequacy to satisfaction /optimisation

• Ease of modeling