Modelling with Finite Domains

Domains and Labelling Complex Constraints Labelling Different Problem Modellings Efficiency Using SICStus Prolog

Smuggler’s Knapsack Problem

Smuggler with knapsack with capacity 9, who needs to choose items to smuggle to make profit at least 30

object profit size

whiskey 15 4

perfume 10 3

cigarettes 7 2

4 3 2 9 15 10 7 30W P C W P C+ + ≤ ∧ + + ≥

Simple Example

Goal for the smugglers knapsack problem:

[W,P,C] :: [0..9],

4*W + 3*P + 2*C ≤ 9,

15*W + 10*P + 7*C ≥ 30.

The CLP(FD) system returns unknown.

But how do we find a solution?

Invoke a complete (backtracking) solver

D W D P D C( ) [ .. ], ( ) [ .. ], ( ) [ .. ]= = =0 2 0 3 0 4

give variable domains

Labelling

Built-in predicate labeling invokes the complete constraint solver

labeling(Vs) takes a list of finite domain variables Vs and finds a solution

[W,P,C] :: [0..9], 4*W + 3*P + 2*C ≤ 9,15*W + 10*P + 7*C ≥ 30, labeling([W,P,C]).

has solutions:

W P C W P C

W P C W P C

= ∧ = ∧ = = ∧ = ∧ == ∧ = ∧ = = ∧ = ∧ =0 1 3 0 3 01 1 1 2 0 0

Constrain and Generate

Typical form of a finite domain program

Define variables and domains

Add constraints to model the problem

Call labelling to invoke complete solver

Modelling 4-Queens

Write a CLP(FD) program queens(Q1,Q2,Q3,Q4) to solve the 4-Queens Problem

Hint: Use all_different_neq.

Q1 Q2 Q3 Q4

1

2

3

4

4-Queens (Cont.)

queens(Q1,Q2,Q3,Q4) :-[Q1,Q2,Q3,Q4] :: [1..4],all_different_neq(Q1,Q2,Q3,Q4), all_different_neq(Q1,Q2+1,Q3+2,Q4+3),all_different_neq(Q1,Q2-1,Q3-2,Q4-3),labeling([Q1,Q2,Q3,Q4] ).

Labelling

Most CLP(FD) goals end with a call to labelling which performs a backtracking search.

Typical programs spend most of their time in labelling so it is important that it is efficient.

One great strength of CLP is that the programmer can write their own labelling predicate which takes advantage of the problem structure. I.e. they can program their own search strategy.

To help CLP(FD) provides the built-in predicates dom(V,D) list of values D in current domain V maxdomain(V,M) current maximum value M of V mindomain(V,M) current minimum value M of V

Labelling

Using these it is straightforward to program the built-in labelling predicate.

labeling iterates through each variable V in the list of variables to be labelled, calling indomain(V) to try each of the (remaining) values in the domain of V.

labeling([]).labeling([V|Vs]) :- indomain(V), labeling(Vs).

indomain(V) :- dom(V,D),

member(V,D).

try values for V

try values in domain

Send+More=Money Example

S E N D

M O R E

M O N E Y

+=

Cryptarithmetic problem, each digit is different and the equation holds

smm(S,E,N,D,M,O,R,Y) :-[S,E,N,D,M,O,R,Y] :: [0..9],constrain(S,E,N,D,M,O,R,Y),labeling([S,E,N,D,M,O,R,Y]).

constrain(S,E,N,D,M,O,R,Y) :-S ≠ 0, M ≠ 0, all_different_neq([S,E,N,D,M,O,R,Y]), 1000*S + 100*E + 10*N + D+ 1000*M + 100*O + 10*R + E= 10000*M + 1000*O + 100*R + 10*E + Y.

Send+More=Money Example (Cont.)

After domain declarations:

D S D E D N D D D M D O D R D Y( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) [ .. ]= = = = = = = = 0 9

After then all_different_neq adds disequations (no change)final equation: (one propagation rule)

S M≠ ∧ ≠0 0 D S D M( ) ( ) [ .. ]= = 1 9

MD S D E D D D R

D O D N D Y≤

+ + +− − −

19

919000

19000

1900

110

1100

19000

(max , ) (max , ) (max , ) (max , )(min , ) (min , ) (min , )

With the current domain:

M ≤ + + + − − − =×99

91 99000

99000

9900

010

0100

09000 1102.

Send+More=Money Example (Cont.)

Hence D(M) = [1..1]Propagation continues arriving at

D S D E D N D D

D M D O D R D Y

( ) [ .. ] ( ) [ .. ] ( ) [5.. ] ( ) [ .. ]

( ) [ .. ] ( ) [ .. ] ( ) [ .. ] ( ) [ .. ]

= = = == = = =9 9 4 7 8 2 81 1 0 0 2 8 2 9

Note: 3 variables are fixed, all domains reduced

Next labelling is called...

Send+More=Money Example (Cont.)

The call to labelling([S,E,N,D,M,O,R,Y]) Calls indomain(S) which calls member(S,[9])

-- which succeeds adding the constraint S=9 Calls indomain(S) which calls member(E,[4,5,6,7])

-- tries adding E=4. Consistency checks lead to failure

-- tries adding E=5. This succeeds giving

Calls indomain(N) which calls member(N,[6]). This adds N=6. Calls indomain(D) which calls member(D,[7]). This adds D=7. Etc..

D S D E D N D D

D M D O D R D Y

( ) [ .. ] ( ) [ .. ] ( ) [5.. ] ( ) [ .. ]

( ) [ .. ] ( ) [ .. ] ( ) [ .. ] ( ) [ .. ]

= = = == = = =9 9 4 7 8 2 81 1 0 0 2 8 2 9

D S D E D N D D

D M D O D R D Y

( ) [ .. ] ( ) [5.. ] ( ) [ .. ] ( ) [ .. ]

( ) [ .. ] ( ) [ .. ] ( ) [8.. ] ( ) [ .. ]

= = = == = = =9 9 5 6 6 7 71 1 0 0 8 2 2

Generate and Test

Methodology without constraints:

first generate a valuation and

then test if it is a solution

smm(S,E,N,D,M,O,R,Y) :-[S,E,N,D,M,O,R,Y] :: [0..9],labeling([S,E,N,D,M,O,R,Y]),constrain(S,E,N,D,M,O,R,Y).

This program requires 95671082 choices before finding the solution, the previous required 2!

Labelling

There are two choices made in labelling which variable to label which value in the domain to try

Default labelling try variables in order of the given list try value in order min to max (returned by dom)

We can program different strategies. These can lead to dramatic performance improvement.

First Fail Labelling

One useful heuristic is the first-fail principle

“To succeed, try first where you are most likely to fail”

labellingff(Vars) :-

if Vars = [] then return

else choose V Vars with smallest domain

call indomain(V)

compute Rest = Vars \{V}

call labellingff(Rest)

First-Fail Labelling

labelingff([]).labelingff(Vs) :- deleteff(Vs,V,R),

indomain(V), labelingff(R).

deleteff([V0|Vs],V,R) :- getsize(V0,S), minsize(Vs,S,V0,V,R)).

minsize([],_,V,V,[]).minsize([V1|Vs],S0,V0,V,[V0|R]) :-

getsize(V1,S1), S1 < S0,minsize(Vs,S1,V1,V,R).

minsize([V1|Vs],S0,V0,V,[V1|R]) :-getsize(V1,S1), S1 ≥ S0,minsize(Vs,S0,V0,V,R).

Choice of Domain Value

Value choice only effects order in which branches are explored

Problem specific knowledge may lead to finding a solution faster

N-Queens Example

Q1 Q2 Q3 Q4

1

2

3

4

Problem of placing N queens on an NxN chessboard so that none can take another.

For large N solutions are more likely with values in the middle of variable domains

Middle-Out Ordering

indomain_mid(V) :- ordervalues(V,L), member(V,L).

ordervalues(V,L) :-dom(V,D), length(D,N), H = N div 2,halvelist(H,D,[],F,S), merge(S,F,L).

halvelist(0,L2,L1,L1,L2).halvelist(N,[E|R],L0,L1,L2) :-

N ≥ 1, N1 = N - 1,halvelist(N1,R,[E|L0],L1,L2).

merge([],L,L).merge([X|L1],L2,[X|L3]) :- merge(L2,L1,L3).

Labelling Efficiency

We can use both variable and value ordering together

Different number of choices for N-queens in different ranges

Efficiency

Apart from programming labelling we can do a number of other things to reduce the search space:

Order rules and literals in rule bodies correctly. Add redundant constraints. Use complex constraints such as all_different since they

have better propagation behaviour. Use reified constraints. Try different models for the problem. Extend the constraint solver to provide a problem specific

complex primitive constraint. Use incomplete heuristic techniques.

Reified Constraints

A reified constraint c B holds if the Boolean variable B reflects the truth of constraint c.I.e. B=1 if c holds and B=0 if ¬c.

Write a constraint apart(F1,F2,K12) which holds if F1 and F2 are at least K12 apart.

Better to use reified constraint -- choice is delayed until labelling

apart(F1,F2,K12) :- F1 ≥ F2+K12.apart(F1,F2,K12) :- F2 ≥ F1+K12.

apart(F1,F2,K12) :-[B1,B2] :: [0,1],(F1 ≥ F2+K12 B1),(F2 ≥ F1+K12 B2),1 ≤ B1 + B2.

Different Problem Modellings

Different views of the problem lead to different models Depending on solver capabilities one model may require

less search to find answer Look for model with fewer variables Look for more direct mapping to primitive constraints. Empirical comparison may be worthwhile

Different Problem Modellings

Simple assignment problem:

• four workers w1,w2,w3,w4 and • four products p1,p2,p3,p4. • Assign workers to products to make profit ≥ 19

p p p p

w

w

w

w

1 2 3 4

1 7 1 3 4

2 8 2 5 1

3 4 3 7 2

4 3 1 6 3

Profit matrix is:

Operations Research Model

p p p p

w

w

w

w

1 2 3 4

1 7 1 3 4

2 8 2 5 1

3 4 3 7 2

4 3 1 6 3

16 Boolean variables Bij meaning worker i is assigned product j

∧∧

= =

= =

∑∑

=

=

=

+ + ++ + + ++ + + ++ + + +

≥

i ijj

j ijj

B

B

P

B B B B

B B B B

B B B B

B B B B

P

14

1

4

14

1

4

11 12 13 14

21 22 23 24

31 32 33 34

41 42 43 44

1

1

7 3 4

8 2 5

4 3 7 2

3 6 3

19

11 prim. constraints

28 choices to find all four solutions

B23

Better Model

Make use of disequality and complex constraints. Four variables W1,W2,W3,W4 corresponding to workers

p p p p

w

w

w

w

1 2 3 4

1 7 1 3 4

2 8 2 5 1

3 4 3 7 2

4 3 1 6 3

alldifferent W W W W

element W WP

element W WP

element W WP

element W WP

P WP WP WP WP

P

([ , , , ])

( ,[ , , , ], )

( ,[8, , , ], )

( ,[ , , , ], )

( ,[ , , , ], )

1 2 3 4

1 7 1 3 4 1

2 2 5 1 2

3 4 3 7 2 3

4 3 1 6 3 4

1 2 3 4

19

∧∧∧∧∧

= + + + ∧≥

7 prim. constraints

14 choices to find all four solutions

W1

W2

W3

W4

1 2 3 4

Different Model

Four variables T1,T2,T3,T4 corresponding to products.

p p p p

w

w

w

w

1 2 3 4

1 7 1 3 4

2 8 2 5 1

3 4 3 7 2

4 3 1 6 3

alldifferent T T T T

element T TP

element T TP

element T TP

element T TP

P TP TP TP TP

P

([ , , , ])

( ,[ , , , ], )

( ,[ , , , ], )

( ,[ , , , ], )

( ,[ , , , ], )

1 2 3 4

1 7 8 4 3 1

2 1 2 31 2

3 3 5 7 6 3

4 4 1 2 3 4

1 2 3 4

19

∧∧∧∧∧

= + + + ∧≥

7 prim. constraints

7 choices to find all four solutions

1

2

3

4

T1 T2 T3 T4

Optimisation

So far we have focussed on satisfaction: finding a solution. CLP can also be used for optimisation. We use the built in constraint minimize(G,F). This finds the

solution to goal G which minimizes the objective function F. Usually a variant of the backtracking integer optimizer is used.

Optimisation

Consider the Smuggler’s Knapsack Problem

[W,P,C] :: [0..9], 4*W + 3*P + 2*C <= 9,

15*W + 10*P + 7*C >= 30,

minimize(labeling([W,P,C]), -15*W-10*P-7*C).

The answer is

W=1 P=1 C=1 L=-32.

Branch & bound (rept.)

1 2 3 4 5 6

2

4

6

8

1 2 3 4 5 6

2

4

6

8

maxz=8x+5y

such that x+y≤6

9x+5y≤45

x,y≥0, integer

Let P be a linear integer program. To solve P

1. solve Ps linear relaxation P ’.2a. If the solution of P ’ is integer-valued,

it is the optimum of P2b. If the solution contains a fractional

variable v create two subproblems:

3. solve the subproblems recursively. 4. The solution to P

is the better solution of P ’a,P ’b

P'a: P∧xi ≤ v⎣ ⎦P'b: P∧xi ≥ v⎡ ⎤

optimum: z=1654

x=154, y=9

4

1 2 3 4 5 6

2

4

6

8

10

1 2 3 4 5 6

2

4

6

8

10

Branch & bound (rept.)

1 2 3 4 5 6

2

4

6

8

1 2 3 4 5 6

2

4

6

8 maxz=8x+5y

such thatx+y≤6

9x+5y≤45

x,y≥0

optimum: z=1654

x=154, y=9

4

x≤3 x≥4 maxz=8x+5y

such thatx+y≤69x+5y≤45

x≥4 (*)x,y≥0

optimum: z=4

x=4, y=95

2 4 6 8 10

2

4

6

8

10

2 4 6 8 10

2

4

6

8

10

y≥2

y≤1

maxz=8x+5y

such thatx+y≤69x+5y≤45

x≥4

y≤1 (*)x,y≥0

optimum: z=3659

x=409, y=1

Branch & Bound in OPL (rept.)

A reflective function allows direct access to the linear relaxation (Simplex value).

Branch and bound could therefore be simulated by the search instruction

while not bound(x) dolet v=simplexValue(x) inlet f=floor(v) inlet c=ceiling(v) in

tryx <= f | x >= c

endtry;

Branch & Bound in CLP

To solve P with relaxed optimum x=v solve•P1: (P and x<=floor(v))•P2: (P and x>=ceiling(v))•compare optimum of P1 with optimum of P2

At any point in a CLP computation, the conjunction of all accumulated constraints must be consistent.

•otherwise backtracking will be triggered•backtracking will “forget” all intermediate results

The problem (P1 and P2) is inconsistent, because of(X<=floor(v) and x>=ceiling(v)).

•therefore a “naive” version of branch and bound cannot be implemented “naively”

Problem Representation in CLP

Solution: For each new sub-problem create a problem instance with new, fresh variables.

Problem representation problem([X,Y], -Y) :- 2*Y <=3*X-3, X+Y<=5.

lower and upper bound are either integersor the atom ‘unbound’ for unbound, e.g.[ b(3, 7), b(4, unbound) ]

Objective Constraints

Bounds representation [ b(v1low, v1high), b(v2low, v2high), ... ]

Problem Variables

Constructing a fresh Problem Instance

bounded_problem(Vs, Bounds, Objective) :-bounds(Vs, Bounds),problem(Vs, Objective).

bounds([], []).bounds([V|Vs], [b(L,U)|Bs]) :-

(L=unbound -> true ; L <= V),(U=unbound -> true ; U >= V),bounds(Vs, Bs).

problem([X,Y], -Y) :- 2*Y <=3*X-3, X+Y<=5.

bounded_problem(Xs, [b(unbound,unbound), b(unbound,unbound)], F) Xs is the list of problem variables only constrained by the body of problem/2.

bounded _problem(Xs, [b(unbound,2), b(unbound, unbound)], F)Xs is a list of variables [X,Y] constrained by the body of problem/2 andthe additional bound X<=2.

Constructing Sub Problems

new_bounds([], [], [], []).new_bounds([V | Vs], [B | Bs], [BL | BLs], [BR | BRs]) :-

(integer(V) ->BL = B, BR = B,new_bounds(Vs, Bs, BLs, BRs)

;B=(L,U),Vlow = floor(V), Vhigh = ceiling(V),BL = b(L,Vlow), BR = b(Vhigh, U),BLs = Bs, BRs = Bs

).

•Split on the first fractional variable•Copy all other bounds

Example: new_bounds([2.6, 2.4], // relaxation optimum [b(unbound, unbound), b(unbound, unbound)], // original problem [b(unbound, 2), b(unbound, unbound)], // sub problem A

[b(3, unbound), b(unbound, unbound)]) // sub problem B

Exploring the Sub Problems

bnb(CBest, CVal, Bnds, Best, BestVals) :-(minimize((F<CBest, bounded_problem(Vs, Bnds, F)), F)->

new_bounds(Vs, Bnds, BndsL, BndsR),(BndsL=BndsR ->

Best=F, BestVals=Vs ;

bnb(CBest, CVals, BndsL, LBest, LVals),bnb(LBest, LVals, BndsR, Best, BestVals)

;Best=CBest, BestVals=CVals)

).

bnb(Current Best Objective Value, Current Best Valuation, Bounds Structure, Improved Objective Value, Improved Valuation) :-

Generate fresh problem instance and solve relaxationif improved solution found

split relaxation on first fractional variableif no factional variable

return last solutionelse call b&b on both sub problems

else return last solution

Using SICStus Prolog as CLP(FD)

SICStus Prolog is a CLP language which uses libraries to mimic CLP(R) and CLP(FD).

To enter CLP(FD) mode

?- use_module(library(clpfd)). Syntax is similar to examples except

=, <, ≠, ≤, are written #=, #<, #\=, #=< (not #<=), #<=> labeling(L) is labeling([],L) (the first argument is a list of options) Use domain(ListVar,min,max) to initialize domains. All FD constraints must be made explicit.

Example SICStus CLP(FD) Program

smm(S,E,N,D,M,O,R,Y) :-domain([S,E,N,D,M,O,R,Y], 0, 9),constrain(S,E,N,D,M,O,R,Y),labeling([], [S,E,N,D,M,O,R,Y]).

constrain(S,E,N,D,M,O,R,Y) :-S #\= 0, M #\= 0, all_different([S,E,N,D,M,O,R,Y]), 1000*S + 100*E + 10*N + D+ 1000*M + 100*O + 10*R + E#= 10000*M + 1000*O + 100*R + 10*E + Y.

Using SICStus Prolog as CLP(QR)

To enter CLP(R) mode (ie linear programming with floating point arithmetic)

?- use_module(library(clpr)). To enter CLP(Q) mode

(ie linear programming with rational arithmetic)

?- use_module(library(clpq)).

Syntax is similar to Prolog but all constraints must be made explicit with braces: {X+Y =< 2} etc minimization is called for with the goal minimize(+Objective) MIP minimization is triggered with bb_inf(+Ints, +Objective, -Inf)

Ints is the list of integral decision variables, eg [X,Y]

Example SICStus CLP(R) Program

?- {X>=0, Y>=0, X+Y =< 6, 9*X+5*Y =< 45},

minimize(8*X+5*Y).X = 3.75,Y = 2.25

?- {X>=0, Y>=0, X+Y =< 6, 9*X+5*Y =< 45},

bb_inf([X,Y], 8*X+5*Y, Opt).Opt = -40.0,{X+Y=<6.0},{X+0.5555555555555556*Y=<5.0},{X>=0.0},{Y>=0.0}

1 2 3 4 5 6

2

4

6

8

1 2 3 4 5 6

2

4

6

8

maxz=8x+5y

such that x+y≤6

9x+5y≤45

x,y≥0, integer

Complex FD-Constraints in Sicstus

Labeling Options in Sicstus

VariableOrder

ValueOrder

Labeling Options in Sicstus

Reflective FD-Constraints in Sicstus

Summary

CLP is a powerful modelling language for describing constraint problems.

However it is also simple: it allows programmers to define their own application specific constraints, that’s all!

CLP is generic in the underlying constraint domain and solver. However, backtracking search is always provided.

CLP(FD) provides consistency techniques which can be used with backtracking search to solve CSPs.

Summary (Cont.)

But CLP is not just a modelling language: It is a full-fledged programming language with complex data structures.

It can be used to program Search strategies Heuristics Constraint solvers

The key to efficient solution of CSPs is the programmibility of CLP.