An Overview of Soft Constraints

An Overview of Soft Constraints

Presented by Tony SchneiderSources• “Soft Constraints,” Meseguer, Rossi, & Schiex, CP Handbook, Chapter 9, 281—328, 2006• “Solving Weighted CSP by Maintaining Arc Consistency,” Larrosa & Schiex, Artificial

Intelligence, Vol 159(1-2), pages 1—26, 2004.

Tony Schneider--Soft Constraints 2

Motivation

• Why Soft Constraints?– Find best answer as opposed to any answer– Systems where constraints existence is probabilistic – Over-constrained problems

• Optimality– Requires exploration of entire search space (i.e., finding

all-solutions)– Necessitates efficient pruning methods– At least as hard as classical CSPs

March 11+13, 2013


Outline

• Background Information Section 9.1

• Specific Soft Constraint Frameworks Section 9.2

– Fuzzy & weighted• Local Consistency in Soft Constraints [Larrosa+ AI04]

• Generic Soft Constraint Frameworks Section 9.3

– Semiring-based & valued• Systematic Search in Soft Constraints Section 9.5

• Wrap up

March 11+13, 2013


Notation• tV is a partial solution over V variables• Constraint

– c∈C is denoted ⟨R,V or ⟩ RV

– R=relation(c), V=scope(c)– For W⊆V, tv[W]= πW(tV)

• For a constraint c: ⟨R,W , ⟩ Rw

– W⊆V ⟺ c is completely assigned by tV

– tv[W]∈Rw ⟺ tV satisfies c

– tv[W] ∉ RW ⟺ tV violates c

• tv is consistent all constraints completely assigned by it are ⟺satisfied

March 11+13, 2013


Fuzzy Set

• What is a fuzzy set?– Changes question of satisfaction from• “Does this value belong to this set?” to• “How much does this value belong to this set?”

– Formally• μe(a) = 0 : Value a doesn’t belong to set e at all

• μe(a) = 1 : Value a completely belongs to set e

– Acceptance value ranges from [0,1]

March 11+13, 2013


Fuzzy Constraints

• A fuzzy constraint: a relation RV and a function

maps a membership degree to every possible tuple for the variables in V

• A fuzzy CSP is a CSP with fuzzy constraints

March 11+13, 2013


Combining Fuzzy Constraints

• Conjunctive Combination (RV ⊗ RW)– Combines two relations into a new relation RV⋃W

– Formally:

where

March 11+13, 2013


Conjunctive Combination: Example

March 11+13, 2013

R1

x1 x2 μ(a)

0 0 .2

0 1 .3

1 0 .4

1 1 .8

R2

x2 x3 μ(a)

0 0 .9

0 1 .1

1 0 .7

1 1 .6

R1 2⋃x1 x2 x3 μ(a)

0 0 0 .2

0 0 1 .1

0 1 0 .3

0 1 1 .3

1 0 0 .4

… … … …

1 1 1 .6


Solutions in Fuzzy CSPs

• The satisfaction rate of a complete assignment is the value of the least satisfied constraint

• A complete assignment with– non-zero satisfaction rate is a solution– the maximum satisfaction rate is an optimal solution

March 11+13, 2013


Advantages of Fuzzy CSPs

• Good for applications where system is only as strong as its weakest link– Space Engineering– Medical Applications

• Can easily represent classical constraints• Can use other operators besides min

extensions

March 11+13, 2013


Disadvantages of Fuzzy CSPs

• Consider two assignments t, t’t: μR1(t) = .5 μR2(t) = 1.0t’: μR1(t’) = .5 μR2(t’) = .5

• Each would be considered optimal solutions…– but… t is superior WRT μR2

• Solution? Fuzzy lexicographic constraints…– Sort μ values in increasing order– Break ties in minimum preference with next lowest μ

March 11+13, 2013


Outline






• Wrap up

March 11+13, 2013


Weighted Constraints

• Another soft constraint framework

• Suitable for applications dealing with minimizing cost or penalties, e.g., – best price for some combination of items– minimizing travel costs

March 11+13, 2013

Constraints Fuzzy Weighted

… have membership values costs

Optimal solution maximizes values minimizes cost


Solutions in Weighted Constraint Networks

• Weighted CSP = CSP + a weighted constraint – Weighted constraints: c,w⟨ ⟩– w(c) ∈ N, Z, R is the cost of the constraint c

• Cost (assignment) = ∑ of violated constraints• Optimal solution is a complete assignment

with minimal cost– ∀c∈C, w(c) 1 weighted CSP MaxCSP≣ ⇒ ≣

March 11+13, 2013


k-Weighted Constraints

• Refinement of weighted constraints• Defined by the tuple ⟨X,D,C,K where ⟩– C is a set of k-weighted constraints– k is an integer

• k acts as an upper bound for acceptable costs• Weights are mapped to tuples in C

March 11+13, 2013


Outline






• Wrap up

March 11+13, 2013


Local Consistency in SCSPs

• Concept similar to hard-CSPs– Used to reduce search space – Detect inconsistencies early

• However– Requires consideration of constraint costs– Values can only be pruned when node inconsistent

March 11+13, 2013


Node Consistency in WCSPs

• Every variable v has a unary constraint Cv

– A value a v∈ is node consistent (NC) if Cv(a) < ⟙– A variable is NC if its values are NC– A problem is NC if its variables are NC

March 11+13, 2013

Cx

x Cx(a)

a 0b 1

c 3

Is x NC if = 4?⟙Is x NC if = 3?⟙

Yes!

No. Prune value c.


Arc Consistency in WCSPs

• A value a in variable i is arc consistent (AC) with respect to Cij if– It is node consistent– It has at least one support b in Dj s.t. Cij(a,b) = ⟘

• A variable is AC if its values are AC• A problem is AC if its variables are AC• However, arc inconsistent values cannot be

removed as in hard-CSPs– Why?

March 11+13, 2013



March 11+13, 2013

Cx

x Cx(a)

a 0

b 1

Cy

y Cy(a)

a 2

b 1

c 2

Cxy

x y Cxy(a)

a a 1

a b 2

a c 1

b a 0

b b 0

b c 2

• Is value ‘b’ x arc consistent WRT variable y?∈• Is value ‘a’ x arc consistent WRT variable y?∈Yes NoA CSP where• X = {x, y}• D = {a, b, c}• C = {Cxy}• ⟙ = 3



• So why bother with AC?– Use AC to push costs of binary constraints onto unary ones

• To make value a ∈x arc consistent WRT Cxy

– Find minimum cost of assigning a in Cxy

– Add αa to unary constraint Cx(a)

– Subtract αa from cost of all constraints in Cxy

March 11+13, 2013


WCSP Arc Consistency Preliminary

• Subtraction in WCSPs is defined by

• And addition by

March 11+13, 2013


WCSP Arc Consistency Example

March 11+13, 2013

Cx

x Cx(a)

a 0

b 1

Cy

y Cy(a)

a 2

b 1

c 2

Cxy

x y Cxy(a)

a a 1

a b 2

a c 1

b a 0

b b 0

b c 2

• Goal: Make variable y arc consistentIs a y∈ AC? YesIs b y∈ AC? YesIs c y∈ AC? No

A CSP where• X = {x, y}• D = {a, b, c}• C = {Cxy}• ⟙ = 3



March 11+13, 2013

Cx

x Cx(a)

a 0

b 1

Cy

y Cy(a)

a 2

b 1

Cxy

x y Cxy(a)

a a 1

a b 2

a c 1

b a 0

b b 0

b c 2

1. Find minimum cost of assigning a in Cxy

2. Add αa to unary constraint Cy(a)3. Subtract αa from cost of constraints in Cxy

c 2c 3

Cxy

x y Cxy(a)

a a 1

a b 2

a c 0

b a 0

b b 0

b c 1

αa = 1



c 3


March 11+13, 2013

Cx

x Cx(a)

a 0

b 1

Cy

y Cy(a)

a 2

b 1

Is variable y arc consistent?

Cxy

x y Cxy(a)

a a 1

a b 2

a c 0

b a 0

b b 0

b c 1

No. Value c is not node consistent.

Cxy

x y Cxy(a)

a a 1

a b 2

b a 0

b b 0


Is the problem arc consistent? No. Value a in x has no support.

Tony Schneider--Soft Constraints


March 11+13, 2013 26

Cx

x Cx(a)

a 0

b 1

Cy

y Cy(a)

a 2

b 1

1. Find minimum cost of assigning a in Cxy

2. Add αa to unary constraint Cx(a)3. Subtract αa from cost of constraints in Cxy

Cxy

x y Cxy(a)

a a 1

a b 2

b a 0

b b 0

Cxy

x y Cxy(a)

a a 0

a b 1

b a 0

b b 0

αa = 1

Cx

x Cx(a)

a 1

b 1




March 11+13, 2013

Cx

x Cx(a)

a 1

b 1

Cy

y Cy(a)

a 2

b 1

Cxy

x y Cxy(a)

a a 1

a b 2

a c 0

b a 0

Cxy

x y Cxy(a)

a a 0

a b 1

b a 0

b b 0

Why does this work?



Weighted Arc Consistency Intuition

• Force supports for arc inconsistent values• Arc inconsistent values can’t simply be removed– Cost of binary constraint is taken on by unary– Other tuples with the previously inconsistent value

adjusted to compensate for the higher unary cost

• Caveat: order of evaluation can produce different (but equal) structures

March 11+13, 2013


Equivalent WCSP Networks

March 11+13, 2013

Same resultant network from previous exampleResultant AC network if x was made AC first instead of (y,c)


W-AC3 and W-AC2001

• Operate in a similar fashion to their hard-CSP counterparts– Maintain a queue of revised variables– Make each adjacent variable of variables in the queue

arc-consistent• Force support for arc inconsistent values (i.e., push binary

constraints onto unary)– Prune node inconsistent values in the variable

• If variable is pruned, add it to the queue

• Time complexity: O(ed3) for each algorithm

March 11+13, 2013


Outline






• Wrap up

March 11+13, 2013


Generic Frameworks

• Motivation– Generic algorithms & properties• One algorithm for many similar, specific frameworks

– Allows easy comparison between frameworks• Two main generic frameworks

1. Semiring-Based Constraints2. Valued constraints

March 11+13, 2013


Semiring: Preliminaries• Operator Properties

– Commutative a +s b = b +s a– Associative (a +s b) +s c = (a +s b) +s c– Idempotent a +s a = a

• Annihilators– A value b whose application to another value a destroys a – E.g., 0 is the annihilator for integer multiplication

• Neutral Elements– A value b whose application to another value a results in the value

a

March 11+13, 2013


c-Semirings

• A c-semiring is a tuple E,+⟨ s,×s,0,1 where⟩– E is a set of satisfaction degrees where 0 E, 1 E∈ ∈– Operator +s

• is associative, commutative, and idempotent• is closed in E• has 0 as a neutral element and 1 as an annihilator

– Operator ×s

• is associative and commutative• is closed in E• has 1 as a neutral element and 0 as an annihilator

– ×s distributes over +s

March 11+13, 2013


c-Semiring Operators

• ×s is used to combine preferences• Let– RC be completely satisfied – RU be completely unsatisfied– Rl have satisfaction degree l– vals(R) be the preference level of relation R

• Then– vals(RC) ×s vals(RU)= 0– vals(RC) ×s vals(Rl) = l– vals(Rl) ×s vals( Rl) = l

March 11+13, 2013


c-Semiring Operators

• +s is used to order preference levels• Let a and b be two preference levels– a +s b = a b ≼s a– a +s b = b a ≼s b– a +s b = c and a ≠ c, b ≠ c Incomparable

• +s Induces a partial ordering over preferences– Specifically a lattice where +s is the least upper bound

(LUB) of {a,b}

March 11+13, 2013


Partial Orders in c-Semirings

• Allow for more expressive modeling

• Optimization of multiple criteria

• Example– maximizing route scenery– minimizing cost & time

March 11+13, 2013

⟨0.5, 50, ∞⟩ ⟨0.4, 40, 900⟩

⟨0.0, ∞, ∞⟩

⟨0.75, 50, 400⟩ ⟨0.5, 40, 800⟩⟨1.0, 0, 0⟩

Satisfaction level s, c, t⟨ ⟩


Semiring Constraint Networks

• A semiring constraint network is a tuple ⟨X,D,C,S where⟩– X, D : same as in classical CSPs– C is a set of soft constraints– S = E,+⟨ s,×s,0,1 is a c-semiring⟩

March 11+13, 2013


Soft Constraints

• A soft constraint: a function f on a set of variables V ⊆ X

f maps a semiring value to every possible tuple for the variables in V

• A soft constraint is a pair ⟨f,V denoted as ⟩ fv

March 11+13, 2013


Outline






• Wrap up

March 11+13, 2013


Overview of Valued Constraints

• An alternative to semiring-based constraints– Elements represent violation degrees– Totally ordered• Can represent any totally ordered semiring-based

constraint

• Uses a valuation structure instead of a semiring

March 11+13, 2013


Valuation Structure

• A valuation structure is a tuple <E, , ⊕ ≼v, , > where⟘ ⟙– E is a set of values totally ordered by ≼v

– ⟘ is a minimum element in E– ⟙ is a maximum element in E– ⊕ is an operator which

• is associative, commutative, and monotonic• is closed in E• Has as a neutral element and as an annihilator⟘ ⟙

• Monotonicity– a ≼v b ((⟶ a ⊕ c) ≼v (b ⊕ c))

March 11+13, 2013


Valued Constraint Network• Same definition as semiring CN, but the valuation structure

replaces the semiring

• Valuation structures can always be represented as semirings: ⟘ = 1 = 0⟙ ⊕v = ×s a ≼v b iff a +s b = a

• When totally ordered, semiring CNs have corresponding valued CNs 1 = ⟘ 0 = ⟙ ×s = ⊕v a +s b = min≼v{a, b}

March 11+13, 2013


Operations on Soft Constraints

• Combination / Join– Combines two constraints without losing information– Take two soft constraints fV and f’V’

and

March 11+13, 2013



• Combination / Join Example in Fuzzy CSPs (×s = min)

March 11+13, 2013

R1

x1 x2 μ(a)

0 0 .2

0 1 .3

1 0 .4

1 1 .8

R2

x2 x3 μ(a)

0 0 .9

0 1 .1

1 0 .7

1 1 .6

R1 2⋃x1 x2 x3 μ(a)

0 0 0 .2

0 0 1 .1

0 1 0 .3

0 1 1 .3

1 0 0 .4

… … … …

1 1 1 .6

×s =



• Projection– Eliminates one or more variables from a constraint– Take some soft constraint fV

and

March 11+13, 2013



• Projection Example in Fuzzy CSPs (+s = max), W = {x2, x3}

March 11+13, 2013

Rx1 x2 x3 μ(a)

0 0 0 .2

0 0 1 .1

0 1 0 .3

0 1 1 .3

1 0 0 .4

… … … …

1 1 1 .6

Gx2 x3 μ(a)

0 0 .2

0 1 .1

1 0 .3

1 1 .3

0 0 .4

… … …

1 1 .6

R[W] or R[-x1]

Tony Schneider--Soft Constraints



48

March 11+13, 2013

Rx1 x2 x3 μ(a)

0 0 0 .2

0 0 1 .1

0 1 0 .3

0 1 1 .3

1 0 0 .4

… … … …

1 1 1 .6

R[W] or R[-x1]

Gx2 x3 μ(a)

0 1 .1

1 0 .3

1 1 .3

0 0 .4

… … …

1 1 .6




March 11+13, 2013

Rx1 x2 x3 μ(a)

0 0 0 .2

0 0 1 .1

0 1 0 .3

0 1 1 .3

1 0 0 .4

… … … …

1 1 1 .6

R[W] or R[-x1]

Gx2 x3 μ(a)

0 1 .1

1 0 .3

0 0 .4

… … …

1 1 .6


Soft Constraint Solutions

• ⨝f∈C f is a soft constraint associates a consistency level with a complete assignment t– vals(t) is its preference level

– ⨝f∈C f [ø] is the network’s consistency level

• A solution to a semiring SCSP is a complete assignment t such that ⨝f∈C f[t] ≠ 0

• An optimal solution t is one s.t. no other assignment t’ satisfies vals(t) ≺s vals(t’)

March 11+13, 2013


Generic Frameworks Wrap-up• Provide a generalized description of SCSPs• Two types

– Valued– Semiring

• Can readily transition between various frameworks

March 11+13, 2013

Semiring E ×s +s ≽s 0 1

Valued E ⊕ minv ≼v ⟙ ⟘Probabilistic [0,1] xy max ≥ 1 0

k-weighted {0,…,k} +k min ≤ k 0

Fuzzy [0,1] min max ≥ 0 1

Classical {t, f} ∧ ∨ t ≽s f f t


Outline






• Wrap up

March 11+13, 2013


k-Weighted Constraints (Review)

• Refinement of weighted constraints • Defined by the tuple ⟨X,D,C,K where ⟩– X, D variables and domains as in classical CSPs– C is a set of k-weighted constraints– k is an integer

• k acts as an upper bound for acceptable costs• Weights are mapped to tuples in C

March 11+13, 2013


Search in Soft CSPs

• Discussion limited to binary, k-weighted SCSPs• Systematic Search explores entire state space– Guaranteed to find optimal solution

• Local search – Limited by some resources (usually time)– May not find best solution

• Local & Systematic can cooperate to improve performance

March 11+13, 2013


Systematic Search in Soft CSPs

• A first approach: Depth-First Branch & Bound (DFBB)– Explores the search tree in a DFS manner– Keeps track of two bounds

• Bounds– lb(n): Best possible violation degree at node n if

search continues– ub: Maximum acceptable violation degree

March 11+13, 2013


Depth-First Branch & Bound

• Main idea– Assign a variable xi at search level i– If lb(i) > ub, then backtrack

Else continue until a solution is found & update ub– Continue until search space is exhausted

• Improvements focus on better estimations of the lower bound– If CP is the set of all fully assigned constraints, then

March 11+13, 2013

Tony Schneider--Soft Constraints 57March 11+13, 2013

R1

x1 x2 w0 0 20 1 3

K = 10

lb1 =

ub =

R2

x1 x3 w0 0 7

0 1 4

R3

x2 x4 w0 0 1

0 1 9

1 0 0

1 1 1

R4

x3 x4 w0 0 6

0 1 5

1 0 3

1 1 4 100 1

0 0 1 1 0 11 0

10

0x1

x2

x3

x4

10

-

CP = {R1, R2, R3, R4} {R1} {ø} {R1, R2}

22 + 7 = 92 + 7 + 1 + 6 = 16

Branch and Bound: lb1(t) Example


R1

x1 x2 w0 0 20 1 3

R2

x1 x3 w0 0 70 1 4

R3

x2 x4 w0 0 10 1 9

1 0 0

1 1 1

R4

x3 x4 w0 0 60 1 5

1 0 3

1 1 4 100 1

0 0 1 1 0 11 0

10

0x1

x2

x3

x4

Skipping Ahead…nodes visited = 13



Improving on DFBB

• Improving search– lb1 used costs of completely assigned constraints– Use a more sophisticated lower bound

• Partial Forward Checking (PFC )– Look-ahead to get costs from unassigned variables– Includes costs from partially assigned constraints

March 11+13, 2013


Partial Forward Checking (PFC)

• Cp is the set of fully assigned constraints• F is the set of unassigned variables• icja is the inconsistency count from F

• lb2(t) replaces lb1(t) in the DFBB search

March 11+13, 2013


R1

x1 x2 w0 0 20 1 3

K = 10

lb2 =

ub =

R2

x1 x3 w0 0 7

0 1 4

R3

x2 x4 w0 0 1

0 1 9

1 0 0

1 1 1

R4

x3 x4 w0 0 6

0 1 5

1 0 3

1 1 4 100 1

0 0 1 1 0 11 0

10

0x1

x2

x3

x4

10

F = {x2, x3, x4}

0 + …


CP = {ø} 0 + min(2, 3) + …0 + 2 + …0 + 2 + min(7,4) + …0 + 2 + 4 + …0 + 2 + 4 = 6


R1

x1 x2 w0 0 20 1 3

K = 10

lb2 =

ub =

R2

x1 x3 w0 0 7

0 1 4

R3

x2 x4 w0 0 1

0 1 9

1 0 0

1 1 1

R4

x3 x4 w0 0 6

0 1 5

1 0 3

1 1 4 100 1

0 0 1 1 0 11 0

10

0x1

x2

x3

x4

10

F = {x3, x4}


CP = {R1} 2 + …2 + min(7,4) + …2 + 4 + min(1, 9)2 + 4 + …2 + 4 + 1 = 7


R1

x1 x2 w0 0 20 1 3

K = 10

lb2 =

ub =

R2

x1 x3 w0 0 7

0 1 4

R3

x2 x4 w0 0 1

0 1 9

1 0 0

1 1 1

R4

x3 x4 w0 0 6

0 1 5

1 0 3

1 1 4 100 1

0 0 1 1 0 11 0

10

0x1

x2

x3

x4

10

F = {x4}


CP = {R1, R2} 2 + 7 + …2 + 7 + min(5, 6)2 + 7 + 5 = 14


R1

x1 x2 w0 0 20 1 3

R2

x1 x3 w0 0 70 1 4

R3

x2 x4 w0 0 10 1 9

1 0 0

1 1 1

R4

x3 x4 w0 0 60 1 5

1 0 3

1 1 4 100 1

0 0 1 1 0 11 0

10

0x1

x2

x3

x4

Skipping Ahead…nodes visited = 7



Improving on PFC: Directional Arc Inconsistency (DAI)

• Extend lb(t) to include costs from completely unassigned constraints

• Assumes a static variable ordering

• dacja is the cost from values arc inconsistent with current assignment, and below it in the ordering

March 11+13, 2013


R1

x1 x2 w0 0 20 1 3

K = 10

ub =

R2

x1 x3 w0 0 7

0 1 4

R3

x2 x4 w0 0 1

0 1 9

1 0 0

1 1 1

R4

x3 x4 w0 0 6

0 1 5

1 0 3

1 1 4

10


lb3 = 0 + …

F = {x2, x3, x4}

x1

x2

x3

x4lb3 = 0 + min((2 + 1), (2+9), …) + …lb3 = 0 + min((2 + 1), (2+9), (3+0), (3+1)) + …lb3 = 0 + 3 + …lb3 = 0 + 3 + min((7 + 6), (7+5), …) + …lb3 = 0 + 3 + min((7 + 6), (7+5), (4+3), (4+4)) + …lb3 = 0 + 3 + 7 + …

Already Inconsistent. Nodes Visited = 1Instantiate x1 with 0No fully assigned constraints yetWhat happens if 0 is assigned to x2?

Contribution from icja plus contribution from dacja

What happens if 1 is assigned to x2?What happens if 0 is assigned to x3?What happens if 1 is assigned to x3?


Local Search

• Used in conjunction with DFBB– Preprocessing– May decrease size of upper bound

• Examples– Hill Climbing– Stochastic search– Tabu Search

March 11+13, 2013


Wrap Up

• Several ways to model soft constraints– Specific frameworks

• Fuzzy, Weighted, Probabilistic– Generic frameworks

• c-semiring and valued

• Search in SCSPs relies on bounds for pruning• Local search improves on initial upper bound• Arc consistency – Can’t be used to prune domain values– Can be used to increase node costs (which can be pruned)

March 11+13, 2013


Questions?

March 11+13, 2013

Thanks!


Stronger WCSP Node Consistency

• Previous example after arc consistency:

• However, no matter which value in y is assigned, it will always incur a cost of 1 from x due to Cx

March 11+13, 2013

Cx

x Cx(a)

a 1

b 1

Cy

y Cy(a)

a 2

b 1

Cxy

x y Cxy(a)

a a 1

a b 2

b a 0

b b 0


Revised Node Consistency: NC*

• Let Cø be a global constant value initially equal to ⟘

• A value a v∈ is NC* if Cv(a) C⊕ ø< ⟙• A variable v is NC* if – its values are NC* and– there exists a value a v such that ∈ Cv(a) = ⟘

• A problem is NC* if its variables are NC*

March 11+13, 2013


NC* Algorithm

• For each variable i X∈– Find minimum unary cost for variable i

– Add α to Cø

– Subtract α from every unary cost in Ci

March 11+13, 2013


W-AC*3

• Similar to W-AC3, but– Each variable must be NC*, not NC– Problem must initially be NC*– Cø is updated after supports are forced in a

variable– Anytime Cø is changed, every domain is checked

for NC* inconsistent values• Time complexity: O(n2d2 + ed3)

March 11+13, 2013

Documents

An Overview of Soft Constraints