Constraint Systems Laboratory
Robert J. Woodward
Constraint Systems LaboratoryDepartment of Computer Science & Engineering
University of Nebraska-Lincoln
Acknowledgements• Software platform developed in collaboration with Shant Karakashian (to a large extent) & Chris Reeson• Elizabeth Claassen & David B. Marx of the Department of Statistics @ UNL• Experiments conducted at UNL’s Holland Computing Center• NSF Graduate Research Fellowship & NSF Grant No. RI-111795
11/28/2011 Woodward-MS Thesis Defense 1
Relational Neighborhood Inverse Consistencyfor Constraint Satisfaction:
A Structure-Based Approach for Adjusting Consistency & Managing Propagation
Constraint Systems Laboratory
Main Contributions1. Relational Neighborhood Inverse Consistency
(RNIC)– Characterization on binary & non-binary CSPs– An algorithm for enforcing RNIC– Comparison to other consistency properties
2. Variations of RNIC– Reformulation by redundancy removal, triangulation,
both– A strategy for selecting the appropriate variation
3. Managing constraint propagation– Four queue-management strategies (QMSs)
4. Empirical evaluations on benchmark problems
11/28/2011 Woodward-MS Thesis Defense 2
Constraint Systems Laboratory
Outline• Background• Relational Neighborhood Inverse Consistency (RNIC)
– Property, characterization• Dual Graphs of Binary CSPs
– Complete constraint network– Non-complete constraint network– RNIC on binary CSPs
• Enforcing RNIC– Algorithm for RNIC– Dual-graph reformulation– Selection strategy
• Evaluating RNIC• Propagation-Queue Management• Conclusions & Future Work
11/28/2011 Woodward-MS Thesis Defense 3
Constraint Systems Laboratory
Constraint Satisfaction Problem• CSP
– Variables, Domains – Constraints: Relations & scopes
• Representation– Hypergraph– Dual graph
• Solved with– Search– Enforcing consistency– Lookahead = Search + enforcing consistency
11/28/2011 Woodward-MS Thesis Defense 4
R4
BCD
ABDE
CF
EFAB
R3 R1
R2
C
F
E
BD
AB
DADA AD B
R5
R6
R3
AB
C D
E
FR1
R4R2 R5
R6
Hypergraph
Dual graph
• Warning– Consistency properties vs. algorithms
Constraint Systems Laboratory
Neighborhood Inverse Consistency• Property [Freuder+ 96]
↪ Every value can be extended to a solution in its variable’s neighborhood
↪ Domain-based property
• Algorithm⧾No space overhead
⧾Adapts to graph connectivity
11/28/2011 Woodward-MS Thesis Defense 5
0,1,2
0,1,2
0,1,2
0,1,2
R0 R1 R3
R2
R4A
B
C
D
R3
AB
C D
E
FR1
R4R2 R5
R6• Binary CSPs [Debruyene+ 01]⧿Not effective on sparse problems
⧿Too costly on dense problems
• Non-binary CSPs?⧿Neighborhoods likely too large
Constraint Systems Laboratory
Relational NIC
11/28/2011 Woodward-MS Thesis Defense 6
• Property↪ Every tuple can be extended to a
solution in its relation’s neighborhood↪ Relation-based property
• Algorithm– Operates on dual graph– Filters relations– Does not alter topology of graphs
R4
BCD
ABDE
CF
EFAB
R3 R1
R2
C
F
E
BD
AB
DADA AD B
R5
R6
R3
AB
C D
E
FR1
R4R2 R5
Hypergraph
Dual graph
• Domain filtering– Property: RNIC+DF– Algorithm: Projection
Constraint Systems Laboratory
From NIC to RNIC• Neighborhood Inverse Consistency (NIC)
[Freuder+ 96]– Proposed for binary CSPs – Operates on constraint graph– Filters domain of variables
• Relational Neighborhood Inverse Consistency (RNIC)– Proposed for both binary & non-binary CSPs– Operates on dual graph– Filters relations; last step projects updated relations on domains
• Both – Adapt consistency level to local topology of constraint network– Add no new relations (constraint synthesis)
• NIC was shown to be ineffective or costly, we show that RNIC is worthwhile
11/28/2011 Woodward-MS Thesis Defense 7
Constraint Systems Laboratory
Characterizing RNIC: Binary CSPs
• On binary CSPs
[Luchtel, 2011]– NIC (on the constraint graph) and RNIC (on
the dual graph) are not comparable– Empirically, RNIC does more filtering than
NIC
11/28/2011 Woodward-MS Thesis Defense 8
Constraint Systems Laboratory
GAC, SGAC• Variable-based properties• So far, most popular for non-
binary CSPs
Characterizing RNIC (I): Nonbinary CSPs
R(*,m)C
[Karakashian+ 10] • Relation-based property• Every tuple has a support in
every subproblem induced by a combination of m connected relations
11/28/2011 Woodward-MS Thesis Defense 9
GACR(*,2)C+DF
SGAC
R(*,3)C R(*,δ+1)CR(*,2)C
p’p : p is strictly weaker than p’
R4
R3 R1
R2
R5
R6
Constraint Systems Laboratory
Characterizing RNIC (II): Nonbinary CSPs
• The fuller picture, details are in the thesis
– w: Property weakened by redundancy removal– tri: Property strengthened by triangulation– δ: Degree of dual network
11/28/2011 Woodward-MS Thesis Defense 10
R(*,3)C
wRNIC
R(*,4)C
RNIC
wtriRNICtriRNIC
R(*,δ+1)CwR(*,3)C
wR(*,4)C
wR(*,δ+1)C
R(*,2)C≡wR(*,2)C
Constraint Systems Laboratory
Outline• Background• Relational Neighborhood Inverse Consistency
– Property, characterization• Dual Graphs of Binary CSPs
– Complete constraint network– Non-complete constraint network– RNIC on binary CSPs
• Enforcing RNIC– Algorithm for RNIC– Dual-graph reformulation– Selection strategy
• Evaluating RNIC• Propagation-Queue Management• Conclusions & Future Work
11/28/2011 Woodward-MS Thesis Defense 11
Constraint Systems Laboratory
Complete Binary CSPs• Triangle-shaped grid• n-1 vertices for V1
– C1,i i [2,∈ n]– Completely connected
• n-1 vertices for Vi≥2
– Centered on C1,i
– i-2 along horizontal– n-i along vertical– Completely connected
• Not shown for clarity
11/28/2011 Woodward-MS Thesis Defense 12
V1
V2
V3
Vn-1
Vn
uh
uv
C1,i
Vn Cn-1,nC1,n
V1 Vn-1
Vn Vn C3,n C2,n
V2V3
V5 V5 V5
C1,2
C2,3C1,3
C3,4C1,4 C2,4
C4,5C1,5
V1
C3,5 C2,5
V1
V1 V2
V2
V2
V3 V3
V4
V3V4
V4
V5 V2V3V4V1
V1
Constraint Systems Laboratory
• A triangle-shaped grid– Every CSP variable annotates
a chain of length n-2– Remove edges that link two
non-consecutive vertices
Complete Binary CSPs: RR (I)• An edge is redundant if
– There exists an alternate path between two vertices
– Shared variables appearin every vertex in the path
11/28/2011 Woodward-MS Thesis Defense 13
Vn Cn-1,nC1,n
V1 Vn-1
Vn Vn C3,n C2,n
V2V3
V5 V5 V5
C2,3C1,3
C3,4 C2,4
C4,5C1,5
V1
C3,5 C2,5
V1
V1 V2
V2
V2
V3 V3
V4
V3V4
V4
V5 V2V3V4V1
V1
uh
uv
C1,i C1,2
C1,4
R4
BCD
ABDEAB
R3
BD
A
DADA A B
R5
R6
D
B
Constraint Systems Laboratory
Complete Binary CSPs: RR (II)
11/28/2011 Woodward-MS Thesis Defense 14
V1
V2
V3 V4
C10
V5C9 C8
C7 C1
C3
C4
C6
C2
C5
C1
C2
C3
C5
C6
C4
V1V1
V2
V2
V3V3
V4
V4V5C7
C10
C8
C9
V4
V2
V5
V5
V4
V3
• A redundancy-free dual graph is not unique• No chain for V2, but a star
Constraint Systems Laboratory
Non-complete Binary CSPs• Non-complete binary CSP
– Is a complete binary constraint graph with missing edges
• In the dual graph– There are missing dual vertices– The dual vertices with variable
Vi in their scope are completely connected
• Redundancy-free dual graph– Can still form a chain using
alternate edges
11/28/2011 Woodward-MS Thesis Defense 15
V1
V2
V3 V4
C1,4
V5C2,5
C3,5
C1,2 C1,5
C2,3
C3,4
V5 V5 V5
C1,2
C2,3C1,3
C3,4C1,4 C2,4
C4,5C1,5
V1
C3,5 C2,5
V1
V1 V2
V2
V2
V3 V3
V4
V3V4
V4
C1,3
C4,5
C2,4
Constraint Systems Laboratory
RNIC on Binary CSPs• After RR, RNIC is never stronger than R(*,3)C• Configurations for R(*,4)C
– C1 has three adjacent constraints C2, C3, C4
– C1 is not an articulation point
• Two configurations, neither is possible
11/28/2011 Woodward-MS Thesis Defense 16
C1
C3
C2 C4
V1 V1V2
C1
C3
C2 C4
V1 V2V1
Constraint Systems Laboratory
Outline• Background• Relational Neighborhood Inverse Consistency
– Property, characterization• Dual Graphs of Binary CSPs
– Complete constraint network– Non-complete constraint network– RNIC on binary CSPs
• Enforcing RNIC– Algorithm for RNIC– Dual-graph reformulation– Selection strategy
• Evaluating RNIC• Propagation-Queue Management• Conclusions & Future Work
11/28/2011 Woodward-MS Thesis Defense 17
Constraint Systems Laboratory
Algorithm for Enforcing RNIC• Two queues
1. Q: relations to be updated2. Qt(R): The tuples of relation R
whose supports must be verified• SEARCHSUPPORT(τ,R)
– Backtrack search on Neigh(R)1. Loop until all Qt( )⋅ are empty
11/28/2011 Woodward-MS Thesis Defense 18
• Complexity– Space: O(ketδ)– Time: O(tδ+1eδ)– Efficient for a fixed δ
..…
Neigh(R)R
ττi
✗
√
Constraint Systems Laboratory
Improving Algorithm’s Performance1. Use IndexTree [Karakashian+ AAAI10]
– To quickly check consistency of 2 tuples
2. Dynamically detect dangles– Tree structures may show in subproblem @ each instantiation– Apply directional arc consistency
11/28/2011 Woodward-MS Thesis Defense 19
R4
R3 R1
R2
R5
R6
Note that exploiting dangles is– Not useful in R(*,m)C: small value of m, subproblem size– Not applicable to GAC: does not operate on dual graph
Constraint Systems Laboratory
Reformulating the Dual Graph
11/28/2011 Woodward-MS Thesis Defense 20
R4
BCD
ABDE
CF
EFAB
R3 R1
R2
C
F
E
BD
A
DADA A B
R5
R6
• Cycles of length ≥ 4– Hampers propagation– RNICR(*,3)C
• Triangulation (triRNIC)– Triangulate dual graph
RR+Triangulation (wtriRNIC)• Local, complementary, do not ‘clash’
wRNICRNIC
wtriRNICtriRNIC
R4
BCD
ABDE
CF
EFAB
R3 R1
R2
C
F
E
BD
AB
DADA AD B
R5
R6
• High degree– Large neighborhoods– High computational cost
• Redundancy Removal (wRNIC)– Use minimal dual graph
D
B
Constraint Systems Laboratory
Selection Strategy: Which? When?• Density of dual graph ≥ 15% is too dense
– Remove redundant edges
• Triangulation increases density no more than two fold– Reformulate by triangulation
• Each reformulation executed at most once
11/28/2011 Woodward-MS Thesis Defense 21
No
YesNo Yes
Yes
No
dGo ≥ 15%
dGtri ≤ 2 dGo dGwtri ≤ 2 dGw
Go GwtriGwGtri
Start
Constraint Systems Laboratory
Outline• Background• Relational Neighborhood Inverse Consistency
– Property, characterization• Dual Graphs of Binary CSPs
– Complete constraint network– Non-complete constraint network– RNIC on binary CSPs
• Enforcing RNIC– Algorithm for RNIC– Dual-graph reformulation– Selection strategy
• Evaluating RNIC• Propagation-Queue Management• Conclusions & Future Work
11/28/2011 Woodward-MS Thesis Defense 22
Constraint Systems Laboratory
Experimental Setup• Backtrack search with full lookahead• We compare
– wR(*,m)C for m = 2,3,4– GAC– RNIC and its variations
• General conclusion– GAC best on random problems– RNIC-based best on structured/quasi-
structued problems
• We focus on non-binary results (960 instances)
– triRNIC theoretically has the least number of nodes visited
– selRNIC solves most instances backtrack free (652 instances)
11/28/2011 Woodward-MS Thesis Defense 23
Category #Binary #Non-binaryAcademic 31 0
Assignment 7 50Boolean 0 160
Crossword 0 258Latin square 50 0
Quasi-random 390 25Random 980 290
TSP 0 30Unsolvable
Memory 10 60All timed out 447 87
Constraint Systems Laboratory
Experimental Results• Statistical analysis on CP benchmarks• Time: Censored data calculated mean• Rank: Censored data rank based on
probability of survival data analysis• #F: Number of instances fastest
11/28/2011 Woodward-MS Thesis Defense 24
Algorithm Time Rank #F [ ]⋅ CPU #C [ ]⋅ Completion #BT-free169 instances: aim-100,aim-200,lexVg,modifiedRenault,ssa
wR(*,2)C 944924 3 52 A 138 B 79wR(*,3)C 925004 4 8 B 134 B 92wR(*,4)C 1161261 5 2 B 132 B 108
GAC 1711511 7 83 C 119 C 33RNIC 6161391 8 19 C 100 C 66
triRNIC 3017169 9 9 C 84 C 80wRNIC 1184844 6 8 B 131 B 84
wtriRNIC 937904 2 3 B 144 B 129selRNIC 751586 1 17 A 159 A 142
• [ ]⋅ CPU: Equivalence classes based on CPU
• [ ]⋅ Completion: Equivalence classes based on completion• #C: Number of instances completed• #BT-free: # instances solved backtrack free
Constraint Systems Laboratory
Outline• Background• Relational Neighborhood Inverse Consistency
– Property, characterization• Dual Graphs of Binary CSPs
– Complete constraint network– Non-complete constraint network– RNIC on binary CSPs
• Enforcing RNIC– Algorithm for RNIC– Dual-graph reformulation– Selection strategy
• Evaluating RNIC• Propagation-Queue Management• Conclusions & Future Work
11/28/2011 Woodward-MS Thesis Defense 25
Constraint Systems Laboratory
Propagation-Queue Management• Three directions for
ordering the relations:1. Arbitrary ordering
(previous)2. Perfect elimination
ordering (PEO) of some triangulation
3. Ordering of the maximal cliques, corresponds to a tree-decomposition ordering (TD)
11/28/2011 Woodward-MS Thesis Defense 26
R4
BCD
ABDE
CF
EF
AB
R3
R1
R2
ADR5
R6
PEO Maximal cliques
Ord
erin
g
R1,R2,R4
C3
C4
R6,R4
R5,R4
R1,R3,R4
C2
C1
R4
BCD
ABDE
CF
EFAB
R3 R1
R2
ADR5
R6
Constraint Systems Laboratory
Queue-Management Strategies
11/28/2011 Woodward-MS Thesis Defense 27
R4
BCD
ABDE
CF
EF
AB
R3
R1
R2
ADR5
R6
PEOO
rder
ing
R4
BCD
ABDE
CF
EFAB
R3 R1
R2
ADR5
R6
Constraint Systems Laboratory
R1,R2,R4
C3
C4
R6,R4
R5,R4
R1,R3,R4
C2
C1
Queue-Management Strategies
11/28/2011 Woodward-MS Thesis Defense 28
maximal cliquesO
rder
ing
R4
BCD
ABDE
CF
EFAB
R3 R1
R2
ADR5
R6
Constraint Systems Laboratory
Propagation-Queue Management• Statistical analysis on CP benchmarks• Time: Censored data rank based on
probability of survival data analysis
triRNIC Pre-processing SAT UNSATStrategy Time [ ]⋅ CPU % [ ]⋅ CPU [ ]⋅ CPU
QMSa 1,410,292 C - A CQMSPEO 1,186,691 B 16% A BQMSTD 765,976 A 46% A A
11/28/2011 Woodward-MS Thesis Defense 29
wtriRNIC Pre-processing SAT UNSATStrategy Time [ ]⋅ CPU % [ ]⋅ CPU [ ]⋅ CPU
QMSa 479,725 A - A BQMSPEO 467,747 A 2% A AQMSTD 476,604 A 1% A B
triRNIC Search SAT UNSATStrategy Time [ ]⋅ CPU % [ ]⋅ CPU [ ]⋅ CPU
QMSa 1,243,917 C - A CQMSPEO 900,069 B 28% A BQMSTD 416,464 A 67% A AQMSLTD 403,766 A 68% A AQMSL2TD 434,479 A 65% A A
wtriRNIC Search SAT UNSATStrategy Time [ ]⋅ CPU % [ ]⋅ CPU [ ]⋅ CPU
QMSa 628,523 C - A CQMSPEO 582,629 B 7% A BQMSTD 519,578 A 17% A AQMSLTD 602,437 C 4% B AQMSL2TD 575,277 C 8% B A
• [ ]⋅ CPU: Equivalence classes based on CPU• %: Percent increased gain by the algorithm
Constraint Systems Laboratory
Conclusions
• RNIC• Structure of binary dual graph• Algorithm for enforcing RNIC
– Polynomial for fixed-degree dual graphs– BT-free search: hints to problem tractability
• Various reformulations of the dual graph• Adaptive, unifying, self-regulatory, automatic
strategy• New propagation-queue management strategies• Empirical evidence, supported by statistics
11/28/2011 Woodward-MS Thesis Defense 30
Constraint Systems Laboratory
Future Work• Extension to singleton-type consistencies• Extension to constraints defined in intension
– Possible by only domain filtering (weakening)• Study influence of redundancy removal
algorithms– Redundancy removal algorithm of [Janssen+ 89]
seems to favor grids• Evaluate new queue-management strategies
on other consistency algorithms
11/28/2011 Woodward-MS Thesis Defense 31
Constraint Systems Laboratory
Thank You!
Questions?
11/28/2011 Woodward-MS Thesis Defense 32