1. Constraint Systems Laboratory Robert J. Woodward Constraint Systems Laboratory Department of Computer Science & Engineering University of Nebraska-Lincoln Acknowledgements Software platform developed in collaboration with Shant Karakashian (to a large extent) & Chris Reeson Elizabeth Claassen & David B. Marx of the Department of Statistics @ UNL Experiments conducted at UNL’s Holland Computing Center NSF Graduate Research Fellowship & NSF Grant No. RI-111795 11/28/2011Woodward-MS Thesis Defense1 Relational Neighborhood Inverse Consistency for Constraint Satisfaction: A Structure-Based Approach for Adjusting Consistency & Managing Propagation

2. Constraint Systems Laboratory Main Contributions 1.Relational Neighborhood Inverse Consistency (RNIC) –Characterization on binary & non-binary CSPs –An algorithm for enforcing RNIC –Comparison to other consistency properties 2.Variations of RNIC –Reformulation by redundancy removal, triangulation, both –A strategy for selecting the appropriate variation 3.Managing constraint propagation –Four queue-management strategies (QMSs) 4.Empirical evaluations on benchmark problems 11/28/2011Woodward-MS Thesis Defense2

3. Constraint Systems Laboratory Outline Background Relational Neighborhood Inverse Consistency (RNIC) –Property, characterization Dual Graphs of Binary CSPs –Complete constraint network –Non-complete constraint network –RNIC on binary CSPs Enforcing RNIC –Algorithm for RNIC –Dual-graph reformulation –Selection strategy Evaluating RNIC Propagation-Queue Management Conclusions & Future Work 11/28/2011Woodward-MS Thesis Defense3

4. Constraint Systems Laboratory Constraint Satisfaction Problem CSP –Variables, Domains –Constraints: Relations & scopes Representation –Hypergraph –Dual graph Solved with –Search –Enforcing consistency –Lookahead = Search + enforcing consistency 11/28/2011Woodward-MS Thesis Defense4 R4R4 BCD ABDE CF EF AB R3R3 R1R1 R2R2 C F E BD AB D AD A B R5R5 R6R6 R3R3 A B C D E F R1R1 R4R4 R2R2 R5R5 R6R6 Hypergraph Dual graph Warning –Consistency properties vs. algorithms

5. Constraint Systems Laboratory Neighborhood Inverse Consistency Property [Freuder+ 96] ↪ Every value can be extended to a solution in its variable’s neighborhood ↪ Domain-based property Algorithm  No space overhead  Adapts to graph connectivity 11/28/2011Woodward-MS Thesis Defense5 0,1,2 R0R0 R1R1 R3R3 R2R2 R4R4 A B C D R3R3 A B C D E F R1R1 R4R4 R2R2 R5R5 R6R6 Binary CSPs [Debruyene+ 01]  Not effective on sparse problems  Too costly on dense problems Non-binary CSPs?  Neighborhoods likely too large

6. Constraint Systems Laboratory Relational NIC 11/28/2011Woodward-MS Thesis Defense6 Property ↪ Every tuple can be extended to a solution in its relation’s neighborhood ↪ Relation-based property Algorithm –Operates on dual graph –Filters relations –Does not alter topology of graphs R4R4 BCD ABDE CF EF AB R3R3 R1R1 R2R2 C F E BD AB D AD A B R5R5 R6R6 R3R3 A B C D E F R1R1 R4R4 R2R2 R5R5 Hypergraph Dual graph Domain filtering –Property: RNIC+DF –Algorithm: Projection

7. Constraint Systems Laboratory From NIC to RNIC Neighborhood Inverse Consistency (NIC) [Freuder+ 96] –Proposed for binary CSPs –Operates on constraint graph –Filters domain of variables Relational Neighborhood Inverse Consistency (RNIC) –Proposed for both binary & non-binary CSPs –Operates on dual graph –Filters relations; last step projects updated relations on domains Both –Adapt consistency level to local topology of constraint network –Add no new relations (constraint synthesis) NIC was shown to be ineffective or costly, we show that RNIC is worthwhile 11/28/2011Woodward-MS Thesis Defense7

8. Constraint Systems Laboratory Characterizing RNIC: Binary CSPs On binary CSPs [Luchtel, 2011] –NIC (on the constraint graph) and RNIC (on the dual graph) are not comparable –Empirically, RNIC does more filtering than NIC 11/28/2011Woodward-MS Thesis Defense8

9. Constraint Systems Laboratory GAC, SGAC Variable-based properties So far, most popular for non- binary CSPs Characterizing RNIC (I): Nonbinary CSPs R(*,m)C [Karakashian+ 10] Relation-based property Every tuple has a support in every subproblem induced by a combination of m connected relations 11/28/2011Woodward-MS Thesis Defense9 GAC R(*,2)C+DF SGAC R(*,3)CR(*,δ+1)CR(*,2)C p’p : p is strictly weaker than p’ R4R4 R3R3 R1R1 R2R2 R5R5 R6R6

10. Constraint Systems Laboratory Characterizing RNIC (II): Nonbinary CSPs The fuller picture, details are in the thesis –w: Property weakened by redundancy removal –tri: Property strengthened by triangulation –δ: Degree of dual network 11/28/2011Woodward-MS Thesis Defense10 R(*,3)C wRNIC R(*,4)C RNIC wtriRNIC triRNIC R(*,δ+1)C wR(*,3)C wR(*,4)C wR(*,δ+1)C R(*,2)C ≡ wR(*,2)C

11. Constraint Systems Laboratory Outline Background Relational Neighborhood Inverse Consistency –Property, characterization Dual Graphs of Binary CSPs –Complete constraint network –Non-complete constraint network –RNIC on binary CSPs Enforcing RNIC –Algorithm for RNIC –Dual-graph reformulation –Selection strategy Evaluating RNIC Propagation-Queue Management Conclusions & Future Work 11/28/2011Woodward-MS Thesis Defense11

12. Constraint Systems Laboratory Complete Binary CSPs Triangle-shaped grid n-1 vertices for V 1 –C 1,i i ∈ [2,n] –Completely connected n-1 vertices for V i≥2 –Centered on C 1,i –i-2 along horizontal –n-i along vertical –Completely connected Not shown for clarity 11/28/2011Woodward-MS Thesis Defense12 V1V1 V2V2 V3V3 V n-1 VnVn uhuh uvuv C 1,i VnVn C n-1,n C 1,n V1V1 V n-1 VnVn VnVn C 3,n C 2,n V2V2 V3V3 V5V5 V5V5 V5V5 C 1,2 C 2,3 C 1,3 C 3,4 C 1,4 C 2,4 C 4,5 C 1,5 V1V1 C 3,5 C 2,5 V1V1 V1V1 V2V2 V2V2 V2V2 V3V3 V3V3 V4V4 V3V3 V4V4 V4V4 V5V5 V2V2 V3V3 V4V4 V1V1 V1V1

13. Constraint Systems Laboratory A triangle-shaped grid –Every CSP variable annotates a chain of length n-2 –Remove edges that link two non-consecutive vertices Complete Binary CSPs: RR (I) An edge is redundant if –There exists an alternate path between two vertices –Shared variables appear in every vertex in the path 11/28/2011Woodward-MS Thesis Defense13 VnVn C n-1,n C 1,n V1V1 V n-1 VnVn VnVn C 3,n C 2,n V2V2 V3V3 V5V5 V5V5 V5V5 C 2,3 C 1,3 C 3,4 C 2,4 C 4,5 C 1,5 V1V1 C 3,5 C 2,5 V1V1 V1V1 V2V2 V2V2 V2V2 V3V3 V3V3 V4V4 V3V3 V4V4 V4V4 V5V5 V2V2 V3V3 V4V4 V1V1 V1V1 uhuh uvuv C 1,i C 1,2 C 1,4 R4R4 BCD ABDEAB R3R3 BD A D AD A A B R5R5 R6R6 D B

14. Constraint Systems Laboratory Complete Binary CSPs: RR (II) 11/28/2011Woodward-MS Thesis Defense14 V1V1 V2V2 V3V3 V4V4 C 10 V5V5 C9C9 C8C8 C7C7 C1C1 C3C3 C4C4 C6C6 C2C2 C5C5 C1C1 C2C2 C3C3 C5C5 C6C6 C4C4 V1V1 V1V1 V2V2 V2V2 V3V3 V3V3 V4V4 V4V4 V5V5 C7C7 C8C8 C9C9 V4V4 V2V2 V5V5 V5V5 V4V4 V3V3 A redundancy-free dual graph is not unique No chain for V 2, but a star

15. Constraint Systems Laboratory Non-complete Binary CSPs Non-complete binary CSP –Is a complete binary constraint graph with missing edges In the dual graph –There are missing dual vertices –The dual vertices with variable V i in their scope are completely connected Redundancy-free dual graph –Can still form a chain using alternate edges 11/28/2011Woodward-MS Thesis Defense15 V1V1 V2V2 V3V3 V4V4 C 1,4 V5V5 C 2,5 C 3,5 C 1,2 C 1,5 C 2,3 C 3,4 V5V5 V5V5 V5V5 C 1,2 C 2,3 C 1,3 C 3,4 C 1,4 C 2,4 C 4,5 C 1,5 V1V1 C 3,5 C 2,5 V1V1 V1V1 V2V2 V2V2 V2V2 V3V3 V3V3 V4V4 V3V3 V4V4 V4V4 C 1,3 C 4,5 C 2,4

16. Constraint Systems Laboratory RNIC on Binary CSPs After RR, RNIC is never stronger than R(*,3)C Configurations for R(*,4)C –C 1 has three adjacent constraints C 2, C 3, C 4 –C 1 is not an articulation point Two configurations, neither is possible 11/28/2011Woodward-MS Thesis Defense16 C1C1 C3C3 C2C2 C4C4 V1V1 V1V1 V2V2 C1C1 C3C3 C2C2 C4C4 V1V1 V2V2 V1V1

17. Constraint Systems Laboratory Outline Background Relational Neighborhood Inverse Consistency –Property, characterization Dual Graphs of Binary CSPs –Complete constraint network –Non-complete constraint network –RNIC on binary CSPs Enforcing RNIC –Algorithm for RNIC –Dual-graph reformulation –Selection strategy Evaluating RNIC Propagation-Queue Management Conclusions & Future Work 11/28/2011Woodward-MS Thesis Defense17

18. Constraint Systems Laboratory Algorithm for Enforcing RNIC Two queues 1.Q : relations to be updated 2.Q t (R) : The tuples of relation R whose supports must be verified S EARCH S UPPORT ( τ, R ) –Backtrack search on Neigh( R ) Loop until all Q t ( ⋅ ) are empty 11/28/2011Woodward-MS Thesis Defense18 Complexity –Space: O ( ketδ ) –Time: O ( t δ+1 eδ ) –Efficient for a fixed δ..… Neigh( R ) R τ τiτi ✗ √

19. Constraint Systems Laboratory Improving Algorithm’s Performance 1.Use IndexTree [Karakashian+ AAAI10] –To quickly check consistency of 2 tuples 2.Dynamically detect dangles –Tree structures may show in subproblem @ each instantiation –Apply directional arc consistency 11/28/2011Woodward-MS Thesis Defense19 R4R4 R3R3 R1R1 R2R2 R5R5 R6R6 Note that exploiting dangles is –Not useful in R(*,m)C: small value of m, subproblem size –Not applicable to GAC: does not operate on dual graph

20. Constraint Systems Laboratory Reformulating the Dual Graph 11/28/2011Woodward-MS Thesis Defense20 R4R4 BCD ABDE CF EFAB R3R3 R1R1 R2R2 C F E BD A D AD A A B R5R5 R6R6 Cycles of length ≥ 4 –Hampers propagation –RNIC  R(*,3)C Triangulation (triRNIC) –Triangulate dual graph RR+Triangulation (wtriRNIC) Local, complementary, do not ‘clash’ wRNIC RNIC wtriRNIC triRNIC R4R4 BCD ABDE CF EFAB R3R3 R1R1 R2R2 C F E BD AB D AD A B R5R5 R6R6 High degree –Large neighborhoods –High computational cost Redundancy Removal (wRNIC) –Use minimal dual graph D B

21. Constraint Systems Laboratory Selection Strategy: Which? When? Density of dual graph ≥ 15% is too dense –Remove redundant edges Triangulation increases density no more than two fold –Reformulate by triangulation Each reformulation executed at most once 11/28/2011Woodward-MS Thesis Defense21 No YesNo Yes No d G o ≥ 15% d G tri ≤ 2 d G o d Gw tri ≤ 2 d G w GoGo G wtri GwGw G tri Start

22. Constraint Systems Laboratory Outline Background Relational Neighborhood Inverse Consistency –Property, characterization Dual Graphs of Binary CSPs –Complete constraint network –Non-complete constraint network –RNIC on binary CSPs Enforcing RNIC –Algorithm for RNIC –Dual-graph reformulation –Selection strategy Evaluating RNIC Propagation-Queue Management Conclusions & Future Work 11/28/2011Woodward-MS Thesis Defense22

23. Constraint Systems Laboratory Experimental Setup Backtrack search with full lookahead We compare –wR(*,m)C for m = 2,3,4 –GAC –RNIC and its variations General conclusion –GAC best on random problems –RNIC-based best on structured/quasi- structued problems We focus on non-binary results (960 instances) –triRNIC theoretically has the least number of nodes visited –selRNIC solves most instances backtrack free (652 instances) 11/28/2011Woodward-MS Thesis Defense23 Category#Binary#Non-binary Academic310 Assignment750 Boolean0160 Crossword0258 Latin square500 Quasi-random39025 Random980290 TSP030 Unsolvable Memory1060 All timed out44787

24. Constraint Systems Laboratory Experimental Results Statistical analysis on CP benchmarks Time: Censored data calculated mean Rank: Censored data rank based on probability of survival data analysis #F: Number of instances fastest 11/28/2011Woodward-MS Thesis Defense24 AlgorithmTimeRank#F [ ⋅ ] CPU #C [ ⋅ ] Completion #BT-free 169 instances: aim-100,aim-200,lexVg,modifiedRenault,ssa wR(*,2)C944924352A138B79 wR(*,3)C92500448B134B92 wR(*,4)C116126152B132B108 GAC1711511783C119C33 RNIC6161391819C100C66 triRNIC301716999C84C80 wRNIC118484468B131B84 wtriRNIC93790423B144B129 selRNIC751586117A159A142 [ ⋅ ] CPU : Equivalence classes based on CPU [ ⋅ ] Completion : Equivalence classes based on completion #C: Number of instances completed #BT-free: # instances solved backtrack free

25. Constraint Systems Laboratory Outline Background Relational Neighborhood Inverse Consistency –Property, characterization Dual Graphs of Binary CSPs –Complete constraint network –Non-complete constraint network –RNIC on binary CSPs Enforcing RNIC –Algorithm for RNIC –Dual-graph reformulation –Selection strategy Evaluating RNIC Propagation-Queue Management Conclusions & Future Work 11/28/2011Woodward-MS Thesis Defense25

26. Constraint Systems Laboratory Propagation-Queue Management Three directions for ordering the relations: 1.Arbitrary ordering (previous) 2.Perfect elimination ordering (PEO) of some triangulation 3.Ordering of the maximal cliques, corresponds to a tree-decomposition ordering (TD) 11/28/2011Woodward-MS Thesis Defense26 R4R4 BCD ABDE CF EF AB R3R3 R1R1 R2R2 AD R5R5 R6R6 PEO Maximal cliques Ordering R 1,R 2,R 4 C3C3 C4C4 R 6,R 4 R 5,R 4 R 1,R 3,R 4 C2C2 C1C1 R4R4 BCD ABDE CF EF AB R3R3 R1R1 R2R2 AD R5R5 R6R6

27. Constraint Systems Laboratory Queue-Management Strategies 11/28/2011Woodward-MS Thesis Defense27 R4R4 BCD ABDE CF EF AB R3R3 R1R1 R2R2 AD R5R5 R6R6 PEO Ordering Exact QMS a Arbitrary ordering of relations QMS PEO Perfect elimination ordering QMS TD Sequence of maximal cliques The cliques revised in sequence Each clique is revised until quiescence Revised back and forth until quiescence Lazy QMS LTD Same as QMS TD, except Cliques are traversed only once QMS L 2 T D Same as QMS TD, except traverses each clique only once each relation only once R4R4 BCD ABDE CF EF AB R3R3 R1R1 R2R2 AD R5R5 R6R6

28. Constraint Systems Laboratory R 1,R 2,R 4 C3C3 C4C4 R 6,R 4 R 5,R 4 R 1,R 3,R 4 C2C2 C1C1 Queue-Management Strategies Exact QMS a Arbitrary ordering of relations QMS PEO Perfect elimination ordering QMS TD Sequence of maximal cliques The cliques revised in sequence Each clique is revised until quiescence Revised back and forth until quiescence Lazy QMS LTD Same as QMS TD, except Cliques are traversed only once QMS L 2 T D Same as QMS TD, except traverses each clique only once each relation only once 11/28/2011Woodward-MS Thesis Defense28 maximal cliques Ordering R4R4 BCD ABDE CF EF AB R3R3 R1R1 R2R2 AD R5R5 R6R6

29. Constraint Systems Laboratory Propagation-Queue Management Statistical analysis on CP benchmarks Time: Censored data rank based on probability of survival data analysis triRNIC Pre-processingSATUNSAT StrategyTime [ ⋅ ] CP U % QMS a 1,410,292C-AC QMS PEO 1,186,691B16%AB QMS TD 765,976A46%AA 11/28/2011Woodward-MS Thesis Defense29 wtriRNIC Pre-processingSATUNSAT StrategyTime [ ⋅ ] CPU % QMS a 479,725A-AB QMS PEO 467,747A2%AA QMS TD 476,604A1%AB triRNIC SearchSATUNSAT StrategyTime [ ⋅ ] CPU % QMS a 1,243,917C-AC QMS PEO 900,069B28%AB QMS TD 416,464A67%AA QMS LTD 403,766A68%AA QMS L2TD 434,479A65%AA wtriRNIC SearchSATUNSAT StrategyTime [ ⋅ ] CPU % QMS a 628,523C-AC QMS PEO 582,629B7%AB QMS TD 519,578A17%AA QMS LTD 602,437C4%BA QMS L2TD 575,277C8%BA [ ⋅ ] CPU : Equivalence classes based on CPU %: Percent increased gain by the algorithm

30. Constraint Systems Laboratory Conclusions RNIC Structure of binary dual graph Algorithm for enforcing RNIC –Polynomial for fixed-degree dual graphs –BT-free search: hints to problem tractability Various reformulations of the dual graph Adaptive, unifying, self-regulatory, automatic strategy New propagation-queue management strategies Empirical evidence, supported by statistics 11/28/2011Woodward-MS Thesis Defense30

31. Constraint Systems Laboratory Future Work Extension to singleton-type consistencies Extension to constraints defined in intension –Possible by only domain filtering (weakening) Study influence of redundancy removal algorithms –Redundancy removal algorithm of [Janssen+ 89] seems to favor grids Evaluate new queue-management strategies on other consistency algorithms 11/28/2011Woodward-MS Thesis Defense31

32. Constraint Systems Laboratory Thank You! Questions? 11/28/2011Woodward-MS Thesis Defense32