1. Constraint Systems Laboratory Robert J. Woodward Joint work with Shant Karakashian, Daniel Geschwender, Christopher Reeson, and Berthe Y. Choueiry @ UNL Christian Bessiere @ LIRMM-CNRS Support Experiments conducted at UNL’s Holland Computing Center NSF Graduate Research Fellowship & NSF Grant No. RI-111795 18 Oct. 2013Coconut Talk1 Recent Advances in High-Level Relational Consistency

2. Constraint Systems Laboratory Publications Relational m-wise consistency, R( ∗,m)C –Relational Consistency by Constraint Filtering[SAC 10] –A First Practical Algorithm for High Levels of Relational Consistency[AAAI 10] –Improving the Performance of Consistency Algorithms by Localizing and Bolstering Propagation in a Tree Decomposition[AAAI 13] Relational Neighborhood Inverse Consistency, RNIC –Solving Difficult CSPs with Relational Neighborhood Inverse Consistency[AAAI 11] –Adaptive Neighborhood Inverse Consistency as Lookahead for Non-Binary CSPs [AAAI-SA 11] –Reformulating the Dual Graphs of CSPs to Improve the Performance of Relational Neighborhood Inverse Consistency[SARA 11] –Revisiting Neighborhood Inverse Consistency on Binary CSPs[CP 12] –Selecting the Appropriate Consistency Algorithm for CSPs Using Machine Learning Classifiers[AAAI- SA13] MS thesis, Woodward, Dec 2011 PhD thesis, Karakashian, May 2013 Papers and slides available on lab website, consystlab.unl.educonsystlab.unl.edu 18 Oct. 2013Coconut Talk2

3. Constraint Systems Laboratory Overview Background Relational m-wise consistency, R( ∗,m)C [SAC10,AAAI10] –Property, Algorithm, Weakening –Characterization, Evaluating Relational Neighborhood Inverse Consistency (RNIC) [AAAI11,SARA11] –Property, Algorithm –Dual-graph reformulation, Characterization, Selection strategy –Evaluating Dual Graphs of Binary CSPs [CP2012] –Complete constraint network, Non-complete constraint network –RNIC on binary CSPs –Characterization, Evaluating Conclusions 18 Oct. 2013Coconut Talk3

4. Constraint Systems Laboratory Constraint Satisfaction Problem CSP –Variables, Domains –Constraints: Relations & scopes Representation –Hypergraph –Dual graph Solved with –Search –Enforcing consistency –Lookahead = Search + enforcing consistency 18 Oct. 2013Coconut Talk4 R4R4 BCD ABDE CF EF AB R3R3 R1R1 R2R2 C F E BD AB D AD A B R5R5 R6R6 R3R3 A B C D E F R1R1 R4R4 R2R2 R5R5 R6R6 Hypergraph Dual graph Key to our research –Operate on the dual graph

5. Constraint Systems Laboratory Relational m-wise consistency, R( ∗,m)C 18 Oct. 2013Coconut Talk5 [SAC 2010, AAAI 2010] A parameterized relational consistency property Definition –For every set of m constraints –every tuple in a relation can be extended to an assignment –of variables in the scopes of the other m-1 relations R( ∗,m)C ≡ every m relations form a minimal CSP

6. Constraint Systems Laboratory Algorithms for Enforcing R( ∗,m)C 18 Oct. 2013Coconut Talk6 P ER T UPLE –For each tuple find a solution for the variables in the m-1 relations –Many satisfiability searches Effective when there are many solutions Each search is quick & easy t1t1 titi t2t2 t3t3 A LL S OL –Find all solutions of problem induced by m relations, & keep their tuples –A single exhaustive search Effective when there are few or no solutions Hybrid Solvers (portfolio based) [+Scott]

7. Constraint Systems Laboratory Index-Tree Data Structure Goal: quickly find matching tuples in other relations Given two relations, R 1 & R 2 For a given tuple in R 1, find matching tuples in R 2 18 Oct. 2013Coconut Talk7 R2R2 ABCD t1t1 0010 t2t2 0110 t3t3 0111 t4t4 1111 0 0 1 1 1 1 1 1 t1t1 t2t2 t3t3 t4t4 A B C Root R1R1 XABC τ1τ1 0001 τ2τ2 1001 τ3τ3 0000 τ4τ4 0011

8. Constraint Systems Laboratory Weaken R( ∗,m)C by removing redundant edges [Jégou 89] R 1 R 2 R 3 R 1 R 2 R 4 R 1 R 2 R 5 R 1 R 3 R 4 R 2 R 3 R 4 R 2 R 4 R 5 R 3 R 4 R 5 Weakening R( ∗,m)C 8Coconut Talk R 1 R 2 R 3 R 1 R 2 R 5 R 1 R 3 R 4 R 2 R 4 R 5 R 3 R 4 R 5 C G ABD ACEG BCF ADE CFG R1R1 R3R3 R2R2 R4R4 R5R5 C CF CG AD AE ABD ACEG BCF ADE CFG R1R1 R3R3 R2R2 R4R4 R5R5 B 18 Oct. 2013 R( ∗,3)CwR( ∗,3)C C A minimal dual graph EA DA A B C F

9. Constraint Systems Laboratory Characterizing R( ∗,m)C 18 Oct. 2013Coconut Talk9 GACmaxRPWC R3C R( ∗,2)C wR( ∗,2)C R2C R( ∗,3)CR( ∗,4)C R4C wR( ∗,3)CwR( ∗,4)C R( ∗,m)C RmCRmC wR( ∗,m)C GAC [Waltz 75] maxRPWC [Bessiere+ 08] RmC: Relational m Consistency [Dechter+ 97] [Jégou 89] p’p’p : p is strictly weaker than p’

10. Constraint Systems Laboratory Empirical Evaluations (1) 18 Oct. 2013Coconut Talk10 AlgorithmAvg. #Nodes Avg. Time sec #Complete d #Fastest#BF SAT aim-100 (instances: 16, vars: 100, dom: 2, rels: 307, arity: 3) GAC9,459,773. 0 759.71541 wR( ∗,2)C 234,526.7125.61675 wR( ∗,3)C 3,979.119.41637 wR( ∗,4)C 559.126.31629 SAT modifiedRenault (instances: 19, vars: 110, dom: 42, rels: 128, arity: 10) GAC1,171,458. 4 108.517145 wR( ∗,2)C 211.55.01957 wR( ∗,3)C 110.413.319014 wR( ∗,4)C 110.281.319016

11. Constraint Systems Laboratory Empirical Evaluations (2) 18 Oct. 2013Coconut Talk11 AlgorithmAvg. #Nodes Avg. Time sec #Complete d #Fastest#BF UNSAT aim-100 (instances: 8, vars: 100, dom: 2, rels: 173, arity: 3) GAC--000 wR( ∗,2)C 4,619,373. 0 2,016.8310 wR( ∗,3)C 18,766.697.4430 wR( ∗,4)C 18,685.3944.2411 UNSAT modifiedRenault (instances: 31, vars: 111, dom: 42, rels: 130, arity: 10) GAC1,171,458. 4 782.3920 wR( ∗,2)C 487.05.2282025 wR( ∗,3)C 0.09.630228 wR( ∗,4)C 0.044.2312

12. Constraint Systems Laboratory Overview Background Relational Consistency R( ∗,m)C [SAC10,AAAI10] –Property, Algorithm, Weakening –Characterization, Evaluating Relational Neighborhood Inverse Consistency (RNIC) [AAAI11,SARA11] –Property, Algorithm –Dual-graph reformulation, Characterization, Selection strategy –Evaluating Dual Graphs of Binary CSPs [CP2012] –Complete constraint network, Non-complete constraint network –RNIC on binary CSPs –Characterization, Evaluating Conclusions 18 Oct. 2013Coconut Talk12

13. Constraint Systems Laboratory Neighborhood Inverse Consistency Property [Freuder+ 96] ↪ Every value can be extended to a solution in its variable’s neighborhood ↪ Domain-based property Algorithm  No space overhead  Adapts to graph connectivity 18 Oct. 2013Coconut Talk13 0,1,2 R0R0 R1R1 R3R3 R2R2 R4R4 A B C D R3R3 A B C D E F R1R1 R4R4 R2R2 R5R5 R6R6 Binary CSPs [Debruyene+ 01]  Not effective on sparse problems  Too costly on dense problems Non-binary CSPs?  Neighborhoods likely too large

14. Constraint Systems Laboratory Relational NIC 18 Oct. 2013Coconut Talk14 Property ↪ Every tuple can be extended to a solution in its relation’s neighborhood ↪ Relation-based property Algorithm –Operates on dual graph –Filters relations –Does not alter topology of graphs R4R4 BCD ABDE CF EF AB R3R3 R1R1 R2R2 C F E BD AB D AD A B R5R5 R6R6 R3R3 A B C D E F R1R1 R4R4 R2R2 R5R5 Hypergraph Dual graph Domain filtering –Property: RNIC+DF –Algorithm: Projection

15. Constraint Systems Laboratory From NIC to RNIC Neighborhood Inverse Consistency (NIC) [Freuder+ 96] –Proposed for binary CSPs –Operates on constraint graph –Filters domain of variables Relational Neighborhood Inverse Consistency (RNIC) –Proposed for both binary & non-binary CSPs –Operates on dual graph –Filters relations; last step projects updated relations on domains Both –Adapt consistency level to local topology of constraint network –Add no new relations (no constraint synthesis) 18 Oct. 2013Coconut Talk15

16. Constraint Systems Laboratory Algorithm for Enforcing RNIC Two queues 1.Q : relations to be updated 2.Q t (R) : The tuples of relation R whose supports must be verified S EARCH S UPPORT ( τ, R ) –Backtrack search on Neigh( R ) Loop until all Q t ( ⋅ ) are empty 18 Oct. 2013Coconut Talk16 Complexity –Space: O ( ketδ ) –Time: O ( t δ+1 eδ ) –Efficient for a fixed δ..… Neigh( R ) R τ τiτi ✗ √

17. Constraint Systems Laboratory Improving Algorithm’s Performance Dynamically detect dangles –Tree structures may show in subproblem @ each instantiation –Apply directional arc consistency 18 Oct. 2013Coconut Talk17 R4R4 R3R3 R1R1 R2R2 R5R5 R6R6 Note that exploiting dangles is –Not useful in R( ∗,m)C: small value of m, subproblem size –Not applicable to GAC: does not operate on dual graph

18. Constraint Systems Laboratory High density –Large neighborhoods –Higher cost of RNIC Minimal dual graph –Equivalent CSP –Computed efficiently [Janssen+ 89] Run algorithm on a minimal dual graph  Smaller neighborhoods, solution set not affected  wRNIC: a strictly weaker property Reformulation: Removing Redundant Edges 18 Oct. 2013Coconut Talk18 d Go = 60% d Gw = 40% wRNICRNIC R4R4 BCD ABDE CF EFAB R3R3 R1R1 R2R2 C F E BD A D AD A A B R5R5 R6R6 D B

19. Constraint Systems Laboratory Reformulation: Triangulation Cycles of length ≥ 4 –Hampers propagation Triangulating dual graph –Equivalent CSP –Min-fill heuristic Run algorithm on a triangulated dual graph  Created loops enhance propagation –triRNIC: a strictly stronger property 18 Oct. 2013Coconut Talk19 R4R4 BCD ABDE CF EFAB R3R3 R1R1 R2R2 C F E BD AB D AD A B R5R5 R6R6 d Go = 60% d Gtri = 67% wRNIC RNIC triRNIC

20. Constraint Systems Laboratory Reformulation: RR & Triangulation Fixing the dual graph –RR copes with high density –Triangulation boosts propagation RR+Tri –Both operate locally –Are complementary, do not ‘clash’ Run algorithm on a RR+tri dual graph –CSP solution set is not affected –wtriRNIC is not comparable with RNIC 18 Oct. 2013Coconut Talk20 R4R4 BCD ABDE CF EFAB R3R3 R1R1 R2R2 C F E BD AB D AD A B R5R5 R6R6 d Go = 60% d Gwtri = 47% R4R4 BCD ABDE CF EFAB R3R3 R1R1 R2R2 C F E BD AB D AD A B R5R5 R6R6 wRNIC RNIC wtriRNIC triRNIC

21. Constraint Systems Laboratory Selection Strategy: Which? When? Density of dual graph ≥ 15% is too dense –Remove redundant edges Triangulation increases density no more than two fold –Reformulate by triangulation Each reformulation executed at most once 18 Oct. 2013Coconut Talk21 No YesNo Yes No d G o ≥ 15% d G tri ≤ 2 d G o d Gw tri ≤ 2 d G w GoGo G wtri GwGw G tri Start

22. Constraint Systems Laboratory GAC, SGAC Variable-based properties Characterizing RNIC R( ∗,m)C Relation-based property 18 Oct. 2013Coconut Talk22 GAC R(*,2)C+DF SGAC R(*,3)CR(*,δ+1)CR(*,2)C p’p’p : p is strictly weaker than p’

23. Constraint Systems Laboratory Characterizing RNIC The fuller picture –w: Property weakened by redundancy removal –tri: Property strengthened by triangulation –δ: Degree of dual network 18 Oct. 2013Coconut Talk23 R(*,3)C wRNIC R(*,4)C RNIC wtriRNIC triRNIC R(*,δ+1)C wR(*,3)C wR(*,4)C wR(*,δ+1)C R(*,2)C ≡ wR(*,2)C

24. Constraint Systems Laboratory Experimental Setup Backtrack search with full lookahead We compare –wR( ∗,m)C for m = 2,3,4 –GAC –RNIC and its variations General conclusion –GAC best on random problems –RNIC-based best on structured/quasi- structued problems We focus on non-binary results (960 instances) –triRNIC theoretically has the least number of nodes visited –selRNIC solves most instances backtrack free (652 instances) 18 Oct. 2013Coconut Talk24 Category#Binary#Non-binary Academic310 Assignment750 Boolean0160 Crossword0258 Latin square500 Quasi-random39025 Random980290 TSP030 Unsolvable Memory1060 All timed out44787

25. Constraint Systems Laboratory Experimental Results Statistical analysis on CP benchmarks Time: Censored data calculated mean Rank: Censored data rank based on probability of survival data analysis #F: Number of instances fastest 18 Oct. 2013Coconut Talk25 AlgorithmTimeRank#F [ ⋅ ] CPU #C [ ⋅ ] Completion #BT-free 169 instances: aim-100,aim-200,lexVg,modifiedRenault,ssa wR( ∗,2)C 944,924352A138B79 wR( ∗,3)C 925,00448B134B92 wR( ∗,4)C 1,161,26152B132B108 GAC1,711,511783C119C33 RNIC6,161,391819C100C66 triRNIC3,017,16999C84C80 wRNIC1,184,84468B131B84 wtriRNIC937,90423B144B129 selRNIC751,586117A159A142 [ ⋅ ] CPU : Equivalence classes based on CPU [ ⋅ ] Completion : Equivalence classes based on completion #C: Number of instances completed #BT-free: # instances solved backtrack free

26. Constraint Systems Laboratory Overview Background Relational Consistency R( ∗,m)C [SAC10,AAAI10] –Property, Algorithm, Weakening –Characterization, Evaluating Relational Neighborhood Inverse Consistency (RNIC) [AAAI11,SARA11] –Property, Algorithm –Dual-graph reformulation, Characterization, Selection strategy –Evaluating Dual Graphs of Binary CSPs [CP2012] –Complete constraint network, Non-complete constraint network –RNIC on binary CSPs –Characterization, Evaluating Conclusions 18 Oct. 2013Coconut Talk26

27. Constraint Systems Laboratory Neighborhood Inverse Consistency 18 Oct. 2013Coconut Talk27 Relational NIC [Woodward+ AAAI 11] –Reformulation of NIC [Freuder & Elfe, AAAI 96] –Defined for dual graph –Algorithm operates on dual graph & filter relations (not domains!) –Initially designed for non-binary CSPs How about RNIC on binary CSPs? –Impact of the structure of the dual graph R3R3 R2R2 R1R1 R4R4 A B C D

28. Constraint Systems Laboratory Complete Constraint Graph 18 Oct. 2013Coconut Talk28 V1V1 V2V2 V3V3 V n-1 VnVn C 1,2 VnVn C n-1,n C 1,n V1V1 V n-1 VnVn C 3,n C 2,n V2V2 V3V3 V5V5 V5V5 V5V5 C 2,3 C 1,3 C 3,4 C 1,4 C 2,4 C 4,5 C 1,5 V1V1 C 3,5 C 2,5 V1V1 V1V1 V2V2 V2V2 V2V2 V3V3 V3V3 V4V4 V3V3 V4V4 V4V4 V5V5 V2V2 V3V3 V4V4 V1V1 V1V1 VnVn VnVn C 4,n V4V4 C i-2,i C 1,i ViVi ViVi C 3,i C 2,i C i,i+1 C i,n-1 C i,n ViVi ViVi ViVi ViVi (i-2) vertices (n-i) vertices ViVi ViVi ViVi Dual Graph: Triangle shaped grids

29. Constraint Systems Laboratory Minimal Dual Graph 18 Oct. 2013Coconut Talk29 V1V1 V2V2 V3V3 V n-1 VnVn VnVn C n-1,n C 1,n V1V1 V n-1 VnVn VnVn C 3,n C 2,n V2V2 V3V3 V5V5 V5V5 V5V5 C 1,2 C 2,3 C 1,3 C 3,4 C 1,4 C 2,4 C 4,5 C 1,5 V1V1 C 3,5 C 2,5 V1V1 V1V1 V2V2 V2V2 V2V2 V3V3 V3V3 V4V4 V3V3 V4V4 V4V4 V5V5 V2V2 V3V3 V4V4 V1V1 V1V1 C i-2,i C 1,i ViVi ViVi C 3,i C 2,i C i,i+1 C i,n-1 C i,n ViVi ViVi ViVi ViVi (i-2) vertices (n-i) vertices ViVi ViVi ViVi VnVn C 4,n V4V4 Dual Graph: Triangle shaped grids

30. Constraint Systems Laboratory Minimal Dual Graph … can be a triangle- shaped grid (planar) 18 Oct. 2013Coconut Talk30 V5V5 V5V5 V5V5 C 1,2 C 2,3 C 1,3 C 3,4 C 1,4 C 2,4 C 4,5 C 1,5 V1V1 C 3,5 C 2,5 V1V1 V1V1 V2V2 V2V2 V2V2 V3V3 V3V3 V4V4 V3V3 V4V4 V4V4 … but does not have to be –Star on V 2 –Cycle of size 6 C 1,5 C 1,3 C 3,5 C 2,4 C 4,5 C 3,4 V1V1 V1V1 V2V2 V2V2 V3V3 V3V3 V4V4 V4V4 V5V5 C 1,2 C 1,4 C 2,3 C 2,5 V1V1 V2V2 V5V5 V5V5 V4V4 V3V3 V1V1 V2V2 V3V3 V4V4 C 1,4 V5V5 C 2,5 C 3,5 C 1,2 C 1,5 C 2,3 C 3,4 C 4,5 C 1,3 C 2,4

31. Constraint Systems Laboratory Non-Complete Constraint Graph Can still be a triangle-shaped grid –Have a chain of vertices –of length ≤ n-1 18 Oct. 2013Coconut Talk31 V1V1 V2V2 V3V3 V4V4 C 1,4 V5V5 C 2,5 C 3,5 C 1,2 C 1,5 C 2,3 C 3,4 C 1,2 C 2,3 C 3,4 C 1,4 C 1,5 V1V1 C 3,5 C 2,5 V1V1 V2V2 V2V2 V3V3 V3V3 V4V4 V5V5 V5V5

32. Constraint Systems Laboratory wRNIC on Binary CSPs 18 Oct. 2013Coconut Talk32 C1C1 C3C3 C2C2 C4C4 V1V1 V1V1 V2V2 V3V3 C1C1 C3C3 C2C2 C4C4 V1V1 V2V2 V1V1 V3V3 On a binary CSP, RNIC enforced on the minimal dual graph (wRNIC) is never strictly stronger than R(*,3)C. wRNIC can never consider more than 3 relations In either case, it is not possible to have an edge between C 3 & C 4 (a common variable to C 3 & C 4 ) while keeping C 3 as a binary constraint

33. Constraint Systems Laboratory NIC, sCDC, and RNIC not comparable NIC Property [Freuder & Elfe, AAAI 96] ↪ Every value can be extended to a solution in its variable’s neighborhood 18 Oct. 2013Coconut Talk33 A B C D R3R3 R2R2 R1R1 R4R4 sCDC Property [Lecoutre+, JAIR 11] ↪ An instantiation {(x,a),(y,b)} is DC iff (y,b) holds in SAC when x=a and (x,a) holds in SAC when y=b and (x,y) in scope of some constraint. Further, the problem is also AC. RNIC Property [Woodward+, AAAI 11] ↪ Every tuple can be extended to a solution in its relation’s neighborhood ↪ wRNIC, triRNIC, wtriRNIC enforce RNIC on a minimal, triangulated, and minimal triangulated dual graph, respectively ↪ selRNIC automatically selects the RNIC variant based on the density of the dual graph

34. Constraint Systems Laboratory Experimental Results (CPU Time) 18 Oct. 2013Coconut Talk34 Benchmark# inst.AC3.1sCDC1NICselRNIC CPU Time (msec) NIC Quickest bqwh-16-106100/1003,5053,8601,4703,608 bqwh-18-141100/10068,62982,77238,87777,981 coloring-sgb-queen12/50680,140(+3) -(+9) 57,545634,029 coloring-sgb-games3/441,31733,307(+1) 86041,747 rand-2-2310/101,467,2461,460,089987,3121,171,444 rand-2-243/10567,620677,253(+7) 3,456,437677,883 sCDC Quickest driver2/7(+5) 70,990(+5) 17,070358,790(+4) 185,220 ehi-8587/100(+13) 27,304(+13) 573513,459(+13) 75,847 ehi-9089/100(+11) 34,687(+11) 605713,045(+11) 90,891 frb35-1710/1041,24938,927179,76373,119 RNIC Quickest composed-25-1-2510/102263351,457114 composed-25-1-210/102232831,45088 composed-25-1-409/10(+1) 288(+1) 357120,544(+1) 137 composed-25-1-8010/10223417(+1) -190 composed-75-1-2510/102,7011,444363,785305 composed-75-1-210/102,3491,73348,249292 composed-75-1-407/10(+1) 1,924(+3) 1,647631,040(+3) 286 composed-75-1-8010/101,4841,473(+1) -397

35. Constraint Systems Laboratory Experimental Results (BT-free, #NV) 18 Oct. 2013Coconut Talk35 Benchmark# inst.AC3.1sCDC1NICselRNICAC3.1sCDC1NICselRNIC BT-Free#NV NIC Quickest bqwh-16-106100/10003851,8071,8817391,310 bqwh-18-141100/100001025,28325,99812,49022,518 coloring-sgb-queen12/501-16191,853-15,79891,853 coloring-sgb-games3/4114114,368 4014,368 rand-2-2310/1000100471,111 12471,111 rand-2-243/1000100222,085 24222,085 sCDC Quickest driver2/712113,8934093,763 ehi-8587/1000100871001,425000 ehi-9089/1000100891001,298000 frb35-1710/10000024,491 24,346 RNIC Quickest composed-25-1-2510/10010 153000 composed-25-1-210/10010 162000 composed-25-1-409/100109 172000 composed-25-1-8010/10010- 1120-0 composed-75-1-2510/10010 345000 composed-75-1-210/10010 346000 composed-75-1-407/100107 335000 composed-75-1-8010/10010- 1990-0

36. Constraint Systems Laboratory Conclusions Introduced R( ∗,m)C, RNIC Algorithm for enforcing R( ∗,m)C and RNIC –BT-free search: hints to problem tractability Various reformulations of the dual graph Adaptive, unifying, self-regulatory, automatic strategy for RNIC Structure of binary dual graph Empirical evidence, supported by statistics 18 Oct. 2013Coconut Talk36

37. Constraint Systems Laboratory Thank You! Questions? 18 Oct. 2013Coconut Talk37

38. Constraint Systems Laboratory Enforcing R(*,m)C on the Induced Dual CSP P ω R1R1 R3R3 R5R5 For each τ in R Assign τ as a value for R Solve P ω (with τ fixed) with forward checking Extract from Q Q ω1ω1 ω2ω2 ω3ω3 AB CBCB R 1 : A BR 2 : B C R 5 : C F G C If no solution found: delete τ Define CSP P ω DE CBCB R 3 : D ER 4 : E F R 5 : C F G C Add to Q: R i ≠R’, R i ∈ ω’ and R’ ∈ ω’ 38 ω1ω1 ω2ω2 ω3ω3 R2R2 R4R4 BC EF CFG 18 Oct. 2013Coconut Talk