1. Constraint Systems Laboratory 10/24/2015Bayer–MS Thesis Defense1 Reformulating Constraint Satisfaction Problems with Application to Geospatial Reasoning Ken Bayer Constraint Systems Laboratory Department of Computer Science & Engineering University of Nebraska-Lincoln Supported by NSF CAREER award #0133568 & AFOSR grants FA9550-04-1-0105, FA9550-07-1-0416

2. Constraint Systems Laboratory Main contributions 1.Four new reformulation techniques for CSPs –Query reformulation –Domain reformulation –Constraint relaxation –Reformulation via symmetry detection 2.BID as a CSP [Michalowski & Knoblock, AAAI 05] –Improved constraint model –Showed that original BID is in P –Custom solver 3.Application of the reformulations to BID problem 10/24/20152Bayer–MS Thesis Defense

3. Constraint Systems Laboratory Outline Background BID model & custom solver Reformulation techniques –Description –General use in CSPs –Application to BID –Evaluation on real-world BID data Conclusions & future work 10/24/20153Bayer–MS Thesis Defense

4. Constraint Systems Laboratory Motivation: finding my house 10/24/2015Bayer–MS Thesis Defense4 Google Maps Yahoo Maps Actual location Microsoft Live Local (as of November 2006)

5. Constraint Systems Laboratory Abstraction & Reformulation … may be an approximation Original formulation Original query Reformulated formulation Reformulated query Original problemReformulated problem Reformulation technique Solutions( P r )  (Solutions( P o )) Solutions( P o ) Original spaceReformulated space 10/24/20155Bayer–MS Thesis Defense

6. Constraint Systems Laboratory Constraint Satisfaction Problems Formulation : F = ( V, D, C ) –V = set of variables –D = set of their domains –C = set of constraints restricting the acceptable combination of values for variables Query: All consistent solutions, a single solution, etc. Solved with –Constraint propagation –Search 10/24/20156Bayer–MS Thesis Defense

7. Constraint Systems Laboratory Constraint propagation Remove values that cannot appear in any solution Arc consistency 10/24/20157Bayer–MS Thesis Defense

8. Constraint Systems Laboratory Building Identification (BID) problem Data input: map layout + phone book Basic numbering rules Additional information 10/24/20158Bayer–MS Thesis Defense B6 B8 B2 B4 B5 B3 B9 B10B7 B1 S1S2 S3 Si = Building = Corner building = Street S1#1,S1#4, S1#8,S2#7, S2#8,S3#1, S3#2,S3#3, S3#15 Phone Book

9. Constraint Systems Laboratory Basic numbering rules Ordering: Increasing/decreasing numbers along a street Parity: Odd/even numbers on opposing sides of a street Phone book: Complete/incomplete –Assumption: all addresses in phone-book must be used Ordering B1 << B2B3 Odd Even Parity B1 B2 B3 B4 Phone book # 17 # 29 # 54 B1B2 B3 B4 10/24/20159Bayer–MS Thesis Defense

10. Constraint Systems Laboratory Additional information Landmarks B1B2 1600 Pennsylvania Avenue Gridlines B1B2 S1 S1 #1xxS1 #2xx 10/24/201510Bayer–MS Thesis Defense

11. Constraint Systems Laboratory Query 1.Given an address, what buildings could it be? 2.Given a building, what addresses could it have? 10/24/201511Bayer–MS Thesis Defense B6 B8 B2 B4 B5 B3 B9 B10B7 B1 S1S2 S3 Si = Building = Corner building = Street S1#1,S1#4, S1#8,S2#7, S2#8,S3#1, S3#2,S3#3, S3#15 S1#1, S3#1, S3#15 Phone Book

12. Constraint Systems Laboratory Outline Background BID model & custom solver Reformulation techniques Conclusions & future work 10/24/20151210/24/201512 Bayer–MS Thesis Defense

13. Constraint Systems Laboratory CSP model: variables Orientation variables –OddOnNorth, OddOnEast, IncreasingNorth, IncreasingEast –Domains: {true, false} Corner variables –Corner buildings –Domains: set of streets Building variables –Corner & non-corner buildings –Domains: set of addresses D B1c = { S1, S2 } D B1 = { S1 # 2, S1 # 4,…, S1 # 228, S2 # 5, S2 # 9,…, S2 # 25 } S1 S2 D B2 = { S2 # 5, S2 # 9,…, S2 # 25 } B2 B1 B1c 10/24/201513Bayer–MS Thesis Defense

14. Constraint Systems Laboratory CSP model: constraints S1 S2 B2 OddOnNorth B1 B1c B3 B4 B5 B1 B2 IncreasingEast 10/24/201514Bayer–MS Thesis Defense

15. Constraint Systems Laboratory Example constraint network 10/24/201515Bayer–MS Thesis Defense B6 B8 B2 B4 B5 B3 B9 B10B7 B1 S1S2 S3 Si = Building = Corner building = Street S1#1,S1#4, S1#8,S2#7, S2#8,S3#1, S3#2,S3#3, S3#15

16. Constraint Systems Laboratory 1.Orientations vary per street (e.g., Belgrade) 2.Non-corner building on two streets 3.Corner building on more than two streets  All gracefully handled by the model Special configurations 10/24/201516Bayer–MS Thesis Defense

17. Constraint Systems Laboratory Custom solver Backtrack search Conflict-directed backjumping Forward checking (nFC3) Domains implemented as intervals (box consistency) Variable ordering 1.Orientation variables 2.Corner variables 3.Building variables Backdoor variables –Orientation + corner variables Orientation & corner variables Building variables Filter values 10/24/201517Bayer–MS Thesis Defense

18. Constraint Systems Laboratory Backdoor variables We instantiate only orientation & corner variables We guarantee solvability without instantiating building variables B6B8B11B2B4 B5 B3 B9 B10 B7 B1 B6B8B11B2B4 B5 B3 B9 B10 B7 B1 10/24/201518Bayer–MS Thesis Defense

19. Constraint Systems Laboratory Features of new model & solver Model –Reflects topology –Reduces number of variables –Reduces constraint arity –Constraints can be declared locally & in restricted ‘contexts’ (feature important for Michalowski’s work) Solver –Exploits structure of problem –Implements domains as possibly infinite intervals –Incorporates all reformulations (to be introduced) Improvement over previous work [Michalowski+, 05] 10/24/201519Bayer–MS Thesis Defense

20. Constraint Systems Laboratory Outline Background BID model & custom solver Reformulation techniques –Query reformulation –AllDiff-Atmost & domain reformulation –Constraint relaxation –Reformulation via symmetry detection Conclusions & future work 10/24/20152010/24/201520Bayer–MS Thesis Defense

21. Constraint Systems Laboratory Query in the BID Problem: BID instances have many solutions We only need to know which values appear in at least one solution B1B2B3B4 Phone book: {4,8} B1B2B3B4 2468 24810 24812 481012 46810 46812 10/24/201521Bayer–MS Thesis Defense

22. Constraint Systems Laboratory Query reformulation Query: Find all solutions, collect values for variables Query: For each variable- value combination, determine satisfiability Solving the BIDReformulated BID Query reformulation Original queryReformulated query Single counting problemMany satisfiability problems All solutionsPer-variable solution Exhaustive searchOne path Impractical when there are many solutions Costly when there are few solutions 10/24/201522Bayer–MS Thesis Defense

23. Constraint Systems Laboratory Evaluations: real-world data from El Segundo [Shewale] Case studyPhone bookNumber of… completenessbuildingscorner bldgsblocks NSeg125-c100.0% 125174 NSeg125-i45.6% NSeg206-c100.0% 206287 NSeg206-I50.5% SSeg131-c100.0% 131368 SSeg131-i60.3% SSeg178-c100.0% 1784612 SSeg178-i65.6% Previous work did not scale up beyond 34 bldgs, 7 corner bldgs, 1 block All techniques tested return same solutions 10/24/201523Bayer–MS Thesis Defense

24. Constraint Systems Laboratory Evaluation: query reformulation Case studyOriginal queryNew query [s] NSeg125-i>1 week744.7 NSeg206-i>1 week14,818.9 SSeg131-i>1 week66,901.1 SSeg178-i>1 week119,002.4 Case studyOriginal query [s]New query [s] NSeg125-c1.5139.2 NSeg206-c20.24,971.2 SSeg131-c1123.438,618.4 SSeg178-c3291.2117,279.1 Incomplete phone book → many solutions → better performance Complete phone book → few solutions → worse performance 10/24/201524Bayer–MS Thesis Defense

25. Constraint Systems Laboratory Generalizing query reformulation Relational (i,m)-consistency, R(i,m)C –Given m constraints, let s be the size of their scope –Compute all solutions of length s –To generate tuples of length i (i.e., constraints of arity i) –Space: O(d s ) Query reformulation –For each combination of values for i variables –Try to extend to one solution of length s –Space: O( ( ) d i ), i < s Per-variable solution computes R(1,| C |)C s i 10/24/201525Bayer–MS Thesis Defense

26. Constraint Systems Laboratory Outline Background BID model & custom solver Reformulation techniques –Query reformulation –AllDiff-Atmost & domain reformulation –Constraint relaxation –Reformulation via symmetry detection Conclusions & future work 10/24/20152610/24/201526Bayer–MS Thesis Defense

27. Constraint Systems Laboratory Domain reformulation Domains in the BID are large Min/max value? Enumerate? B1B2B3B4 Phonebook = {3,8} [3,8] [0,245] [0, ] B1B2B3B4 {1,2,3,…,8} 10/24/201527Bayer–MS Thesis Defense

28. Constraint Systems Laboratory AllDiff-Atmost constraint AllDiff-Atmost( A,k,d) –For a set of variables A, ensure that the variables in A collectively are not assigned more than k values from the set d Three expansion slots { High end graphics card, Low end graphics card, Sound card, 10MB ethernet card, 100MB ethernet card, 1GB ethernet card, …} At most one network card 10/24/201528Bayer–MS Thesis Defense

29. Constraint Systems Laboratory AllDiff-Atmost reformulation Replaces –interval d of values (potentially infinite) –with k symbolic values 10/24/201529Bayer–MS Thesis Defense

30. Constraint Systems Laboratory AllDiff-Atmost in the BID B1B2B3B4 Phone book: {12,28} B5 Cannot use more than –3 addresses less than 12 –3 addresses in (12,28) –3 addresses more than 28 Even side 1228303234 1214162836 1012142028 2461228 ……1228… { s 1, s 2, s 3, 12, s 4, s 5, s 6, 28, s 7, s 8, s 9 } { 2, 4, …, 10, 12, 14, …, 26, 28, 30, …, 998, 1000 } Original domain Reformulated domain 10/24/201530Bayer–MS Thesis Defense

31. Constraint Systems Laboratory Evaluation : domain reformulation Reduced domain size → improved search performance Case study Phone-book completeness Average domain sizeRuntime [s] OriginalReformulatedOriginalReformulated NSeg125-i45.6%1103.1 236.12943.7744.7 NSeg206-i50.5%1102.0438.814,818.95533.8 SSeg131-i60.3%792.9192.967,910.166,901.1 SSeg178-i65.6%785.5186.3119,002.4117,826.7 10/24/201531Bayer–MS Thesis Defense

32. Constraint Systems Laboratory Outline Background BID model & custom solver Reformulation techniques –Query reformulation –AllDiff-Atmost & domain reformulation –Constraint relaxation –Reformulation via symmetry detection Conclusions & future work 10/24/20153210/24/201532Bayer–MS Thesis Defense

33. Constraint Systems Laboratory Constraint relaxation Resource allocation problems are often matching problems with additional constraints Alice Charlie Bob Computer sales TV sales Inventory or Additional constraint: We don’t want Bob and Charlie both working sales at the same time 10/24/201533Bayer–MS Thesis Defense

34. Constraint Systems Laboratory BID as a matching problem Assume we have no grid constraints 10/24/201534Bayer–MS Thesis Defense B6 B8 B2 B4 B5 B3 B9 B10B7 B1 S1 S2 S3 S1#1,S1#4, S1#8,S2#7, S2#8,S3#1, S3#2,S3#3, S3#15

35. Constraint Systems Laboratory BID w/o grid constraints BID instances without grid constraints can be solved in polynomial time Case studyRuntime [s] BT searchMatching NSeg125-c139.24.8 NSeg206-c4971.216.3 SSeg131-c38618.37.3 SSeg178-c117279.122.5 NSeg125-i744.72.5 NSeg206-i5533.88.5 SSeg131-i38618.37.3 SSeg178-i117826.74.9 10/24/201535Bayer–MS Thesis Defense

36. Constraint Systems Laboratory BID w/ grid constraints The matching reformulation is a necessary approximation Solutions to BID instance Solutions to matching reformulation Reformulation No solution to matching reformulation → no solution to the original BID 10/24/201536Bayer–MS Thesis Defense

37. Constraint Systems Laboratory Lookahead : relaxation during search Filter vvps that do not appear in any maximum matching [Régin, 1994] Instantiated variables (corners) Matching relaxation Uninstantiated variables Filter values 10/24/201537Bayer–MS Thesis Defense

38. Constraint Systems Laboratory Using the relaxation prior to search Preproc1 –Prior to running per-variable solution loop –Remove vvps that don’t appear in any max. matching [Régin, 1994] BID instance + vvp Relax as matching Preproc2 Is matching solvable? Search determines solvability of CSP Yes No BID cannot have a solution Preproc2 –Prior to testing a CSP for solvability –Necessary approximation determines unsolvability 10/24/201538Bayer–MS Thesis Defense

39. Constraint Systems Laboratory Evaluation: constraint relaxation Generally, improves performance Rarely, the overhead exceeds the gains Case Study BT Preproc2 +BT % (from BT) Lkhd +BT % (from BT) Lkhd +Preproc1&2 + BT % (from Lkhd+BT) NSeg125-i1232.51159.16.0%726.641.0%701.13.5% NSeg206-c2277.5614.273.0%1559.231.5%443.871.5% SSeg178-i138404.2103244.725.4%121492.412.2%85185.929.9% 10/24/201539Bayer–MS Thesis Defense Case Study BT Preproc2 +BT % (from BT) Lkhd +BT % (from BT) Lkhd +Preproc1&2 + BT % (from Lkhd+BT) NSeg125-i100.833.267.1%140.2 - 39.0%39.671.8% NSeg206-c114405.9114141.30.2%107896.35.7%108646.6-0.7%

40. Constraint Systems Laboratory Outline Background BID model & custom solver Reformulation techniques –Query reformulation –AllDiff-Atmost & domain reformulation –Constraint relaxation –Reformulation via symmetry detection Conclusions & future work 10/24/20154010/24/201540Bayer–MS Thesis Defense

41. Constraint Systems Laboratory Find all maximum matchings … alternating cycles: … even alternating paths starting @ free vertex: Find one maximum matching [Hopcroft+Karp, 73] Identify… [Berge, 73] 10/24/201541Bayer–MS Thesis Defense

42. Constraint Systems Laboratory Symmetric maximum matchings  = =  () All matchings can be produced using the sets of disjoint alternating paths & cycles  Compact representation S 10/24/201542Bayer–MS Thesis Defense

43. Constraint Systems Laboratory Some symmetric solutions do not break the grid constraints –Use symmetry breaking constraints to avoid exploring them Some do, we do not know how to use them… Symmetric matchings in BID 10/24/201543Bayer–MS Thesis Defense 

44. Constraint Systems Laboratory Conclusions We proposed four reformulation techniques We described their usefulness for general CSPs We demonstrated their effectiveness on the BID Lesson: reformulation is an effective approach to improve the scalability of complex systems 10/24/201544Bayer–MS Thesis Defense

45. Constraint Systems Laboratory Future work Empirically evaluate our new algorithm for relational (i,m)-consistency Identify other problems that benefit from a matching relaxation Exploit the symmetries we identified Enhance the model by inferring more constraints [Michalowski] 10/24/201545Bayer–MS Thesis Defense

46. Constraint Systems Laboratory Questions? 10/24/201546Bayer–MS Thesis Defense

47. Constraint Systems Laboratory Overall architecture 10/24/201547Bayer–MS Thesis Defense

48. Constraint Systems Laboratory Using query reformulation A B E D F C ABCDEF ……………… ……………… ……………… AB …… …… …… … AC …… …… …… DB …… …… RC (i,m) : New algor.: Space O(d s ) Space O( ( ) d i ) EC …… …… …… FB …… …… Goal: Compute R(2,7)C with i=2, m=7 … … Note: i < s s i s=6 10/24/201548Bayer–MS Thesis Defense