1. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 1 Foundations of Constraint Processing CSCE421/821, Fall 2004: www.cse.unl.edu/~choueiry/F04-421-821/ Berthe Y. Choueiry (Shu-we-ri) Avery Hall, Room 123B choueiry@cse.unl.edu Tel: +1(402)472-5444 Consistency Algorithms

2. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 2 Lecture Sources Required reading 1.Algorithms for Constraint Satisfaction Problems, Mackworth and Freuder AIJ'85 2.Sections 3.1, 3.2, 3.3. Chapter 3. Constraint Processing. Dechter Recommended 1.Sections 3.4—3.10. Chapter 3. Constraint Processing. Dechter 2.Networks of Constraints: Fundamental Properties and Application to Picture Processing, Montanari, Information Sciences 74 3.Consistency in Networks of Relations, Mackworth AIJ'77 4.Constraint Propagation with Interval Labels, Davis, AIJ'87 5.Path Consistency on Triangulated Constraint Graphs, Bliek & Sam- Haroud IJCAI'99

3. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 3 Outline 1.Motivation and material 2.Node and arc consistency and their complexity 3.Criteria for performance comparison and CSP parameters. Which AC algorithm to choose? Project results may tell! 4.Path consistency and its complexity 5.K-consistency, minimality and decomposability 6.Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc.

4. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 4 Consistency checking Motivation CSP are solved with search (conditioning) Search performance is affected by: - problem size - amount of backtracking Backtracking may yield Thrashing Exploring non-promising sub-trees and rediscovering the same inconsistencies over and over again  Malady of backtrack search

5. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 5 (Some) causes of thrashing Search order: V1, V2, V3, V4, V5 What happens in search if we: do not check the constraint on V3 (node inconsistency) do not check constraint between V3 and V5 (arc inconsistency) do not check constraints between V3—V4, V3—V5, and V4—V5 (path inconsistency) {1, 2} V4 {1, 2, 3} { 0, 1, 2} { 1, 2} V5 {1, 2, 3} V1 V2 V3<V2 V3>0 V3<V4 V5<V4 V3<V5 V3<V1 V3

6. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 6 Consistency checking Goal : eliminate inconsistent combinations Algorithms (for binary constraints): Node consistency Arc consistency (AC-1, AC-2, AC-3,..., AC-7, AC-3.1, etc.) Path consistency (PC-1, PC-2, DPC, PPC, etc.) Constraints of arbitrary arity: –Waltz algorithm ancestor of AC's –Generalized arc-consistency (Dechter, Section 3.5.1 ) –Relational m-consistency (Dechter, Section 8.1)

7. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 7 Overview of Recommended Reading Davis, AIJ'77 –focuses on the Waltz algorithm –studies its performance and quiescence for given: constraint types (order relations, bounded diff, algebraic, etc.) domains types (continuous or finite) Mackworth, AIJ'77 –presents NC, AC-1, AC-2, PC-1, PC-2 Mackworth and Freuder, AIJ'85 –studies their complexity  Mackworth and Freuder concentrate on finite domains  More people work on finite domains than on continuous ones  Continuous domains can be quite tough

8. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 8 Constraint Propagation with Interval Labels Ernest Davis `Old' paper (1987): terminology slightly different Interval labels: continuous domains –Section 8: sign labels (discrete) Concerned with Waltz algorithm (e.g., quiescence, completeness) Constraint types vs. domain types (Table 3) Addresses applications from reasoning –about Physical Systems (circuit, SPAM ) –about time relations ( TMM )  Advice: read Section 3, more if you wish

9. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 9 Waltz algorithm for label inference Refine (C(V i, V k, V m, V n ), V i ) - finds a new label for V i consistent with C. Revise (C (V i, V k, V m, V n )) refines the domains of V i, V k, V m, V n by iterating over these variables until quiescence (no domains is further refined) Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised Waltz Algorithm  ancestor of arc-consistency

10. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 10 WARNING → Completeness: Running the Waltz Algorithm does not solve the problem. A=2 and B=3 is still not a solution! → Quiescence: The Waltz algorithm may go into infinite loops even if problem is solvable x  [0, 100] x = y y  [0, 100] x = 2y → Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm f (constraint types, domain types), see Table 3

11. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 11 Importance of this paper  Establishes that constraints of bounded differences (temporal reasoning) can be efficiently solved by the Waltz algorithm ( O(n 3 ), n number of variables)  Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains). This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm.

12. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 12 Consistency checking algorithms Examining finite, binary CSPs and their complexity Notation: Given variables i, j, k with values x, y, z, the predicate P ijk (x, y, z) is true iff the 3-tuple  x, y, z   C ijk Node consistency: checking P i (x) Arc consistency: checking P ij (x,y) Path consistency: bit-matrix manipulation

13. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 13 Worst-case complexity as a function of the input parameters Upper bound: f(n) is O(g(n)) means that f(n)  c.g(n) f grows as g or slower Lower bound: f(n) is  (h(n)) means that f(n)  c.h(n ) f grows as g or faster Input parameters for a CSP: n = number of variables a = (max) size of a domain d k = degree of V k (  n-1) e = number of edges (or constraints)  [ (n-1), n(n-1)/2 ] Worst-case asymptotic complexity time space

14. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 14 Outline 1.Motivation and material 2.Node & arc consistency and their complexity 3.Criteria for performance comparison and CSP parameters. Which AC algorithm to choose? Project results may tell! 4.Path consistency and its complexity 5.K-consistency, minimality and decomposability 6.Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc.

15. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 15 Node consistency (NC) Procedure NC(i): D i  D i  { x | P i (x) } Begin for i  1 until n do NC(i) end

16. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 16 Complexity of NC Procedure of NC(i) D i  D i  { x | P i (x) } Begin for i  1 until n do NC(i) end For each variable, we check a values We have n variables, we do n.a checks NC is O(a.n)

17. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 17 Arc-consistency Adapted from Dechter Definition: Given a constraint graph G, A variable V i is arc-consistent relative to V j iff for every value a  D Vi, there exists a value b  D Vj | (a, b)  C Vi,Vj. The constraint C Vi,Vj is arc-consistent iff –V i is arc-consistent relative to V j and –V j is arc-consistent relative to V i. A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent 1 2 2 3 1 2 3 1 2 3 Vi Vj Vi

18. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 18 Procedure Revise Revises the domains of a variable i Procedure Revise(i,j): Begin DELETE  false for each x  D i do if there is no y  D j such that P ij (x, y) then begin delete x from D i DELETE  true end return DELETE end Revise is directional What is the complexity of Revise ? {a 2 }

19. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 19 Revise : example R. Dechter Apply the Revise procedure to the following example

20. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 20 Solution : D Vi  D Vi   Vi (C Vi,Vj D Vi ) This is actually equivalent to D Vi   Vi (C Vi,Vj D Vi ) Effect of Revise Adapted from Rina Dechter Question: Given two variables V i and V j their domains D Vi and D Vj, and the constraint C Vi,Vj, write the effect of the Revise procedure as a sequence of operations in relational algebra Hint: Think about the domain D Vi as a unary constraint C Vi and consider the composition of this unary constraint and the binary one.. 1 2 2 3 1 2 3 1 2 3 Vi Vj Vi

21. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 21 Arc consistency (AC-1) Procedure AC-1: 1 begin 2 for i  1 until n do NC(i) 3 Q  {(i, j) | (i,j)  arcs(G), i  j } 4 repeat 5 begin 6 CHANGE  false 7 for each (i, j)  Q do CHANGE  ( REVISE(i, j) or CHANGE ) 8 end 9 until ¬ CHANGE 10 end AC-1 does not update Q, the queue of arcs No algorithm can have time complexity below O(ea 2 )

22. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 22 Arc consistency 1.AC may discover the solution Example borrowed from Dechter

23. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 23 Arc consistency 2. AC may discover inconsistency Example borrowed from Dechter

24. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 24 NC & AC Example courtesy of M. Fromherz In the temporal problem below 1.AC propagates bound 2.Unary constraint x>3 is imposed 3.AC propagates bounds again [0, 10] x  y-3 x y [0, 7] [3, 10] x  y-3 x y [4, 7] [7, 10] x  y-3 x y

25. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 25 Complexity of AC-1 Procedure AC-1: 1 begin 2 for i  1 until n do NC(i) 3 Q  {(i, j) | (i,j)  arcs(G), i  j 4 repeat 5 begin 6 CHANGE  false 7 for each (i, j)  Q do CHANGE  (REVISE(i, j) or CHANGE) 8 end 9 until ¬ CHANGE 10 end Note : Q is not modified and |Q| = 2e 4  9 repeats at most n·a times Each iteration has |Q| = 2e calls to REVISE Revise requires at most a 2 checks of P ij  AC-1 is O(a 3 · n · e)

26. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 26 AC versions AC-1 does not update Q, the queue of arcs AC-2 iterates over arcs connected to at least one node whose domain has been modified. Nodes are ordered. AC-3 same as AC-2, nodes are not ordered

27. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 27 Arc consistency (AC-3) AC-3 iterates over arcs connected to at least one node whose domain has been modified Procedure AC-3: 1 begin 2 for i  1 until n do NC(i) 3 Q  { (i, j) | (i, j)  arcs(G), i  j } 4 While Q is not empty do 5 begin 6 select and delete any arc (k, m) from Q 7 If Revise(k, m) then Q  Q  { (i, k) | (i, k)  arcs(G), i  k, i  m } 8 end 9 end

28. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 28 Complexity of AC-3 Procedure AC-3: 1 begin 2 for i  1 until n do NC(i) 3 Q  {(i, j) | (i, j)  arcs(G), i  j } 4 While Q is not empty do 5 begin 6select and delete any arc (k, m) from Q 7 If Revise(k, m) then Q  Q  { (i, k) | (i, k)  arcs(G), i  k, i  m } 8 end 9 end First |Q| = 2e, then it grows and shrinks 4  8 Worst case: –1 element is deleted from D Vk per iteration –none of the arcs added is in Q Line 7: a·(d k - 1) Lines 4 - 8: Revise : a 2 checks of P ij  AC-3 is O(a 2 (2e + a(2e-n)))  Connected graph (e  n – 1), AC-3 is O(a 3 e)  If AC-p, AC-3 is O(a 2 e)  AC-3 is  (a 2 e)  Complete graph: O(a 3 n 2 )

29. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 29 Example: Apply AC-3 Example: Apply AC-3 Thanks to Xu Lin D V1 = {1, 2, 3, 4, 5} D V2 = {1, 2, 3, 4, 5} D V3 = {1, 2, 3, 4, 5} D V4 = {1, 2, 3, 4, 5} C V2,V3 = {(2, 2), (4, 5), (2, 5), (3, 5), (2, 3), (5, 1), (1, 2), (5, 3), (2, 1), (1, 1)} C V1,V3 = {(5, 5), (2, 4), (3, 5), (3, 3), (5, 3), (4, 4), (5, 4), (3, 4), (1, 1), (3, 1)} C V2,V4 = {(1, 2), (3, 2), (3, 1), (4, 5), (2, 3), (4, 1), (1, 1), (4, 3), (2, 2), (1, 5)} { 1, 2, 3, 4, 5 } V1 V3 V2 V4 V1 V3 V2 V4 { 1, 3, 5 }{ 1, 2, 3, 5 } { 1, 2, 3, 4 } { 1, 3, 5 }

30. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 30 Applying AC-3 Thanks to Xu Lin Queue = {C V2, V4, C V4,V2, C V1,V3, C V2,V3, C V3, V1, C V3,V2 } Revise (V 2,V 4 ): D V2  D V2 \ {5} = {1, 2, 3, 4} Queue = {C V4,V2, C V1,V3, C V2,V3, C V3, V1, C V3, V2 } Revise (V 4, V 2 ): D V4  D V4 \ {4} = {1, 2, 3, 5} Queue = {C V1,V3, C V2,V3, C V3, V1, C V3, V2 } Revise (V 1, V 3 ): D V1  {1, 2, 3, 4, 5} Queue = {C V2,V3, C V3, V1, C V3, V2 } Revise (V 2, V 3 ): D V2  {1, 2, 3, 4}

31. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 31 Applying AC-3 (cont.) Thanks to Xu Lin Queue = {C V3, V1, C V3, V2 } Revise (V 3, V 1 ): D V3  D V3 \ {2} = {1, 3, 4, 5} Queue = {C V2, V3, C V3, V2 } Revise (V 2, V 3 ): D V2  D V2 Queue = {C V3, V2 } Revise (V 3, V 2 ): D V3  {1, 3, 5} Queue = {C V1, V3 } Revise (V 1, V 3 ): D V1  {1, 3, 5} END

32. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 32 Main Improvements Mohr & Henderson (AIJ 86): AC-3 O(a 3 e)  AC-4 O(a 2 e) AC-4 is optimal Trade repetition of consistency-check operations with heavy bookkeeping on which and how many values support a given value for a given variable Data structures it uses: –m : values that are active, not filtered –s : lists all vvp's that support a given vvp –counter : given a vvp, provides the number of support provided by a given variable How it proceeds: 1.Generates data structures 2.Prepares data structures 3.Iterates over constraints while updating support in data structures

33. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 33 Step 1: Generate 3 data structures m and s have as many rows as there are vvp’s in the problem counter has as many rows as there are tuples in the constraints 04),,( 01,( 01,( 02,( 03,( 31 32 24 24 24 VV VV VV VV VV counter 

34. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 34 Step 2: Prepare data structures Data structures: s and counter. Checks for every constraint C Vi,Vj all tuples V i =a i, V j =b j_ ) When the tuple is allowed, then update: –s(V j,b j )  s(V j,b j )  {(V i, a i )} and –counter(V j,b j )  (V j,b j ) + 1 Update counter ((V 2, V 3 ), 2) to value 1 Update counter ((V 3, V 2 ), 2) to value 1 Update s-htable (V 2, 2) to value ((V 3, 2)) Update s-htable (V 3, 2) to value ((V 2, 2)) Update counter ((V 2, V3), 4) to value 1 Update counter ((V 3, V2), 5) to value 1 Update s-htable (V 2, 4) to value ((V 3, 5)) Update s-htable (V 3, 5) to value ((V 2, 4))

35. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 35 Constraints C V2,V3 and C V3,V2 S (V4, 5)Nil (V3, 2)((V2, 1), (V2, 2)) (V3, 3)((V2, 5), (V2, 2)) (V2, 2)((V3,1), (V3,3), (V3, 5), (V3,2)) (V3, 1)((V2, 1), (V2, 2), (V2, 5)) (V2, 3)((V3, 5)) (V1, 2)Nil (V2, 1)((V3, 1), (V3, 2)) (V1, 3)Nil (V3, 4)Nil (V4, 2)Nil (V3, 5)((V2, 3), (V2, 2), (V2, 4)) (v4, 3)Nil (V1, 1)Nil (V2, 4)((V3, 5)) (V2, 5)((V3, 3), (V3, 1)) (V4, 1)Nil (V1, 4)Nil (V1, 5)Nil (V4, 4)Nil Counter (V4, V2), 30 (V4, V2), 20 (V4, V2), 10 (V2, V3), 12 (V2, V3), 34 (V4, V2), 51 (V2, V4), 10 (V2, V3), 40 (V4, V2), 41 (V2, V4), 20 (V2, V3), 50 (V2, V4), 32 (V2, V4), 40 (V2, V4), 50 (V3, V1), 10 (V3, V1), 20 (V3, V1), 30 …… (V3, V2), 40 Etc…Etc. Updating m Note that (V3, V2),4  0 thus we remove 4 from the domain of V3 and update (V3, 4)  nil in m Updating counter Since 4 is removed from D V3 then for every (V k, l) | (V3, 4)  s [(V k, l)], we decrement counter [(V k, V3), l] by 1

36. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 36 Summary of arc-consistency algorithms AC-4 is optimal (worst-case) [Mohr & Henderson, AIJ 86] Warning: worst-case complexity is pessimistic. Better worst-case complexity: AC-4 Better average behavior: AC-3 [Wallace, IJCAI 93] AC-5: special constraints [Van Hentenryck, Deville, Teng 92] functional, anti-functional, and monotonic constraints AC-6, AC-7: general but rely heavily on data structures for bookkeeping [Bessière & Régin] Now, back to AC-3: AC-2000, AC-2001, AC-3.1, AC3.3 Non-binary constraints: –GAC (general) [Mohr & Masini 1988], –all-different (dedicated) [R é gin, 94]

37. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 37 Outline 1.Motivation and material 2.Node and arc consistency and their complexity 3.Criteria for performance comparison & CSP parameters. Which AC algorithm to choose? Project results may tell! 4.Path consistency and its complexity 5.K-consistency, minimality and decomposability 6.Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc.

38. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 38 CSP parameters ‹n, a, t, d› n is number of variables a is maximum domain size t is constraint tightness: d is constraint density where e is the #constraints, e min =(n-1), and e max = n(n-1)/2 Lately, we use constraint ratio p = e/e max → Constraints in random problems often generated uniform → Use only connected graphs (throw the unconnected ones away)

39. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 39 Criteria for performance comparison 1.Bounding time and space complexity (theoretical) –worst-case, –average-case, –best- case 2.Counting the number of constraint checks (theoretical, empirical) 3.Measuring CPU time (empirical)

40. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 40 Performance comparison Courtesy of Lin XU AC-7 AC-4 AC-3

41. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 41 Wisdom (?) Free adaptation from Bessière When a single constraint check is very expensive to make, use AC-7. When there is a lot of propagation, use AC-4 When little propagation and constraint checks are cheap, use AC-3x Advice? Instructor's personal opinion Use AC-3..

42. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 42 Outline 1.Motivation and material 2.Node and arc consistency and their complexity 3.Criteria for performance comparison and CSP parameters. Which AC algorithm to choose? Project results may tell! 4.Path consistency and its complexity 5.K-consistency, minimality and decomposability 6.Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc.

43. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 43 AC is not enough Example borrowed from Dechter Arc-consistent? Satisfiable?  seek higher levels of consistency V 1 a b V 2 V 3 = V 1 b a a b b a V 2 V 3 = =

44. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 44 Consistency of a path A path (V 0, V 1, V 2, …, V m ) of length m is consistent iff for any value x  D V0 and for any value y  D Vm that are consistent (i.e., P V0 Vm (x, y))  a sequence of values z 1, z 2, …, z m-1 in the domains of variables V 1, V 2, …, V m-1, such that all constraints between them (along the path, not across it) are satisfied (i.e., P V0 V1 (x, z 1 )  P V1 V2 (z 1, z 2 )  …  P Vm-1 Vm (z m-1, z m ) ) for all x  D V0 for all y  D Vm V0 Vm Vm-1 V2 V1

45. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 45 Note  The same variable can appear more than once in the path  Every time, it may have a different value  Constraints considered: P V0,Vm and those along the path  All other constraints are neglected for all x  D V0 for all y  D Vm V0 Vm Vm-1 V2 V1

46. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 46 Example: consistency of a path  Check path length = 2, 3, 4, 5, 6,.... {a, b, c} V2 {a, b, c} V3 {a, b, c} V1 {a, b, c} V4 {a, b, c} V7 {a, b, c} V5 {a, b, c} V6 All mutex constraints            

47. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 47 Path consistency: definition  A path of length m is path consistent  A CSP is path consistent Property of a CSP Definition: A CSP is path consistent (PC) iff every path is consistent (i.e., any length of path) Question: should we enumerate every path of any length? Answer: No, only length 2, thanks to [Mackworth AIJ'77]

48. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 48 Making a CSP Path Consistent (PC) Special case: Complete graph Theorem: In a complete graph, if every path of length 2 is consistent, the network is path consistent [Mackworth AIJ'77]  PC-1: two operations, composition and intersection  Proof by induction. Special case: Triangulated graph Theorem: In a triangulated graph, if every path of length 2 is consistent, the network is path consistent [Bliek & Sam-Haroud ‘99]  PPC (partially path consistent)  PC

49. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 49 Tools for PC-1 Two operators 1.Constraint composition: ( ) R 13 = R 12 R 23 2.Constraint intersection: (  ) R 13  R 13, old  R 13, induced

50. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 50 Path consistency (PC-1) Achieved by composition and intersection (of binary relations expressed as matrices) over all paths of length two. Procedure PC-1: 1 Begin 2 Y n  R 3 repeat 4 begin 5 Y 0  Y n 6 For k  1 until n do 7 For i  1 until n do 8 For j  1 until n do 9 Y l ij  Y l-1 ij  Y l-1 ik Y l-1 kk Y l-1 kj 10 end 11 until Y n = Y 0 12 Y  Y n 10 end

51. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 51 Properties of PC-1 Discrete CSPs [Montanari'74] 1.PC-1 terminates 2.PC-1 results in a path consistent CSP PC-1 terminates. It is complete, sound (for finding PC network) PC-2: Improves to PC-1similarly to (AC-1  AC-3) Complexity of PC-1..

52. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 52 Complexity of PC-1 Procedure PC-1: 1 Begin 2 Y n  R 3repeat 4 begin 5 Y 0  Y n 6 For k  1 until n do 7 For i  1 until n do 8 For j  1 until n do 9 Y l ij  Y l-1 ij  Y l-1 ik Y l-1 kk Y l-1 kj 10 end 11 until Y n = Y 0 12 Y  Y n 10 end Line 9: a 3 Lines 6–10: n 3. a 3 Line 3: at most n 2 relations x a 2 elements PC-1 is O(a 5 n 5 ) PC-2 is O(a 5 n 3 ) and  (a 3 n 3 )

53. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 53 Some improvements Mohr & Henderson (AIJ 86) –PC-2 O(a 5 n 3 )  PC-3 O(a 3 n 3 ) –Open question: PC-3 optimal? Han & Lee (AIJ 88) –PC-3 is incorrect –PC-4 O(a 3 n 3 ) space and time Singh (ICTAI 95) –PC-5 uses ideas of AC-6 (support bookkeeping) Note: PC is seldom used in practical applications unless in presence of special ty pe of constraints (e.g., bounded difference)

54. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 54 Path consistency as inference of binary constraints Path consistency corresponds to inferring a new constraint (alternatively, tightening an existing constraint) between every two variables given the constraints that link them to a third variable  Considers all subgraphs of 3 variables  3-consistency B C A B < C A < B B C A A < C B < C

55. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 55 Path consistency as inference of binary constraints Another example: V4V4 V3V3 a b V4V4 V2V2 V1V1 V3V3     V2V2 V1V1     = =

56. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 56 Question Adapted from Dechter Given three variables V i, V k, and V j and the constraints C Vi,Vk, C Vi,Vj, and C Vk,Vj, write the effect of PC as a sequence of operations in relational algebra. B C A B < C A < B A + 3 > C B C A A < B A + 3 > C A < C B < C B C A A < B -3 < A –C < 0 B < C Solution: C Vi,Vj  C Vi,Vj   ij (C Vi,Vk C Vk,Vj )

57. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 57 Outline 1.Motivation and material 2.Node and arc consistency and their complexity 3.Criteria for performance comparison and CSP parameters. Which AC algorithm to choose? Project results may tell! 4.Path consistency and its complexity 5.K-consistency, Minimality & decomposability 6.Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc.

58. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 58 Important properties of a CSP 1. (Arc Consistency, AC) 2.Path Consistency, PC 3.(strong) K-consistency 4.Minimality 5.Decomposability (ref. lossless join decomposition)

59. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 59 Is this CSP 3-consistent? is it 2-consistent? Definitions: k-consistency vs. strong k-consistency k-consistency: any consistent assignment for (k-1)-variable can be extended to any k th variable strong k-consistency: has to be j-consistent for all j  k Warning Does 3-consistency guarantee 2 consistency? {red, blue} A B   { red } {red, blue} { red } C

60. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 60 Minimality PC tightens the binary constraints The tightest possible binary constraints yield the minimal network Minimal network a.k.a. central problem Given two values for two variables, if they are consistent, then they appear in at least one solution. Note: Minimal  path consistent The definition of minimal CSP is concerned with binary CSPs

61. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 61 Minimal CSP Minimal network a.k.a. central problem Given two values for two variables, if they are consistent, then they appear in at least one solution. Informally In a minimal CSP the remainder of the CSP does not add any further constraint to the direct constraint C Vi, Vj between the two variables V i and V j [Mackworth AIJ'77] A minimal CSP is perfectly explicit: as far as the pair V i and V j is concerned, the rest of the network does not add any further constraint to the direct constraint C Vi, Vj [Montanari'74] The binary constraints are explicit as possible. [Montanari'74]

62. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 62 Decomposability Any combination of values for k variables that satisfy the constraints between them can be extended to a solution. Decomposability generalizes minimality Minimality: any consistent combination of values for any 2 variables is extendable to a solution Decomposability: any consistent combination of values for any k variables is extendable to a solution  Decomposable  Minimal  Path Consistent Strong n-consistent  n-consistent  Solvable 

63. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 63 Decomposability In general: Decomposability  minimality  path consistent PC is used to approximate minimality (which is the central problem) When is the approximation the real thing? Special cases: When composition distributes over intersection, [Montanari'74] PC-1 on the completed graph guarantees minimality and decomposability When constraints are convex [Bliek & Sam-Haroud 99] PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)

64. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 64 Relations to (theory of) DB CSPDatabase MinimalPair-wise consistent DecomposableComplete join

65. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 65 Path Consistency CSP 0 Complete the graph PC-1 CSP 0 Triangulate the graph PPC Result: CSP 0 is not path consistent or PC-1(CSP 0 ) is path consistent 1.It is tight, but could be tighter 2.The graph is complete Result: CSP 0 is not path consistent or PPC(CSP 0 ) is path consistent 1.It is generally less tight than PC-1(CSP 0 ) 2.The graph is triangulated Watch for  PPC

66. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 66 Path Consistency CSP a Complete the graph PC-1/2/4 Result: [Montanari 74] CSP a is not path consistent or PC-1(CSP a ) is 1.Path consistent, minimal & decomposable 2.Graph is complete With a  (b  c)=(a  b)  (b  c) CSP b Triangulate the graph PPC Constraint are convex Result: [B&S-H 99] CSP b is not path consistent or PPC(CSP b ) is 1.Path consistent, minimal, decomposable 2.Existing edges are as tight as PC-1(CSP b ) 3.Graph is triangulated

67. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 67 PC guarantees minimality, decomposability In PC-1, two operations: R 01 (R 13  R' 13 ) = (R 01 R 13 )  (R 01 R’ 13 ) When ( ) distributes over (  ), then [Montanari'74] 1.PC guarantees that CSP is minimal and decomposable 2.1 loop in PC-1 is sufficient (PC collapses to Floyd-Warshall) Intersection,  Composition, R’ 13 R 13 R 23 R 12 V1V1 V3V3 V2V2 V0V0 R 01

68. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 68 Temporal reasoning constraints of bounded difference Variables: X, Y, Z, etc. Constraints: a  Y-X  b, i.e. Y-X = [a, b] = I Composition: I 1 I 2 = [a 1, b 1 ] [a 2, b 2 ] = [a 1 + a 2, b 1 +b 2 ] Interpretation: –intervals indicate distances –composition is triangle inequality. Intersection: I 1  I 2 = [max(a 1, a 2 ), min(b 1, b 2 )] Distributivity: I 1 (I 2  I 3 ) = (I 1 I 2 )  (I 1 I 3 ) Proof: left as an exercise

69. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 69 Composition of intervals + : R’ 13 = R 12 + R 23 = [4, 12] R 01 + R 13 = [2,5] + [3, 5] = [5, 10] R 01 + R' 13 = [2,5] + [4, 12] = [6, 17] Intersection of intervals: R 13  R' 13 = [4, 12]  [3, 5] = [4, 5] R 01 + (R 13  R' 13 ) = (R 01 + R 13 )  (R 01 + R' 13 ) R 01 + (R 13  R' 13 ) = [2, 5] + [4, 5] = [6, 10] (R 01 + R 13 )  (R 01 + R' 13 ) = [5, 10]  [6,17] = [6, 10] Path consistency guarantees minimality Side note: PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path), where composition is ‘scalar addition’ and intersection is ‘scalar minimal’ Example constraints of bounded difference (temporal reasoning) R’ 13 R 13 =[3,5] R 23 =[1,8] R 12 =[3,4] V1V1 V3V3 V2V2 V0V0 R 01 =[2,5] [

70. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 70 Convex constraints : temporal reasoning (again!) Thanks to Xu Lin (2002) Constraints of bounded difference are convex We triangulate the graph (good heuristics exist) Apply PPC: restrict propagations in PC to triangles of the graph (and not in the complete graph) According to [Bliek & Sam-Haroud 99] PPC becomes equivalent to PC, thus it guarantees minimality and decomposability

71. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 71 Constraint propagation courtesy of Dechter After Arc-consistency: After Path-consistency: Are these CSPs the same? –Which one is more explicit? –Are they equivalent? The more propagation, –the more explicit the constraints –the more search is directed towards a solution 123123 123123 1212 2323 ( 0, 1 )    

72. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 72 Arc-consistent? Path-consistent? PC can detect unsatisfiability V3V3 a b V4V4 V2V2 V1V1      

73. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 73 PC is not enough Arc-consistent? Path-consistent? Satisfiable?  we should seek (even) higher levels of consistency  k-consistency, k = 1, 2, 3, …. {a, b, c} V2 {a, b, c} V3 {a, b, c} V1 {a, b, c} V4 {a, b, c} V7 {a, b, c} V5 {a, b, c} V6 All mutex constraints                 

74. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 74 k- consistency 2-consistency: D Vi  D Vi   Vi (C Vi,Vj D Vj ) Projection of binary constraint is used to tighten unary constraint 3-consistency: C Vi,Vj  C Vi,Vj   Vi,Vj (C Vi,Vk C Vk,Vj ) Projection of ternary constraint is used to tighten binary constraint k-consistency: [Freuder 78, 82] Projection of k-ary constraint is used to tighten (k-1)-ary constraint

75. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 75 k-consistency Definition: A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables, there exists an instantiation of any k th variable such that the k values taken together satisfy all the constraints among the k variables. Example: (courtesy of Dechter) Q Q Q Q Q 3-consistent? 4-consistent?

76. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 76 Summary : the higher the consistency level, the better New terms: AC, PC, minimal CSP, decomposable CSP, complete graph, triangulated graph, Revise, AC-1, AC-3, PC-1, PC-2, PPC Complexity of AC, PC algorithms AC tightens domains, PC tightens binary constraints Consistency operations in terms of composition, intersection in7 relational algebra Path consistency approximates minimality, which approximates decomposability Situations when path consistency guarantees minimality and decomposability Criteria for performance comparison Parameters for random CSPs

77. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 77 Outline 1.Motivation and material 2.Node and arc consistency and their complexity 3.Criteria for performance comparison and CSP parameters. Which AC algorithm to choose? Project results may tell! 4.Path consistency and its complexity 5.K-consistency, Minimality & decomposability 6.Other related results: box consistency, higher levels of consistency, (i, j)-consistency, etc.

78. Foundations of Constraint Processing, Fall 2004 Oct 5, 2004Consistency: Properties & Algorithms 78 Related results Continuous domains: box consistency Higher consistency levels: –(strong) k-consistency, (i,j)-consistency [Freuder] Integration with search: –AC: Forward Checking (FC) [Haralick & Elliott] Directional Arc-Consistency (DAC) [Dechter] Maintaining arc consistency (MAC) [Sabin & Freuder ] –PC: Directed path consistency (DPC) [Dechter] Non-binary constraints: –Generalized arc consistency (GAC) [Mohr & Masini] –Relational consistency [Dechter] Non-binary constraints and integration with search: nFC 0, nFC 1, …, nFC 5 etc. Tightness  Density  LowHigh LowFCMAC HighFC