Consistency Algorithms Foundations of Constraint Processing CSCE421/821, Spring 2008: www.cse.unl.edu/~choueiry/S08-421-821/ Berthe Y. Choueiry (Shu-we-ri) Avery Hall, Room 123B choueiry@cse.unl.edu Tel: +1(402)472-5444 Consistency: Properties & Algorithms

Properties & Algorithms Lecture Sources Required reading Algorithms for Constraint Satisfaction Problems, Mackworth and Freuder AIJ'85 Sections 3.1, 3.2, 3.3. Chapter 3. Constraint Processing. Dechter Recommended Sections 3.4—3.10. Chapter 3. Constraint Processing. Dechter Networks of Constraints: Fundamental Properties and Application to Picture Processing, Montanari, Information Sciences 74 Consistency in Networks of Relations, Mackworth AIJ'77 Constraint Propagation with Interval Labels, Davis, AIJ'87 Path Consistency on Triangulated Constraint Graphs, Bliek & Sam-Haroud IJCAI'99 Consistency: Properties & Algorithms

Properties & Algorithms Outline Motivation and material Node and arc consistency and their complexity Criteria for performance comparison and CSP parameters. Path consistency and its complexity Consistency properties of a CSP Other related results: box consistency, constraint synthesis, look-ahead, non-binary constraints, etc. Consistency: Properties & Algorithms

Properties & Algorithms Consistency checking Motivation CSP are solved with search (conditioning) Search performance is affected by: - problem size - amount of backtracking Backtracking may yield Thrashing Exploring non-promising sub-trees and rediscovering the same inconsistencies over and over again  Malady of backtrack search Consistency: Properties & Algorithms

(Some) causes of thrashing Search order: V1, V2, V3, V4, V5 What happens in search if we: do not check the constraint on V3 (node inconsistency) do not check constraint between V3 and V5 (arc inconsistency) do not check constraints between V3—V4, V3—V5, and V4—V5 (path inconsistency) {1, 2} V4 {1, 2, 3} { 0, 1, 2} { 1, 2} V5 V1 V2 V3<V2 V3>0 V3<V4 V5<V4 V3<V5 V3<V1 V3 Consistency: Properties & Algorithms

Properties & Algorithms Consistency checking Goal: eliminate inconsistent combinations Algorithms (for binary constraints): Node consistency Arc consistency (AC-1, AC-2, AC-3, ..., AC-7, AC-3.1, etc.) Path consistency (PC-1, PC-2, DPC, PPC, etc.) Constraints of arbitrary arity: Waltz algorithm ancestor of AC's Generalized arc-consistency (Dechter, Section 3.5.1 ) Relational m-consistency (Dechter, Section 8.1) Consistency: Properties & Algorithms

Overview of Recommended Reading Davis, AIJ'77 focuses on the Waltz algorithm studies its performance and quiescence for given: constraint types (order relations, bounded diff, algebraic, etc.) domains types (continuous or finite) Mackworth, AIJ'77 presents NC, AC-1, AC-2, PC-1, PC-2 Mackworth and Freuder, AIJ'85 studies their complexity  Mackworth and Freuder concentrate on finite domains  More people work on finite domains than on continuous ones  Continuous domains can be quite tough Consistency: Properties & Algorithms

Constraint Propagation with Interval Labels Ernest Davis `Old' paper (1987): terminology slightly different Interval labels: continuous domains Section 8: sign labels (discrete) Concerned with Waltz algorithm (e.g., quiescence, completeness) Constraint types vs. domain types (Table 3) Addresses applications from reasoning about Physical Systems (circuit, SPAM) about time relations (TMM)  Advice: read Section 3, more if you wish Consistency: Properties & Algorithms

Waltz algorithm for label inference Refine(C(Vi, Vk, Vm, Vn), Vi) - finds a new label for Vi consistent with C. Revise(C (Vi, Vk, Vm, Vn)) refines the domains of Vi, Vk, Vm, Vn by iterating over these variables until quiescence (no domains is further refined) Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised Waltz Algorithm  ancestor of arc-consistency Consistency: Properties & Algorithms

WARNING f(constraint types, domain types), see Table 3 → Completeness: Running the Waltz Algorithm does not solve the problem. A=2 and B=3 is still not a solution! → Quiescence: The Waltz algorithm may go into infinite loops even if problem is solvable x  [0, 100] x = y y  [0, 100] x = 2y → Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm f(constraint types, domain types), see Table 3 Consistency: Properties & Algorithms

Importance of this paper  Establishes that constraints of bounded differences (temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3), n number of variables)  Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains). This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm. Consistency: Properties & Algorithms

Basic consistency algorithms Examining finite, binary CSPs and their complexity Notation: Given variables i, j, k with values x, y, z, the predicate Pijk(x, y, z) is true iff the 3-tuple x, y, z  Cijk Node consistency: checking Pi(x) Arc consistency: checking Pij(x,y) Path consistency: bit-matrix manipulation Consistency: Properties & Algorithms

Worst-case asymptotic complexity time space Worst-case complexity as a function of the input parameters Upper bound: f(n) is O(g(n)) means that f(n)  c.g(n) f grows as g or slower Lower bound: f(n) is  (h(n)) means that f(n)  c.h(n) f grows as g or faster Input parameters for a CSP: n = number of variables a = (max) size of a domain dk = degree of Vk ( n-1) e = number of edges (or constraints)  [(n-1), n(n-1)/2] Consistency: Properties & Algorithms

Properties & Algorithms Outline Motivation and material Node & arc consistency and their complexity: NC, AC-1, AC-3, AC-4 Criteria for performance comparison and CSP parameters. Path consistency and its complexity Consistency properties of a CSP Other related results: box consistency, constraint synthesis, look-ahead, non-binary constraints, etc. Consistency: Properties & Algorithms

Properties & Algorithms Node consistency (NC) Procedure NC(i): Di  Di  { x | Pi(x) } Begin for i  1 until n do NC(i) end Consistency: Properties & Algorithms

Properties & Algorithms Complexity of NC Procedure of NC(i) Di Di  { x | Pi(x) } Begin for i  1 until n do NC(i) end For each variable, we check a values We have n variables, we do n.a checks NC is O(a.n) Consistency: Properties & Algorithms

Arc-consistency Adapted from Dechter Definition: Given a constraint graph G, A variable Vi is arc-consistent relative to Vj iff for every value aDVi, there exists a value bDVj | (a, b)CVi,Vj. The constraint CVi,Vj is arc-consistent iff Vi is arc-consistent relative to Vj and Vj is arc-consistent relative to Vi. A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent 1 2 3 Vi Vj Consistency: Properties & Algorithms

Properties & Algorithms Procedure Revise Revises the domains of a variable i Procedure Revise(i,j): Begin DELETE  false for each x  Di do if there is no y  Dj such that Pij(x, y) then begin delete x from Di DELETE  true end return DELETE Revise is directional What is the complexity of Revise? {a2} Consistency: Properties & Algorithms

Revise: example R. Dechter Apply the Revise procedure to the following example Consistency: Properties & Algorithms

Effect of Revise Adapted from Dechter Question: Given two variables Vi and Vj their domains DVi and DVj, and the constraint CVi,Vj, write the effect of the Revise procedure as a sequence of operations in relational algebra Hint: Think about the domain DVi as a unary constraint CVi and consider the composition of this unary constraint and the binary one.. 1 2 3 Vi Vj Solution: DVi  DVi  Vi (CVi,Vj DVi) This is actually equivalent to DVi  Vi (CVi,Vj DVi) Consistency: Properties & Algorithms

Properties & Algorithms Arc consistency (AC-1) Procedure AC-1: 1 begin 2 for i  1 until n do NC(i) 3 Q  {(i, j) | (i,j)  arcs(G), i  j } 4 repeat 5 begin 6 CHANGE  false 7 for each (i, j)  Q do CHANGE  ( REVISE(i, j) or CHANGE ) 8 end 9 until ¬ CHANGE 10 end AC-1 does not update Q, the queue of arcs No algorithm can have time complexity below O(ea2) Alert: Algorithms do not, but should, test for empty domains Consistency: Properties & Algorithms

Properties & Algorithms Arc consistency AC may discover the solution Example borrowed from Dechter V2 V3 V1 Consistency: Properties & Algorithms

Properties & Algorithms Arc consistency 2. AC may discover inconsistency Example borrowed from Dechter Consistency: Properties & Algorithms

NC & AC Example courtesy of M. Fromherz In the temporal problem below AC propagates bound Unary constraint x>3 is imposed AC propagates bounds again [0, 10] x  y-3 x y [0, 7] [3, 10] x  y-3 x y [4, 7] [7, 10] x  y-3 x y Consistency: Properties & Algorithms

Properties & Algorithms Complexity of AC-1 Procedure AC-1: 1 begin 2 for i  1 until n do NC(i) 3 Q  {(i, j) | (i,j)  arcs(G), i  j 4 repeat 5 begin 6 CHANGE  false 7 for each (i, j)  Q do CHANGE  (REVISE(i, j) or CHANGE) 8 end 9 until ¬ CHANGE 10 end Note: Q is not modified and |Q| = 2e 4  9 repeats at most n·a times Each iteration has |Q| = 2e calls to REVISE Revise requires at most a2 checks of Pij  AC-1 is O(a3 · n · e) Consistency: Properties & Algorithms

Properties & Algorithms AC versions AC-1 does not update Q, the queue of arcs AC-2 iterates over arcs connected to at least one node whose domain has been modified. Nodes are ordered. AC-3 same as AC-2, nodes are not ordered Consistency: Properties & Algorithms

Properties & Algorithms Arc consistency (AC-3) AC-3 iterates over arcs connected to at least one node whose domain has been modified Procedure AC-3: 1 begin 2 for i  1 until n do NC(i) 3 Q  { (i, j) | (i, j)  arcs(G), i  j } 4 While Q is not empty do 5 begin 6 select and delete any arc (k, m) from Q 7 If Revise(k, m) then Q  Q  { (i, k) | (i, k)  arcs(G), i  k, i  m } 8 end 9 end Consistency: Properties & Algorithms

Properties & Algorithms Complexity of AC-3 Procedure AC-3: 1 begin 2 for i  1 until n do NC(i) 3 Q  {(i, j) | (i, j) arcs(G), i  j } 4 While Q is not empty do 5 begin select and delete any arc (k, m) from Q 7 If Revise(k, m) then Q  Q  { (i, k) | (i, k)  arcs(G), i  k, i  m } 8 end 9 end First |Q| = 2e, then it grows and shrinks 48 Worst case: 1 element is deleted from DVk per iteration none of the arcs added is in Q Line 7: a·(dk - 1) Lines 4 - 8: Revise: a2 checks of Pij  AC-3 is O(a2(2e + a(2e-n))) Connected graph (e  n – 1), AC-3 is O(a3e) If AC-p, AC-3 is O(a2e)  AC-3 is (a2e) Complete graph: O(a3n2) Consistency: Properties & Algorithms

Properties & Algorithms Example: Apply AC-3 Example: Apply AC-3 Thanks to Xu Lin DV1 = {1, 2, 3, 4, 5} DV2 = {1, 2, 3, 4, 5} DV3 = {1, 2, 3, 4, 5} DV4 = {1, 2, 3, 4, 5} CV2,V3 = {(2, 2), (4, 5), (2, 5), (3, 5), (2, 3), (5, 1), (1, 2), (5, 3), (2, 1), (1, 1)} CV1,V3 = {(5, 5), (2, 4), (3, 5), (3, 3), (5, 3), (4, 4), (5, 4), (3, 4), (1, 1), (3, 1)} CV2,V4 = {(1, 2), (3, 2), (3, 1), (4, 5), (2, 3), (4, 1), (1, 1), (4, 3), (2, 2), (1, 5)} { 1, 2, 3, 4, 5 } V1 V3 V2 V4 { 1, 3, 5 } { 1, 2, 3, 5 } { 1, 2, 3, 4 } Consistency: Properties & Algorithms

Applying AC-3 Thanks to Xu Lin Queue = {CV2, V4, CV4,V2, CV1,V3, CV2,V3, CV3, V1, CV3,V2} Revise(V2,V4): DV2 DV2 \ {5} = {1, 2, 3, 4} Queue = {CV4,V2, CV1,V3, CV2,V3, CV3, V1, CV3, V2} Revise(V4, V2): DV4  DV4 \ {4} = {1, 2, 3, 5} Queue = {CV1,V3, CV2,V3, CV3, V1, CV3, V2} Revise(V1, V3): DV1  {1, 2, 3, 4, 5} Queue = {CV2,V3, CV3, V1, CV3, V2} Revise(V2, V3): DV2  {1, 2, 3, 4} Consistency: Properties & Algorithms

Applying AC-3 (cont.) Thanks to Xu Lin Queue = {CV3, V1, CV3, V2} Revise(V3, V1): DV3  DV3 \ {2} = {1, 3, 4, 5} Queue = {CV2, V3, CV3, V2} Revise(V2, V3): DV2  DV2 Queue = {CV3, V2} Revise(V3, V2): DV3  {1, 3, 5} Queue = {CV1, V3} Revise(V1, V3): DV1  {1, 3, 5} END Consistency: Properties & Algorithms

Properties & Algorithms Main Improvements Mohr & Henderson (AIJ 86): AC-3 O(a3e)  AC-4 O(a2e) AC-4 is optimal Trade repetition of consistency-check operations with heavy bookkeeping on which and how many values support a given value for a given variable Data structures it uses: m: values that are active, not filtered s: lists all vvp's that support a given vvp counter: given a vvp, provides the number of support provided by a given variable How it proceeds: Generates data structures Prepares data structures Iterates over constraints while updating support in data structures Consistency: Properties & Algorithms

Step 1: Generate 3 data structures m and s have as many rows as there are vvp’s in the problem counter has as many rows as there are tuples in the constraints 4 ), , ( 1 2 3 V counter M Consistency: Properties & Algorithms

Step 2: Prepare data structures Data structures: s and counter. Checks for every constraint CVi,Vj all tuples Vi=ai, Vj=bj_) When the tuple is allowed, then update: s(Vj,bj)  s(Vj,bj)  {(Vi, ai)} and counter(Vj,bj)  (Vj,bj) + 1 Update counter ((V2, V3), 2) to value 1 Update counter ((V3, V2), 2) to value 1 Update s-htable (V2, 2) to value ((V3, 2)) Update s-htable (V3, 2) to value ((V2, 2)) Update counter ((V2, V3), 4) to value 1 Update counter ((V3, V2), 5) to value 1 Update s-htable (V2, 4) to value ((V3, 5)) Update s-htable (V3, 5) to value ((V2, 4)) Consistency: Properties & Algorithms

Constraints CV2,V3 and CV3,V2 Nil (V3, 2) ((V2, 1), (V2, 2)) (V3, 3) ((V2, 5), (V2, 2)) (V2, 2) ((V3,1), (V3,3), (V3, 5), (V3,2)) (V3, 1) ((V2, 1), (V2, 2), (V2, 5)) (V2, 3) ((V3, 5)) (V1, 2) (V2, 1) ((V3, 1), (V3, 2)) (V1, 3) (V3, 4) (V4, 2) (V3, 5) ((V2, 3), (V2, 2), (V2, 4)) (v4, 3) (V1, 1) (V2, 4) (V2, 5) ((V3, 3), (V3, 1)) (V4, 1) (V1, 4) (V1, 5) (V4, 4) Counter (V4, V2), 3 (V4, V2), 2 (V4, V2), 1 (V2, V3), 1 2 (V2, V3), 3 4 (V4, V2), 5 1 (V2, V4), 1 (V2, V3), 4 (V4, V2), 4 (V2, V4), 2 (V2, V3), 5 (V2, V4), 3 (V2, V4), 4 (V2, V4), 5 (V3, V1), 1 (V3, V1), 2 (V3, V1), 3 … (V3, V2), 4 Etc… Etc. Updating m Note that (V3, V2),4  0 thus we remove 4 from the domain of V3 and update (V3, 4)  nil in m Updating counter Since 4 is removed from DV3 then for every (Vk, l) | (V3, 4)  s[(Vk, l)], we decrement counter[(Vk, V3), l] by 1 Consistency: Properties & Algorithms

Summary of arc-consistency algorithms AC-4 is optimal (worst-case) [Mohr & Henderson, AIJ 86] Warning: worst-case complexity is pessimistic. Better worst-case complexity: AC-4 Better average behavior: AC-3 [Wallace, IJCAI 93] AC-5: special constraints [Van Hentenryck, Deville, Teng 92] functional, anti-functional, and monotonic constraints AC-6, AC-7: general but rely heavily on data structures for bookkeeping [Bessière & Régin] Now, back to AC-3: AC-2000, AC-2001, AC-3.1, AC3.3, etc. Non-binary constraints: GAC (general) [Mohr & Masini 1988] all-different (dedicated) [Régin, 94] Consistency: Properties & Algorithms

Properties & Algorithms Outline Motivation and material Node and arc consistency and their complexity Criteria for performance comparison & CSP parameters. Which AC algorithm to choose? Path consistency and its complexity Consistency properties of a CSP Other related results: box consistency, constraint synthesis, look-ahead, non-binary CSPs, etc. Consistency: Properties & Algorithms

Properties & Algorithms CSP parameters ‹n, a, t, d› n is number of variables a is maximum domain size t is constraint tightness: d is constraint density where e is the #constraints, emin=(n-1), and emax = n(n-1)/2 Lately, we use constraint ratio p = e/emax → Constraints in random problems often generated uniform → Use only connected graphs (throw the unconnected ones away) Consistency: Properties & Algorithms

Criteria for performance comparison Bounding time and space complexity (theoretical) worst-case average-case best- case Counting #CC and #NV (theoretical, empirical) Measuring CPU time (empirical) Consistency: Properties & Algorithms

Performance comparison Courtesy of Lin XU AC-3, AC-7, AC-4 on n=10,a=10,t,d=0.6 displaying #CC and CPU time AC-4 AC-7 AC-3 Consistency: Properties & Algorithms

Performance comparison Courtesy of Shant AC3, AC3.1, AC4 on on n=40,a=16,t,d=0.15 displaying #CC and CPU time Consistency: Properties & Algorithms

Wisdom (?) Free adaptation from Bessière When a single constraint check is very expensive to make, use AC-7. When there is a lot of propagation, use AC-4 When little propagation and constraint checks are cheap, use AC-3x Advice? Instructor's personal opinion Use AC-3.. Consistency: Properties & Algorithms

Properties & Algorithms Outline Motivation and material Node and arc consistency and their complexity Criteria for performance comparison and CSP parameters. Path consistency and its complexity Consistency properties of CSPs Other related results: box consistency, constraint synthesis, look-ahead, non-binary CSPs, etc. Consistency: Properties & Algorithms

AC is not enough Example borrowed from Dechter Arc-consistent? Satisfiable?  seek higher levels of consistency V V 1 1 b a a b = = V V V V 2 2 3 3 a b a b b a a b = Consistency: Properties & Algorithms

Properties & Algorithms Consistency of a path A path (V0, V1, V2, …, Vm) of length m is consistent iff for any value xDV0 and for any value yDVm that are consistent (i.e., PV0 Vm(x, y))  a sequence of values z1, z2, … , zm-1 in the domains of variables V1, V2, …, Vm-1, such that all constraints between them (along the path, not across it) are satisfied (i.e., PV0 V1(x, z1)  PV1 V2(z1, z2)  …  PVm-1 Vm(zm-1, zm) ) for all x  DV0 for all y  DVm V0 Vm Vm-1 V2 V1 Consistency: Properties & Algorithms

Properties & Algorithms Note  The same variable can appear more than once in the path  Every time, it may have a different value  Constraints considered: PV0,Vm and those along the path  All other constraints are neglected for all x  DV0 for all y  DVm V0 Vm Vm-1 V2 V1 Consistency: Properties & Algorithms

Example: consistency of a path  Check path length = 2, 3, 4, 5, 6, .... {a, b, c} V2 V3 V1 V4 V7 V5 V6 All mutex constraints             Consistency: Properties & Algorithms

Path consistency: definition  A path of length m is path consistent  A CSP is path consistent Property of a CSP Definition: A CSP is path consistent (PC) iff every path is consistent (i.e., any length of path) Question: should we enumerate every path of any length? Answer: No, only length 2, thanks to [Mackworth AIJ'77] Consistency: Properties & Algorithms

Making a CSP Path Consistent (PC) Special case: Complete graph Theorem: In a complete graph, if every path of length 2 is consistent, the network is path consistent [Mackworth AIJ'77]  PC-1: two operations, composition and intersection  Proof by induction. Special case: Triangulated graph Theorem: In a triangulated graph, if every path of length 2 is consistent, the network is path consistent [Bliek & Sam-Haroud ‘99]  PPC (partially path consistent)  PC Consistency: Properties & Algorithms

Properties & Algorithms Tools for PC-1 Two operators Constraint composition: ( • ) R13 = R12 • R23 Constraint intersection: (  ) R13 R13, old  R13, induced Consistency: Properties & Algorithms

Path consistency (PC-1) Achieved by composition and intersection (of binary relations expressed as matrices) over all paths of length two. Procedure PC-1: 1 Begin 2 Yn  R 3 repeat 4 begin 5 Y0  Yn 6 For k  1 until n do 7 For i  1 until n do 8 For j  1 until n do 9 Ylij  Yl-1ij  Yl-1ik • Yl-1kk • Yl-1kj 10 end 11 until Yn = Y0 12 Y  Yn 10 end Consistency: Properties & Algorithms

Properties & Algorithms Properties of PC-1 Discrete CSPs [Montanari'74] PC-1 terminates PC-1 results in a path consistent CSP PC-1 terminates. It is complete, sound (for finding PC network) PC-2: Improves to PC-1similarly to (AC-1  AC-3) Complexity of PC-1.. Consistency: Properties & Algorithms

Properties & Algorithms Complexity of PC-1 Procedure PC-1: 1 Begin 2 Yn  R 3 repeat 4 begin 5 Y0  Yn 6 For k  1 until n do 7 For i  1 until n do 8 For j  1 until n do 9 Ylij  Yl-1ij  Yl-1ik • Yl-1kk • Yl-1kj 10 end 11 until Yn = Y0 12 Y  Yn 10 end Line 9: a3 Lines 6–10: n3. a3 Line 3: at most n2 relations x a2 elements PC-1 is O(a5n5) PC-2 is O(a5n3) and (a3n3) Consistency: Properties & Algorithms

Properties & Algorithms Some improvements Mohr & Henderson (AIJ 86) PC-2 O(a5n3)  PC-3 O(a3n3) Open question: PC-3 optimal? Han & Lee (AIJ 88) PC-3 is incorrect PC-4 O(a3n3) space and time Singh (ICTAI 95) PC-5 uses ideas of AC-6 (support bookkeeping) Note: PC is seldom used in practical applications unless in presence of special type of constraints (e.g., bounded difference) Consistency: Properties & Algorithms

Path consistency as inference of binary constraints B < C A < B A < C Path consistency corresponds to inferring a new constraint (alternatively, tightening an existing constraint) between every two variables given the constraints that link them to a third variable  Considers all subgraphs of 3 variables  3-consistency B < C Consistency: Properties & Algorithms

Path consistency as inference of binary constraints Another example: V4 V3 a b V2 V1  = Consistency: Properties & Algorithms

Question Adapted from Dechter Given three variables Vi, Vk, and Vj and the constraints CVi,Vk, CVi,Vj, and CVk,Vj, write the effect of PC as a sequence of operations in relational algebra. B B B C A A < B A + 3 > C A < C B < C A < B A < B A A B < C B < C C C -3 < A –C < 0 A + 3 > C Solution: CVi,Vj  CVi,Vj  ij(CVi,Vk CVk,Vj) Consistency: Properties & Algorithms

Constraint propagation courtesy of Dechter After Arc-consistency: After Path-consistency: Are these CSPs the same? Which one is more explicit? Are they equivalent? The more propagation, the more explicit the constraints the more search is directed towards a solution 1 2 3 1 2 3 ( 0, 1 )  ( 0, 1 )  Consistency: Properties & Algorithms

PC can detect unsatisfiability Arc-consistent? Path-consistent? V1 a b    V2 V3 a b  a b a b   a b V4 Consistency: Properties & Algorithms

Properties & Algorithms Outline Motivation and material Node and arc consistency and their complexity Criteria for performance comparison and CSP parameters. Which AC algorithm to choose? Project results may tell! Path consistency and its complexity Consistency properties of a CSP Other related results: box consistency, constraint synthesis, look-ahead, non-binary constraints, etc. Consistency: Properties & Algorithms

Consistency properties of a CSP (Arc Consistency, AC) (Path Consistency, PC) Minimality Decomposability (ref. lossless join decomposition) When PC approximates minimality and decomposability. Example: temporal reasoning (strong) k-consistency (i,j)-consistency Consistency: Properties & Algorithms

Properties & Algorithms Minimality PC tightens the binary constraints The tightest possible binary constraints yield the minimal network Minimal network a.k.a. central problem Given two values for two variables, if they are consistent, then they appear in at least one solution. Note: Minimal  path consistent The definition of minimal CSP is concerned with binary CSPs Consistency: Properties & Algorithms

Properties & Algorithms Minimal CSP Minimal network a.k.a. central problem Given two values for two variables, if they are consistent, then they appear in at least one solution. Informally In a minimal CSP the remainder of the CSP does not add any further constraint to the direct constraint CVi, Vj between the two variables Vi and Vj [Mackworth AIJ'77] A minimal CSP is perfectly explicit: as far as the pair Vi and Vj is concerned, the rest of the network does not add any further constraint to the direct constraint CVi, Vj [Montanari'74] The binary constraints are explicit as possible. [Montanari'74] Consistency: Properties & Algorithms

Properties & Algorithms Decomposability Any combination of values for k variables that satisfy the constraints between them can be extended to a solution. Decomposability generalizes minimality Minimality: any consistent combination of values for any 2 variables is extendable to a solution Decomposability: any consistent combination of values for any k variables is extendable to a solution Decomposable  Minimal  Path Consistent   Strong n-consistent  n-consistent  Solvable Consistency: Properties & Algorithms

Relations to (theory of) DB CSP Database Minimal Pair-wise consistent Decomposable Complete join Consistency: Properties & Algorithms

Properties & Algorithms PC approximates.. In general: Decomposability  minimality  path consistent PC is used to approximate minimality (which is the central problem) When is the approximation the real thing? Special cases: When composition distributes over intersection, [Montanari'74] PC-1 on the completed graph guarantees minimality and decomposability When constraints are convex [Bliek & Sam-Haroud 99] PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible) Consistency: Properties & Algorithms

PC: algorithms and property CSP0 Complete the graph PC-1 Result: CSP0 is not path consistent or PC-1(CSP0) is path consistent It is tight, but could be tighter The graph is complete CSP0 Triangulate the graph PPC Result: CSP0 is not path consistent or PPC(CSP0) is path consistent It is generally less tight than PC-1(CSP0) The graph is triangulated Watch for PPC Consistency: Properties & Algorithms

PC algorithms: special cases CSPa Complete the graph PC-1/2/4 Result: [Montanari 74] CSPa is not path consistent or PC-1(CSPa) is Path consistent, minimal & decomposable Graph is complete With a(bc)=(ab)(bc) CSPb Result: [B&S-H 99] CSPb is not path consistent or PPC(CSPb) is Path consistent, minimal, decomposable Existing edges are as tight as PC-1(CSPb) Graph is triangulated Triangulatethe graph PPC Constraint are convex Consistency: Properties & Algorithms

Special conditions: examples Distributivity property 1 loop in PC-1 (PC-3) is sufficient Exploiting special conditions in temporal reasoning: Temporal constraints in the Simple Temporal Problem (STP): composition & intersection Composition distributes over intersection PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path) Convex constraints PPC Consistency: Properties & Algorithms

Distributivity property Intersection,  Composition, • In PC-1, two operations: RAB • (RBC  R'BC) = (RAB • RBC)  (RAB • R’BC) When ( • ) distributes over (  ), then [Montanari'74] PC-1 guarantees that CSP is minimal and decomposable 1 loop in PC-1 is sufficient B RAB R’BC RBC A C Consistency: Properties & Algorithms

Temporal reasoning constraints of bounded difference Variables: X, Y, Z, etc. Constraints: a  Y-X  b, i.e. Y-X = [a, b] = I Composition: I 1 • I2 = [a1, b1] • [a2, b2] = [a1+ a2, b1+b2] Interpretation: intervals indicate distances composition is triangle inequality. Intersection: I1  I2 = [max(a1, a2), min(b1, b2)] Distributivity: I1 • (I2  I3) = (I1 • I2)  (I1 • I3) Proof: left as an exercise Consistency: Properties & Algorithms

Example: constraints of bounded difference V1 V3 V2 V0 R01=[2,5] Composition of intervals + : R’13 = R12 + R23 = [4, 12] R01 + R13 = [2,5] + [3, 5] = [5, 10] R01 + R'13 = [2,5] + [4, 12] = [6, 17] Intersection of intervals: R13  R'13 = [4, 12]  [3, 5] = [4, 5] R01 + (R13  R'13) = (R01 + R13)  (R01 + R'13) R01 + (R13  R'13) = [2, 5] + [4, 5] = [6, 10] (R01 + R13)  (R01 + R'13) = [5, 10]  [6,17] = [6, 10] Path consistency guarantees minimality and decomposability [ Consistency: Properties & Algorithms

Properties & Algorithms PC-1 on the STP PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path), where composition is ‘scalar addition’ and intersection is ‘scalar minimal’ Consistency: Properties & Algorithms

Convex constraints: temporal reasoning (again!) Thanks to Xu Lin (2002) Constraints of bounded difference are convex We triangulate the graph (good heuristics exist) Apply PPC: restrict propagations in PC to triangles of the graph (and not in the complete graph) According to [Bliek & Sam-Haroud 99] PPC becomes equivalent to PC, thus it guarantees minimality and decomposability Consistency: Properties & Algorithms

Properties & Algorithms PC is not enough {a, b, c} V2 V3 V1 V4 V7 V5 V6 All mutex constraints Arc-consistent? Path-consistent? Satisfiable?  we should seek (even) higher levels of consistency  k-consistency, k = 1, 2, 3, ….                  Consistency: Properties & Algorithms

Properties & Algorithms k-consistency A CSP is k-consistent iff given any consistent instantiation of any (k-1) distinct variables there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables. Example: (courtesy of Dechter) 3-consistent? 4-consistent? Q Q Q Q Q Consistency: Properties & Algorithms

Warning: Does 3-consistency guarantee 2-consistency? B {red, blue} {red, blue}   A C { red } { red } Question: Is this CSP 3-consistent? is it 2-consistent? Lesson: 3-consistency does not guarantee 2-consistency Consistency: Properties & Algorithms

(Strong) k-consistency: definition Definitions: k-consistency: any consistent assignment for (k-1)-variable can be extended to any kth variable strong k-consistency: has to be j-consistent for all j  k Globally consistent: i-consistent for all 1in Consistency: Properties & Algorithms

(i,j)-consistency: definition every consistent instantiation to i variables can be extended to every j variables Consequently: 2-consistency: i=1, j=1 3-consistency: i=2, j=1 k-consistency: i=k-1, j=1 Minimality: i=2, j=n-2 Decomposability: i=k, j=n-k Consistency: Properties & Algorithms

Consistency properties & algorithms: summary New terms: AC, PC, minimal CSP, decomposable CSP, complete graph, triangulated graph, Revise, AC-1, AC-3, PC-1, PC-2, PPC Complexity of AC, PC algorithms AC tightens domains, PC tightens binary constraints Consistency operations in terms of composition, intersection in relational algebra Path consistency approximates minimality, which approximates decomposability Situations when path consistency guarantees minimality and decomposability Strong k-consistency and (i,j)-consistency Consistency: Properties & Algorithms

Properties & Algorithms Outline Motivation and material Node and arc consistency and their complexity Criteria for performance comparison and CSP parameters. Which AC algorithm to choose? Project results may tell! Path consistency and its complexity Consistency properties of a CSP Other related results: Constraint synthesis, Box consistency Singleton Arc Consistency Neighborhood Inverse Consisten Look-ahead, Non-binary CSPs, etc. Consistency: Properties & Algorithms

Properties & Algorithms Constraint synthesis k-consistency: an alternative view Constraint of arity (k-1) are used to synthesize a constraint of arity k, which is then used to filter constraints of arity (k-1) Examples: 2-consistency: DVi  DVi  Vi (CVi,Vj DVj) Projection of binary constraint is used to tighten unary constraint 3-consistency: CVi,Vj  CVi,Vj  Vi,Vj(CVi,Vk CVk,Vj) Binary constraints are used to synthesize a ternary constraint Projection of the ternary constraint is used to tighten binary constraint k-consistency: [Freuder 78, 82] Constraints of arity (k-1) are used to synthesize constraints of arity k Projection of k-ary constraint is used to tighten (k-1)-ary constraints Consistency: Properties & Algorithms

Solving CSPs by Constraint synthesis [Freuder 78] From k=2 to k=n, achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints, then use the k-ary constraints to filter constraints of arity k-1, k-2, etc. Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP Consistency: Properties & Algorithms

Box Consistency (on interval constraints) Domains are (continuous) intervals Historically also called: continuous CSPs, continuous domains Domains are infinite: We cannot enumerate consistent values/tuples [Davis, AIJ 87] (see recommended reading) showed that even AC may be incomplete or not terminate We apply consistency (usually, arc-consistency) on the boundaries of the interval Consistency: Properties & Algorithms

Singleton Arc Consistency (SAC) Repeat until no change occurs Repeat for each variable Repeat for each value in domain Assign this value to this variable. If the CSP is AC, keep the value. Otherwise, remove it. Proposed by Debruyne & Bessière, IJCAI 97 Consistency: Properties & Algorithms

Neighborhood Inverse Consistency (NIC) Repeat until no change occurs Repeat for each variable Consider only the neighborhood of the variable Repeat for each value for the variable If the value appears in any complete solution for the neighborhood, keep it. Otherwise, remove it. Proposed by Freuder & Elfe, AAAI 96 Consistency: Properties & Algorithms

Look-ahead (already discussed) Integration of consistency algorithm with search AC: Forward Checking (FC) [Haralick & Elliott] Directional Arc-Consistency (DAC) [Dechter] Maintaining arc consistency (MAC) [Sabin & Freuder] PC: Directed path consistency (DPC) [Dechter] Consistency: Properties & Algorithms

Properties & Algorithms Non-binary CSPs (Almost) all algorithms and properties discussed so far were restricted to binary CSPs Consistency properties for non-binary CSPs are the topic of current research. Mainly, properties and algorithms for: GAC: generalized arc-consistency [Mohr & Masini] Relational m-consistency [Dechter, Chap 8] Consistency: Properties & Algorithms

Generalized Arc-Consistency Conceptually: project the constraint on each of the variables in its scope to tighten the domain of the variable. When constraint is not defined in extension, GAC may be problematic (e.g., NP-hard in TCSP) Consistency: Properties & Algorithms

Relational consistency Dechter (see Section 8.1.1) generalizes consistency properties to non-binary constraints Relational m-consistency Relational 1-consistency  relational arc-consistency Relational 2-consistency  relational path-consistency Relational (i,m)consistency Relational (1,1)consistency  GAC Consistency: Properties & Algorithms

Look-ahead in non-binary CSPs In binary CSPs, when a variable is instantiated, Revise (Vf, Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc In non-binary CSPs, a constraint that applies to a current variable may involve one or more past variables and one or more future variables. The question is: when to trigger Revise? There are 5 strategies: nFC0, nFC1, …, nFC5 in increasing ‘aggressivity’ [Bessiere et al. AIJ 02 ] For example, nFC0 triggers Revise only when all but one variable has been instantiated Consistency: Properties & Algorithms

Related results: summary Interval constraints: box consistency Solving CSPs by Constraint synthesis: [Freuder] From k=2 to k=n-1, achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints, then filter constraints of arity k-1, k-2, etc. Integration with search: AC: Forward Checking (FC) [Haralick & Elliott] Directional Arc-Consistency (DAC) [Dechter] Maintaining arc consistency (MAC) [Sabin & Freuder] PC: Directed path consistency (DPC) [Dechter] Non-binary constraints: Generalized arc consistency (GAC) [Mohr & Masini] Relational consistency [Dechter] Non-binary constraints and integration with search: nFC0, nFC1, …, nFC5 etc. Consistency: Properties & Algorithms