1. Consistency Algorithms Foundations of Constraint Processing
CSCE421/821, Spring 2008:
Berthe Y. Choueiry (Shu-we-ri)
Avery Hall, Room 123B
Tel: +1(402)
Consistency:
Properties & Algorithms

2. Properties & Algorithms
Lecture Sources
Required reading
Algorithms for Constraint Satisfaction Problems, Mackworth and Freuder AIJ'85
Sections 3.1, 3.2, 3.3. Chapter 3. Constraint Processing. Dechter
Recommended
Sections 3.4—3.10. Chapter 3. Constraint Processing. Dechter
Networks of Constraints: Fundamental Properties and Application to Picture Processing, Montanari, Information Sciences 74
Consistency in Networks of Relations, Mackworth AIJ'77
Constraint Propagation with Interval Labels, Davis, AIJ'87
Path Consistency on Triangulated Constraint Graphs, Bliek & Sam-Haroud IJCAI'99
Consistency:
Properties & Algorithms

3. Properties & Algorithms
Outline
Motivation and material
Node and arc consistency and their complexity
Criteria for performance comparison and CSP parameters.
Path consistency and its complexity
Consistency properties of a CSP
Other related results: box consistency, constraint synthesis, look-ahead, non-binary constraints, etc.
Consistency:
Properties & Algorithms

4. Properties & Algorithms
Consistency checking
Motivation
CSP are solved with search (conditioning)
Search performance is affected by:
- problem size
- amount of backtracking
Backtracking may yield
Thrashing
Exploring non-promising sub-trees and rediscovering the same inconsistencies over and over again
 Malady of backtrack search
Consistency:
Properties & Algorithms

5. (Some) causes of thrashing
Search order:
V1, V2, V3, V4, V5
What happens in search if we:
do not check the constraint on V3 (node inconsistency)
do not check constraint between V3 and V5 (arc inconsistency)
do not check constraints between V3—V4, V3—V5, and V4—V5 (path inconsistency)
{1, 2} V4
{1, 2, 3}
{ 0, 1, 2}
{ 1, 2} V5 V1 V2
V3<V2
V3>0
V3<V4
V5<V4
V3<V5
V3<V1 V3
Consistency:
Properties & Algorithms

6. Properties & Algorithms
Consistency checking
Goal: eliminate inconsistent combinations
Algorithms (for binary constraints):
Node consistency
Arc consistency (AC-1, AC-2, AC-3, ..., AC-7, AC-3.1, etc.)
Path consistency (PC-1, PC-2, DPC, PPC, etc.)
Constraints of arbitrary arity:
Waltz algorithm ancestor of AC's
Generalized arc-consistency (Dechter, Section )
Relational m-consistency (Dechter, Section 8.1)
Consistency:
Properties & Algorithms

7. Overview of Recommended Reading
Davis, AIJ'77
focuses on the Waltz algorithm
studies its performance and quiescence for given:
constraint types (order relations, bounded diff, algebraic, etc.)
domains types (continuous or finite)
Mackworth, AIJ'77
presents NC, AC-1, AC-2, PC-1, PC-2
Mackworth and Freuder, AIJ'85
studies their complexity
 Mackworth and Freuder concentrate on finite domains
 More people work on finite domains than on continuous ones
 Continuous domains can be quite tough
Consistency:
Properties & Algorithms

8. Constraint Propagation with Interval Labels Ernest Davis
`Old' paper (1987): terminology slightly different
Interval labels: continuous domains
Section 8: sign labels (discrete)
Concerned with Waltz algorithm (e.g., quiescence, completeness)
Constraint types vs. domain types (Table 3)
Addresses applications from reasoning
about Physical Systems (circuit, SPAM)
about time relations (TMM)
 Advice: read Section 3, more if you wish
Consistency:
Properties & Algorithms

9. Waltz algorithm for label inference
Refine(C(Vi, Vk, Vm, Vn), Vi) - finds a
new label for Vi consistent with C.
Revise(C (Vi, Vk, Vm, Vn)) refines the domains of Vi, Vk, Vm, Vn by iterating over these variables until quiescence (no domains is further refined)
Waltz Algorithm revises all constraints by iterating over each constraint connected to a variable whose domain has been revised
Waltz Algorithm  ancestor of arc-consistency
Consistency:
Properties & Algorithms

10. WARNING f(constraint types, domain types), see Table 3
→ Completeness: Running the Waltz Algorithm does not solve the problem.
A=2 and B=3 is still not a solution!
→ Quiescence: The Waltz algorithm may go into infinite loops even if problem is solvable
x  [0, 100] x = y
y  [0, 100] x = 2y
→ Davis provides a classification of problems in terms of completeness and quiescence of the Waltz algorithm
f(constraint types, domain types), see Table 3
Consistency:
Properties & Algorithms

11. Importance of this paper
 Establishes that constraints of bounded differences (temporal reasoning) can be efficiently solved by the Waltz algorithm (O(n3), n number of variables)
 Early paper that attracts attention on the difficulty of label propagation in interval labels (continuous domains).
This work has been continued by Faltings (AIJ 92) who studied the early-quiescence problem of the Waltz algorithm.
Consistency:
Properties & Algorithms

12. Basic consistency algorithms
Examining finite, binary CSPs and their complexity
Notation:
Given variables i, j, k with values x, y, z, the predicate
Pijk(x, y, z) is true iff the 3-tuple x, y, z  Cijk
Node consistency: checking Pi(x)
Arc consistency: checking Pij(x,y)
Path consistency: bit-matrix manipulation
Consistency:
Properties & Algorithms

13. Worst-case asymptotic complexity
time
space
Worst-case complexity as a function of the input parameters
Upper bound: f(n) is O(g(n)) means that f(n)  c.g(n)
f grows as g or slower
Lower bound: f(n) is  (h(n)) means that f(n)  c.h(n)
f grows as g or faster
Input parameters for a CSP:
n = number of variables
a = (max) size of a domain
dk = degree of Vk ( n-1)
e = number of edges (or constraints)  [(n-1), n(n-1)/2]
Consistency:
Properties & Algorithms

14. Properties & Algorithms
Outline
Motivation and material
Node & arc consistency and their complexity: NC, AC-1, AC-3, AC-4
Criteria for performance comparison and CSP parameters.
Path consistency and its complexity
Consistency properties of a CSP
Other related results: box consistency, constraint synthesis, look-ahead, non-binary constraints, etc.
Consistency:
Properties & Algorithms

15. Properties & Algorithms
Node consistency (NC)
Procedure NC(i):
Di  Di  { x | Pi(x) }
Begin
for i  1 until n do NC(i)
end
Consistency:
Properties & Algorithms

16. Properties & Algorithms
Complexity of NC
Procedure of NC(i)
Di Di  { x | Pi(x) }
Begin
for i  1 until n do NC(i)
end
For each variable, we check a values
We have n variables, we do n.a checks
NC is O(a.n)
Consistency:
Properties & Algorithms

17. Arc-consistency Adapted from Dechter
Definition: Given a constraint graph G,
A variable Vi is arc-consistent relative to Vj iff for every value aDVi, there exists a value bDVj | (a, b)CVi,Vj.
The constraint CVi,Vj is arc-consistent iff
Vi is arc-consistent relative to Vj and
Vj is arc-consistent relative to Vi.
A binary CSP is arc-consistent iff every constraint (or sub-graph of size 2) is arc-consistent 1 2 3 Vi Vj
Consistency:
Properties & Algorithms

18. Properties & Algorithms
Procedure Revise
Revises the domains of a variable i
Procedure Revise(i,j):
Begin
DELETE  false
for each x  Di do
if there is no y  Dj such that Pij(x, y) then
begin
delete x from Di
DELETE  true
end
return DELETE
Revise is directional
What is the complexity of Revise?
{a2}
Consistency:
Properties & Algorithms

19. Revise: example R. Dechter
Apply the Revise procedure to the following example
Consistency:
Properties & Algorithms

20. Effect of Revise Adapted from Dechter
Question:
Given
two variables Vi and Vj
their domains DVi and DVj, and
the constraint CVi,Vj,
write the effect of the Revise procedure as a sequence of operations in relational algebra
Hint:
Think about the domain DVi as a unary constraint CVi and
consider the composition of this unary constraint and the binary one.. 1 2 3 Vi Vj
Solution: DVi  DVi  Vi (CVi,Vj DVi)
This is actually equivalent to DVi  Vi (CVi,Vj DVi)
Consistency:
Properties & Algorithms

21. Properties & Algorithms
Arc consistency (AC-1)
Procedure AC-1:
1 begin
2 for i  1 until n do NC(i)
3 Q  {(i, j) | (i,j)  arcs(G), i  j }
4 repeat
begin
CHANGE  false
for each (i, j)  Q do CHANGE  ( REVISE(i, j) or CHANGE )
end
9 until ¬ CHANGE
10 end
AC-1 does not update Q, the queue of arcs
No algorithm can have time complexity below O(ea2)
Alert: Algorithms do not, but should, test for empty domains
Consistency:
Properties & Algorithms

22. Properties & Algorithms
Arc consistency
AC may discover the solution
Example borrowed from Dechter V2 V3 V1
Consistency:
Properties & Algorithms

23. Properties & Algorithms
Arc consistency
2. AC may discover inconsistency
Example borrowed from Dechter
Consistency:
Properties & Algorithms

24. NC & AC Example courtesy of M. Fromherz
In the temporal problem below
AC propagates bound
Unary constraint x>3 is imposed
AC propagates bounds again
[0, 10]
x  y-3 x y
[0, 7]
[3, 10]
x  y-3 x y
[4, 7]
[7, 10]
x  y-3 x y
Consistency:
Properties & Algorithms

25. Properties & Algorithms
Complexity of AC-1
Procedure AC-1:
1 begin
2 for i  1 until n do NC(i)
3 Q  {(i, j) | (i,j)  arcs(G), i  j
4 repeat
5 begin
6 CHANGE  false
for each (i, j)  Q do CHANGE  (REVISE(i, j) or CHANGE)
8 end
9 until ¬ CHANGE
10 end
Note: Q is not modified and |Q| = 2e
4  9 repeats at most n·a times
Each iteration has |Q| = 2e calls to REVISE
Revise requires at most a2 checks of Pij
 AC-1 is O(a3 · n · e)
Consistency:
Properties & Algorithms

26. Properties & Algorithms
AC versions
AC-1 does not update Q, the queue of arcs
AC-2 iterates over arcs connected to at least one node whose domain has been modified. Nodes are ordered.
AC-3 same as AC-2, nodes are not ordered
Consistency:
Properties & Algorithms

27. Properties & Algorithms
Arc consistency (AC-3)
AC-3 iterates over arcs connected to at least one node whose domain has been modified
Procedure AC-3:
1 begin
2 for i  1 until n do NC(i)
3 Q  { (i, j) | (i, j)  arcs(G), i  j }
4 While Q is not empty do
5 begin
6 select and delete any arc (k, m) from Q
7 If Revise(k, m) then Q  Q  { (i, k) | (i, k)  arcs(G), i  k, i  m }
8 end
9 end
Consistency:
Properties & Algorithms

28. Properties & Algorithms
Complexity of AC-3
Procedure AC-3:
1 begin
2 for i  1 until n do NC(i)
3 Q  {(i, j) | (i, j) arcs(G), i  j }
4 While Q is not empty do
5 begin
select and delete any arc (k, m) from Q
7 If Revise(k, m) then Q  Q  { (i, k) | (i, k)  arcs(G), i  k, i  m }
8 end
9 end
First |Q| = 2e, then it grows and shrinks 48
Worst case:
1 element is deleted from DVk per iteration
none of the arcs added is in Q
Line 7: a·(dk - 1)
Lines 4 - 8:
Revise: a2 checks of Pij
 AC-3 is O(a2(2e + a(2e-n)))
Connected graph (e  n – 1), AC-3 is O(a3e)
If AC-p, AC-3 is O(a2e)  AC-3 is (a2e)
Complete graph: O(a3n2)
Consistency:
Properties & Algorithms

29. Properties & Algorithms
Example: Apply AC-3
Example: Apply AC Thanks to Xu Lin
DV1 = {1, 2, 3, 4, 5}
DV2 = {1, 2, 3, 4, 5}
DV3 = {1, 2, 3, 4, 5}
DV4 = {1, 2, 3, 4, 5}
CV2,V3 = {(2, 2), (4, 5), (2, 5), (3, 5), (2, 3), (5, 1), (1, 2), (5, 3), (2, 1), (1, 1)}
CV1,V3 = {(5, 5), (2, 4), (3, 5), (3, 3), (5, 3), (4, 4), (5, 4), (3, 4), (1, 1), (3, 1)}
CV2,V4 = {(1, 2), (3, 2), (3, 1), (4, 5), (2, 3), (4, 1), (1, 1), (4, 3), (2, 2), (1, 5)}
{ 1, 2, 3, 4, 5 } V1 V3 V2 V4
{ 1, 3, 5 }
{ 1, 2, 3, 5 }
{ 1, 2, 3, 4 }
Consistency:
Properties & Algorithms

30. Applying AC-3 Thanks to Xu Lin
Queue = {CV2, V4, CV4,V2, CV1,V3, CV2,V3, CV3, V1, CV3,V2}
Revise(V2,V4): DV2 DV2 \ {5} = {1, 2, 3, 4}
Queue = {CV4,V2, CV1,V3, CV2,V3, CV3, V1, CV3, V2}
Revise(V4, V2): DV4  DV4 \ {4} = {1, 2, 3, 5}
Queue = {CV1,V3, CV2,V3, CV3, V1, CV3, V2}
Revise(V1, V3): DV1  {1, 2, 3, 4, 5}
Queue = {CV2,V3, CV3, V1, CV3, V2}
Revise(V2, V3): DV2  {1, 2, 3, 4}
Consistency:
Properties & Algorithms

31. Applying AC-3 (cont.) Thanks to Xu Lin
Queue = {CV3, V1, CV3, V2}
Revise(V3, V1): DV3  DV3 \ {2} = {1, 3, 4, 5}
Queue = {CV2, V3, CV3, V2}
Revise(V2, V3): DV2  DV2
Queue = {CV3, V2}
Revise(V3, V2): DV3  {1, 3, 5}
Queue = {CV1, V3}
Revise(V1, V3): DV1  {1, 3, 5} END
Consistency:
Properties & Algorithms

32. Properties & Algorithms
Main Improvements
Mohr & Henderson (AIJ 86):
AC-3 O(a3e)  AC-4 O(a2e)
AC-4 is optimal
Trade repetition of consistency-check operations with heavy bookkeeping on which and how many values support a given value for a given variable
Data structures it uses:
m: values that are active, not filtered
s: lists all vvp's that support a given vvp
counter: given a vvp, provides the number of support provided by a given variable
How it proceeds:
Generates data structures
Prepares data structures
Iterates over constraints while updating support in data structures
Consistency:
Properties & Algorithms

33. Step 1: Generate 3 data structures
m and s have as many rows as there are vvp’s in the problem
counter has as many rows as there are tuples in the constraints 4 ), , ( 1 2 3 V
counter M
Consistency:
Properties & Algorithms

34. Step 2: Prepare data structures
Data structures: s and counter.
Checks for every constraint CVi,Vj all tuples Vi=ai, Vj=bj_)
When the tuple is allowed, then update:
s(Vj,bj)  s(Vj,bj)  {(Vi, ai)} and
counter(Vj,bj)  (Vj,bj) + 1
Update counter ((V2, V3), 2) to value 1
Update counter ((V3, V2), 2) to value 1
Update s-htable (V2, 2) to value ((V3, 2))
Update s-htable (V3, 2) to value ((V2, 2))
Update counter ((V2, V3), 4) to value 1
Update counter ((V3, V2), 5) to value 1
Update s-htable (V2, 4) to value ((V3, 5))
Update s-htable (V3, 5) to value ((V2, 4))
Consistency:
Properties & Algorithms

35. Constraints CV2,V3 and CV3,V2
Nil
(V3, 2)
((V2, 1), (V2, 2))
(V3, 3)
((V2, 5), (V2, 2))
(V2, 2)
((V3,1), (V3,3), (V3, 5), (V3,2))
(V3, 1)
((V2, 1), (V2, 2), (V2, 5))
(V2, 3)
((V3, 5))
(V1, 2)
(V2, 1)
((V3, 1), (V3, 2))
(V1, 3)
(V3, 4)
(V4, 2)
(V3, 5)
((V2, 3), (V2, 2), (V2, 4))
(v4, 3)
(V1, 1)
(V2, 4)
(V2, 5)
((V3, 3), (V3, 1))
(V4, 1)
(V1, 4)
(V1, 5)
(V4, 4)
Counter
(V4, V2), 3
(V4, V2), 2
(V4, V2), 1
(V2, V3), 1 2
(V2, V3), 3 4
(V4, V2), 5 1
(V2, V4), 1
(V2, V3), 4
(V4, V2), 4
(V2, V4), 2
(V2, V3), 5
(V2, V4), 3
(V2, V4), 4
(V2, V4), 5
(V3, V1), 1
(V3, V1), 2
(V3, V1), 3 …
(V3, V2), 4
Etc…
Etc.
Updating m
Note that (V3, V2),4  0 thus
we remove 4 from the domain of V3
and update (V3, 4)  nil in m
Updating counter
Since 4 is removed from DV3 then
for every (Vk, l) | (V3, 4)  s[(Vk, l)],
we decrement counter[(Vk, V3), l] by 1
Consistency:
Properties & Algorithms

36. Summary of arc-consistency algorithms
AC-4 is optimal (worst-case) [Mohr & Henderson, AIJ 86]
Warning: worst-case complexity is pessimistic.
Better worst-case complexity: AC-4
Better average behavior: AC [Wallace, IJCAI 93]
AC-5: special constraints [Van Hentenryck, Deville, Teng 92]
functional, anti-functional, and monotonic constraints
AC-6, AC-7: general but rely heavily on data structures for bookkeeping [Bessière & Régin]
Now, back to AC-3: AC-2000, AC-2001, AC-3.1, AC3.3, etc.
Non-binary constraints:
GAC (general) [Mohr & Masini 1988]
all-different (dedicated) [Régin, 94]
Consistency:
Properties & Algorithms

37. Properties & Algorithms
Outline
Motivation and material
Node and arc consistency and their complexity
Criteria for performance comparison & CSP parameters.
Which AC algorithm to choose?
Path consistency and its complexity
Consistency properties of a CSP
Other related results: box consistency, constraint synthesis, look-ahead, non-binary CSPs, etc.
Consistency:
Properties & Algorithms

38. Properties & Algorithms
CSP parameters ‹n, a, t, d›
n is number of variables
a is maximum domain size
t is constraint tightness:
d is constraint density
where e is the #constraints, emin=(n-1), and emax = n(n-1)/2
Lately, we use constraint ratio p = e/emax
→ Constraints in random problems often generated uniform
→ Use only connected graphs (throw the unconnected ones away)
Consistency:
Properties & Algorithms

39. Criteria for performance comparison
Bounding time and space complexity (theoretical)
worst-case
average-case
best- case
Counting #CC and #NV (theoretical, empirical)
Measuring CPU time (empirical)
Consistency:
Properties & Algorithms

40. Performance comparison Courtesy of Lin XU
AC-3, AC-7, AC-4 on n=10,a=10,t,d=0.6 displaying #CC and CPU time
AC-4
AC-7
AC-3
Consistency:
Properties & Algorithms

41. Performance comparison Courtesy of Shant
AC3, AC3.1, AC4 on on n=40,a=16,t,d=0.15 displaying #CC and CPU time
Consistency:
Properties & Algorithms

42. Wisdom (?) Free adaptation from Bessière
When a single constraint check is very expensive to make, use AC-7.
When there is a lot of propagation, use AC-4
When little propagation and constraint checks are cheap, use AC-3x
Advice? Instructor's personal opinion
Use AC-3..
Consistency:
Properties & Algorithms

43. Properties & Algorithms
Outline
Motivation and material
Node and arc consistency and their complexity
Criteria for performance comparison and CSP parameters.
Path consistency and its complexity
Consistency properties of CSPs
Other related results: box consistency, constraint synthesis, look-ahead, non-binary CSPs, etc.
Consistency:
Properties & Algorithms

44. AC is not enough Example borrowed from Dechter
Arc-consistent?
Satisfiable?
 seek higher levels of consistency V V 1 1
b a
a b = = V V V V 2 2 3 3
a b
a b
b a
a b =
Consistency:
Properties & Algorithms

45. Properties & Algorithms
Consistency of a path
A path (V0, V1, V2, …, Vm) of length m is consistent iff
for any value xDV0 and for any value yDVm that are consistent (i.e., PV0 Vm(x, y))
 a sequence of values z1, z2, … , zm-1 in the domains of variables V1, V2, …, Vm-1, such that all constraints between them (along the path, not across it) are satisfied
(i.e., PV0 V1(x, z1)  PV1 V2(z1, z2)  …  PVm-1 Vm(zm-1, zm) )
for all x  DV0
for all y  DVm V0 Vm
Vm-1 V2 V1
Consistency:
Properties & Algorithms

46. Properties & Algorithms
Note
 The same variable can appear more than once in the path
 Every time, it may have a different value
 Constraints considered: PV0,Vm and those along the path
 All other constraints are neglected
for all x  DV0
for all y  DVm V0 Vm
Vm-1 V2 V1
Consistency:
Properties & Algorithms

47. Example: consistency of a path
 Check path length = 2, 3, 4, 5, 6, ....
{a, b, c} V2 V3 V1 V4 V7 V5 V6
All mutex constraints            
Consistency:
Properties & Algorithms

48. Path consistency: definition
 A path of length m is path consistent
 A CSP is path consistent Property of a CSP
Definition: A CSP is path consistent (PC) iff every path is
consistent (i.e., any length of path)
Question: should we enumerate every path of any length?
Answer: No, only length 2, thanks to [Mackworth AIJ'77]
Consistency:
Properties & Algorithms

49. Making a CSP Path Consistent (PC)
Special case: Complete graph
Theorem: In a complete graph, if every path of length 2 is consistent, the network is path consistent [Mackworth AIJ'77]
 PC-1: two operations, composition and intersection
 Proof by induction.
Special case: Triangulated graph
Theorem: In a triangulated graph, if every path of length 2 is consistent, the network is path consistent [Bliek & Sam-Haroud ‘99]
 PPC (partially path consistent)  PC
Consistency:
Properties & Algorithms

50. Properties & Algorithms
Tools for PC-1
Two operators
Constraint composition: ( • )
R13 = R12 • R23
Constraint intersection: (  )
R13 R13, old  R13, induced
Consistency:
Properties & Algorithms

51. Path consistency (PC-1)
Achieved by composition and intersection (of binary relations expressed as matrices) over all paths of length two.
Procedure PC-1:
1 Begin
2 Yn  R
3 repeat
begin
Y0  Yn
For k  1 until n do
For i  1 until n do
For j  1 until n do
9 Ylij  Yl-1ij  Yl-1ik • Yl-1kk • Yl-1kj
end
11 until Yn = Y0
12 Y  Yn
10 end
Consistency:
Properties & Algorithms

52. Properties & Algorithms
Properties of PC-1
Discrete CSPs [Montanari'74]
PC-1 terminates
PC-1 results in a path consistent CSP
PC-1 terminates. It is complete, sound (for finding PC network)
PC-2: Improves to PC-1similarly to (AC-1  AC-3)
Complexity of PC-1..
Consistency:
Properties & Algorithms

53. Properties & Algorithms
Complexity of PC-1
Procedure PC-1:
1 Begin
2 Yn  R
3 repeat
4 begin
5 Y0  Yn
6 For k  1 until n do
7 For i  1 until n do
For j  1 until n do
Ylij  Yl-1ij  Yl-1ik • Yl-1kk • Yl-1kj
10 end
11 until Yn = Y0
12 Y  Yn
10 end
Line 9: a3
Lines 6–10: n3. a3
Line 3: at most n2 relations x a2 elements
PC-1 is O(a5n5)
PC-2 is O(a5n3) and (a3n3)
Consistency:
Properties & Algorithms

54. Properties & Algorithms
Some improvements
Mohr & Henderson (AIJ 86)
PC-2 O(a5n3)  PC-3 O(a3n3)
Open question: PC-3 optimal?
Han & Lee (AIJ 88)
PC-3 is incorrect
PC-4 O(a3n3) space and time
Singh (ICTAI 95)
PC-5 uses ideas of AC-6 (support bookkeeping)
Note: PC is seldom used in practical applications unless in presence of special type of constraints (e.g., bounded difference)
Consistency:
Properties & Algorithms

55. Path consistency as inference of binary constraints
B < C
A < B
A < C
Path consistency corresponds to inferring a new constraint
(alternatively, tightening an existing constraint) between every two
variables given the constraints that link them to a third variable
 Considers all subgraphs of 3 variables
 3-consistency
B < C
Consistency:
Properties & Algorithms

56. Path consistency as inference of binary constraints
Another example: V4 V3
a b V2 V1  =
Consistency:
Properties & Algorithms

57. Question Adapted from Dechter
Given three variables Vi, Vk, and Vj and the constraints CVi,Vk, CVi,Vj, and CVk,Vj, write the effect of PC as a sequence of operations in relational algebra. B B B C A
A < B
A + 3 > C
A < C
B < C
A < B
A < B A A
B < C
B < C C C
-3 < A –C < 0
A + 3 > C
Solution: CVi,Vj  CVi,Vj  ij(CVi,Vk CVk,Vj)
Consistency:
Properties & Algorithms

58. Constraint propagation courtesy of Dechter
After Arc-consistency:
After Path-consistency:
Are these CSPs the same?
Which one is more explicit?
Are they equivalent?
The more propagation,
the more explicit the constraints
the more search is directed towards a solution 1 2 3 1 2 3
( 0, 1 ) 
( 0, 1 ) 
Consistency:
Properties & Algorithms

59. PC can detect unsatisfiability
Arc-consistent?
Path-consistent? V1
a b    V2 V3
a b 
a b
a b  
a b V4
Consistency:
Properties & Algorithms

60. Properties & Algorithms
Outline
Motivation and material
Node and arc consistency and their complexity
Criteria for performance comparison and CSP parameters.
Which AC algorithm to choose? Project results may tell!
Path consistency and its complexity
Consistency properties of a CSP
Other related results: box consistency, constraint synthesis, look-ahead, non-binary constraints, etc.
Consistency:
Properties & Algorithms

61. Consistency properties of a CSP
(Arc Consistency, AC)
(Path Consistency, PC)
Minimality
Decomposability (ref. lossless join decomposition)
When PC approximates minimality and decomposability.
Example: temporal reasoning
(strong) k-consistency
(i,j)-consistency
Consistency:
Properties & Algorithms

62. Properties & Algorithms
Minimality
PC tightens the binary constraints
The tightest possible binary constraints yield the minimal network
Minimal network a.k.a. central problem
Given two values for two variables, if they are consistent, then they appear in at least one solution.
Note:
Minimal  path consistent
The definition of minimal CSP is concerned with binary CSPs
Consistency:
Properties & Algorithms

63. Properties & Algorithms
Minimal CSP
Minimal network a.k.a. central problem
Given two values for two variables, if they are consistent, then they appear in at least one solution.
Informally
In a minimal CSP the remainder of the CSP does not add any further constraint to the direct constraint CVi, Vj between the two variables Vi and Vj [Mackworth AIJ'77]
A minimal CSP is perfectly explicit: as far as the pair Vi and Vj is concerned, the rest of the network does not add any further constraint to the direct constraint CVi, Vj [Montanari'74]
The binary constraints are explicit as possible. [Montanari'74]
Consistency:
Properties & Algorithms

64. Properties & Algorithms
Decomposability
Any combination of values for k variables that satisfy the constraints between them can be extended to a solution.
Decomposability generalizes minimality
Minimality: any consistent combination of values for
any 2 variables is extendable to a solution
Decomposability: any consistent combination of values for
any k variables is extendable to a solution
Decomposable 
Minimal 
Path Consistent  
Strong n-consistent 
n-consistent 
Solvable
Consistency:
Properties & Algorithms

65. Relations to (theory of) DB
CSP
Database
Minimal
Pair-wise consistent
Decomposable
Complete join
Consistency:
Properties & Algorithms

66. Properties & Algorithms
PC approximates..
In general:
Decomposability  minimality  path consistent
PC is used to approximate minimality (which is the central problem)
When is the approximation the real thing?
Special cases:
When composition distributes over intersection, [Montanari'74]
PC-1 on the completed graph guarantees minimality and decomposability
When constraints are convex [Bliek & Sam-Haroud 99]
PPC on the triangulated graph guarantees minimality and decomposability (and the existing edges are as tight as possible)
Consistency:
Properties & Algorithms

67. PC: algorithms and property
CSP0
Complete the graph
PC-1
Result:
CSP0 is not path consistent or
PC-1(CSP0) is path consistent
It is tight, but could be tighter
The graph is complete
CSP0
Triangulate the graph
PPC
Result:
CSP0 is not path consistent or
PPC(CSP0) is path consistent
It is generally less tight than PC-1(CSP0)
The graph is triangulated
Watch for PPC
Consistency:
Properties & Algorithms

68. PC algorithms: special cases
CSPa
Complete the graph
PC-1/2/4
Result: [Montanari 74]
CSPa is not path consistent or
PC-1(CSPa) is
Path consistent, minimal & decomposable
Graph is complete
With a(bc)=(ab)(bc)
CSPb
Result: [B&S-H 99]
CSPb is not path consistent or
PPC(CSPb) is
Path consistent, minimal, decomposable
Existing edges are as tight as PC-1(CSPb)
Graph is triangulated
Triangulatethe graph
PPC
Constraint are convex
Consistency:
Properties & Algorithms

69. Special conditions: examples
Distributivity property
1 loop in PC-1 (PC-3) is sufficient
Exploiting special conditions in temporal reasoning:
Temporal constraints in the Simple Temporal Problem (STP): composition & intersection
Composition distributes over intersection
PC-1 is a generalization of the Floyd-Warshall algorithm (all pairs shortest path)
Convex constraints
PPC
Consistency:
Properties & Algorithms

70. Distributivity property
Intersection, 
Composition, •
In PC-1, two operations:
RAB • (RBC  R'BC) = (RAB • RBC)  (RAB • R’BC)
When ( • ) distributes over (  ), then [Montanari'74]
PC-1 guarantees that CSP is minimal and decomposable
1 loop in PC-1 is sufficient B
RAB
R’BC
RBC A C
Consistency:
Properties & Algorithms

71. Temporal reasoning constraints of bounded difference
Variables: X, Y, Z, etc.
Constraints: a  Y-X  b, i.e. Y-X = [a, b] = I
Composition: I 1 • I2 = [a1, b1] • [a2, b2] = [a1+ a2, b1+b2]
Interpretation:
intervals indicate distances
composition is triangle inequality.
Intersection: I1  I2 = [max(a1, a2), min(b1, b2)]
Distributivity: I1 • (I2  I3) = (I1 • I2)  (I1 • I3)
Proof: left as an exercise
Consistency:
Properties & Algorithms

72. Example: constraints of bounded differenceV1 V3 V2 V0
R01=[2,5]
Composition of intervals + :
R’13 = R12 + R23 = [4, 12]
R01 + R13 = [2,5] + [3, 5] = [5, 10]
R01 + R'13 = [2,5] + [4, 12] = [6, 17]
Intersection of intervals: R13  R'13 = [4, 12]  [3, 5] = [4, 5]
R01 + (R13  R'13) = (R01 + R13)  (R01 + R'13)
R01 + (R13  R'13) = [2, 5] + [4, 5] = [6, 10]
(R01 + R13)  (R01 + R'13) = [5, 10]  [6,17] = [6, 10]
Path consistency guarantees minimality and decomposability [
Consistency:
Properties & Algorithms

73. Properties & Algorithms
PC-1 on the STP
PC-1 generalizes Floyd-Warshall algorithm (all-pairs shortest path), where
composition is ‘scalar addition’ and
intersection is ‘scalar minimal’
Consistency:
Properties & Algorithms

74. Convex constraints: temporal reasoning (again!)
Thanks to Xu Lin (2002)
Constraints of bounded difference are convex
We triangulate the graph (good heuristics exist)
Apply PPC: restrict propagations in PC to triangles of the graph (and not in the complete graph)
According to [Bliek & Sam-Haroud 99] PPC becomes equivalent to PC, thus it guarantees minimality and decomposability
Consistency:
Properties & Algorithms

75. Properties & Algorithms
PC is not enough
{a, b, c} V2 V3 V1 V4 V7 V5 V6
All mutex constraints
Arc-consistent?
Path-consistent?
Satisfiable?
 we should seek (even) higher levels of consistency
 k-consistency, k = 1, 2, 3, ….                 
Consistency:
Properties & Algorithms

76. Properties & Algorithms
k-consistency
A CSP is k-consistent iff
given any consistent instantiation of any (k-1) distinct variables
there exists an instantiation of any kth variable such that the k values taken together satisfy all the constraints among the k variables.
Example: (courtesy of Dechter)
3-consistent?
4-consistent? Q Q Q Q Q
Consistency:
Properties & Algorithms

77. Warning: Does 3-consistency guarantee 2-consistency?B
{red, blue}
{red, blue}   A C
{ red }
{ red }
Question:
Is this CSP 3-consistent?
is it 2-consistent?
Lesson:
3-consistency does not guarantee 2-consistency
Consistency:
Properties & Algorithms

78. (Strong) k-consistency: definition
Definitions:
k-consistency:
any consistent assignment for (k-1)-variable can be extended to any kth variable
strong k-consistency:
has to be j-consistent for all j  k
Globally consistent:
i-consistent for all 1in
Consistency:
Properties & Algorithms

79. (i,j)-consistency: definition
every consistent instantiation to i variables
can be extended to every j variables
Consequently:
2-consistency: i=1, j=1
3-consistency: i=2, j=1
k-consistency: i=k-1, j=1
Minimality: i=2, j=n-2
Decomposability: i=k, j=n-k
Consistency:
Properties & Algorithms

80. Consistency properties & algorithms: summary
New terms: AC, PC, minimal CSP, decomposable CSP, complete graph, triangulated graph, Revise, AC-1, AC-3, PC-1, PC-2, PPC
Complexity of AC, PC algorithms
AC tightens domains, PC tightens binary constraints
Consistency operations in terms of composition, intersection in relational algebra
Path consistency approximates minimality, which approximates decomposability
Situations when path consistency guarantees minimality and decomposability
Strong k-consistency and (i,j)-consistency
Consistency:
Properties & Algorithms

81. Properties & Algorithms
Outline
Motivation and material
Node and arc consistency and their complexity
Criteria for performance comparison and CSP parameters.
Which AC algorithm to choose? Project results may tell!
Path consistency and its complexity
Consistency properties of a CSP
Other related results:
Constraint synthesis,
Box consistency
Singleton Arc Consistency
Neighborhood Inverse Consisten
Look-ahead,
Non-binary CSPs, etc.
Consistency:
Properties & Algorithms

82. Properties & Algorithms
Constraint synthesis
k-consistency: an alternative view
Constraint of arity (k-1) are used to synthesize a constraint of arity k, which is then used to filter constraints of arity (k-1)
Examples:
2-consistency: DVi  DVi  Vi (CVi,Vj DVj)
Projection of binary constraint is used to tighten unary constraint
3-consistency: CVi,Vj  CVi,Vj  Vi,Vj(CVi,Vk CVk,Vj)
Binary constraints are used to synthesize a ternary constraint
Projection of the ternary constraint is used to tighten binary constraint
k-consistency: [Freuder 78, 82]
Constraints of arity (k-1) are used to synthesize constraints of arity k
Projection of k-ary constraint is used to tighten (k-1)-ary constraints
Consistency:
Properties & Algorithms

83. Solving CSPs by Constraint synthesis [Freuder 78]
From k=2 to k=n,
achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints,
then use the k-ary constraints to filter constraints of arity k-1, k-2, etc.
Process ends with a unique n-ary constraint whose tuples are all the solutions to the CSP
Consistency:
Properties & Algorithms

84. Box Consistency (on interval constraints)
Domains are (continuous) intervals
Historically also called: continuous CSPs, continuous domains
Domains are infinite:
We cannot enumerate consistent values/tuples
[Davis, AIJ 87] (see recommended reading) showed that even AC may be incomplete or not terminate
We apply consistency (usually, arc-consistency) on the boundaries of the interval
Consistency:
Properties & Algorithms

85. Singleton Arc Consistency (SAC)
Repeat until no change occurs
Repeat for each variable
Repeat for each value in domain
Assign this value to this variable.
If the CSP is AC, keep the value. Otherwise, remove it.
Proposed by Debruyne & Bessière, IJCAI 97
Consistency:
Properties & Algorithms

86. Neighborhood Inverse Consistency (NIC)
Repeat until no change occurs
Repeat for each variable
Consider only the neighborhood of the variable
Repeat for each value for the variable
If the value appears in any complete solution for the neighborhood, keep it. Otherwise, remove it.
Proposed by Freuder & Elfe, AAAI 96
Consistency:
Properties & Algorithms

87. Look-ahead (already discussed)
Integration of consistency algorithm with search
AC:
Forward Checking (FC) [Haralick & Elliott]
Directional Arc-Consistency (DAC) [Dechter]
Maintaining arc consistency (MAC) [Sabin & Freuder]
PC:
Directed path consistency (DPC) [Dechter]
Consistency:
Properties & Algorithms

88. Properties & Algorithms
Non-binary CSPs
(Almost) all algorithms and properties discussed so far were restricted to binary CSPs
Consistency properties for non-binary CSPs are the topic of current research. Mainly, properties and algorithms for:
GAC: generalized arc-consistency [Mohr & Masini]
Relational m-consistency [Dechter, Chap 8]
Consistency:
Properties & Algorithms

89. Generalized Arc-Consistency
Conceptually: project the constraint on each of the variables in its scope to tighten the domain of the variable.
When constraint is not defined in extension, GAC may be problematic (e.g., NP-hard in TCSP)
Consistency:
Properties & Algorithms

90. Relational consistency
Dechter (see Section 8.1.1) generalizes consistency properties to non-binary constraints
Relational m-consistency
Relational 1-consistency  relational arc-consistency
Relational 2-consistency  relational path-consistency
Relational (i,m)consistency
Relational (1,1)consistency  GAC
Consistency:
Properties & Algorithms

91. Look-ahead in non-binary CSPs
In binary CSPs, when a variable is instantiated, Revise (Vf, Vc) updates the domain of a future variable Vf that is a neighbor to the current variable Vc
In non-binary CSPs, a constraint that applies to a current variable may involve one or more past variables and one or more future variables. The question is: when to trigger Revise?
There are 5 strategies: nFC0, nFC1, …, nFC5 in increasing ‘aggressivity’ [Bessiere et al. AIJ 02 ]
For example, nFC0 triggers Revise only when all but one variable has been instantiated
Consistency:
Properties & Algorithms

92. Related results: summary
Interval constraints: box consistency
Solving CSPs by Constraint synthesis: [Freuder]
From k=2 to k=n-1, achieve k-consistency by using (k-1)-arity constraints to synthesize k-arity constraints, then filter constraints of arity k-1, k-2, etc.
Integration with search:
AC:
Forward Checking (FC) [Haralick & Elliott]
Directional Arc-Consistency (DAC) [Dechter]
Maintaining arc consistency (MAC) [Sabin & Freuder]
PC: Directed path consistency (DPC) [Dechter]
Non-binary constraints:
Generalized arc consistency (GAC) [Mohr & Masini]
Relational consistency [Dechter]
Non-binary constraints and integration with search:
nFC0, nFC1, …, nFC5
etc.
Consistency:
Properties & Algorithms