1. CS B551: Elements of Artificial Intelligence
Instructor: Kris Hauser

2. Constraint Propagation …
… is the process of determining how the constraints and the possible values of one variable affect the possible values of other variables
It is an important form of “least-commitment” reasoning 2 2

3. Forward Checking Whenever a pair (Xv) is added to assignment A do:
For each variable Y not in A do:
For every constraint C relating Y to the variables in A do:
Remove all values from Y’s domain that do not satisfy C
n = number of variables
d = size of initial domains
s = maximum number of constraints involving a given variable (s  n-1)
Forward checking takes O(nsd) time 3

4. Forward Checking in Map Coloring
Empty set: the current assignment {(WA  R), (Q  G), (V  B)}
does not lead to a solution WA NT Q
NSW V SA T
RGB R GB G B RB 4 4

5. Forward Checking in Map Coloring
Contradiction that forward
checking did not detect T WA NT SA Q
NSW V WA NT Q
NSW V SA T
RGB R GB G B RB 5 5

6. Forward Checking in Map Coloring
Contradiction that forward
checking did not detect T WA NT SA Q
NSW V
Detecting this contradiction requires a more powerful constraint propagation technique WA NT Q
NSW V SA T
RGB R GB G B RB 6 6

7. Constraint Propagation for Binary Constraints
REMOVE-VALUES(X,Y)
removed  false
For every value v in the domain of Y do
If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
Remove v from Y‘s domain
removed  true
Return removed 7

8. Constraint Propagation for Binary Constraints
AC3
Initialize queue Q with all variables (not yet instantiated)
While Q   do
X  Remove(Q)
For every (not yet instantiated) variable Y related to X by a (binary) constraint do
If REMOVE-VALUES(X,Y) then
If Y’s domain =  then exit
Insert(Y,Q) 8

9. Edge Labeling
We consider an image of a scene composed of polyhedral objects such that each vertex is the endpoint of exactly three edges 9 9

10. Edge Labeling
An “edge extractor” has accurately extracted all the visible edges in the image. The problem is to label each edge as convex (+), concave (-), or occluding () such that the complete labeling is physically possible 10 10

11. Concave
edges
Occluding
edges
Convex
edges 11 11

12. + + + - - + + + The arrow is oriented such that the object
is on the right of
the occluding
edge + + + - - + + + 12 12

13. One Possible Edge Labeling+ - 13 13

14. Junction Types L
Fork T Y 14 14

15. - - - - Junction Label Sets + + + + (Waltz, 1975; Mackworth, 1977) 15

16. Edge Labeling as a CSP A variable is associated with each junction
The domain of a variable is the label set associated with the junction type
Constraints: The values assigned to two adjacent junctions must give the same label to the joining edge 16 16

17. AC3 Applied to Edge Labeling
Q = (X1, X2, X3, ...) X5 X3 X1 X2
X12 X4 X8 17 17

18. AC3 Applied to Edge Labeling
Q = (X1, ...) + - X5 X1 + - 18 18

19. Q = (X1, ...) + - X5 X1 + - 19 19

20. Q = (X5, ...) + - X5 + - 20 20

21. Q = (X5, ...) + + X5 + - X3 21 21

22. Q = (X5, ...) + + X5 + - X3 22 22

23. Q = (X3, ...) + + + - X3 23 23

24. Q = (X3, ...) + + X3 + - + X8 + 24 24

25. Q = (X3, ...) + + X3 + - + X8 + 25 25

26. Q = (X8, ...) + + + - + X8 + 26 26

27. Q = (X8, ...) + - + + +
X12 - + X8 27 27

28. Complexity Analysis of AC3
n = number of variables
d = size of initial domains
s = maximum number of constraints involving a given variable (s  n-1)
Each variables is inserted in Q up to d times
REMOVE-VALUES takes O(d2) time
AC3 takes O(ndsd2) = O(nsd3) time
Usually more expensive than forward checking
AC3
Initialize queue Q with all variables (not yet instantiated)
While Q   do
X  Remove(Q)
For every (not yet instantiated) variable Y related to X by a (binary) constraint do
If REMOVE-VALUES(X,Y) then
If Y’s domain =  then exit
Insert(Y,Q)
REMOVE-VALUES(X,Y)
removed  false
For every value v in the domain of Y do
If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
Remove v from Y‘s domain
removed  true
Return removed 28

29. Is AC3 all that we need?
No !!
AC3 can’t detect all contradictions among binary constraints X Z Y
XY
XZ
YZ
{1, 2} 29 29

30. Is AC3 all that we need?
No !!
AC3 can’t detect all contradictions among binary constraints X Z Y
XY
XZ
YZ
{1, 2}
REMOVE-VALUES(X,Y)
removed  false
For every value v in the domain of Y do
If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
Remove v from Y‘s domain
removed  true
Return removed 30 30

31. Is AC3 all that we need?
No !!
AC3 can’t detect all contradictions among binary constraints X Z Y
XY
XZ
YZ
{1, 2}
REMOVE-VALUES(X,Y,Z)
removed  false
For every value w in the domain of Z do
If there is no pair (u,v) of values in the domains of X and Y verifying the constraint on (X,Y) such that the constraints on (X,Z) and (Y,Z) are satisfied then
Remove w from Z‘s domain
removed  true
Return removed
REMOVE-VALUES(X,Y)
removed  false
For every value v in the domain of Y do
If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
Remove v from Y‘s domain
removed  true
Return removed 31 31

32. Is AC3 all that we need?
No !!
AC3 can’t detect all contradictions among binary constraints
Not all constraints are binary X Z Y
XY
XZ
YZ
{1, 2} 32 32

33. Tradeoff
Generalizing the constraint propagation algorithm increases its time complexity
Tradeoff between time spent in backtracking search and time spent in constraint propagation
A good tradeoff when all or most constraints are binary is often to combine backtracking with forward checking and/or AC3 (with REMOVE- VALUES for two variables) 33 33

34. Modified Backtracking Algorithm with AC3
CSP-BACKTRACKING(A, var-domains)
If assignment A is complete then return A
Run AC3 and update var-domains accordingly
If a variable has an empty domain then return failure
X  select a variable not in A
D  select an ordering on the domain of X
For each value v in D do
Add (Xv) to A
var-domains  forward checking(var-domains, X, v, A)
If no variable has an empty domain then (i) result  CSP-BACKTRACKING(A, var-domains) (ii) If result  failure then return result
Remove (Xv) from A
Return failure 34

35. A Complete Example: 4-Queens Problem1 3 2 4 X1
{1,2,3,4} X3 X4 X2
1) The modified backtracking algorithm starts by calling AC3, which removes no value 35 35

36. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2
2) The backtracking algorithm then selects a variable and a value for this variable. No heuristic helps in this selection. X1 and the value 1 are arbitrarily selected 36 36

37. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2
3) The algorithm performs forward checking, which eliminates 2 values in each other variable’s domain 37

38. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2 4) The algorithm calls AC3 1 238 38

39. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2
REMOVE-VALUES(X,Y)
removed  false
For every value v in the domain of Y do
If there is no value u in the domain of X such that the constraint on (X,Y) is satisfied then
Remove v from Y‘s domain
removed  true
Return removed
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
X2 = 3 is
incompatible
with any of the remaining values of X3
4) The algorithm calls AC3, which eliminates 3 from the domain of X2 39 39

40. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2
4) The algorithm calls AC3, which eliminates 3 from the domain of X2, and 2 from the domain of X3 40 40

41. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2
The algorithm calls AC3, which eliminates 3 from the domain of X2, and 2 from the domain of X3, and 4 from the domain of X3 41 41

42. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2
The domain of X3 is empty  backtracking 42 42

43. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2
The algorithm removes 1 from X1’s domain and assign 2 to X1 43 43

44. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2
The algorithm performs forward checking 44 44

45. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2 The algorithm calls AC3 1 2 3 445 45

46. 4-Queens Problem X1 {1,2,3,4} X3 X4 X2
The algorithm calls AC3, which reduces the domains of X3 and X4 to a single value 46 46

47. Exploiting the Structure of CSP
If the constraint graph contains several components, then solve one independent CSP per component T WA NT SA Q
NSW V 47 47

48. Exploiting the Structure of CSP
If the constraint graph is a tree, then :
Order the variables from the root to the leaves  (X1, X2, …, Xn)
For j = n, n-1, …, 2 call REMOVE-VALUES(Xj, Xi) where Xi is the parent of Xj
Assign any valid value to X1
For j = 2, …, n do Assign any value to Xj consistent with the value assigned to its parent Xi X Y Z U V W
 (X, Y, Z, U, V, W) 48 48

49. Exploiting the Structure of CSP
Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph WA NT SA Q
NSW V 49 49

50. Exploiting the Structure of CSP
Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph
If the graph becomes a tree, then proceed
as shown in previous
slide WA NT Q
NSW V 50 50

51. 8
Representation
Variables Xij for i, j in {1,..,9}
Domains {1,…,9}
Constraints:
XijXik, for jk
Xij  Xkj, for ik
XijXmn, for (i,j), (m,n) in same cell 2 2 7 5 8 9 9 5 7 4 3 6 2 9

52. 8
Representation
Variables Xij for i, j in {1,..,9}
Domains {1,…,9}
Constraints:
XijXik, for jk
Xij  Xkj, for ik
XijXmn, for (i,j), (m,n) in same cell 2 2 7 5 8 9 9 5 7 4 3 6 2 9

53. Can we detect this using constraint propagation?8
Representation
Variables Xij for i, j in {1,..,9}
Domains {1,…,9}
Constraints:
XijXik, for jk
Xij  Xkj, for ik
XijXmn, for (i,j), (m,n) in same cell 2 2 7 5 8 9 9 5 7 4
Can we detect this using constraint propagation? 3 6 2 9

54. Must detect 9-way interactions8
Representation
Variables Xij for i, j in {1,..,9}
Domains {1,…,9}
Constraints:
XijXik, for jk
Xij  Xkj, for ik
XijXmn, for (i,j), (m,n) in same cell 2 2 7 5 8 9 9 5 7 4
Must detect 9-way interactions 3 6 2 9

55. 8 Representation 2 2 2 7 5 8 9 9 5 7 4 3 6 2 9 C39=5 C15=4 C33=8 …
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
???
R18=3 2
R22=8
R32=6 2 7
R37=9 … 5 8 9 9 5 7 4 3 6 2 9

56. X18 = (1,3) 8 Representation 2 2 2 7 5 8 9 9 5 7 4 3 6 2 9 C39=5 C15=4…
C12=9
C28=4
C38=1
X18 = (1,3) 8
Representation 2
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
???
R18=3 2
R22=8
R32=6 2 7
R37=9 … 5 8 9 9 5 7 4 3 6 2 9

57. X18 = (1,3) 8 X22 = (3,3) Representation 2 2 2 7 5 8 9 9 5 7 4 3 6 2 9
C39=5
C15=4
C33=8 …
C12=9
C28=4
C38=1
X18 = (1,3) 8
X22 = (3,3)
Representation 2
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
???
R18=3 2
R22=8
R32=6 2 7
R37=9 … 5 8 9 9 5 7 4 3 6 2 9

58. X18 = (1,3) 8 X22 = (3,3) X32 = (2,3) X37 = (3,3) Representation 2 2 2
C39=5
C15=4
C33=8 …
C12=9
C28=4
C38=1
X18 = (1,3) 8
X22 = (3,3)
X32 = (2,3)
X37 = (3,3)
Representation 2
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
???
R18=3 2
R22=8
R32=6 2 7
R37=9 … 5 8 9 9 5 7 4 3 6 2 9

59. X18 = (1,3) 8 X22 = (3,3) X32 = (2,3) X37 = (3,3) Representation 2 2 2
C39=5
C15=4
C33=8 …
C12=9
C28=4
C38=1
X18 = (1,3) 8
X22 = (3,3)
X32 = (2,3)
X37 = (3,3)
Representation 2
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
Rij=k  Ckj=I
Similar constraints between X’s and R’s, X’s and C’s
Rij  Rik for j  k
Cij  Cik for j  k
Xij  Xik for j  k
R18=3 2
R22=8
R32=6 2 7
R37=9 … 5 8 9 9 5 7 4 3 6 2 9

60. 8 Representation 2 2 2 7 5 8 9 9 5 7 4 3 6 2 9 C12 = 9 R12= {1-9}
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
Rij=k  Ckj=I
Similar constraints between X’s and R’s, X’s and C’s
Rij  Rik for j  k
Cij  Cik for j  k
Xij  Xik for j  k 2 2 7 5 8 9 9 5 7 4 3 6 2 9

61. 8 Representation 2 2 2 7 5 8 9 9 5 7 4 3 6 2 9 C12 = 9 C22 = {1-9}
{2-9} 8
Representation 2
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
Rij=k  Ckj=i
Similar constraints between X’s and R’s, X’s and C’s
Rij  Rik for j  k
Cij  Cik for j  k
Xij  Xik for j  k 2 2 7 5 8 9 9 5 7 4 3 6 2 9

62. 8 Representation 2 2 2 7 5 8 9 9 5 7 4 3 6 2 9 X32 = (2,2) C12 = 9
{2-9}
Representation 2
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
Rij=k  Ckj=I
Similar constraints between X’s and R’s, X’s and C’s
Rij  Rik for j  k
Cij  Cik for j  k
Xij  Xik for j  k 2 2 7 5 8 9 9 5 7 4 3 6 2 9

63. 8 Representation 2 2 2 7 5 8 9 9 5 7 4 3 6 2 9 X32 = (2,2) C12 = 9
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
Rij=k  Ckj=I
Similar constraints between X’s and R’s, X’s and C’s
Rij  Rik for j  k
Cij  Cik for j  k
Xij  Xik for j  k 2 2 7 5 8 9 9 5 7 4 3 6 2 9

64. 2 8 Representation 2 2 2 7 5 8 9 9 5 7 4 3 6 2 9 X32 = (2,2) C12 = 9
Variables Rij in {1,…,9} Cij in {1,…,9} Xij in {(1,1),…,(3,3)}
Constraints:
Rij=k  Ckj=I
Similar constraints between X’s and R’s, X’s and C’s
Rij  Rik for j  k
Cij  Cik for j  k
Xij  Xik for j  k 2 2 7 5 8 9 9 5 7 4 3 6 2 9

65. Local Search for CSPs Init: Make an arbitrary assignment
Repeat: Modify some variable to reduce # of violated constraints

66. Boolean Satisfiability Problems
Highly successful local search algorithms
WalkSAT
See R&N 7.3
p constraints of form ui*  uj*  uk*= 1
where ui* is either ui or ui
n variables ui, …, un

67. Observations…
If a CSP has few constraints, local search solves it quickly
Random starting assignment not too far from a solution
Million-queens puzzles solved in < 1min, c. 1990
If a CSP has many constraints, local search solves it quickly
Constraints “guide” solver to a solution (if one exists)

68. Hard Sudoku’s
Human solvers: Lot of logic (deep constraint propagation)
1 . . | | . . 2
. 9 . | | . 5 .
. . 6 | | 7 . .
. 5 . | | . . .
. . . | | . . .
. . . | | . 4 .
7 . . | | 6 . .
. 3 . | | . 8 .
. . 2 | | . . 1
Computer solvers: Lot of backtracking

69. Hard & Easy 3-SAT Problems
Let R = # of constraints / # of variables
As n , the fraction of hard problems reduces to 0

70. Recap Constraint propagation, AC3 Taking advantage of CSP structure
Local search for CSPs

71. Next Class
Intro to uncertainty
R&N 4.3-4,