1. Constraint Propagation

2. Constraint Propagation …
… is the process of determining how the possible values of one variable affect the possible values of other variables
Constraint Propagation

3. Constraint Propagation
Forward Checking
After a variable X is assigned a value v, look at each unassigned variable Y that is connected to X by a constraint and deletes from Y’s domain any value that is inconsistent with v
Constraint Propagation

4. Constraint Propagation
Map Coloring T WA NT SA Q
NSW V WA NT Q
NSW V SA T
RGB
Constraint Propagation

5. Constraint Propagation
Map Coloring T WA NT SA Q
NSW V WA NT Q
NSW V SA T
RGB R GB
Constraint Propagation

6. Constraint Propagation
Map Coloring T WA NT SA Q
NSW V WA NT Q
NSW V SA T
RGB R GB B G RB
Constraint Propagation

7. Constraint Propagation
Map Coloring T WA NT SA Q
NSW V
Impossible assignments that forward
checking do not detect WA NT Q
NSW V SA T
RGB R GB B G RB
Constraint Propagation

8. Example: Street Puzzle1 2 3 4 5
Ni = {English, Spaniard, Japanese, Italian, Norwegian}
Ci = {Red, Green, White, Yellow, Blue}
Di = {Tea, Coffee, Milk, Fruit-juice, Water}
Ji = {Painter, Sculptor, Diplomat, Violonist, Doctor}
Ai = {Dog, Snails, Fox, Horse, Zebra}

9. Example: Street Puzzle1 2 3 4 5
Ni = {English, Spaniard, Japanese, Italian, Norwegian}
Ci = {Red, Green, White, Yellow, Blue}
Di = {Tea, Coffee, Milk, Fruit-juice, Water}
Ji = {Painter, Sculptor, Diplomat, Violonist, Doctor}
Ai = {Dog, Snails, Fox, Horse, Zebra}
The Englishman lives in the Red house
The Spaniard has a Dog
The Japanese is a Painter
The Italian drinks Tea
The Norwegian lives in the first house on the left
The owner of the Green house drinks Coffee
The Green house is on the right of the White house
The Sculptor breeds Snails
The Diplomat lives in the Yellow house
The owner of the middle house drinks Milk
The Norwegian lives next door to the Blue house
The Violonist drinks Fruit juice
The Fox is in the house next to the Doctor’s
The Horse is next to the Diplomat’s
Who owns the Zebra?
Who drinks Water?

10. Example: Task Scheduling
T1 must be done during T3
T2 must be achieved before T1 starts
T2 must overlap with T3
T4 must start after T1 is complete
Are the constraints compatible?
Find the temporal relation between every two tasks

11. Constraint Propagation

12. Edge Labeling in Computer Vision
Russell and Norvig: Chapter 24, pages
Constraint Propagation

13. Constraint Propagation
Edge Labeling
Constraint Propagation

14. Constraint Propagation
Edge Labeling
Constraint Propagation

15. Constraint Propagation
Edge Labeling + –
Constraint Propagation

16. Constraint Propagation
Edge Labeling + -
Constraint Propagation

17. Constraint Propagation
Junction Label Sets + - - + + - - +
(Waltz, 1975; Mackworth, 1977)
Constraint Propagation

18. Constraint Propagation
Edge Labeling as a CSP
A variable is associated with each junction
The domain of a variable is the label set of the corresponding junction
Each constraint imposes that the values given to two adjacent junctions give the same label to the joining edge
Constraint Propagation

19. Constraint Propagation
Edge Labeling + - + -
Constraint Propagation

20. Constraint Propagation
Edge Labeling + + + -
Constraint Propagation

21. Constraint Propagation
Edge Labeling + + + - + +
Constraint Propagation

22. Constraint Propagation
Edge Labeling + - + + + - +
Constraint Propagation

23. Removal of Arc Inconsistencies
REMOVE-ARC-INCONSISTENCIES(J,K)
removed  false
X  label set of J
Y  label set of K
For every label y in Y do
If there exists no label x in X such that the constraint (x,y) is satisfied then
Remove y from Y
If Y is empty then contradiction  true
removed  true
Label set of K  Y
Return removed
Constraint Propagation

24. CP Algorithm for Edge Labeling
Associate with every junction its label set
contradiction  false
Q  stack of all junctions
while Q is not empty and not contradiction do
J  UNSTACK(Q)
For every junction K adjacent to J do
If REMOVE-ARC-INCONSISTENCIES(J,K) then
STACK(K,Q)
(Waltz, 1975; Mackworth, 1977)
Constraint Propagation

25. General CP for Binary Constraints
Algorithm AC3
contradiction  false
Q  stack of all variables
while Q is not empty and not contradiction do
X  UNSTACK(Q)
For every variable Y adjacent to X do
If REMOVE-ARC-INCONSISTENCIES(X,Y) then
STACK(Y,Q)
Constraint Propagation

26. General CP for Binary Constraints
Algorithm AC3
contradiction  false
Q  stack of all variables
while Q is not empty and not contradiction do
X  UNSTACK(Q)
For every variable Y adjacent to X do
If REMOVE-ARC-INCONSISTENCY(X,Y) then
STACK(Y,Q)
REMOVE-ARC-INCONSISTENCY(X,Y)
removed  false
For every value y in the domain of Y do
If there exists no value x in the domain of X such that the constraints on (x,y) is satisfied then
Remove y from the domain of Y
If Y is empty then contradiction  true
removed  true
Return removed
Constraint Propagation

27. Complexity Analysis of AC3
n = number of variables
d = number of values per variable
s = maximum number of constraints on a pair of variables
Each variables is inserted in Q up to d times
REMOVE-ARC-INCONSISTENCY takes O(d2) time
CP takes O(n s d3) time
Constraint Propagation

28. Constraint Propagation
Is AC3 All What is Needed?
NO! X Y Z
X  Y
X  Z
Y  Z
{1, 2}
Constraint Propagation

29. Constraint Propagation
Solving a CSP
Interweave constraint propagation, e.g.,
forward checking
AC3
and backtracking
+ Take advantage of the CSP structure
Constraint Propagation

30. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

31. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

32. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

33. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

34. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

35. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

36. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

37. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

38. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

39. Constraint Propagation
4-Queens Problem X1
{1,2,3,4} X3 X4 X2 1 3 2 4
Constraint Propagation

40. Constraint Propagation
Structure of CSP
If the constraint graph contains multiple components, then one independent CSP per component T WA NT SA Q
NSW V
Constraint Propagation

41. Constraint Propagation
Structure of CSP
If the constraint graph contains multiple components, then one independent CSP per component
If the constraint graph is a tree (no loop), then the CSP can be solved efficiently
Constraint Propagation

42. Constraint Propagation
Constraint Tree X Y Z U V W
 (X, Y, Z, U, V, W)
Constraint Propagation

43. Constraint Propagation
Constraint Tree
Order the variables from the root to the leaves  (X1, X2, …, Xn)
For j = n, n-1, …, 2 do REMOVE-ARC-INCONSISTENCY(Xj, Xi) where Xi is the parent of Xj
Assign any legal value to X1
For j = 2, …, n do
assign any value to Xj consistent with the value assigned to Xi, where Xi is the parent of Xj
Constraint Propagation

44. Constraint Propagation
Structure of CSP
If the constraint graph contains multiple components, then one independent CSP per component
If the constraint graph is a tree, then the CSP can be solved efficiently
Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph WA NT SA Q
NSW V
Constraint Propagation

45. Constraint Propagation
Structure of CSP
If the constraint graph contains multiple components, then one independent CSP per component
If the constraint graph is a tree, then the CSP can be solved in linear time
Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph WA NT Q
NSW V
Constraint Propagation

46. Backtracking Algorithm
partial assignment
of variables
CSP-BACKTRACKING({})
CSP-BACKTRACKING(a)
If a is complete then return a
X  select unassigned variable
D  select an ordering for the domain of X
For each value v in D do
If v is consistent with a then
Add (X= v) to a
result  CSP-BACKTRACKING(a)
If result  failure then return result
Return failure

47. Map Coloring {} WA=red WA=green WA=blue NT=green NT=blue Q=red Q=blueSA Q
NSW V T

48. Questions Which variable X should be assigned a value next?
In which order should its domain D be sorted?

49. Questions Which variable X should be assigned a value next?
In which order should its domain D be sorted?
What are the implications of a partial assignment for yet unassigned variables? ( Constraint Propagation -- see next class)

50. Choice of Variable
Map coloring WA NT WA NT SA Q
NSW V T SA

51. Choice of Variable Most-constrained-variable heuristic:
Select a variable with the fewest remaining values

52. Choice of Variable Most-constraining-variable heuristic:WA NT SA Q
NSW V T SA
Most-constraining-variable heuristic:
Select the variable that is involved in the largest number of constraints on other unassigned variables

53. Choice of Value WA NT WA NT SA Q
NSW V T {}

54. Choice of Value Least-constraining-value heuristic:WA NT WA NT SA Q
NSW V T
{blue}
Least-constraining-value heuristic:
Prefer the value that leaves the largest subset of legal values for other unassigned variables

55. Local Search for CSP Pick initial complete assignment (at random)1 2 3 2
Pick initial complete assignment (at random)
Repeat
Pick a conflicted variable var (at random)
Set the new value of var to minimize the number of conflicts
If the new assignment is not conflicting then return it
(min-conflicts heuristics)

56. Remark
Local search with min-conflict heuristic works extremely well for million-queen problems
The reason: Solutions are densely distributed in the O(nn) space, which means that on the average a solution is a few steps away from a randomly picked assignment

57. Applications CSP techniques allow solving very complex problems
Numerous applications, e.g.:
Crew assignments to flights
Management of transportation fleet
Flight/rail schedules
Task scheduling in port operations
Design
Brain surgery

58. Stereotaxic Brain Surgery

59. Stereotaxic Brain Surgery
• < Tumor < 2200
2000 < B2 + B4 < 2200
2000 < B4 < 2200
2000 < B3 + B4 < 2200
2000 < B3 < 2200
2000 < B1 + B3 + B4 < 2200
2000 < B1 + B4 < 2200
2000 < B1 + B2 + B4 < 2200
2000 < B1 < 2200
2000 < B1 + B2 < 2200
• 0 < Critical < 500
0 < B2 < 500 T C B1 B2 B3 B4

60.  Constraint Programming
“Constraint programming represents one of the closest approaches computer science has yet made to the Holy Grail of programming: the user states the problem, the computer solves it.”
Eugene C. Freuder, Constraints, April 1997

61. Over-Constrained Problems
Weaken an over-constrained problem by:
Enlarging the domain of a variable
Loosening a constraint
Removing a variable
Removing a constraint
Constraint Propagation

62. Non-Binary Constraints
So far, all constraints have been binary (two variables) or unary (one variable)
Constraints with more than 2 variables would be difficult to propagate
Theoretically, one can reduce a constraint with k>2 variables to a set of binary constraints by introducing additional variables
Constraint Propagation

63. When to Use CSP Techniques?
When the problem can be expressed by a set of variables with constraints on their values
When constraints are relatively simple (e.g., binary)
When constraints propagate well (AC3 eliminates many values)
When the solutions are “densely” distributed in the space of possible assignments
Constraint Propagation

64. Constraint Propagation
Summary
Forward checking
Constraint propagation
Edge labeling in Computer Vision
Interweaving CP and backtracking
Exploiting CSP structure
Weakening over-constrained CSP
Constraint Propagation

65. Game Playing

66. Games as search problems
Chess, Go
Simulation of war (war game)
스타크래프트의 전투
Claude Shannon, Alan Turing  Chess program (1950년대)
Constraint Propagation

67. Constraint Propagation
Contingency problems
The opponent introduces uncertainty
마이티에서는 co-work이 필요
고스톱에서는 co-work방지가 필요
Hard to solve  in chess, possible nodes, 1040 different legal positions
Time limits  how to make the best use of time to reach good decisions
Pruning, heuristic evaluation function
Constraint Propagation

68. Perfect decisions in two person games
The initial state, A set of operators, A terminal test, A utility function (payoff function)
Mini-max algorithm,
Negmax algorithms
Constraint Propagation

69. Mini-max algorithm (AND-OR tree)
Constraint Propagation

70. Constraint Propagation
상대방의 관점
Constraint Propagation

71. Constraint Propagation
Negmax
Knuth and Moore (1975)
F(n) = f(n), if n has no successors
F(n) = max{-F(n1), …, -F(nk)},
if n has successors n1, …, nk
Constraint Propagation

72. Constraint Propagation
The Negmax formalism
Constraint Propagation

73. Constraint Propagation
Imperfect Decisions
utility function  evaluation
terminal test  cutoff test
Evaluation function ::: an estimate of the utility of the game from a given position
Chess  material value (장기도 유사)
Weighted linear function 
w1f1+w2f2+….+wnfn
Constraint Propagation

74. Constraint Propagation
Cutting off search
To set a fixed depth limit, so that the cutoff test succeeds for all nodes at or below depth d  iterative deepening until time runs out  위험이 있을 수 있다
Quiescent posiiton ::: unlikely to exhibit wild swings in value in near future
Quiescent search :: Non-quiescent search  extra search to find quiescent position
Horizon problem
Constraint Propagation

75. Constraint Propagation
Alpha-beta pruning
Eliminate unnecessary evaluations
Pruning
Constraint Propagation

76. Constraint Propagation
Alpha-beta pruning
Alpha cutoff Beta cutoff
Constraint Propagation

77. Negmax representation
Constraint Propagation

78. Constraint Propagation
Example
Constraint Propagation

79. Constraint Propagation
Games with Chance
Chance nodes  expected value
Backgammon, 윷놀이
Expectimax value
Constraint Propagation

80. Constraint Propagation
A backgammon position
Constraint Propagation

81. Constraint Propagation
Comparision
MAX A A A A 2 1 2 1
1.3 21
40.9
DICE
2.1 .9 .1 .9 .1 .9 .1 .9 .1
MIN 20 30 1
400 2 3 1 4 20 20 30 30 1 1
400
400 2 2 3 3 1 1 4 4
Constraint Propagation

82. Constraint Propagation
Monte Carlo Search
Constraint Propagation

83. Monte Carlo Search (Algo.)
Four steps are applied per search iteration: 1) Selection: Starting at the root node, a child selection policy is recursively applied to descend through the tree until the most urgent expandable node is reached. A node is expandable if it represents a nonterminal state and has unvisited (i.e. unexpanded) children.

84. Monte Carlo Search(Algo.)
2) Expansion: One (or more) child nodes are added to expand the tree, according to the available actions. 3) Simulation: A simulation is run from the new node(s) according to the default policy to produce an outcome. 4) Backpropagation: The simulation result is “backed up” (i.e. backpropagated) through the selected nodes to update their statistics.

85. Monte Carlo Search (policy)
1) Tree Policy: Select or create a leaf node from the nodes already contained within the search tree (selection and expansion). 2) Play out the domain from a given non-terminal state to produce a value estimate (simulation).