1. Lecture 7 Constraint Satisfaction Problems
CSE 573
Artificial Intelligence I
Henry Kautz
Fall 2001

2. But first… Why is driving to Spokane like playing Backgammon?
States = cities
Start state = starting city
Goal state test = is state the destination city?
Operators = move to an adjacent city; cost = distance
Output: a shortest path from start state to goal state
SUPPOSE WE REVERSE START AND GOAL STATES?

3. Backtracking Constraint Satisfaction Algorithms
Roadmap
Tree Search (guessing)
BFS
DFS
Iterative Deepening
Bidirectional
Best-first search A*
Game tree
Davis-Putnam (logic)
Cutset conditioning (probability)
Inference (simplifying)
Forward Checking
Path Consistency (Waltz labeling, temporal algebra)
Resolution
“Bucket Algorithm”
Iterative improvement (wandering)
Hillclimbing (gradient descent)
Walksat & Simulated annealing
Genetic algorithms
Monte-Carlo Methods
Backtracking Constraint Satisfaction Algorithms
Today we are going to start talking about a inference method

4. Map Coloring
We want to color map with 3 (RED, BLUE, GREEN) colors using DFS.
Suppose we color Spain RED, then try France, RED, and so on for every country.
Eventually we reach a leaf, apply the test for a goal state, and start backing up.
What’s wrong?
But how to make the COMPUTER notice that?
Must make constraint that no adjacent states same color DECLARATIVE
so that computer can reason about it.
ALSO: if Spain RED, France BLUE, France MUST BE GREEN!
Want to INFER this fact!

5. CSP V is a set of variables v1, v2, …, vn
D is a set of finite domains D1, D2, …, Dn
C is a set of constraints C1, C2, …, Cm
Each constraint specifies a restriction over joint values of a subset of the varibles
E.g.:
v1 is Spain, v2 is France, v3 is Germany, …
Di = { Red, Blue, Green} for all i
For each adjacent vi, vj there is a constraint Ck
(vi,vj) in { (R,G), (R,B), (G,R), (G,B), (B,R), (B,G) }
Important special case:
Discrete Binary CSP

6. Variations Find a solution that satisfies all constraints
Find all solutions
Find a “tightest form” for each constraint
(v1,v2) in { (R,G), (R,B), (G,R), (G,B), (B,R), (B,G) } 
(v1,v2) in { (R,G), (R,B), (B,G) }
Find a solution that minimizes some additional objective function
Note: in general path length to solution is not important!

7. Exploiting CSP Structure
Interleave inference and search
At each internal node:
Select unassigned variable
Select a value in domain
Backtracking: try another value
Branching factor?
How to best select a variable?
At each node:
Propagate Constraints
Branching factor is number of possible values for the chosen variable.
Thus: to minimize branching factor, choose a variable with the smallest domain!
MOST constrained variable heuristic! We will see other good reasons for this!

8. Running Example: 4 Queens
Variables:
Constraints: Q
VARIABLES: one for each COLUMN, which ROW that queen appears in
Q1 in { 1, 2, 3, 4 }
Q2 in { 1, 2, 3, 4 }
CONSTRAINTS: Non-attacking relationship
SEE NEXT SLIDE

9. Running Example: 4 Queens
Variables:
Q1 in {1,2,3,4}
Q2 in {1,2,3,4}
Q3 in {1,2,3,4}
Constraints: Q1 Q2 1 3 4 2
VARIABLES: one for each COLUMN, which ROW that queen appears in
Q1 in { 1, 2, 3, 4 }
Q2 in { 1, 2, 3, 4 }
CONSTRAINTS: Non-attacking relationship

10. Constraint Checking Q Q x x Q Q x Q x Q x allows us to backtrack early
6 GUESSES

11. Forward Checking x Q x Q Q x Q x
When a variable is set, immediately remove inconsistent values from the domains
of all other variables
A kind of LOOKAHEAD
Need to keep track of CURRENT domains
4 GUESSES

12. Arc Consistency Only one guess! x Q x Q x Q x Q
Q3=3 inconsistent with Q4 in {2,3,4}
Q2=1 and Q2=2 inconsistent with Q3 in {1}
ARC CONSISTENCY – reducing domain for one
variable may reduce domains for others! So, ITERATE forward checking!
Only ONE guess – FIRST square!
Only one guess!

13. Huffman-Clowes Labeling
                               
Huffman-Clowes Labeling + - + + + + +

14. Waltz’s Filtering: Arc-Consistency
Lines: variables
Conjunctions: constraints
Initially Di = {+,-, ,  )
Repeat until no changes:
Choose edge (variable)
Delete labels on edge not consistent with both endpoints

15. No labeling!
                          
                            

16. Path Consistency Path consistency (3-consistency):
Check every triple of variables
More expensive!
k-consistency:
n-consistency: backtrack-free search

17. Variable and Value Selection
Select variable with smallest domain
Which values to try first?
Why different?
Tie breaking?
Recall why: first reason is to minimize branching factor.
Second reason is that if domain is small, are MORE LIKELY to cause more PROPAGATION,
and SET OTHER VARIABLES.
Called the MOST CONSTRAINED VARIABLE heuristic.
Try value first that is MOST LIKELY to actually appear in a solution. BEST CASE: no backtracking!
How to guess which value? One that causes the LEAST TIGHTENING of the domains of OTHER variables.
LEAST CONSTRAINED VALUE heuristic.
Why OPPOSITE?
Because you MUST eventually satisfy EVERY clause, so MIGHT AS WELL start with one that is HARDEST
to satisfy.
BUT you do not need to use every VALUE: so might as well try to AVOID considering UNLIKELY value!
Tie breaking: DETERMINISTIC or RANDOMIZED.
Which better?
DETERMINISTIC more easily becomes STUCK; Should break ties RANDOMLY.
We will see how we can further LEVERAGE randomness.
FOR WEB VERSION: copy this slide, WITH ANSWERS filled in.

18. Variable and Value Selection
Select variable with smallest domain
Minimize branching factor
Most likely to propagate: most constrained variable heuristic
Which values to try first?
Most likely value for solution
Least propagation! Least constrained variable
Why different?
Every constraint must be eventually satisfied
Not every value must be assigned to a variable!
Tie breaking?
In general randomized tie breaking best – less likely to get stuck on same bad pattern of choices
Recall why: first reason is to minimize branching factor.
Second reason is that if domain is small, are MORE LIKELY to cause more PROPAGATION,
and SET OTHER VARIABLES.
Called the MOST CONSTRAINED VARIABLE heuristic.
Try value first that is MOST LIKELY to actually appear in a solution. BEST CASE: no backtracking!
How to guess which value? One that causes the LEAST TIGHTENING of the domains of OTHER variables.
LEAST CONSTRAINED VALUE heuristic.
Why OPPOSITE?
Because you MUST eventually satisfy EVERY clause, so MIGHT AS WELL start with one that is HARDEST
to satisfy.
BUT you do not need to use every VALUE: so might as well try to AVOID considering UNLIKELY value!
Tie breaking: DETERMINISTIC or RANDOMIZED.
Which better?
DETERMINISTIC more easily becomes STUCK; Should break ties RANDOMLY.
We will see how we can further LEVERAGE randomness.
FOR WEB VERSION: copy this slide, WITH ANSWERS filled in.

19. N-queens Demo Board size 15 Delay 6
Deterministic vs. Randomized tie breaking

20. Inference in CSP’s: So Far…
Constraint checking against partial assignments
Forward checking: each time a variable is instantiated, remove other inconsistent values
Keep track of current domains of vars
Arc consistency:
Iterate forward checking until no more changes
For each pair (Vi,Vj): for every value d in the current domain of Vi there is some value y in the domain of Vj such that Vi=x and Vj=y is permitted by the binary constraint between Vi and Vj
Variable / value selection

21. Coming Up Leveraging randomized tie-breaking Satisfiability
Boolean (0/1) variables
Non-binary constraints
Iterative Repair and Problem Hardness