Lecture 7 Constraint Satisfaction Problems CSE 573 Artificial Intelligence I Henry Kautz Fall 2001

But first… Why is driving to Spokane like playing Backgammon? States = cities Start state = starting city Goal state test = is state the destination city? Operators = move to an adjacent city; cost = distance Output: a shortest path from start state to goal state SUPPOSE WE REVERSE START AND GOAL STATES?

Backtracking Constraint Satisfaction Algorithms Roadmap Tree Search (guessing) BFS DFS Iterative Deepening Bidirectional Best-first search A* Game tree Davis-Putnam (logic) Cutset conditioning (probability) Inference (simplifying) Forward Checking Path Consistency (Waltz labeling, temporal algebra) Resolution “Bucket Algorithm” Iterative improvement (wandering) Hillclimbing (gradient descent) Walksat & Simulated annealing Genetic algorithms Monte-Carlo Methods Backtracking Constraint Satisfaction Algorithms Today we are going to start talking about a inference method

Map Coloring We want to color map with 3 (RED, BLUE, GREEN) colors using DFS. Suppose we color Spain RED, then try France, RED, and so on for every country. Eventually we reach a leaf, apply the test for a goal state, and start backing up. What’s wrong? But how to make the COMPUTER notice that? Must make constraint that no adjacent states same color DECLARATIVE so that computer can reason about it. ALSO: if Spain RED, France BLUE, France MUST BE GREEN! Want to INFER this fact!

CSP V is a set of variables v1, v2, …, vn D is a set of finite domains D1, D2, …, Dn C is a set of constraints C1, C2, …, Cm Each constraint specifies a restriction over joint values of a subset of the varibles E.g.: v1 is Spain, v2 is France, v3 is Germany, … Di = { Red, Blue, Green} for all i For each adjacent vi, vj there is a constraint Ck (vi,vj) in { (R,G), (R,B), (G,R), (G,B), (B,R), (B,G) } Important special case: Discrete Binary CSP

Variations Find a solution that satisfies all constraints Find all solutions Find a “tightest form” for each constraint (v1,v2) in { (R,G), (R,B), (G,R), (G,B), (B,R), (B,G) }  (v1,v2) in { (R,G), (R,B), (B,G) } Find a solution that minimizes some additional objective function Note: in general path length to solution is not important!

Exploiting CSP Structure Interleave inference and search At each internal node: Select unassigned variable Select a value in domain Backtracking: try another value Branching factor? How to best select a variable? At each node: Propagate Constraints Branching factor is number of possible values for the chosen variable. Thus: to minimize branching factor, choose a variable with the smallest domain! MOST constrained variable heuristic! We will see other good reasons for this!

Running Example: 4 Queens Variables: Constraints: Q VARIABLES: one for each COLUMN, which ROW that queen appears in Q1 in { 1, 2, 3, 4 } Q2 in { 1, 2, 3, 4 } CONSTRAINTS: Non-attacking relationship SEE NEXT SLIDE

Running Example: 4 Queens Variables: Q1 in {1,2,3,4} Q2 in {1,2,3,4} Q3 in {1,2,3,4} Constraints: Q1 Q2 1 3 4 2 VARIABLES: one for each COLUMN, which ROW that queen appears in Q1 in { 1, 2, 3, 4 } Q2 in { 1, 2, 3, 4 } CONSTRAINTS: Non-attacking relationship

Constraint Checking Q Q x x Q Q x Q x Q x allows us to backtrack early 6 GUESSES

Forward Checking x Q x Q Q x Q x When a variable is set, immediately remove inconsistent values from the domains of all other variables A kind of LOOKAHEAD Need to keep track of CURRENT domains 4 GUESSES

Arc Consistency Only one guess! x Q x Q x Q x Q Q3=3 inconsistent with Q4 in {2,3,4} Q2=1 and Q2=2 inconsistent with Q3 in {1} ARC CONSISTENCY – reducing domain for one variable may reduce domains for others! So, ITERATE forward checking! Only ONE guess – FIRST square! Only one guess!

Huffman-Clowes Labeling                                 Huffman-Clowes Labeling + - + + + + +

Waltz’s Filtering: Arc-Consistency Lines: variables Conjunctions: constraints Initially Di = {+,-, ,  ) Repeat until no changes: Choose edge (variable) Delete labels on edge not consistent with both endpoints

No labeling!                                                        

Path Consistency Path consistency (3-consistency): Check every triple of variables More expensive! k-consistency: n-consistency: backtrack-free search

Variable and Value Selection Select variable with smallest domain Which values to try first? Why different? Tie breaking? Recall why: first reason is to minimize branching factor. Second reason is that if domain is small, are MORE LIKELY to cause more PROPAGATION, and SET OTHER VARIABLES. Called the MOST CONSTRAINED VARIABLE heuristic. Try value first that is MOST LIKELY to actually appear in a solution. BEST CASE: no backtracking! How to guess which value? One that causes the LEAST TIGHTENING of the domains of OTHER variables. LEAST CONSTRAINED VALUE heuristic. Why OPPOSITE? Because you MUST eventually satisfy EVERY clause, so MIGHT AS WELL start with one that is HARDEST to satisfy. BUT you do not need to use every VALUE: so might as well try to AVOID considering UNLIKELY value! Tie breaking: DETERMINISTIC or RANDOMIZED. Which better? DETERMINISTIC more easily becomes STUCK; Should break ties RANDOMLY. We will see how we can further LEVERAGE randomness. FOR WEB VERSION: copy this slide, WITH ANSWERS filled in.

Variable and Value Selection Select variable with smallest domain Minimize branching factor Most likely to propagate: most constrained variable heuristic Which values to try first? Most likely value for solution Least propagation! Least constrained variable Why different? Every constraint must be eventually satisfied Not every value must be assigned to a variable! Tie breaking? In general randomized tie breaking best – less likely to get stuck on same bad pattern of choices Recall why: first reason is to minimize branching factor. Second reason is that if domain is small, are MORE LIKELY to cause more PROPAGATION, and SET OTHER VARIABLES. Called the MOST CONSTRAINED VARIABLE heuristic. Try value first that is MOST LIKELY to actually appear in a solution. BEST CASE: no backtracking! How to guess which value? One that causes the LEAST TIGHTENING of the domains of OTHER variables. LEAST CONSTRAINED VALUE heuristic. Why OPPOSITE? Because you MUST eventually satisfy EVERY clause, so MIGHT AS WELL start with one that is HARDEST to satisfy. BUT you do not need to use every VALUE: so might as well try to AVOID considering UNLIKELY value! Tie breaking: DETERMINISTIC or RANDOMIZED. Which better? DETERMINISTIC more easily becomes STUCK; Should break ties RANDOMLY. We will see how we can further LEVERAGE randomness. FOR WEB VERSION: copy this slide, WITH ANSWERS filled in.

N-queens Demo Board size 15 Delay 6 Deterministic vs. Randomized tie breaking

Inference in CSP’s: So Far… Constraint checking against partial assignments Forward checking: each time a variable is instantiated, remove other inconsistent values Keep track of current domains of vars Arc consistency: Iterate forward checking until no more changes For each pair (Vi,Vj): for every value d in the current domain of Vi there is some value y in the domain of Vj such that Vi=x and Vj=y is permitted by the binary constraint between Vi and Vj Variable / value selection

Coming Up Leveraging randomized tie-breaking Satisfiability Boolean (0/1) variables Non-binary constraints Iterative Repair and Problem Hardness