1. Lecture 5: On-line search Constraint Satisfaction Problems
Reading: Sec. 4.5 & Ch. 5, AIMA
Rutgers CS440, Fall 2003

2. Recap Blind search (BFS, DFS) Informed search (Greedy, A*)
Local search (Hill-climbing, SA, GA1)
Today:
On-line search
Constraint satisfaction problems
Rutgers CS440, Fall 2003

3. On-line search So far, off-line search: On-line search:
Observe environment, determine best set of actions (shortest path) that leads from start to goal
Good for static environments
On-line search:
Perform action, observe environment, perform, observe, …
Dynamic, stochastic domains: exploration
Similar to…
…local search, but to get to all successor states, agent has to explore all actions
Rutgers CS440, Fall 2003

4. Depth-first online
Online algorithms cannot simultaneously explore distant states (i.e., jump from current state to a very distant state, like, e.g., A*)
Similar to DFS
Try unexplored actions
Once all tried, and the goal is not reached, backtrack to unbacktracked previous state
Rutgers CS440, Fall 2003

5. DFS-Online function action = DFS-Online(percept)
s = GetState(percept);
if GoalTest(s) return stop;
if NewState(s) unexplored[s] = Actions(s);
if NonEmpty(sp)
Result[sp,ap] = s;
Unbacktracked[s] = sp;
end
If Empty(unexpolred[s])
If Empty(unbacktracked[s]) return stop;
else
a: Result[ s, a ] = Pop( unbacktracked[s] );
a = Pop( unexplored[s] );
sp = s;
return a;
endcc
Rutgers CS440, Fall 2003

6. Maze G s0
Rutgers CS440, Fall 2003

7. Online local search Hill-climbing… Instead… Is an online search!
However, it does not have any memory.
Can one use random restarts?
Instead…
Random wandering by choosing random actions (random walk) --- eventually, visits all points
Augment HC with memory of sorts: keep track of estimates of h(s) and updated them as the search proceeds
LRTA* - learning real-time A*
Rutgers CS440, Fall 2003

8. LRTA* function action = LRTA*(percept) s = GetState(percept);
if GoalTest(s) return stop;
if NewState(s) H[s] = h(s);
if NonEmpty(sp)
Result[sp,ap] = s;
H[sp] = min b Actions(sp) { c(sp,b,s) + H(s) };
end
a = arg min b Actions(s) { c(s,b, Result[s,b] ) + H(Result[s,b] ) };
sp = s;
return a;
endcc
Rutgers CS440, Fall 2003

9. Maze – LRTA* 2 1 G 3 2 1 1 4 3 2 2 2 3 3 4 s0 3 3 3 3 4 4 4 4 4 4 4
Rutgers CS440, Fall 2003

10. Constraint satisfaction problems - CSP
Slightly different from standard search formulation
Standard search: abstract notion of state + successor function + goal test
CSP:
State: a set of variables V = {V1, …, VN} and values that can be assigned to them specified by their domains D = {D1, …, DN}, Vi  Di
Goal: a set of constraints on the values that combinations of V can take
Examples:
Map coloring, scheduling, transportation, configuration, crossword puzzle, N-queens, minesweeper, …
Rutgers CS440, Fall 2003

11. Map coloring Vi = { WA, NT, SA, Q, NSW, V, T } Di = { R, G, B }
Goal / constraint: adjacent regions must have different color
Solution: { (WA,R), (NT,G), (SA,B), (Q,R), (NSW,G), (V,R), (T,G) }
Rutgers CS440, Fall 2003

12. Constraint graph Nodes: variables, links: constraints NT Q WA NSW SA V
Rutgers CS440, Fall 2003

13. Types of CSPs Discrete-valued variables Continuous-valued variables
Finite: O(dN) assignments, d = |D|
Map coloring, Boolean satisfiability
Infinite: D is infinite, e.g., strings or natural numbers
Scheduling
Linear constraints: aV1 + bV2 < V3
Continuous-valued variables
Functional optimization
Linear programming
Rutgers CS440, Fall 2003

14. Types of constraints Unary: V  blue Binary: V  Q Higher order
Preferences: different values of V have different “scores”
Rutgers CS440, Fall 2003

15. CSP as search Generic: fits all CSP Depth N, N = number of variables
function assignment = NaiveCSP(V,D,C)
1) Initial state = {};
2) Successor function: assign value to unassigned variable that does not conflict with current assignments.
3) Goal: Assignment complete?
end
Generic: fits all CSP
Depth N, N = number of variables
DFS, but path irrelevant
Problem: # leaves is n!dN > dN
Luckily, we only need consider one variable per depth! – Assignment is commutative.
Rutgers CS440, Fall 2003

16. Backtracking search DFS with single variable assignment
Rutgers CS440, Fall 2003

17. Map coloring example WA NT SA Q NSW V T NT Q WA
Rutgers CS440, Fall 2003

18. How to improve efficiency?
Address the following questions
Which variable to pick next?
What value to assign next?
Can one detect failure early?
Take advantage of problem structure?
Rutgers CS440, Fall 2003

19. Most constrained variable
Choose next the variable that is most constrained based on current assignment WA NT SA Q
NSW V T NT WA SA
Rutgers CS440, Fall 2003

20. Most constraining variable
Tie breaker among most constrained variables WA NT SA Q
NSW V T NT Q SA
Rutgers CS440, Fall 2003

21. Least constraining value
Select the value that least constrains the other variables (rules our fewest other variables)
Rutgers CS440, Fall 2003

22. Forward checking Terminate search early, if necessary:
keep track of values of unassigned variables
Rutgers CS440, Fall 2003

23. Constraint propagation
FC propagates assignment from assigned to unassigned variables, but does not provide early detection of all failures
NT & SA cannot both be blue!
CP repeatedly enforces constraints
Rutgers CS440, Fall 2003

24. Arc consistency
Simplest form of propagation makes each arcs consistent:
X -> Y is consistent iff for all x  X, there is y  Y that satisfies constraints
Detects failure earlier than FC.
Either preprocessor or after each assignment
AC-3 algorithm
Rutgers CS440, Fall 2003

25. Problem structure
Knowing something about the problem can significantly reduce search time WA NT SA Q
NSW V T
T is independent of the rest! Connected components.
c-components => O(dcn/c)
Rutgers CS440, Fall 2003

26. Tree-structured CSP
If graph is a tree, CSP can be accomplished in O(nd2) WA NT SA Q
NSW V T WA NT SA Q
NSW V
Flatten tree into a “chain” where each node is preceded by its parent
Propagate constraints from last to second node
Make assignment from first to last
Rutgers CS440, Fall 2003

27. Nearly tree-structured problems
What if a graph is not a tree?
Maybe can be made into a tree by constraining it on a variable. WA NT SA Q
NSW V T WA NT SA Q
NSW V T
If c is cutset size, then O(dc (N-c)d2 )
Rutgers CS440, Fall 2003

28. Iterative CSP Local search for CSP
Start from (invalid) initial assignment, modify it to reduce the number of violated constraints
Randomly pick a variable
Reassign a value such that constraints are least violated (min-conflict heuristic)
Hill-climbing with h(s) = min-conflict
Rutgers CS440, Fall 2003