1. Agenda Questions? Topics IDA* Hill climbing (restarts etc)
CSP (Revisit forward checking; iterative appraoches for CSP)

2. Advantages: Can follow gradients in search space; also O(1) space
To specify a hill-climbing
Algorithm you need the
Notions of
1. Set of potential solutions
(and current candidate
solution)
2. A neighborhood of a
given potential solution
3. A “goodness” function
that estimates how good
a given potential solution is
Advantages: Can follow gradients in search space; also
O(1) space
Disadvantages: no guarantee of completeness or optimality

8. In practice, the main issue is the schedule used to decrease T.
The faster T is decreased, the faster the search, but the lower
The probability of reaching global optimum.

9. Hill-climbing in “continuous” search spaces
Example: cube root
Finding using newton-
Raphson approximation
Gradient descent (that you study in calculus of variations) is a special case of hill-climbing search applied to continuous search spaces
The local neighborhood is defined in terms of the “gradient” or derivative of the error function.
Since the error function gradient will be zero near the minimum, and higher farther from it, you tend to take smaller steps near the minimum and larger steps farther away from it. [just as you would want]
Gradient descent is guranteed to converge to the global minimum if alpha (see on the right) is small, and the error function is “uni-modal” (I.e., has only one minimum).
Versions of gradient-descent algorithms will be used in neuralnetwork learning.
Unfortunately, the error function is NOT unimodal for multi-layer neural networks. So, you will have to change the gradient descent with ideas such as “simulated annealing” to increase the chance of reaching global minimum.
Err=
|x3-a|
a1/3 xo X

10. The middle ground between hill-climbing and systematic search
Hill-climbing has a lot of freedom in deciding which node to expand next. But it is incomplete even for finite search spaces.
Good for problems which have solutions, but the solutions are non-uniformly clustered.
Systematic search is complete (because its search tree keeps track of the parts of the space that have been visited).
Good for problems where solutions may not exist,
Or the whole point is to show that there are no solutions (e.g. propositional entailment problem to be discussed later).
or the state-space is densely connected (making repeated exploration of states a big issue).
Smart idea: Try the middle ground between the two?

11. Between Hill-climbing and systematic search
You can reduce the freedom of hill-climbing search to make it more complete
Tabu search
You can increase the freedom of systematic search to make it more flexible in following local gradients
Random restart search

12. Tabu Search
A variant of hill-climbing search that attempts to reduce the chance of revisiting the same states
Idea:
Keep a “Tabu” list of states that have been visited in the past.
Whenever a node in the local neighborhood is found in the tabu list, remove it from consideration (even if it happens to have the best “heuristic” value among all neighbors)
Properties:
As the size of the tabu list grows, hill-climbing will asymptotically become “non-redundant” (won’t look at the same state twice)
In practice, a reasonable sized tabu list (say 100 or so) improves the performance of hill climbing in many problems

13. Random restart search Properties: Variant of depth-first search where
When a node is expanded, its children are first randomly permuted before being introduced into the open list
The permutation may well be “biased” random permutation
Search is “restarted” from scratch anytime a “cutoff” parameter is exceeded
There is a “Cutoff” (which may be in terms of # of backtracks, #of nodes expanded or amount of time elapsed)
Properties:
Because of the “random” permutation, everytime the search is restarted, you are likely to follow different paths through the search tree. This allows you to recover from the bad initial moves.
The higher the cutoff value the lower the amount of restarts (and thus the lower the “freedom” to explore different paths).
When cutoff is infinity, random restart search is just normal depth-first search—it will be systematic and complete
For smaller values of cutoffs, the search has higher freedom, but no guarantee of completeness
A strategy to guarantee asymptotic completeness: (***Didn’t say this in the class)
Start with a low cutoff value, but keep increasing it as time goes on.
Random restart search has been shown to be very good for problems that have a reasonable percentage of “easy to find” solutions (such problems are said to exhibit “heavy-tail” phenomenon). Many real-world problems have this property.

14. Constraint Satisfaction Problems (a brief animated overview)Z X Y
X: red
Y: blue
Z: green
Red
green
blue X
Search backtracking, variable/value heuristics
Inference Consistency enforcement, forward checking Y Z
Coloring Problem
CSP Representation
Variables
Problem Statement
CSP Algorithm
Values
Solution
Constraints
December 2, 1998
Sqalli, Tutorial on Constraint Satisfaction Problems

18. Any qns on HW 2? Project 1? Lecture?
Class of 20th September, 2001
Any qns on HW 2? Project 1? Lecture?

19. Agenda Questions on project 1? (due Sep 27th )
Questions on homework 2? (due Sep 25th)
Any other questions?
Lecture
Next class: Adversarial Search (Chapter 6)
CSP was in Chapter 5
Hill-climbing/Genetic algorithms in Chapter 4

20. Review of CSP/SAT concepts
x,y,u,v: {A,B,C,D,E}
w: {D,E} l : {A,B}
x=A wE
y=B uD
u=C lA
v=D lB
Constraint Satisfaction Problem (CSP)
Given
A set of discrete variables
Legal domains for each of the variables
A set of constraints on values groups of variables can take
Find an assignment of values to all the variables so that none of the constraints are violated
SAT Problem = CSP with boolean variables
A solution:
x=B, y=C, u=D, v=E, w=D, l=B x A N
: {x=A} 1 y B
MAY BE WORTH PUTTING THIS IN TH EMIDDLE OF THE TUTORIAL__ASK FOR A SHOW OF HANDS AS TO HOW MANY KNOW CSPs. IF they don’t;, kEEP the CSP in a different window and show it to them…
MAY BE WORTH PUTTING SOME NUMBERS FROM BACCHUS’s SLIDES. N
: {x=
A & y =
B } 2 v D N
: {x=
A & y =
B & v = D } 3 u C N
: {x=
A & y =
B & v = D & u = C } 4 w E N
: {x=
A & y =
B & v = D & u = C & w= E } w D 5 N
: {x=
A & y =
B & v = D & u = C & w= D } 6

21. y n y y n n n n n n

22. Dynamic variable ordering
Do forward checking
Order variables based on their “live” domains

23. A greedier version of the above:
Ideas for improving convergence:
-- Random restart hill-climbing
After every N iterations,
start with a completely random
assignment
--Probabilistic greedy
-with probability p do what
the greedy strategy suggests
-with probability (1-p) pick a
random variable and change its
value randomly
-- p can increase as the search
progresses
A greedier version of the above:
For each variable v, let l(v) be the value that it can take
so that the number of conflicts are minimized. Let n(v) be the
number of conflicts with this value.
--Pick the variable v with the lowest n(v) value.
--Assign it the value l(v) 1
I pointed out that
The neighborhood
1 is subsumed by
Neighborhood 2 2
This one basically searches the 1-neighborhood of the current assignment
(where k-neighborhood is all assignments that differ from the current assignment in atmost
k-variable values)

25. The phase-transition/Sharp threshold Discussion in the class about
There was a very long
phenomenon
SAT Phase Transition 3 -
~4.3
#clauses / # Variables

26. Class of 25th September, 2001

27. Announcements Project
“Late” submission allowed until 4th october with 10% penalty
Homework 2
“Late” submission allowed until 27th September with 5% penalty
Midterm
Oct 9th or 11th (most likely in-class)
Don’t print hidden slides in lecture notes.
(use the powerpoint print option to supress hidden slides while printing )
I often modify/add slides after the class to reflect additional discussion done in the class.

28. Agenda Local consistency in CSP
Genetic algorithms (variation of Hill Climbing)
IDA* search
Pattern database heuristics
---Start adversarial search…

29. What makes CSP problems hard?
Asignments to individual variables that seem locally consistent are often globally infeasible, causing costly backtracking.
The difficulty of a CSP/SAT problem depends on
Number of variables (propositions)
Number of constraints (clauses)
Arity of constraints
Degree of local consistency
Not very predictive. For example, even if there are too
Many variables, the problem may still be easy because
The constraints are “very few” OR “Very many”
Relative to # variables.
Similarly it may be the case that an n-ary constraint really
Is decomposable into several k-ary constratins (k<<n).

31. Hardness & Local Consistency
An n-variable CSP problem is said to be k-consistent iff every consistent assignment for (k-1) of the n variables can be extended to include any k-th variable
Directional consistency: Assignment to first k-1 variables can be extended to the k-th variable
Strongly k-consistent if it is j-consistent for all j from 1 to k
Higher the level of (strong) consistency of problem, the lesser the amount of backtracking required to solve the problem
A CSP with strong n-consistency can be solved without any backtracking
We can improve the level of consistency of a problem by explicating implicit constraints
Enforcing k-consistency is of O(nk) complexity
Break-even seems to be around k=2 (“arc consistency”) or 3 (“path consistency”)
Use of directional and partial consistency enforcement techniques
One measure of hardness of a problem is the minimum level of consistency that needs to be enforced on the problem so it becomes strongly n-consistent

32. More comments made in the class
Some of this may make
more sense Once we
discuss Propositional logic
More comments made in the class
In general, enforcing consistency involves explicitly adding constraints that hold given the existing constraints
E.g. In enforcing 3-consistency if we find that for a particular 2-label {xi=v1 & xj=v2} there is no possible consistent value of xk, then we write this as an additional constraint
{xi=v1}=> {xj != v2}
When enforcing 2-consistency (or arc-consistency), the new constraints will be of the form xi!=v1 , and so these can be represented by just contracting the domain of xi by pruning v1 from it
Unearthing implicit constraints can also be interpreted as “inferring” new constraints that hold given the existing constraints.
In the context of boolean CSPs (I.e., propositional satisfiability problems), the analogy is even more striking since unearthing new constraints means writing down new clauses (or “facts”) that hold given the existing clauses
This interpretation shows that consistency enforcement is a form of “inference” process.
There is a general idea that in solving a search problem, you can interleave two different processes
“inference” trying to either infer the solution itself or saying no solution exists
“conditioning or enumeration”which attempts to systematically go through potential solutions looking for a real solution.
Good search algorithms interleave both inference and conditioning
E.g. the CSP algorithm we discussed in the class uses backtracking search (enumeration), and forward checking (inference).
ADDED AFTER CLASS
IMPORTANT

33. Enforcing Arc Consistency: An example
When we enforce arc-consistency on the top left CSP (shown as a constraint graph), we get the CSP shown in the bottom left. Notice how the domains have reduced. Here is an explanation of what happens.
Suppose we start from Z. If Z=1, then Y can’t have any valid values. So, we remove 1 from Z’s domain. If Z=2, or 3 then Y can have a valid value (since Y can be 1 or 2).
Now we go to Y. If Y=1, then X can’t have any value. So, we remove 1 from X’s domain. If Y=3, then Z can’t have any value. So, we remove 3 from Y’s domain. So Y has just 2 in its domain!
Now notice that Y’s domain has changed. So, we re-consider Z (since anytime Y’s domain changes, it is possible that Z’s domain gets affected). Sure enough, Z=2 no longer works since Y can only be 2 and so it can’t take any value if Z=2. So, we remove 2 also from Z’s domain. So, Z’s domain is now just 3!
Now, we go to X. X can’t be 2 or 3 (since for either of those values, Y will not have a value. So, X has domain 1!
Notice that in this case, arc-consistency SOLVES the problem, since X,Y and Z have exactly 1 value each and that is the only possible solution. This is not always the case (see next example).
X:{1,2,3}
X<Y
Y:{1,2,3}
Y<Z
Z:{1,2,3}
X:{1}
X<Y
Y:{2}
Y<Z
Z:{3}
ADDED AFTER CLASS
IMPORTANT

34. More on arc-consistency
Arc-consistency doesn’t always imply that the CSP has a solution or that there is no search required. In the previous example, if each variable had domains 1,2,3,4, then at the end of enforcing arc-consistency, each variable will still have 2 values in its domain—thus necessitating search.
Here is another example which shows that the search may find that there is no solution for the CSP, even though it is arc-consistent.
Here is a binary CSP that
Is arc-consistent but has no
Solution.
ADDED AFTER CLASS
IMPORTANT

35. Arc-Consistency > directed arc-consistency > Forward Checking
> : “stronger than”
DAC:For each variable u, we only consider
The effects on the variables that
Come after u in the ordering
After directional arc-consistency
Assuming the variable order
X<Y<Z
X:{1,2}
X:{1,2,3}
X<Y
X<Y
Y:{2}
Y:{1,2,3}
Y<Z
Y<Z
AC is the strongest
It propagates Changes in
all directions Until we
reach a fixed point
(no further changes)
Z:{1,2,3}
Z:{1,2,3}
After forward checking assuming
X<Y<Z, and X has been set to value 1
After arc-consistency
X:{1}
X<Y
X:{1}
Y:{2}
X<Y
FC: We start with the current
Assignment for some of the
Variables, and only consider their
Effects on the future variables.
(only a single level
Propagation is done. After we find
that a value of Y is pruned, we don’t try
To see if that changes domain of Z)
Y<Z
Y:{2,3}
Y<Z
Z:{3}
Z:{1,2,3}
ADDED AFTER CLASS
IMPORTANT

36. Constraint Satisfaction Problems (a brief animated summary)Z X Y
X: red
Y: blue
Z: green
Red
green
blue X
Search backtracking, variable/value heuristics
Inference Consistency enforcement, forward checking Y Z
Coloring Problem
CSP Representation
Variables
Problem Statement
CSP Algorithm
Values
Solution
Constraints

37. Genetic Algorithms Does the evolutionary
metaphor really buy us anything?
-A form of hill climbing
--differences:
Multiple current nodes (seeds)
(instead of a single one)
Neighborhood defined
in terms of combination of
current seeds
--with some randomness
A bit of “evolutionary” metaphors

38. Class of 27th September, 2001

39. IDA* to handle the A* memory problem
Basicaly IDDFS, except instead of the iterations being defined in terms of depth, we define it in terms of f-value
Start with the f cutoff equal to the f-value of the root node
Loop
Generate and search all nodes whose f-values are less than or equal to current cutoff.
Use depth-first search to search the trees in the individual iterations
Keep track of the node N’ which has the smallest f-value that is still larger than the current cutoff. Let this f-value be next-largest-f-value
-- If the search finds a goal node, terminate. If not, set cutoff = next-largest-f-value and go back to Loop
Properties:
Linear memory.
#Iterations in the worst case? =
Bd !! 
(Happens when all nodes have distinct
f-values)

40. Pattern Databases and other more sophisticated approaches for deriving heuristics

41. Manhattan Distance Heuristic1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Manhattan distance is 6+3=9 moves

42. Performance on 15 Puzzle
Random 15 puzzle instances were first solved optimally using IDA* with Manhattan distance heuristic (Korf, 1985).
Optimal solution lengths average 53 moves.
400 million nodes generated on average.
Average solution time is about 50 seconds on current machines.

43. Limitation of Manhattan Distance
To solve a 24-Puzzle instance, IDA* with Manhattan distance would take about 65,000 years on average.
Assumes that each tile moves independently
In fact, tiles interfere with each other.
Accounting for these interactions is the key to more accurate heuristic functions.

44. More Complex Tile Interactions
Only the positions of the fringe tiles and the blank matter. So states that have the fringe tiles and
The blank in the same position are equivalent. So, we can just precompute the values for these.
16!/(16-8)!=519M
More Complex Tile Interactions 3 7 11 12 13 14 15
M.d. is 19 moves, but 31 moves are needed.
M.d. is 20 moves, but 28 moves are needed 3 7 11 12 13 14 15
M.d. is 17 moves, but 27 moves are needed 3 7 11 12 13 14 15

45. Pattern Database Heuristics
Culberson and Schaeffer, 1996
A pattern database is a complete set of such positions, with associated number of moves.
e.g. a 7-tile pattern database for the Fifteen Puzzle contains 519 million entries.

46. Heuristics from Pattern Databases1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
31 moves is a lower bound on the total number of moves needed to solve this particular state.

47. Precomputing Pattern Databases
Entire database is computed with one backward breadth-first search from goal.
All non-pattern tiles are indistinguishable, but all tile moves are counted.
The first time each state is encountered, the total number of moves made so far is stored.
Once computed, the same table is used for all problems with the same goal state.

48. What About the Non-Pattern Tiles?
Given more memory, we can compute additional pattern databases from the remaining tiles.
In fact, we can compute multiple pattern databases from overlapping sets of tiles.
The only limit is the amount of memory available to store the pattern databases.

49. Combining Multiple Databases1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
31 moves needed to solve red tiles
22 moves need to solve blue tiles
Overall heuristic is maximum of 31 moves

50. Applications of Pattern Databases
On 15 puzzle, IDA* with pattern database heuristics is about 10 times faster than with Manhattan distance (Culberson and Schaeffer, 1996).
Pattern databases can also be applied to Rubik’s Cube.

51. On “predicting” the effectiveness of Heuristics
Let us divide the number of nodes expanded nE into
Two parts: nI which is the number of nodes expanded
Whose f-values were strictly less than f* (I.e. the
Cost of the optimal goal), and nG is the # of expanded
Nodes with f-value greater than f*. So, nE=nI+nG
A more informed heuristic is only guaranteed to have
A smaller nI—all bets are off as far as the nG value is
Concerned. In many cases nG may be relatively large
Compared to nI making the nE wind up being higher
For an informed heuristic!
Unfortunately, it is not the case that a heuristic h1 that is more informed than h2 will always do fewer node expansions than h2.
-We can only gurantee that h1 will expand less nodes with f-value less than f* than h2 will
Consider the plot on the right… do you think h1 or h2 is likely to do better in actual search?
The “differentiation” ability of the heuristic—I.e., the ability to tell good nodes from the bad ones-- is also important. But it is harder to measure.
Some new work that does a histogram characterization of the distribution of heuristic values [Korf, 2000]
Nevertheless, informedness of heuristics is a reasonable qualitative measure h* h1
Heuristic value h2
Is h1 better or h2?
Nodes

52. Temporal Constraint Satisfaction Problem (TCSP)
[Dechter, Schwalb, AIJ 1997]
Temporal Constraint Satisfaction Problem (TCSP)
Variables correspond to time points
Constraints retrict allowable distances between time points
Constraints can be represented as sets of intervals
Simple temporal problem (STP)
All constraints have at most one interval (TRACTABLE!!!)
Solution of general TCSP involves enforcing path-constistency
Approximations such as loose path consistency can be used to detect inconsistencies
[0,3] t1
[1,5] [7,9] t0 t2
[ 7,20]
[ 7,12]
[1,12]

53. DDB & EBL x ¬ A y ¬ B v ¬ D u ¬ C w ¬ E w ¬ D
DDB: Skip a level if the regressed
Explanation remains the same
EBL: Store interior node failure
Explanations x A N
: {x=A} 1 y B
x=A
Resume
Search N
: {x=
A & y =
B } 2
y = B & x=A v D N
: {x=
A & y =
B & v = D } 3
y = B & x=A u C
Interior node
explanation N
: {x=
A & y =
B & v = D & u = C }
x=A 4 w E
E4: y = B & x=A
y=B
regression w D N
: {x=
A & y =
B & v = D & u = C & w= E } 5
E5: x=A & w=E N
: {x=
A & y =
B & v = D & u = C & w= D } 6
E6: y = B & w = D