Iterative Improvement Search Including Hill Climbing, Simulated Annealing, WALKsat and more....

An overview of the form of search Given: a set of states (or configuration) a function that evaluates each state (or configuration), Eval(State) Locate: The global maximum, i.e. X* such that Eval(X*) > Eval(X i ) for all X i.

Some problems take this form Find the shortest circuit through the cities A State = Eval(State) = A State = Eval(State) = Find assignments to variables that satisfy clauses boolean satisfiability

Some problems take this form Find the shortest circuit through the cities A State = an ordering of cities to visit {orlando, chicago, etc} Eval(State) = length of path defined by permutation (minimize) A State = a set of assignments to {A,B,C,D,E}, i.e. {1,1,1,1,1}, etc. Eval(State) = # clauses that are satisfied Find assignments to variables that satisfy clauses boolean satisfiability

More problems take this form Assign jobs to machines that minimize run time. A State = Eval(State) = A State = Eval(State) = place components on a chip in a way that is maximally compact & efficient, routing wise

More problems take this form Assign n jobs to m machines that run time is minimized. A State = an assignment of n jobs to m machines Eval(State) = completion time of the jobs (minimize) A State = a placement of components & a routing of interconnections Eval(State) = Distance between components + % of unused space + routing length (minimize) Place components on a chip in a way that is maximally compact & efficient, routing wise

These are problems in which… There is a combinatorial state space being optimized. There is a cost function, Eval(state): state -> real #, to be optimized, or at least a reasonable solution is to be found. Searching all possible states is intractable. Depth first search approaches are too expensive. There’s no known algorithm for finding the optimal solution efficiently. Similar solutions can be said to have similar costs. In general

Iterative improvement search Intuition: consider configurations as though they are laid out on the surface of a landscape. We want to find the highest point in the configuration space. Unlike many other AI search problems like the 8-puzzle, we don’t care how we get there. “Iterative Improvement” methods, generally: Start at a random configuration; repeatedly consider various moves; accept some & reject some. When you’re stuck, restart. These demand a moveset that describe what moves we will consider from any configuration. This moveset is analogous to the successor function in the prior search problems we have covered. Let’s invent movesets for our sample problems.

Some problems take this form Find the shortest circuit through the cities Moveset(state) = Find assignments to variables that satisfy clauses boolean satisfiability

More problems take this form Assign jobs to machines that minimize run time. place components on a chip in a way that is maximally compact & efficient, routing wise Moveset(state) =

Hill climbing Hill-climbing: Attempt to maximize Eval(X) by moving to the highest configuration in our moveset. If they’re all lower, we are stuck at a “local optimum.” 1. Let X := initial state (configuraton) 2. Let E := Eval(X) 3. Let N = moveset_size(X) 4. For ( i = 0 ; i<N ; i := i+1) Let E i := Eval(move(X,i)) 5. If all E i s are ≤ E, terminate, return X 6. Else let i* = argmax i E i 7. X := move(X,i*) 8. E := E i * 9. Goto 3

Hill Climbing Issues & Details Requires no memory (since it involves no backtracking) Is trivial to program The moveset design is critical. This is the real ingenuity – not the decision to use hill-climbing. Evaluation function design is often critical. Problems: dense local optima or plateaux If the number of moves is enormous, the algorithm may be inefficient. What to do? If the number of moves is tiny, the algorithm can get stuck easily. What to do? It’s often cheaper to evaluate an incremental change of a previously evaluated object than to evaluate from scratch. Does hill-climbing permit that? What if approximate evaluation is cheaper than accurate evaluation?

Randomized Hill Climbing 1. Let X := initial state or configuration 2. Let E := Eval(X) 3. Let i = a random move from the moveset 4. Let E i := Eval(move(X,i)) 5. If E < E i then X := move(X,i) E := E i 6. Goto 3 unless bored. What stopping criterion should we use? Any obvious pros or cons compared with our previous hill climber?

Hill climbing example WALKSAT (randomized GSAT): Pick a random unsatisfied clause. Consider 3 moves: flipping each variable. If any improve Eval(X), accept the best. If none improve Eval(X), then 50% of the time, pick the move that is the least bad; 50% of the time, pick a random one. This is the best known algorithm for satisfying Boolean formulae! [Selman, Kautz and Cohen, 1994; see ]

Hill Climbing Example: TSP

3 change example

Hill-climbing Example: TSP