1 Artificial Intelligence CS 165A Tuesday, October 16, 2007  Informed (heuristic) search methods (Ch 4) 1

2 Notes HW#1 –Discussion sessions Wednesday –Office hours What’s an example of a problem where you don’t need to know the path to a goal? (Finding the goal is enough.) –Does this starting position in the 8-puzzle have a solution? –Is this theorem provable? –Traveling salesman problem (“iterative improvement” approach) –Will this move lead me to checkmate, or be checkmated, within N moves?

3 Informed (“Heuristic”) Search If AI is “the attempt to solve NP-complete problems in polynomial time,” blind search doesn’t help Then we typically have two choices –Design algorithms that give good average-case behavior (even if the worst-case behavior is terrible) –Design algorithms that are approximate – give acceptable results (even if not optimal) in acceptable time/memory/etc. Informed search generally tries to do both of these

4 Informed Search (cont.) The path cost g(n) only measures the “past,” it tells nothing about the “future” –I.e., it measures cost from initial state to current state only We wish to minimize the overall cost, so we need an estimate of the “future” cost –I.e., from current state to goal state –Overall cost = past + future costs This estimate is a heuristic – a “rule of thumb” –A function that estimates the solution cost –A rule expressing informal belief

5 How to choose a heuristic? One method: Derive a heuristic from the exact solution cost of a relaxed (less restricted) version of the problem A standard distance metric –E.g., Euclidian distance, Hamming distance, … Use common sense? A good heuristic function must be efficient as well as accurate

6 Heuristics What’s a heuristic for –Driving distance (or time) from city A to city B ? –8-puzzle problem ? –M&C ? –Time to complete homework assignment ? –Robot navigation ? –Reaching the summit ? –Medical diagnosis ? Admissible heuristic –Does not overestimate the cost to reach the goal –“Optimistic” Are the above heuristics admissible?

7 Informed Search Methods Best-first search –Greedy best-first search –A* search Memory bounded search –IDA* –SMA* Iterated improvement algorithms –Hill-climbing –Simulated annealing As with blind search, the strategy is defined by choosing the order of node expansion Not covering in detail

8 Best-First Search Defines a family of search algorithms Uses an evaluation function at each node to estimate the desirability of expanding the node A queuing function sorts the unexpanded nodes in decreasing order of desirability, according to the evaluation function –Therefore, the one with the “highest desirability” is expanded first –We’ve already seen this with Uniform Cost Search  Expand the node n with the lowest g(n)

9 Best-First Search What’s the difference between Q UEUING -F N and E VAL -F N ? –E VAL -F N takes one argument (the node)  Returns a scalar value (e.g., approximate distance to goal) –Q UEUING -F N t akes two arguments: a queue of nodes and a function (e.g., less-than)  Returns a queue of nodes function B EST -F IRST -S EARCH (problem, E VAL -F N ) returns a solution or failure Q UEUING -F N  a function that orders nodes by E VAL -F N return G ENERAL -S EARCH (problem, Q UEUING -F N ) function B EST -F IRST -S EARCH (problem, E VAL -F N ) returns a solution or failure Q UEUING -F N  a function that orders nodes by E VAL -F N return G ENERAL -S EARCH (problem, Q UEUING -F N ) But what evaluation function to use?

10 Greedy Best-First Search Uses a heuristic function, h(n), as the E VAL -F N h(n) estimates of the cost of the best path from state n to a goal state –h(goal) = 0 Greedy search – always expand the node that appears to be closest to the goal (i.e., with the smallest h) –Instant gratification, hence “greedy” function G REEDY -S EARCH (problem, h) returns a solution or failure return B EST -F IRST -S EARCH (problem, h) function G REEDY -S EARCH (problem, h) returns a solution or failure return B EST -F IRST -S EARCH (problem, h) Greedy search often performs well –It doesn’t always find the best solution –It may get stuck –It depends on the particular h function

11 GBFS Example Use h SLD (n) – straight-line distance to goal (admissible?)

12 Sibiu Bucharest h = 253h = 0 OradeaAradFagarasRimnicu Vilcea h = 380h = 366h = 178h = 193 ZerindSibiuTimisoara h = 374h = 253h = 329 h = 366 Arad d = = 450 km Is this the optimal solution? No: Arad  Sibiu  Rimnicu Vilcea  Pitesti  Bucharest 418 km

13 Don’t be greedy? Greedy methods get stuck in local minima (maxima) Robot GOAL

14 Greedy Best-First Search Optimal? Complete? Time complexity? Space complexity? No Exponential: O( b m ) (worst case) Exponential: O( b m ) – keeps all nodes in memory A good heuristic function reduces the (practical) complexity substantially!

15 “A” Search Uniform-cost search minimizes g(n) (“past” cost) Greedy search minimizes h(n) (“expected” or “future” cost) “A Search” combines the two: –Minimize f (n) = g(n) + h(n) –Accounts for the “past” and the “future” –Estimates the cheapest solution (complete path) through node n

16 A* Search “A* Search” is A Search with an admissible h –h is optimistic – it never overestimates the cost to the goal  h(n)  true cost to reach the goal  h is a “Pollyanna” –So f (n) never overestimates the actual cost of the best solution passing through node n function A*-S EARCH (problem, h) returns a solution or failure return B EST -F IRST -S EARCH (problem, f ) function A*-S EARCH (problem, h) returns a solution or failure return B EST -F IRST -S EARCH (problem, f )

17 A* Example f(n) = g(n) + h(n)

18 OradeaAradFagarasRimnicu Vilcea = = = =413 ZerindSibiuTimisoara f = = = =447 f = = 366 Arad A* Example

19 A* Search Optimal? Complete? Time complexity? Space complexity? Yes Exponential; better under some conditions Exponential; keeps all nodes in memory Good news: A* is optimally efficient for any particular h(n) That is, no other optimal algorithm is guaranteed to expand fewer nodes

20 A* Search What if g(n)  0 ? –Greedy best-first search What if h(n)  0 ? –Uniform cost search What if h(n)  0 and g(n)  depth(n) ? –Breadth first search How would you make depth-first search? g(n)  – depth(n)

21 Memory Bounded Search Memory, not computation, is usually the limiting factor in search problems –Certainly true for A* search Why? What takes up memory in A* search? IDA* and SMA* are designed to conserve memory

22 Iterative Deepening A* (IDA*) IDA* is an optimal, memory-bounded, heuristic search algorithm –Requires space proportional to the longest path that it explores –Space estimate: O(bd) Like Iterative Deepening Search –Uses f-cost limit rather than depth-limit –In IDS, depth-limit is incremented after each round –In IDA*, f-cost limit is updated after each round

23 Simplified Memory-Bounded A* (SMA*) IDA* only keeps around the current f-cost limit –Can check the current path for repeated states, but future paths may repeat states already expanded SMA* uses more memory to keep track of repeated states –Up to the limit of allocated memory –Nodes with high f-cost are dropped from the queue when memory is filled (“forgotten nodes”) Optimality and completeness depends on how much memory is available with respect to the optimal solution –Produces the best solution that can be reached given the available memory

24 The cost of being informed Typical performance of informed search methods is much better than uninformed methods –Assuming reasonable heuristics exist However, there is a tradeoff involved –Evaluating the desirability of a node (h) can be a non-trivial problem –E.g., theorem proving  How much closer to the theorem are we if we apply this rule? –Cost of evaluation (heuristic) can be high  It can be a difficult search problem itself

25 Cost tradeoff Cost “Informedness” Cost of expanding nodes (rule application) Cost of evaluating nodes (control strategy) Overall cost There may be different optima for computation and memory

26 Iterative Improvement Algorithms An iterative improvement algorithm starts with a (possibly random) proposed solution, and then makes modifications to improve its quality –Each state is a (proposed) solution –Usually keeps information on current state only Generally for problems in which non-optimal solutions are known, or easily generated –Task: Find the solution that best satisfies the goal test –State space for an IIA = set of all (proposed) solutions –Examples: VLSI layout, TSP, n-queens –Not appropriate for all problems! S0S0 S1S1 S2S2 SnSn

27 Example n-Queens problem: Put n queens on an n x n chess board with no two queens on the same row, column, or diagonal Start -5 1 iteration -3 Goal 0

28 Example Traveling Salesman Problem (TSP) –Start with any path through the cities –Change two links at a time

29 Search vs. iterative improvement Search Iterative improvement

30 Iterative Improvement Algorithms Two classes of iterative improvement algorithms –Hill-climbing –Simulated annealing Analogy: –You are placed at a random point in some unfamiliar terrain (the solution space) and told to reach the highest peak. It is dark, and you have only a very weak flashlight (no map, compass, etc.). –What should you do?

31 Hill Climbing, a.k.a. Gradient Descent Strategy: Move in the direction of increasing value (decreasing cost) –Assumes a reasonable evaluation method!!!  n-queens? TSP? VLSI layout?

32 Hill climbing example “Solution space” Measure of value

33 Hill climbing issues “Solution space” Strategy: –Climb until goal or stuck –If stuck, restart in random location

34 Hill climbing issues Does not maintain a search tree –Evaluates the successor states, and keeps only the best one –Greedy strategy Drawbacks –Local maxima –Plateaus and ridges Can randomize (re-)starting locations and local strategies when stuck –“Random restart hill-climbing” –But how to know when you’re stuck?

35 Simulated Annealing Similar to hill-climbing –But includes a random element –Sometimes take “bad” steps to escape local maxima –Motivated by the roughly analogous physical process of annealing: heating and then slowly cooling a substance to obtain a strong crystalline structure Analogy with physical annealing: –T is temperature, E is energy  A schedule determines the rate at which T is lowered –Value(state) measures the state “goodness” –  E measures increase in “goodness” resulting from new state