1. Heuristic Search Foundations of Artificial Intelligence

2. Announcement Mirror Site Now Available http://facweb.cs.depaul.edu/mobasher/classes/CS480/ The old site on maya.cs.depaul.edu will soon be de-commissioned.

3. Topics Heuristic Search What is a heuristic What is a heuristic Best-First Search and Hill-Climbing Best-First Search and Hill-Climbing A* Search A* Search

4. Heuristic Search Problem with uniform cost search We are only considering the cost so far, not the expected cost of getting to the goal node We are only considering the cost so far, not the expected cost of getting to the goal node But, we don’t know before hand the cost of getting to the goal from a previous state But, we don’t know before hand the cost of getting to the goal from a previous stateSolution: Need to estimate for each state the cost of getting from there to a goal state Need to estimate for each state the cost of getting from there to a goal state Use “heuristic” information to guess which nodes to expand next Use “heuristic” information to guess which nodes to expand next a heuristic is in the form of an evaluation function based on domain-specific information related to the problem. a heuristic is in the form of an evaluation function based on domain-specific information related to the problem. gives us a way to evaluate a node “locally” based on an estimate of the cost to get from the node to a goal node (the idea is to find the least cost path to a goal node). gives us a way to evaluate a node “locally” based on an estimate of the cost to get from the node to a goal node (the idea is to find the least cost path to a goal node).

5. Evaluation Functions h(n) is the heuristic function g(n): cost of the best path found so far between the initial node and n f(n) = h(n)  greedy best-first search f(n) = g(n) + h(n)  A* search

6. Best-First Search Basic Idea: always expand the node that minimizes (or maximizes) the evaluation function f(n) Greedy strategy: f(n) = h(n), where h(n) estimates the cost of getting from the node n to the goal if we keep nodes in memory (on the queue) for backtracking, then this is called (Greedy) Best-First search; if no queue and we stop as soon as f(n) is worse for the children than the parent, then this is called Hill-Climbing. if we keep nodes in memory (on the queue) for backtracking, then this is called (Greedy) Best-First search; if no queue and we stop as soon as f(n) is worse for the children than the parent, then this is called Hill-Climbing. What happens if always try at each step to move closer to the goal node? The BFS algorithm in this case will find the longer solution path, since it will begin by moving forward and then be committed to this choice. What about hill-climbing? The BFS algorithm in this case will find the longer solution path, since it will begin by moving forward and then be committed to this choice. What about hill-climbing?

7. Hill-Climbing This simple policy has three well-known drawbacks: 1. Local Maxima: a local maximum as opposed to global maximum. 2. Plateaus: An area of the search space where evaluation function is flat, thus requiring random walk. 3. Ridge: Where there are steep slopes and the search direction is not towards the top but towards the side. It is simply a loop that continually moves in the direction of the best value. No search tree is maintained. One important refinement is that when there is more than one best successor to choose from, the algorithm can select among them at random.

8. Best-First Search The evaluation function f maps each search node n to positive real number f(n) Traditionally, the smaller f(n), the more promising n Best-first search sorts the search queue at each step in increasing order of f random order is assumed among nodes with equal values of f random order is assumed among nodes with equal values of f

9. Best-First Search The evaluation function f maps each search node n to positive real number f(n) Traditionally, the smaller f(n), the more promising n Best-first search sorts the search queue at each step in increasing order of f random order is assumed among nodes with equal values of f random order is assumed among nodes with equal values of f “Best” only refers to the value of f, not to the quality of the actual path. Best-first search does not generate optimal paths in general “Best” only refers to the value of f, not to the quality of the actual path. Best-first search does not generate optimal paths in general

10. Best-First Search Example (Romania) Suppose we don’t know the actual distances beforehand, but can figure out the straight line distances from a map

11. Best-First Search Example (Romania) Suppose we don’t know the actual distances beforehand, but can figure out the straight line distances from a map Heuristic evaluation function: h(n) = straight-line distance between n h(n) = straight-line distance between n and Bucharest and Bucharest h(n) is a heuristic because it is an estimate of the actual cost of getting from n to the goal Note that h(goal) = 0 always

12. Greedy Best-First Search Arad h(n) = 366 <== Arad <==

13. Greedy Best-First Search Arad Sibiu Timisoara Zerind h(n) = 366 h(n) = 253h(n) = 329 h(n) = 374 <== Sibiu, Timisoara, Zerind <==

14. Greedy Best-First Search Arad Sibiu Timisoara Zerind Oradea Fagaras Arad Rimnicu h(n) = 366 h(n) = 253h(n) = 329 h(n) = 374 366 178 380 193 <== Fagaras, Rimnicu, Timisoara, Zerind, Oradea <==

15. Greedy Best-First Search Arad Sibiu Timisoara Zerind Oradea Fagaras Rimnicu h(n) = 366 h(n) = 253h(n) = 329 h(n) = 374 178 380 193 Bucharest Sibiu 253 h(n) = 0 <== Bucharest, Rimnicu, Timisoara, Zerind, Oradea <==

16. Greedy Best-First Search Arad Sibiu Timisoara Zerind Oradea Fagaras Rimnicu h(n) = 366 h(n) = 253h(n) = 329 h(n) = 374 178 380 193 Bucharest Sibiu 253 h(n) = 0 Actual cost of the solution: Arad => Sibiu => Fagaras => Bucharest is 140 + 99 + 211 = 450 But, consider the path: Arad => Sibiu => Rimnicu => Pitesti => Bucharest with the cost 418 – So we got a suboptimal solution Actual cost of the solution: Arad => Sibiu => Fagaras => Bucharest is 140 + 99 + 211 = 450 But, consider the path: Arad => Sibiu => Rimnicu => Pitesti => Bucharest with the cost 418 – So we got a suboptimal solution

17. Heuristics for 8-Puzzle Problem In total, there are a possible of 9! or 362,880 possible states. However, with a good heuristic function, it is possible to reduce this state to less than 50. Some possible heuristics for 8-Puzzle: h1(n) = no. of misplaced tiles h1(n) = no. of misplaced tiles may have many plateaus (indistinguishable states) doesn’t captures the number of moves to get to the right place h2(n) = sum of Manhattan distances (i.e., no. of squares from desired location of each tile) h2(n) = sum of Manhattan distances (i.e., no. of squares from desired location of each tile) doesn’t capture the importance of sequencing tiles (putting them in the right order)

18. Heuristics for 8-Puzzle Problem h1(s) = 7 h2(s) = 4 + 2 + 2 + 3 + 3 + 0 + 2 + 2 = 18 h1(s) = 7 h2(s) = 4 + 2 + 2 + 3 + 3 + 0 + 2 + 2 = 18 54 3 618 72 12 6 8 3 4 75 s = start state g = goal state

19. 18 19 17 16 17 15 16 14 13 14 Part of the search tree generated by Best-First search using h2 = sum of Manhattan distances.

20. Goal Node Part of the search tree generated by Best-First search using h2 = sum of Manhattan distances. What will happen with hill-climbing? Part of the search tree generated by Best-First search using h2 = sum of Manhattan distances. What will happen with hill-climbing? 18 19 17 16 17 15 16 14 13 14 13 12 13 11 Initial Node Heuristics Search in 8-Puzzle

21. Properties of Best-First (Greedy) Search Complete? No - can get stuck in loops No - can get stuck in loops e.g., Iasi => Neamt => Iasi => Neamt => … e.g., Iasi => Neamt => Iasi => Neamt => … It is complete in finite space with repeated-state checking It is complete in finite space with repeated-state checking Time Complexity: In worst case: O (b m ) In worst case: O (b m ) but good heuristic can give dramatic improvement but good heuristic can give dramatic improvement Space Complexity: In worst case: O (b m ) In worst case: O (b m ) keeps all nodes in memory keeps all nodes in memory Optimal: No

22. A* Search (most popular algorithm in AI) Basic Idea: avoid expanding paths that are already expensive Evaluation function: f(n) = g(n) + h(n) g(n) = cost so far to reach n g(n) = cost so far to reach n h(n) = estimated cost to goal from n h(n) = estimated cost to goal from n f(n) = estimated total cost of path through n to reach the goal f(n) = estimated total cost of path through n to reach the goal Admissible heuristics i.e., h(n)  h*(n), for all n, where h*(n) is the true cost from n i.e., h(n)  h*(n), for all n, where h*(n) is the true cost from n Ex: straight-line distance never overestimates the actual road distance Ex: h1 and h2 is 8-puzzle never overestimate the actual no. of moves A* search is optimal (finds lowest cost solution) if h(n) is admissible A* search is optimal (finds lowest cost solution) if h(n) is admissible however, the number of nodes expanded depends on how good the heuristic is best case: h(n) = h*(n) for all n  A* will find the best solution with no search if h(n) > h*(n) for some n, then A* if h(n) > h*(n) for some n, then A* might still work, but might not find any solution at all

23. A* Search Arad Sibiu Timisoara Zerind Oradea Fagaras Arad Rimnicu f(n) = 366 h(n) = 374 f(n) = 449 Pitesti Craiova 75 h(n) = 329 f(n) = 447 118 140 h(n) = 253 f(n) = 393 140 151 99 80 f(n) = 646 f(n) = 417 f(n) = 661 f(n) = 413 Sibiu 146 97 80 f(n) = 526 f(n) = 415 f(n) = 553

24. A* Search Sibiu Oradea Fagaras Arad Rimnicu Pitesti Craiova h(n) = 253 f(n) = 393 140 151 99 80 f(n) = 646 f(n) = 417f(n) = 661 f(n) = 413 Sibiu 146 97 80 f(n) = 526 f(n) = 415 f(n) = 553 Craiova Rimnicu Bucharest 97 138 101 f(n) = 607f(n) = 615 f(n) = 418...

25. A* Search Sibiu Oradea Fagaras Arad Rimnicu Pitesti Craiova h(n) = 253 f(n) = 393 140 151 99 80 f(n) = 646 f(n) = 417 f(n) = 526 f(n) = 413 Sibiu 146 97 80 f(n) = 526 f(n) = 415 f(n) = 553 Craiova Rimnicu Bucharest 97 138 101 f(n) = 607f(n) = 615 f(n) = 418 Bucharest Sibiu 99 211 f(n) = 591 f(n) = 450...

26. A* Search Sibiu Oradea Fagaras Arad Rimnicu Pitesti Craiova h(n) = 253 f(n) = 393 140 151 99 80 f(n) = 646 f(n) = 417 f(n) = 526 f(n) = 413 Sibiu 146 97 80 f(n) = 526 f(n) = 415 f(n) = 553 Craiova Rimnicu Bucharest 97 138 101 f(n) = 607f(n) = 615 f(n) = 418 Bucharest Sibiu 99 211 f(n) = 591 f(n) = 450...

27. A* search for an instance of 8-puzzle with h1 (sum of misplaced tiles). g(n) assumes each move has a cost of 1. Here we assume repeated state checking. f(n) = g(n) + h(n)

28. A* search for an instance of 8-puzzle with h1 (sum of misplaced tiles). g(n) assumes each move has a cost of 1. Here we assume repeated state checking. f(n) = g(n) + h(n) Order of expansion

29. A* search for an instance of 8-puzzle with h1 (sum of misplaced tiles). g(n) assumes each move has a cost of 1. Here we assume repeated state checking. f(n) = g(n) + h(n)

30. A* search for an instance of 8-puzzle with h1 (sum of misplaced tiles). g(n) assumes each move has a cost of 1. Here we assume repeated state checking. f(n) = g(n) + h(n)

31. A* search for an instance of 8-puzzle with h1 (sum of misplaced tiles). g(n) assumes each move has a cost of 1. Here we assume repeated state checking. f(n) = g(n) + h(n)

32. A* search for an instance of 8-puzzle with h1 (sum of misplaced tiles). g(n) assumes each move has a cost of 1. Here we assume repeated state checking. f(n) = g(n) + h(n)

33. A* search for an instance of 8-puzzle with h1 (sum of misplaced tiles). g(n) assumes each move has a cost of 1. Here we assume repeated state checking. f(n) = g(n) + h(n)

34. A* search for an instance of 8-puzzle with h1 (sum of misplaced tiles). g(n) assumes each move has a cost of 1. Here we assume repeated state checking. f(n) = g(n) + h(n) Note: at level 2 there are two nodes listed with f(n) = 5. Depending on which node is we put in front of the queue, the algorithm will either expand 6 or 7 nodes. Here we have assumed the worse case, and thus the tree shows that 6 nodes were expanded 7

35. A* and Repeated States c = 1 100 2 1 2 h = 100 0 90 1 The heuristic h is clearly admissible

36. A* and Repeated States c = 1 100 2 1 2 h = 100 0 90 1104 4+90 f = 1+100 2+1 ? If we discard this new node, then the search algorithm expands the goal node next and returns a non-optimal solution

37. 1 100 2 1 2 0 90 1104 4+90 1+100 2+1 2+90 102 Instead, if we do not discard nodes revisiting states, the search terminates with an optimal solution A* and Repeated States

38. It is not harmful to discard a node revisiting a state if the new path to this state has higher cost than the previous one A* remains optimal, but the size of the search tree can still be exponential in the worst case Fortunately, for a large family of admissible heuristics – consistent heuristics – there is a much easier way of dealing with revisited states

39. A* and Consistency (Monotonicity) A heuristic h(n) is consistent if for every node n and every successor of n, succ(n): h(n) – h(succ(n))  cost( n  succ(n) ) h(n) – h(succ(n))  cost( n  succ(n) ) i.e., decrease in heuristic value due to an action is never more than the cost of the action i.e., decrease in heuristic value due to an action is never more than the cost of the action All consistent heuristics are admissible For an admissible heuristic the values of f(n) along any path are non-decreasing (monotonincity) the values of f(n) along any path are non-decreasing (monotonincity) A* expands nodes according to non-decreasing order of f(n) A* expands nodes according to non-decreasing order of f(n)

40. A* and Consistency (Monotonicity) 123 456 78 12 3 4 5 67 8 STATE (N) goal  h 1 (N) = number of misplaced tiles  h 2 (N) = sum of the (Manhattan) distances of every tile to its goal position are both consistent

41. A* and Repeated States Dealing with Repeated States: If a newly generated state was previously expanded, then discard the new state If a newly generated state was previously expanded, then discard the new state If multiple (unexpanded) instances of a state end up on the queue, we only keep the instance that has the smallest f value. If multiple (unexpanded) instances of a state end up on the queue, we only keep the instance that has the smallest f value. Theorem: If h is consistent, then whenever A* expands a node, it has already found an optimal path to this node’s state

42. A* and “Informedness” Finding good heuristics for a problem relax restrictions on operators: in general, the cost of an exact solution to a relaxed problem is a good heuristic for the original problem relax restrictions on operators: in general, the cost of an exact solution to a relaxed problem is a good heuristic for the original problem e.g., sum of Manhattan distances in 8-puzzle gives the exact solution to the relaxed version of the problem where a tile can move in any direction, even onto occupied squares use statistical information from training examples to predict the correct heuristic value for the nodes (this may result in an inadmissible heuristic function) use statistical information from training examples to predict the correct heuristic value for the nodes (this may result in an inadmissible heuristic function)

43. A* and “Informedness” (Cont.) Multiple admissible heuristics? given admissible heuristics h 1 and h 2, if h 1 (n) > h 2 (n), for all n, then h 2 dominates h 1 given admissible heuristics h 1 and h 2, if h 1 (n) > h 2 (n), for all n, then h 2 dominates h 1 the dominating admissible heuristic usually expands fewer nodes (it is more informed) the dominating admissible heuristic usually expands fewer nodes (it is more informed) if there are several admissible heuristics none of which dominates any others, we can take the composite heuristic: h(n) = max ( h 1 (n), h 2 (n), …, h k (n) ) if there are several admissible heuristics none of which dominates any others, we can take the composite heuristic: h(n) = max ( h 1 (n), h 2 (n), …, h k (n) ) A* Efficiency and Informedness A* expands every node n for which f(n) < f* A* expands every node n for which f(n) < f* i.e., every node with h(n) < f* - g(n) will be expanded i.e., every node with h(n) < f* - g(n) will be expanded so, if h 1 (n)  h 2 (n), for all n, every node expanded by h 1 must also be expanded by h 2 so, if h 1 (n)  h 2 (n), for all n, every node expanded by h 1 must also be expanded by h 2 Moral of the story: heuristic functions with higher values work better so long as they are admissible (in other words, we want our heuristic to get as close as possible to h*).

44. Efficiency of A* Comparison of search costs and effective branching factors for the Iterative Deepening search and A* algorithms with h1 and h2 for 8-puzzle. d is the average depth of the search tree. Data are averaged over 100 instances of the problem, for various solution lengths.

45. IDA* Algorithm A potential problem with A* is memory since it reduces to breadth-first search (when h = 0), it will potentially use memory that is exponential to the depth of the optimal goal node since it reduces to breadth-first search (when h = 0), it will potentially use memory that is exponential to the depth of the optimal goal node iterative deepening can again help, but now we prune the nodes for which the nearest goal node can be shown to lie below the cutoff depth iterative deepening can again help, but now we prune the nodes for which the nearest goal node can be shown to lie below the cutoff depth note that individual iterations perform a depth- first search; heuristic function is used to prune nodes, but not to determine the order of node expansion note that individual iterations perform a depth- first search; heuristic function is used to prune nodes, but not to determine the order of node expansion

46. IDA* Algorithm Sketch of IDA* Algorithm: 1. Set c = 1; this is the current cutoff value. 2. Set L to be the list of initial nodes. 3.Let n be the first node on L. 4. If L is empty, increment c and return to step 2. 5. If n is a goal node, stop and return the path from initial node to n. 6. Otherwise, remove n from L. Add to front of L every child node n’ of n for which f(n’)  c. Return to step 3. Sketch of IDA* Algorithm: 1. Set c = 1; this is the current cutoff value. 2. Set L to be the list of initial nodes. 3.Let n be the first node on L. 4. If L is empty, increment c and return to step 2. 5. If n is a goal node, stop and return the path from initial node to n. 6. Otherwise, remove n from L. Add to front of L every child node n’ of n for which f(n’)  c. Return to step 3.

47. When to Use Search Techniques? The search space is small, and No other technique is available, or No other technique is available, or Developing a more efficient technique is not worth the effort Developing a more efficient technique is not worth the effort The search space is large, and No other available technique is available, and No other available technique is available, and There exist “good” heuristics There exist “good” heuristics

48. Exercise Consider the problem of solving a cross-word puzzle initial state is an empty board with some possible blocked cells initial state is an empty board with some possible blocked cells a goal state is board configuration filled in with legal English words: a goal state is board configuration filled in with legal English words: How can this problem be viewed as a search problem? What are the operators? How can we measure path costs? Etc. How can this problem be viewed as a search problem? What are the operators? How can we measure path costs? Etc. Assuming we have dictionary of 100,000 words, what would be a good (uninformed) search strategy to use? Why? Assuming we have dictionary of 100,000 words, what would be a good (uninformed) search strategy to use? Why? What might be some good heuristics to use for this problem? What might be some good heuristics to use for this problem? How effective might hill-climbing strategies work in solving this problem? How can we handle the local minima problem? Propose a solution and discuss its effectiveness. How effective might hill-climbing strategies work in solving this problem? How can we handle the local minima problem? Propose a solution and discuss its effectiveness.