1. CS B551: E LEMENT SOF A RTIFICIAL I NTELLIGENCE Instructor: Kris Hauser http://cs.indiana.edu/~hauserk 1

2. R ECAP Blind Search Breadth first (complete, optimal) Depth first (incomplete, not optimal) Iterative deepening (complete, optimal) Nonuniform costs Revisited states 2

3. T HREE S EARCH A LGORITHMS Search #1: unit costs With or without detecting revisited states Search #2: non-unit costs, not detecting revisited states Goal test when node is expanded, not generated Search #3: non-unit costs, detecting revisited states When two nodes in fringe share same state, the one with the lowest path cost is kept 3

4. H EURISTIC S EARCH 4

5. B LIND VS. H EURISTIC S TRATEGIES Blind (or un-informed) strategies do not exploit state descriptions to order FRINGE. They only exploit the positions of the nodes in the search tree Heuristic (or informed) strategies exploit state descriptions to order FRINGE (the most promising nodes are placed at the beginning of FRINGE) 5

6. E XAMPLE 6 For a blind strategy, N 1 and N 2 are just two nodes (at some position in the search tree) Goal state N1N1 N2N2 STATE 1 2 34 56 7 8 123 45 678 123 456 78

7. E XAMPLE 7 For a heuristic strategy counting the number of misplaced tiles, N 2 is more promising than N 1 Goal state N1N1 N2N2 STATE 1 2 34 56 7 8 123 45 678 123 456 78

8. B EST -F IRST S EARCH It exploits state description to estimate how good each search node is An evaluation function f maps each node N of the search tree to a real number f(N) 0 [Traditionally, f(N) is an estimated cost; so, the smaller f(N), the more promising N] Best-first search sorts FRINGE in increasing f [Arbitrary order is assumed among nodes with equal f] 8

9. B EST -F IRST S EARCH It exploits state description to estimate how good each search node is An evaluation function f maps each node N of the search tree to a real number f(N) 0 [Traditionally, f(N) is an estimated cost; so, the smaller f(N), the more promising N] Best-first search sorts FRINGE in increasing f [Arbitrary order is assumed among nodes with equal f] 9 Best does not refer to the quality of the generated path Best-first search does not generate optimal paths in general Best does not refer to the quality of the generated path Best-first search does not generate optimal paths in general

10. H OW TO CONSTRUCT F ? Typically, f(N) estimates: either the cost of a solution path through N Then f(N) = g(N) + h(N), where g(N) is the cost of the path from the initial node to N h(N) is an estimate of the cost of a path from N to a goal node or the cost of a path from N to a goal node Then f(N) = h(N) Greedy best-first-search But there are no limitations on f. Any function of your choice is acceptable. But will it help the search algorithm? 10

11. H OW TO CONSTRUCT F ? Typically, f(N) estimates: either the cost of a solution path through N Then f(N) = g(N) + h(N), where g(N) is the cost of the path from the initial node to N h(N) is an estimate of the cost of a path from N to a goal node or the cost of a path from N to a goal node Then f(N) = h(N) Greedy best-first-search But there are no limitations on f. Any function of your choice is acceptable. But will it help the search algorithm? 11 Heuristic function

12. H EURISTIC F UNCTION The heuristic function h(N) 0 estimates the cost to go from STATE(N) to a goal state Its value is independent of the current search tree; it depends only on STATE(N) and the goal test GOAL? Example: h 1 (N) = number of misplaced numbered tiles = 6 [Why is it an estimate of the distance to the goal?] 12 14 7 5 2 63 8 STATE (N) 64 7 1 5 2 8 3 Goal state

13. O THER E XAMPLES h 1 (N) = number of misplaced numbered tiles = 6 h 2 (N) = sum of the (Manhattan) distance of every numbered tile to its goal position = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 13 h 3 (N) = sum of permutation inversions = n 5 + n 8 + n 4 + n 2 + n 1 + n 7 + n 3 + n 6 = 4 + 6 + 3 + 1 + 0 + 2 + 0 + 0 = 16 13 14 7 5 2 63 8 STATE (N) 64 7 1 5 2 8 3 Goal state

14. 8-P UZZLE 14 45 5 3 3 4 34 4 212 0 3 4 3 f(N) = h(N) = number of misplaced numbered tiles The white tile is the empty tile

15. 8-P UZZLE 15 0+41+5 1+3 3+3 3+4 3+24+15+2 5+0 2+3 2+4 2+3 f(N) = g(N) + h(N) with h(N) = number of misplaced numbered tiles

16. 8-P UZZLE 16 2 0 f(N) = h(N) = distances of numbered tiles to their goals 56 6 4 4 21 5 5 3

17. R OBOT N AVIGATION 17 xNxN yNyN N xgxg ygyg (L 2 or Euclidean distance) h 2 (N) = |x N -x g | + |y N -y g | (L 1 or Manhattan distance)

18. B EST -F IRST E FFICIENCY 18 f(N) = h(N) = straight distance to the goal Local-minimum problem

19. A DMISSIBLE H EURISTIC Let h * (N) be the cost of the optimal path from N to a goal node The heuristic function h(N) is admissible if: 0 h(N) h * (N) An admissible heuristic function is always optimistic !

20. A DMISSIBLE H EURISTIC Let h * (N) be the cost of the optimal path from N to a goal node The heuristic function h(N) is admissible if: 0 h(N) h * (N) An admissible heuristic function is always optimistic ! 20 G is a goal node h(G) = 0

21. 8-P UZZLE H EURISTICS h 1 (N) = number of misplaced tiles = 6 is ??? 21 14 7 5 2 63 8 STATE (N) 64 7 1 5 2 8 3 Goal state

22. 8-P UZZLE H EURISTICS h 1 (N) = number of misplaced tiles = 6 is admissible h 2 (N) = sum of the (Manhattan) distances of every tile to its goal position = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 13 is ??? 22 14 7 5 2 63 8 STATE (N) 64 7 1 5 2 8 3 Goal state

23. 8-P UZZLE H EURISTICS h 1 (N) = number of misplaced tiles = 6 is admissible h 2 (N) = sum of the (Manhattan) distances of every tile to its goal position = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 13 is admissible h 3 (N) = sum of permutation inversions = 4 + 6 + 3 + 1 + 0 + 2 + 0 + 0 = 16 is ??? 23 14 7 5 2 63 8 STATE (N) 64 7 1 5 2 8 3 Goal state

24. 8-P UZZLE H EURISTICS h 1 (N) = number of misplaced tiles = 6 is admissible h 2 (N) = sum of the (Manhattan) distances of every tile to its goal position = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1 = 13 is admissible h 3 (N) = sum of permutation inversions = 4 + 6 + 3 + 1 + 0 + 2 + 0 + 0 = 16 is not admissible 24 14 7 5 2 63 8 STATE (N) 64 7 1 5 2 8 3 Goal state

25. R OBOT N AVIGATION H EURISTICS 25 Cost of one horizontal/vertical step = 1 Cost of one diagonal step = is admissible

26. R OBOT N AVIGATION H EURISTICS 26 Cost of one horizontal/vertical step = 1 Cost of one diagonal step = h 2 (N) = |x N -x g | + |y N -y g | is ???

27. R OBOT N AVIGATION H EURISTICS 27 Cost of one horizontal/vertical step = 1 Cost of one diagonal step = h 2 (N) = |x N -x g | + |y N -y g | is admissible if moving along diagonals is not allowed, and not admissible otherwise h * (I) = 4 2 h 2 (I) = 8

28. H OW TO CREATE AN ADMISSIBLE H ? An admissible heuristic can usually be seen as the cost of an optimal solution to a relaxed problem (one obtained by removing constraints) In robot navigation: The Manhattan distance corresponds to removing the obstacles The Euclidean distance corresponds to removing both the obstacles and the constraint that the robot moves on a grid More on this topic later 28

29. A* S EARCH ( MOST POPULAR ALGORITHM IN AI) f(N) = g(N) + h(N), where: g(N) = cost of best path found so far to N h(N) = admissible heuristic function for all arcs: c(N,N) > 0 SEARCH#2 algorithm is used Best-first search is then called A* search 29

30. 8-P UZZLE 30 0+41+5 1+3 3+3 3+4 3+24+15+2 5+0 2+3 2+4 2+3 f(N) = g(N) + h(N) with h(N) = number of misplaced numbered tiles

31. R OBOT N AVIGATION 31

32. R OBOT N AVIGATION 32 0211 587 7 3 4 7 6 7 632 8 6 45 233 36524435 546 5 6 4 5 f(N) = h(N), with h(N) = Manhattan distance to the goal (not A*)

33. R OBOT N AVIGATION 33 0211 587 7 3 4 7 6 7 632 8 6 45 233 36524435 546 5 6 4 5 f(N) = h(N), with h(N) = Manhattan distance to the goal (not A*) 7 0

34. R OBOT N AVIGATION 34 f(N) = g(N)+h(N), with h(N) = Manhattan distance to goal (A*) 0211 587 7 3 4 7 6 7 632 8 6 45 233 36524435 546 5 6 4 5 7+0 6+1 8+1 7+0 7+2 6+17+2 6+1 8+1 7+2 8+37+26+3 5+4 4+5 3+6 2+78+37+4 6+5 5+6 6+35+62+73+8 4+7 5+64+7 3+8 4+73+8 2+9 3+10 2+9 3+82+9 1+10 0+11

35. R ESULT #1 A* is complete and optimal [This result holds if nodes revisiting states are not discarded] 35

36. P ROOF (1/2) If a solution exists, A* terminates and returns a solution 36 - For each node N on the fringe, f(N) = g(N)+h(N) g(N) d(N), where d(N) is the depth of N in the tree

37. P ROOF (1/2) If a solution exists, A* terminates and returns a solution 37 - For each node N on the fringe, f(N) = g(N)+h(N) g(N) d(N), where d(N) is the depth of N in the tree - As long as A* hasnt terminated, a node K on the fringe lies on a solution path K

38. P ROOF (1/2) If a solution exists, A* terminates and returns a solution 38 - For each node N on the fringe, f(N) = g(N)+h(N) g(N) d(N), where d(N) is the depth of N in the tree - As long as A* hasnt terminated, a node K on the fringe lies on a solution path - Since each node expansion increases the length of one path, K will eventually be selected for expansion, unless a solution is found along another path K

39. P ROOF (2/2) Whenever A* chooses to expand a goal node, the path to this node is optimal 39 K - C*= cost of the optimal solution path - G: non-optimal goal node in the fringe f(G) = g(G) + h(G) = g(G) C* - A node K in the fringe lies on an optimal path: f(K) = g(K) + h(K) C* - So, G will not be selected for expansion G

40. C OMPLEXITY OF A* A* expands all nodes with f(N) < C* A* may expand non-solution nodes with f(N) = C* May be an exponential number of nodes unless the heuristic is sufficiently accurate Within O(log h*(N)) of h*(N) When a problem has no solution, A* runs forever if the state space is infinite or if states can be revisited an arbitrary number of times. In other cases, it may take a huge amount of time to terminate 40

41. W HAT TO DO WITH REVISITED STATES ? 41 c = 1 100 2 1 2 h = 100 0 90 1 The heuristic h is clearly admissible

42. W HAT TO DO WITH REVISITED STATES ? 42 c = 1 100 2 1 2 h = 100 0 90 1104 4+90 f = 1+100 2+1 ? If we discard this new node, then the search algorithm expands the goal node next and returns a non-optimal solution

43. It is not harmful to discard a node revisiting a state if the cost of the new path to this state is cost of the previous path [so, in particular, one can discard a node if it re-visits a state already visited by one of its ancestors] A* remains optimal, but states can still be re- visited multiple times [the size of the search tree can still be exponential in the number of visited states] Fortunately, for a large family of admissible heuristics – consistent heuristics – there is a much more efficient way to handle revisited states 43

44. C ONSISTENT H EURISTIC An admissible heuristic h is consistent (or monotone) if for each node N and each child N of N: 44 (triangle inequality) N N h(N) c(N,N) Intuition: a consistent heuristics becomes more precise as we get deeper in the search tree h(N) c(N,N) + h(N)

45. C ONSISTENCY V IOLATION 45 N N h(N) =100 h(N) =10 c(N,N) =10 (triangle inequality) If h tells that N is 100 units from the goal, then moving from N along an arc costing 10 units should not lead to a node N that h estimates to be 10 units away from the goal

46. C ONSISTENT H EURISTIC ( ALTERNATIVE DEFINITION ) A heuristic h is consistent (or monotone) if 1. for each node N and each child N of N: h(N) c(N,N) + h(N) 2. for each goal node G: h(G) = 0 46 (triangle inequality) N N h(N) c(N,N) A consistent heuristic is also admissible

47. A DMISSIBILITY AND C ONSISTENCY A consistent heuristic is also admissible An admissible heuristic may not be consistent, but many admissible heuristics are consistent 47

48. 8-P UZZLE 48 123 456 78 12 3 4 5 67 8 STATE (N) goal h 1 (N) = number of misplaced tiles h 2 (N) = sum of the (Manhattan) distances of every tile to its goal position are both consistent (why?) N N h(N) c(N,N) h(N) c(N,N) + h(N)

49. R OBOT N AVIGATION 49 Cost of one horizontal/vertical step = 1 Cost of one diagonal step = h 2 (N) = |x N -x g | + |y N -y g | is consistent is consistent if moving along diagonals is not allowed, and not consistent otherwise N N h(N) c(N,N) h(N) c(N,N) + h(N)

50. R ESULT #2 If h is consistent, then whenever A* expands a node, it has already found an optimal path to this nodes state 50

51. P ROOF (1/2) 1. Consider a node N and its child N Since h is consistent: h(N) c(N,N)+h(N) f(N) = g(N)+h(N) g(N)+c(N,N)+h(N) = f(N) So, f is non-decreasing along any path 51 N N

52. P ROOF (2/2) 2. If a node K is selected for expansion, then any other node N in the fringe verifies f(N) f(K) If one node N lies on another path to the state of K, the cost of this other path is no smaller than that of the path to K: f(N) f(N) f(K) and h(N) = h(K) So, g(N) g(K) 52 KN N S

53. P ROOF (2/2) 2. If a node K is selected for expansion, then any other node N in the fringe verifies f(N) f(K) If one node N lies on another path to the state of K, the cost of this other path is no smaller than that of the path to K: f(N) f(N) f(K) and h(N) = h(K) So, g(N) g(K) 53 KN N S If h is consistent, then whenever A* expands a node, it has already found an optimal path to this nodes state Result #2

54. I MPLICATION OF R ESULT #2 54 NN1N1 SS1S1 The path to N is the optimal path to S N2N2 N 2 can be discarded

55. R EVISITED S TATES WITH C ONSISTENT H EURISTIC (S EARCH #3) When a node is expanded, store its state into CLOSED When a new node N is generated: If STATE(N) is in CLOSED, discard N If there exists a node N in the fringe such that STATE(N) = STATE(N), discard the node – N or N – with the largest f (or, equivalently, g) 55

56. A* IS OPTIMAL IF h admissible (but not necessarily consistent) Revisited states not detected Search #2 is used h is consistent Revisited states detected Search #3 is used 56

57. H EURISTIC A CCURACY Let h 1 and h 2 be two consistent heuristics such that for all nodes N: h 1 (N) h 2 (N) h 2 is said to be more accurate (or more informed) than h 1 57 h 1 (N) = number of misplaced tiles h 2 (N) = sum of distances of every tile to its goal position h 2 is more accurate than h 1 14 7 5 2 63 8 STATE (N) 64 7 1 5 2 8 3 Goal state

58. R ESULT #3 Let h 2 be more accurate than h 1 Let A 1 * be A* using h 1 and A 2 * be A* using h 2 Whenever a solution exists, all the nodes expanded by A 2 *, except possibly for some nodes such that f 1 (N) = f 2 (N) = C* (cost of optimal solution) are also expanded by A 1 * 58

59. P ROOF C* = h*(initial-node) [cost of optimal solution] Every node N such that f(N) C* is eventually expanded. No node N such that f(N) > C* is ever expanded Every node N such that h(N) C* g(N) is eventually expanded. So, every node N such that h 2 (N) C* g(N) is expanded by A 2 *. Since h 1 (N) h 2 (N), N is also expanded by A 1 * If there are several nodes N such that f 1 (N) = f 2 (N) = C* (such nodes include the optimal goal nodes, if there exists a solution), A 1 * and A 2 * may or may not expand them in the same order (until one goal node is expanded) 59

60. H OW TO CREATE GOOD HEURISTICS ? By solving relaxed problems at each node In the 8-puzzle, the sum of the distances of each tile to its goal position (h 2 ) corresponds to solving 8 simple problems: It ignores negative interactions among tiles 60 14 7 5 2 63 8 64 7 1 5 2 8 3 5 5 d i is the length of the shortest path to move tile i to its goal position, ignoring the other tiles, e.g., d 5 = 2 h 2 = i=1,...8 d i

61. C AN WE DO BETTER ? For example, we could consider two more complex relaxed problems: h = d1234 + d5678 [disjoint pattern heuristic] 61 14 7 5 2 63 8 64 7 1 5 2 8 3 3 214 4 123 d 1234 = length of the shortest path to move tiles 1, 2, 3, and 4 to their goal positions, ignoring the other tiles 6 7 5 87 5 6 8 d 5678

62. C AN WE DO BETTER ? For example, we could consider two more complex relaxed problems: h = d1234 + d5678 [disjoint pattern heuristic] How to compute d 1234 and d 5678 ? 62 14 7 5 2 63 8 64 7 1 5 2 8 3 3 214 4 123 d 1234 = length of the shortest path to move tiles 1, 2, 3, and 4 to their goal positions, ignoring the other tiles 6 7 5 87 5 6 8 d 5678

63. C AN WE DO BETTER ? For example, we could consider two more complex relaxed problems: h = d1234 + d5678 [disjoint pattern heuristic] These distances are pre-computed and stored [Each requires generating a tree of 3,024 nodes/states (breadth-first search)] 63 14 7 5 2 63 8 64 7 1 5 2 8 3 3 214 4 123 d 1234 = length of the shortest path to move tiles 1, 2, 3, and 4 to their goal positions, ignoring the other tiles 6 7 5 87 5 6 8 d 5678

64. C AN WE DO BETTER ? For example, we could consider two more complex relaxed problems: h = d1234 + d5678 [disjoint pattern heuristic] These distances are pre-computed and stored [Each requires generating a tree of 3,024 nodes/states (breadth-first search)] 64 14 7 5 2 63 8 64 7 1 5 2 8 3 3 214 4 123 d 1234 = length of the shortest path to move tiles 1, 2, 3, and 4 to their goal positions, ignoring the other tiles 6 7 5 87 5 6 8 d 5678 Several order-of-magnitude speedups for the 15- and 24-puzzle (see R&N)

65. I TERATIVE D EEPENING A* (IDA*) Idea: Reduce memory requirement of A* by applying cutoff on values of f Consistent heuristic function h Algorithm IDA*: Initialize cutoff to f(initial-node) Repeat: Perform depth-first search by expanding all nodes N such that f(N) cutoff Reset cutoff to smallest value f of non-expanded (leaf) nodes 65

66. 8-P UZZLE 66 4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles Cutoff=4

67. 8-P UZZLE 67 4 4 6 Cutoff=4 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

68. 8-P UZZLE 68 4 4 6 Cutoff=4 6 5 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

69. 8-P UZZLE 69 4 4 6 Cutoff=4 6 5 5 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

70. 8-P UZZLE 70 4 4 6 Cutoff=4 6 5 56 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

71. 8-P UZZLE 71 4 6 Cutoff=5 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

72. 8-P UZZLE 72 4 4 6 Cutoff=5 6 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

73. 8-P UZZLE 73 4 4 6 Cutoff=5 6 5 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

74. 8-P UZZLE 74 4 4 6 Cutoff=5 6 57 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

75. 8-P UZZLE 75 4 4 6 Cutoff=5 6 5 7 5 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

76. 8-P UZZLE 76 4 4 6 Cutoff=5 6 5 7 55 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles

77. 8-P UZZLE 77 4 4 6 Cutoff=5 6 5 7 55 f(N) = g(N) + h(N) with h(N) = number of misplaced tiles 5

78. A DVANTAGES /D RAWBACKS OF IDA* Advantages: Still complete and optimal Requires less memory than A* Avoid the overhead to sort the fringe Drawbacks: Cant avoid revisiting states not on the current path Available memory is poorly used Non-unit costs? 78

79. M EMORY -B OUNDED S EARCH Proceed like A* until memory is full No more nodes can be added to search tree Drop node in fringe with highest f(N) Place parent back in fringe with backed-up f(P) min(f(P),f(N)) Extreme example: RBFS Only keeps nodes in path to current node 79

80. R ECAP Proving properties of A* performance Admissible heuristics: optimality Consistent heuristics: revisited states 80

81. N EXT C LASS Beyond classical search R&N 4.1-5, 6.1-3 81

82. E FFECTIVE B RANCHING F ACTOR It is used as a measure the effectiveness of a heuristic Let n be the total number of nodes expanded by A* for a particular problem and d the depth of the solution The effective branching factor b * is defined by n = 1 + b * + ( b * ) 2 +...+ ( b * ) d 82

83. E XPERIMENTAL R ESULTS ( SEE R&N FOR DETAILS ) 8-puzzle with: h 1 = number of misplaced tiles h 2 = sum of distances of tiles to their goal positions Random generation of many problem instances Average effective branching factors (number of expanded nodes): 83 dIDSA1*A1*A2*A2* 22.451.79 62.731.341.30 122.78 (3,644,035)1.42 (227)1.24 (73) 16--1.451.25 20--1.471.27 24--1.48 (39,135)1.26 (1,641)