1. SEARCH Heng Ji jih@rpi.edu Feb 02/05, 2016

2. Search We will consider the problem of designing goal- based agents in fully observable, deterministic, discrete, known environments Example: Start state Goal state

3. Search We will consider the problem of designing goal- based agents in fully observable, deterministic, discrete, known environments The solution is a fixed sequence of actions Search is the process of looking for the sequence of actions that reaches the goal Once the agent begins executing the search solution, it can ignore its percepts (open-loop system)

4. Search problem components Initial state Actions Transition model What is the result of performing a given action in a given state? Goal state Path cost Assume that it is a sum of nonnegative step costs The optimal solution is the sequence of actions that gives the lowest path cost for reaching the goal Initial state Goal state

5. Example: Romania Initial state Arad Actions Go from one city to another Transition model If you go from city A to city B, you end up in city B Goal state Bucharest Path cost Sum of edge costs On vacation in Romania; currently in Arad Flight leaves tomorrow from Bucharest

6. State space The initial state, actions, and transition model define the state space of the problem The set of all states reachable from initial state by any sequence of actions Can be represented as a directed graph where the nodes are states and links between nodes are actions What is the state space for the Romania problem?

7. Search Given: Initial state Actions Transition model Goal state Path cost How do we find the optimal solution? How about building the state space and then using Dijkstra’s shortest path algorithm? The state space may be huge! Complexity of Dijkstra’s is O(E + V log V), where V is the size of the state space

8. Tree Search Let’s begin at the start node and expand it by making a list of all possible successor states Maintain a fringe or a list of unexpanded states At each step, pick a state from the fringe to expand Keep going until you reach the goal state Try to expand as few states as possible

9. Search tree “What if” tree of possible actions and outcomes The root node corresponds to the starting state The children of a node correspond to the successor states of that node’s state A path through the tree corresponds to a sequence of actions A solution is a path ending in the goal state Nodes vs. states A state is a representation of a physical configuration, while a node is a data structure that is part of the search tree ……… … Starting state Successo r state Action Goal state

10. Tree Search Algorithm Outline Initialize the fringe using the starting state While the fringe is not empty Choose a fringe node to expand according to search strategy If the node contains the goal state, return solution Else expand the node and add its children to the fringe

11. Tree search example

13. Fringe

14. Search strategies A search strategy is defined by picking the order of node expansion Strategies are evaluated along the following dimensions: Completeness: does it always find a solution if one exists? Optimality: does it always find a least-cost solution? Time complexity: number of nodes generated Space complexity: maximum number of nodes in memory Time and space complexity are measured in terms of b: maximum branching factor of the search tree d: depth of the least-cost solution m: maximum length of any path in the state space (may be infinite)

15. Uninformed search strategies Uninformed search strategies use only the information available in the problem definition Breadth-first search Uniform-cost search Depth-first search Iterative deepening search

16. Review: Search problem formulation Initial state Actions Transition model Goal state Path cost What is the optimal solution? What is the state space?

17. Review: Tree search Initialize the fringe using the starting state While the fringe is not empty Choose a fringe node to expand according to search strategy If the node contains the goal state, return solution Else expand the node and add its children to the fringe To handle repeated states: Keep an explored set; add each node to the explored set every time you expand it Every time you add a node to the fringe, check whether it already exists in the fringe with a higher path cost, and if yes, replace that node with the new one

18. Search strategies A search strategy is defined by picking the order of node expansion Strategies are evaluated along the following dimensions: Completeness: does it always find a solution if one exists? Optimality: does it always find a least-cost solution? Time complexity: number of nodes generated Space complexity: maximum number of nodes in memory Time and space complexity are measured in terms of b: maximum branching factor of the search tree d: depth of the optimal solution m: maximum length of any path in the state space (may be infinite)

19. Uninformed search strategies Uninformed search strategies use only the information available in the problem definition Breadth-first search Uniform-cost search Depth-first search Iterative deepening search

20. Search Algorithm Properties

21. Breadth-First Search

22. Breadth-first search Expand shallowest unexpanded node Implementation: fringe is a FIFO queue, i.e., new successors go at end A DF BC EG

23. Breadth-first search Expand shallowest unexpanded node Implementation: fringe is a FIFO queue, i.e., new successors go at end A DF BC EG

24. Breadth-first search Expand shallowest unexpanded node Implementation: fringe is a FIFO queue, i.e., new successors go at end A DF BC EG

25. Breadth-first search Expand shallowest unexpanded node Implementation: fringe is a FIFO queue, i.e., new successors go at end A DF BC EG

26. Breadth-first search Expand shallowest unexpanded node Implementation: fringe is a FIFO queue, i.e., new successors go at end A DF BC EG

27. Properties of breadth-first search Complete? Yes (if branching factor b is finite) Optimal? Yes – if cost = 1 per step Time? Number of nodes in a b-ary tree of depth d: O(b d ) (d is the depth of the optimal solution) Space? O(bd)O(bd) Space is the bigger problem (more than time)

28. Uniform-cost search Expand least-cost unexpanded node Implementation: fringe is a queue ordered by path cost (priority queue) Equivalent to breadth-first if step costs all equal Complete? Yes, if step cost is greater than some positive constant ε (we don’t want infinite sequences of steps that have a finite total cost) Optimal? Yes – nodes expanded in increasing order of path cost Time? Number of nodes with path cost ≤ cost of optimal solution (C*), O(b C*/ ε ) This can be greater than O(b d ): the search can explore long paths consisting of small steps before exploring shorter paths consisting of larger steps Space? O(b C*/ ε )

29. Depth-First Search

30. Depth-first search Expand deepest unexpanded node Implementation: fringe = LIFO queue, i.e., put successors at front A DF BC EG

31. Depth-first search Expand deepest unexpanded node Implementation: fringe = LIFO queue, i.e., put successors at front A DF BC EG

32. Depth-first search Expand deepest unexpanded node Implementation: fringe = LIFO queue, i.e., put successors at front A DF BC EG

33. Depth-first search Expand deepest unexpanded node Implementation: fringe = LIFO queue, i.e., put successors at front A DF BC EG

34. Depth-first search Expand deepest unexpanded node Implementation: fringe = LIFO queue, i.e., put successors at front A DF BC EG

35. Depth-first search Expand deepest unexpanded node Implementation: fringe = LIFO queue, i.e., put successors at front A DF BC EG

36. Depth-first search Expand deepest unexpanded node Implementation: fringe = LIFO queue, i.e., put successors at front A DF BC EG

37. Depth-first search Expand deepest unexpanded node Implementation: fringe = LIFO queue, i.e., put successors at front A DF BC EG

38. Depth-first search Expand deepest unexpanded node Implementation: fringe = LIFO queue, i.e., put successors at front A DF BC EG

39. Properties of depth-first search Complete? Fails in infinite-depth spaces, spaces with loops Modify to avoid repeated states along path  complete in finite spaces Optimal? No – returns the first solution it finds Time? Could be the time to reach a solution at maximum depth m: O(b m ) Terrible if m is much larger than d But if there are lots of solutions, may be much faster than BFS Space? O(bm), i.e., linear space!

40. Iterative deepening search Use DFS as a subroutine 1. Check the root 2. Do a DFS searching for a path of length 1 3. If there is no path of length 1, do a DFS searching for a path of length 2 4. If there is no path of length 2, do a DFS searching for a path of length 3…

41. Iterative deepening search

45. Properties of iterative deepening search Complete? Yes Optimal? Yes, if step cost = 1 Time? (d+1)b 0 + d b 1 + (d-1)b 2 + … + b d = O(b d ) Space? O(bd)

46. Informed search So far, have assumed that no nongoal state looks better than another Unrealistic Even without knowing the road structure, some locations seem closer to the goal than others Some states of the 8s puzzle seem closer to the goal than others Makes sense to expand closer-seeming nodes first

47. Informed search Idea: give the algorithm “hints” about the desirability of different states Use an evaluation function to rank nodes and select the most promising one for expansion Greedy best-first search A* search

48. Heuristic function Heuristic function h(n) estimates the cost of reaching goal from node n Example: Start state Goal state

49. Heuristics Key notion: heuristic function h(n) gives an estimate of the distance from n to the goal h(n)=0 for goal nodes E.g. straight-line distance for traveling problem A B C F D E 3 4 4 3 9 2 2 start state goal state Say: h(A) = 9, h(B) = 8, h(C) = 9, h(D) = 6, h(E) = 3, h(F) = 0 We’re adding something new to the problem! Can use heuristic to decide which nodes to expand first

50. Heuristic for the Romania problem

51. Greedy best-first search Expand the node that has the lowest value of the heuristic function h(n)

52. Greedy best-first search example

56. Properties of greedy best-first search Complete? No – can get stuck in loops start goal

57. Properties of greedy best-first search Complete? No – can get stuck in loops Optimal? No

58. Properties of greedy best-first search Complete? No – can get stuck in loops Optimal? No Time? Worst case: O(b m ) Best case: O(bd) – If h(n) is 100% accurate Space? Worst case: O(b m )

59. A bad example for greedy A B F D E 3 4 4 7 6 start state goal state Say: h(A) = 9, h(B) = 5, h(D) = 6, h(E) = 3, h(F) = 0 Problem: greedy evaluates the promise of a node only by how far is left to go, does not take cost occurred already into account

60. How can we fix the greedy problem?

61. A * search Idea: avoid expanding paths that are already expensive The evaluation function f(n) is the estimated total cost of the path through node n to the goal: f(n) = g(n) + h(n) g(n): cost so far to reach n (path cost) h(n): estimated cost from n to goal (heuristic)

62. Romania with step costs in km

63. A * search example Open List: Arad We start with our initial state Arad. We make a node and add it to the open list. Since it’s the only thing on the open list, we expand the node. Think of the open list as a priority queue (or heap) that sorts the nodes inside of it according to their g()+h() score.

64. A * search example Open List: Sibiu Timisoara Zerind We add the three nodes we found to the open list. We sort them according to the g()+h() calculation.

65. A * search example Open List: Rimricu Vicea Fagaras Timisoara Zerind Arad Oradea We’ve been to Arad before. Don’t list it again on the open list. When we expand Sibiu, we run into Arad again. But we’ve already expanded this node once; so, we don’t add it to the open list again.

66. A * search example Open List: Rimricu Vicea Fagaras Timisoara Zerind Oradea We see that Rimricu Vicea is at the top of the open list; so, it’s the next node we will expand.

67. A * search example Open List: Fagaras Pitesti Timisoara Zerind Craiova Sibiu Oradea When we expand Rimricu Vicea, we run into Sibiu again. But we’ve already expanded this node once; so, we don’t add it to the open list again.

68. A * search example Open List: Fagaras Pitesti Timisoara Zerind Craiova Oradea Fagaras will be the next node we should expand – it’s at the top of the sorted open list.

69. A * search example Open List: Pitesti Timisoara Zerind Bucharest Craiova Sibiu Oradea When we expand Fagaras, we find Sibiu again. We don’t add it to the open list. We also find Bucharest, but we’re not done. The algorithm doesn’t end until we “expand” the goal node – it has to be at the top of the open list.

70. A * search example Open List: Pitesti Timisoara Zerind Bucharest Craiova Oradea It looks like Pitesti is the next node we should expand.

71. A * search example Open List: Bucharest Timisoara Zerind Craiova Rimricu Vicea Oradea We just found a better value for Bucharest; so, it got moved higher in the list. We also found a worse value for Craiova – we just ignore this. And of course, we ran into Rimricu Vicea again. Since it’s already been expanded once, we don’t re-add it to the Open List.

72. A * search example Open List: Bucharest Timisoara Zerind Craiova Oradea Now it looks like Bucharest is at the top of the open list…

73. A * search example Open List: Bucharest Timisoara Zerind Craiova Oradea Now we “expand” the node for Bucharest. We’re done! (And we know the path that we’ve found is optimal.)

74. Admissible heuristics A heuristic h(n) is admissible if for every node n, h(n) ≤ h * (n), where h * (n) is the true cost to reach the goal state from n An admissible heuristic never overestimates the cost to reach the goal, i.e., it is optimistic Example: straight line distance never overestimates the actual road distance Theorem: If h(n) is admissible, A * is optimal

75. Optimality of A* Suppose A* terminates its search at n* It has found a path whose actual cost f(n*) = g(n*) is lower than the estimated cost f(n) of any path going through any fringe node Since f(n) is an optimistic estimate, there is no way n can have a successor goal state n’ with g(n’) < C* n n*n*f(n*) = C* f(n) > C* n' g(n')  f(n) > C*

76. Optimality of A* A* is optimally efficient – no other tree-based algorithm that uses the same heuristic can expand fewer nodes and still be guaranteed to find the optimal solution Any algorithm that does not expand all nodes with f(n) < C* risks missing the optimal solution

77. Properties of A* Complete? Yes – unless there are infinitely many nodes with f(n) ≤ C* Optimal? Yes Time? Number of nodes for which f(n) ≤ C* (exponential) Space? Exponential

78. Uninformed search strategies AlgorithmComplete?Optimal? Time complexity Space complexity BFS UCS DFS IDS b: maximum branching factor of the search tree d: depth of the optimal solution m: maximum length of any path in the state space C*: cost of optimal solution Yes No Yes If all step costs are equal Yes No O(b d ) O(b m ) O(b d ) O(bm) O(bd) Number of nodes with g(n) ≤ C*

79. All search strategies AlgorithmComplete?Optimal? Time complexity Space complexity BFS UCS DFS IDS Greedy A* Yes No Yes If all step costs are equal Yes No O(b d ) Number of nodes with g(n) ≤ C* O(b m ) O(b d ) O(bm) O(bd) No Worst case: O(b m ) Yes Best case: O(bd) Number of nodes with g(n)+h(n) ≤ C*

80. Exercise Trace from Lugoj to Bucharest

81. Search where the path doesn’t matter So far, looked at problems where the path was the solution Traveling on a graph Eights puzzle However, in many problems, we just want to find a goal state Doesn’t matter how we get there

82. Queens puzzle Q Q Q Q Q Q Q Q Place eight queens on a chessboard so that no two attack each other

83. Backtracking Backtracking is a technique used to solve problems with a large search space, by systematically trying and eliminating possibilities. A standard example of backtracking would be going through a maze. At some point in a maze, you might have two options of which direction to go: Junction Portion A Portion B

84. Backtracking Junction Portion B Portion A One strategy would be to try going through Portion A of the maze. If you get stuck before you find your way out, then you "backtrack" to the junction. At this point in time you know that Portion A will NOT lead you out of the maze, so you then start searching in Portion B

85. Backtracking Clearly, at a single junction you could have even more than 2 choices. The backtracking strategy says to try each choice, one after the other, if you ever get stuck, "backtrack" to the junction and try the next choice. If you try all choices and never found a way out, then there IS no solution to the maze. Junction B C A

86. N-queens problem Find a configuration of n queens not attacking each other

87. What is the maximum number of queens that can be placed on an n x n chessboard such that no two attack one another? The answer is n queens, which gives eight queens for the usual 8x8 board

88. Backtracking – Eight Queens Problem Find an arrangement of 8 queens on a single chess board such that no two queens are attacking one another. In chess, queens can move all the way down any row, column or diagonal (so long as no pieces are in the way). Due to the first two restrictions, it's clear that each row and column of the board will have exactly one queen.

89. 12 Unique Solutions 8-Queens Problem

90. 8-Queens Problem There are 92 distinct solutions There are 12 unique solutions discounting symmetrical answers (rotatations/reflections) How many solutions for 4-Queens? N-Queens? Wikipedia.org

91. Problems N < 4 3 2 1 N < 4 Cannot use N Queens

92. Backtracking in Decision Trees Idea: Start at the root of the decision tree and move downwards, that is, make a sequence of decisions, until you either reach a solution or you enter a situation from where no solution can be reached by any further sequence of decisions. In the latter case, backtrack to the parent of the current node and take a different path downwards from there. If all paths from this node have already been explored, backtrack to its parent. Continue this procedure until you find a solution or establish that no solution exists (there are no more paths to try out).

93. Solution: Backtracking place a queen in the top row, then note the column and diagonals it occupies. then place a queen in the next row down, taking care not to place it in the same column or diagonal. Keep track of the occupied columns and diagonals and move on to the next row. If no position is open in the next row, we back track to the previous row and move the queen over to the next available spot in its row and the process starts over again.

94. Backtracking in Decision Trees Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q place 1 st queen place 2 nd queen place 3 rd queen place 4 th queen empty board

95. The backtracking strategy is as follows: 1) Place a queen on the first available square in row 1. 2) Move onto the next row, placing a queen on the first available square there (that doesn't conflict with the previously placed queens). 3) Continue in this fashion until either: a) you have solved the problem, or b) you get stuck. When you get stuck, remove the queens that got you there, until you get to a row where there is another valid square to try. Backtracking – Eight Queens Problem Animated Example: http://www.hbmeyer.de/backtrac k/achtdamen/eight.htm#up Q Q Q Q Q Q Continue…

96. Exercise – now it’s your turn!

97. Backtracking – Eight Queens Problem The neat thing about coding up backtracking, is that it can be done recursively, without having to do all the bookkeeping at once. Instead, the stack or recursive calls does most of the bookkeeping (ie, keeping track of which queens we've placed, and which combinations we've tried so far, etc.)

98. void solveItRec(int perm[], int location, struct onesquare usedList[]) { if (location == SIZE) { printSol(perm); } for (int i=0; i<SIZE; i++) { if (usedList[i] == false) { if (!conflict(perm, location, i)) { perm[location] = i; usedList[i] = true; solveItRec(perm, location+1, usedList); usedList[i] = false; } perm[] - stores a valid permutation of queens from index 0 to location-1. location – the column we are placing the next queen usedList[] – keeps track of the rows in which the queens have already been placed. Found a solution to the problem, so print it! Loop through possible rows to place this queen. Only try this row if it hasn’t been used Check if this position conflicts with any previous queens on the diagonal 1)mark the queen in this row 2)mark the row as used 3)solve the next column location recursively 4)un-mark the row as used, so we can get ALL possible valid solutions.

99. Backtracking – 8 queens problem - Analysis Another possible brute-force algorithm is generate the permutations of the numbers 1 through 8 (of which there are 8! = 40,320), and uses the elements of each permutation as indices to place a queen on each row. Then it rejects those boards with diagonal attacking positions. The backtracking algorithm, is a slight improvement on the permutation method, constructs the search tree by considering one row of the board at a time, eliminating most non-solution board positions at a very early stage in their construction. Because it rejects row and diagonal attacks even on incomplete boards, it examines only 15,720 possible queen placements. A further improvement which examines only 5,508 possible queen placements is to combine the permutation based method with the early pruning method: The permutations are generated depth-first, and the search space is pruned if the partial permutation produces a diagonal attack

100. 100 Solving Sudoku – Later Steps 112124 1248 12489 uh oh!

101. 101 Sudoku – A Dead End We have reached a dead end in our search With the current set up none of the nine digits work in the top right corner 12489

102. CS314 102 Backing Up When the search reaches a dead end in backs up to the previous cell it was trying to fill and goes onto to the next digit We would back up to the cell with a 9 and that turns out to be a dead end as well so we back up again so the algorithm needs to remember what digit to try next Now in the cell with the 8. We try and 9 and move forward again. 12489 1249

103. Characteristics of Brute Force and Backtracking Brute force algorithms are slow The don't employ a lot of logic For example we know a 6 can't go in the last 3 columns of the first row, but the brute force algorithm will plow ahead any way But, brute force algorithms are fairly easy to implement as a first pass solution many backtracking algorithms are brute force algorithms

104. Key Insights After trying placing a digit in a cell we want to solve the new sudoku board Isn't that a smaller (or simpler version) of the same problem we started with?!?!?!? After placing a number in a cell the we need to remember the next number to try in case things don't work out. We need to know if things worked out (found a solution) or they didn't, and if they didn't try the next number If we try all numbers and none of them work in our cell we need to report back that things didn't work

105. Minesweeper CS314 Recursive Backtracking 105

106. Minesweeper Reveal Algorithm Minesweeper click a cell if bomb game over if cell that has 1 or more bombs on border then reveal the number of bombs that border cell if a cell that has 0 bombs on border then reveal that cell as a blank and click on the 8 surrounding cells

107. CS314 Recursive Backtracking 107 Another Backtracking Problem A Simple Maze Search maze until way out is found. If no way out possible report that.

108. Mazes and Backtracking A final example of something that can be solved using backtracking is a maze. From your start point, you will iterate through each possible starting move. From there, you recursively move forward. If you ever get stuck, the recursion takes you back to where you were, and you try the next possible move. In dealing with a maze, to make sure you don't try too many possibilities, one should mark which locations in the maze have been visited already so that no location in the maze gets visited twice. (If a place has already been visited, there is no point in trying to reach the end of the maze from there again.

109. 109 The Local View North East West Behind me, to the South is a door leading South Which way do I go to get out?

110. Modified Backtracking Algorithm for Maze If the current square is outside, return TRUE to indicate that a solution has been found. If the current square is marked, return FALSE to indicate that this path has been tried. Mark the current square. for (each of the four compass directions) {if ( this direction is not blocked by a wall ) {Move one step in the indicated direction from the current square. Try to solve the maze from there by making a recursive call. If this call shows the maze to be solvable, return TRUE to indicate that fact. } } Unmark the current square. Return FALSE to indicate that none of the four directions led to a solution.

111. CS314 Recursive Backtracking 111 Backtracking in Action The crucial part of the algorithm is the for loop that takes us through the alternatives from the current square. Here we have moved to the North. for (dir = North; dir <= West; dir++) {if (!WallExists(pt, dir)) {if (SolveMaze(AdjacentPoint(pt, dir))) return(TRUE); }

112. CS314 Recursive Backtracking 112 Backtracking in Action Here we have moved North again, but there is a wall to the North. East is also blocked, so we try South. That call discovers that the square is marked, so it just returns.

113. 113 So the next move we can make is West. Where is this leading?

114. CS314 Recursive Backtracking 114 This path reaches a dead end. Time to backtrack! Remember the program stack!

115. CS314 Recursive Backtracking 115 The recursive calls end and return until we find ourselves back here.

116. CS314 Recursive Backtracking 116 And now we try South

117. CS314 Recursive Backtracking 117 Path Eventually Found

118. 118 Other Backtracking Problems Knapsack problem / Exhaustive Search Filling a knapsack. Given a choice of items with various weights and a limited carrying capacity find the optimal load out. 50 lb. knapsack. items are 1 40 lb, 1 32 lb. 2 22 lbs, 1 15 lb, 1 5 lb. A greedy algorithm would choose the 40 lb item first. Then the 5 lb. Load out = 45lb. Exhaustive search 22 + 22 + 5 = 49.

119. Assignment 1