1. Automated Planning and Decision Making Prof. Ronen Brafman Automated Planning and Decision Making 20071 Search

2. Automated Planning and Decision Making2 Introduction  In the past, most problems you encountered were such that the way to solve them was very “ clear ” and straight forward: ○Solving quadratic equation. ○Solving linear equation. ○Matching input to a regular expression  Most of the problems in AI are of different kind. ○NP-Hard problems. ○Require searching in a large solution space.

3. Automated Planning and Decision Making3 Search Problem  A set S of possible states – the search space.  Initial State: s I in S.  Operators - which take us from one state to another (S→S functions)  Goal Conditions.

4. 4 The Search Space  A graph G(V,E) where: ○V={v : v in S} ○E={ : exists o in Operators such that o(v)=u} 4

5. 5 The Task  Find a path in the search space from s I to a state s where the Goal Conditions are valid. ○Sometimes we only care about s, and not the path ○Sometimes we only care whether s exists. ○Sometimes value is associated with states, and we want the best s. ○Sometimes cost is associated with operators, and we want the cheapest path

6. Automated Planning and Decision Making4 Search Problem Example  Finding a solution for Rubik's Cube.  S I : Some state of the cube.  Operators: 90 o rotation of any of the 9 plates.  Goal Conditions: same color for each of the cube's sides.  The Search Space will include: ○A node for any possible state of the cube. ○An edge between two nodes if you can reach from one to the other by a 90 o rotation of some plate.

7. Automated Planning and Decision Making5 Another Example  States: Locations of Tiles.  Operators: Move blank right/left/down/up.  Goal Conditions: As in the picture.  Note: optimal solution of n-Puzzle family is NP-hard… Initial StateGoal State

8. Example: n queens problem ( a constraint satisfaction problem ) 8 States: legal placements of k≤n queens Operators: add a queen Goal condition: n queens placed with no conflicts (no constraint violated)

9. Example: n queens problem An alternative formulation 9 States: placements of n queens – one per column Operators: move one of the queens Goal condition: n queens placed with no conflicts (no constraint violated)

10. Automated Planning and Decision Making6 Solving Search Problems  Apply the Operators on States in order to expand (produce) new States, until we find one that satisfies the Goal Conditions.  At any moment we may have different options to proceed (multiple Operators may apply)  Q: In what order should we expand the States?  Our choice affects: ○Computation time ○Space used ○Whether we reach an optimal solution ○Whether we are guaranteed to find a solution if one exists (completeness)

11. Automated Planning and Decision Making7 Search Tree  You can think of the search as spanning a tree: ○Root: The initial state. ○Children of node v are all states reachable from v by applying an operator.  We can reach the same state via different paths. ○In such a case, we can ignore this duplicate state, and treat it as a leaf node Requires that we remember all states we visited!  Important parameters affecting performance: ○b – Average branching factor. ○d – Depth of the closest solution. ○m – Maximal depth of the tree.  Fringe: set of current leaf (unexpanded) nodes

12. Automated Planning and Decision Making8 Search Methods  Different search methods differ in the order in which they visit/expand the nodes.  It is important to distinguish between: ○The search space ○The order by which we scan that space ○These are orthogonal issues!  Some algorithms are described abstractly by describing the space they generate w/o specifying how it is searched ○You can implement them in different ways by choosing different search algorithms  Blind Search ○No additional information about search states is used.  Informed Search ○Additional information used to improve search efficiency

13. Automated Planning and Decision Making9 Relation to Planning  Simplest (and currently most popular) way to solve planning problem is to search from the initial state to a goal  Search states are states of the world  Operators = actions  There are other formulations! ○For instance: search states = plans  Search is a very general technique – very important for many applications beyond planning

14. Automated Planning and Decision Making10 Blind Search

15. Automated Planning and Decision Making11 Blind Search  Main algorithms: ○DFS – Depth First Search Expand the deepest unexpanded node. Fringe is a LIFO. ○BFS – Breath First Search Expand the shallowest unexpanded node. Fringe is a FIFO. ○IDS – Iterative Deepening Search Combines the advantages of both methods. Avoids the disadvantages of each method.  There many are other variants (e.g., optimizing disk access, parallel search, etc.)

16. Automated Planning and Decision Making12 Breadth First Search 1 4 2 3

17. Automated Planning and Decision Making13 Properties of BFS  Complete? Yes (if b is finite).  Time? 1+b+b 2 +b 3 +…+b(b d -1)=O(b d+1 (  Space? O(b d ) (keeps every node on the fringe).  Optimal? Yes (if cost is 1 per step).  Space is a BIG problem…

18. 18

19. 19

20. Automated Planning and Decision Making14 Depth First Search 5 6 78 1091112 1234

21. Automated Planning and Decision Making15 Properties of DFS  Complete? ○No if given infinite branches (e.g., when we have loops) ○Easy to modify: check for repeat states along path Complete in finite spaces! May require large space to maintain list of visited states  Time? ○Worst case: O(b m ). ○Terrible if m is much larger then d. ○But if solutions occur often or in the “ left ” part of the tree, may be much faster then BFS.  Space? ○O(m) Linear Space!  Optimal? ○No.

22. Automated Planning and Decision Making16 Depth Limited Search  DFS with a depth limit ℓ, i.e., nodes at depth ℓ have no successors.

23. Automated Planning and Decision Making17 Iterative Deepening Search  Increase the depth limit of the DLS with each Iteration:

24. Iterative Deepening Search 24

25. Automated Planning and Decision Making19 Complexity of IDS  Number of nodes generated in a depth-limited search to depth d with branching factor b: N DLS = b 0 + b 1 + b 2 + … + b d-2 + b d-1 + b d  Number of nodes generated in an iterative deepening search to depth d with branching factor b: N IDS = (d+1)b 0 + db 1 + (d-1)b 2 + … + 3b d-2 +2b d-1 + 1b d  For b = 10, d = 5: ○N DLS = 1 + 10 + 100 + 1,000 + 10,000 + 100,000 = 111,111 ○N IDS = 6 + 50 + 400 + 3,000 + 20,000 + 100,000 = 123,456  Overhead = (123,456 - 111,111)/111,111 = 11%

26. Automated Planning and Decision Making20 Properties of IDS  Complete? Yes.  Time? (d+1)b 0 + db 1 + (d-1)b 2 + … + b d = O(b d )  Space? O(d)  Optimal? Yes. If step cost is 1.

27. Automated Planning and Decision Making21 Comparison of the Algorithms Algorithm Criterion BFSUniform CostDFSDLSIDS Complete Yes No Yes Time O(b d+1 )O(b Ciel(C*/ε) )O(b m )O(b l )O(b d ) Space O(b d+1 )O(b Ciel(C*/ε) )O(m)O(l)O(d) Optimal Yes No Yes

28. 28 General Search Scheme  Solve(Nodes) ○if empty Nodes -> return Failure ○else let Node = Select-Node(Nodes) let Rest = Nodes - Node ○if Node is Goal -> return Solution ○else let Children = Expand-Node(Node) let New-Nodes = Add-Nodes(Children, Nodes) Solve(New-Nodes)  Different algorithms obtained by suitable instantiation of Select-Node and Add-Nodes  Nodes are data structures that contain state and bookkeeping info  Initially Nodes = {root}

29. 29 Some Blind Instances of GSS  DFS expands “deepest” node first ○Select-Node : select first Node in Nodes ○Add-Nodes : Puts Children before Nodes ○Implementation: Nodes is a stack (LIFO)  Breadth-First Search expands ’ shallowest ’ nodes first ○Select-Node : select first Node in Nodes ○Add-Nodes : Puts Children after Nodes ○Implementation: Nodes is a queue (FIFO)

30. 30 Bounded GSS  Solve(Nodes,Bound) ○if empty Nodes -> return Failure ○else let Node = Select-Node(Nodes) let Rest = Nodes - Node if f(Node) > Bound –Solve(Rest, Bound) ;;; PRUNE Node else if Node is Goal -> return ProcessSolution(Node,Rest) –else  let Children = Expand-Node(Node)  let New-Nodes = Add-Nodes(Children, Nodes)  Solve(New-Nodes,Bound)  IDS calls Bounded GSS with bound 1,2,3,...  ProcessSolution(Node,Rest) returns Node as solution

31. Automated Planning and Decision Making22 Bidirectional Search  Assumptions: ○There is a small number of states satisfying the goal conditions. ○It is possible to reverse the operators.  Simultaneously run BFS from both directions.  Stop when reaching a state from both directions.  Under the assumption that validating this overlap takes constant time, we get: ○Time: O(b d/2 ) ○Space: O(b d/2 )

32. Automated Planning and Decision Making23 Informed Search

33. Automated Planning and Decision Making24 Informed/Heuristic Search  Heuristic Function h:S→R ○For every state s, h(s) is an estimation of the minimal distance/cost from s to a solution. Distance is only one way to set a price. How to produce h? later on…  Cost Function g:S→R ○For every state s, g(s) is the minimal cost to s from the initial state.  f=g+h, is an estimation of the cost from the initial state to a solution.

34. Properties of Heuristics  h is perfect if h(s) = shortest distance to goal from s (and ∞ if goal is unreachable from s)  The perfect heuristic is denoted by h*  h is safe if h(s)= ∞ implies h*(s)= ∞  h is goal-aware if h(s)=0 for every goal state s  h is admissible if h(s) ≤ h*(s) for all s  h is consistent if h(s) ≤ h(s’)+1 whenever s’ is a child of s 34

35. Automated Planning and Decision Making25 Best First Search  Greedy on h values.  Fringe stored in a queue ordered by h values.  In every step, expand the “ best ” node so far, i.e., the one with the best h value.

36. 36

37. Reaching Bucharest with BFS 37

38. 38

39. Automated Planning and Decision Making28 Properties of Best First  Complete? ○No, can get into loops ○Yes, with duplicate detection and safe h  Time? ○O(b m ). But a good heuristic can give dramatic improvement.  Space? ○O(b m ). Keeps all nodes in memory.  Optimal ○No.

40. Automated Planning and Decision Making29 A* Search  Idea: Avoid expanding paths that are already expensive.  Greedy on f values.  Fringe is stored in a queue ordered by f values.  Recall, f(n)=g(n)+h(n), where: ○g(n): cost so far to reach n. ○h(n): estimated cost from n to goal. ○f(n): estimated total cost of path through n to goal.

41. 41 Slides 5-12 41

42. Bucharest via A* 42

43. Bucharest via A* 43

44. Automated Planning and Decision Making32 Admissible Heuristics  A heuristic h(n) is admissible if for every node n, h(n) ≤ h * (n), where h * (n) is the real cost to reach the goal state from n.  An admissible heuristic never overestimates the cost to reach the goal, i.e., it is optimistic  Theorem: If h(n) is admissible, A * using Tree-Search is optimal. ○Tree search – nodes encountered more than once are not treated as leaf nodes

45. Automated Planning and Decision Making33 Optimality of A* (Proof)  Suppose some suboptimal goal G 2 has been generated and is in the fringe. Let n be an unexpanded node in the fringe such that n is on a shortest path to an optimal goal G.

46. Automated Planning and Decision Making34 Optimality of A* (Proof) 1. f(G 2 ) = g(G 2 )since h(G 2 ) = 0. 2. f(G) = g(G)since h(G) = 0. 3. g(G 2 ) > g(G)since G 2 is suboptimal. 4. f(G 2 ) > f(G)by 1,2,3. 5. h(n) ≤ h * (n)since h is admissible. 6. g(n)+h(n) ≤ g(n)+h * (n)by 5. 7. f(n) ≤ f(G)by definition of f. 8. f(G 2 ) > f(n)by 4,7. Hence A * will never select G 2 for expansion.

47. 47

48. Automated Planning and Decision Making36 Optimality of A*  A * expands nodes in order of increasing f value.  Gradually adds "f-contours" of nodes.  Contour i has all nodes with f=f i, where f i < f i+1.

49. Automated Planning and Decision Making37 Properties of A*  Complete? ○Yes, unless there are infinitely many nodes with f ≤ f(G).  Time? ○Exponential.  Optimal? ○Yes.  Space? ○Keeps all nodes in memory. ○A serious problem in many cases! How can we overcome it?

50. Automated Planning and Decision Making38 Iterative Deepening A* (IDA*)  Combine Iterative Deepening with A* in order to overcome the space problem.  A simple idea: ○Replace upper bound on depth with upper bound on f values. ○If f(n)>U, n is not enqueued. ○If no solution is found given U, increase U.  Another method -- Weighted A* (soon)

51. Automated Planning and Decision Making39 Branch & Bound  Used for finding an optimal solution (i.e., when there are multiple possible solutions, but some are better than others)  Attempts to prune entire sub-trees  Applicable in the context of different search methods (BFS, DFS, …).  Assumes we can generate upper bound U and lower bound L on the value of solutions reachable from n  If for some node n there exists a node n ’ such that U(n)<I(n ’ ), we can prune n.  Often used as follows: ○First, quickly find any solution with value v ’ ○Prune any node n s.t. f(n) > v ’

52. Automated Planning and Decision Making40 Local search algorithms  In many problems, the path to the goal is irrelevant; the goal state itself is all we care about.  Example: n-queens problems  More generally: constraint-satisfaction problems  Local search algorithms keep a single "current" state, and try to improve it.

53. Automated Planning and Decision Making41 Hill-climbing search  “ Like climbing mount Everest in thick fog with amnesia… ”  Improve while you can, i.e. stop when reaching a maxima (minima)  Many variants: tie-breaking rules, restarts

54. Automated Planning and Decision Making42 Hill-climbing search  Problem: depending on initial state, can get stuck in local maxima.

55. Automated Planning and Decision Making44 Solution by Hill-climbing  h = number of pairs of queens that are attacking each other, either directly or indirectly.  Moves: change position of a single queen  On A, h=17.  On B, h=1. local minimum. B A

56. 56

57. 57

58. Automated Planning and Decision Making45 Simulated annealing search  Idea: escape local maxima by allowing some "bad" moves, but gradually decrease their frequency.

59. Automated Planning and Decision Making46 Properties of SAS  One can prove: If T decreases slowly enough, then SAS will find a global optimum with probability approaching 1.  Widely used in VLSI layout, airline scheduling, etc.

60. Automated Planning and Decision Making47 Local beam search  Keep track of k states rather than just one.  Start with k randomly generated states.  At each iteration, all the successors of all k states are generated.  If any one is a goal state, stop; else select the k best successors from the complete list and repeat.

61. 61 Weighted A*  Makes A* more greedy  f(n) = g(n) + W*h(n) for W>1  Gives more weight to the heuristic value, i.e. to getting closer to the goal  Solution is at most factor W optimal  Usually reaches the goal faster and requires less memory, but IDA* uses less memory.

62. 62

63. 63

64. Automated Planning and Decision Making48 Generating a Heuristic Function  Heuristic functions are usually obtained by finding a simple approximation to the problem  Simple heuristics often work well (but not very well)  Examples: ○N-puzzle: Manhattan distance Simplification: assumes that we can move each tile independently. Ignores interactions between tiles A very general idea – ignore certain interactions