1. Quiz 1 : Queue based search  q1a1: Any complete search algorithm must have exponential (or worse) time complexity.  q1a2: DFS requires more space than BFS.  q1a3: BFS always returns the optimal cost path.  q1a4: BFS uses a FIFO queue as fringe.  q1a5: UCS uses a LIFO queue as fringe.  q1a6: BFS has the best time complexity out of UCS, BFS, DFS, ID.  Yes  No  Yes  No

2. CSE511a: Artificial Intelligence Spring 2013 Lecture 3: A* Search 1/24/2012 Robert Pless – Wash U. Multiple slides over the course adapted from Kilian Weinberger, originally by Dan Klein (or Stuart Russell or Andrew Moore)

3. Announcements  Projects:  Project 0 due tomorrow night.  Project 1 (Search) is out soon, will be due Thursday 2/7.  Find project partner  Try pair programming, not divide-and-conquer  Exercise more and try to eat more fruit.

4. Today  A* Search  Heuristic Design

5. Recap: Search  Search problem:  States (configurations of the world)  Successor function: a function from states to lists of (action, state, cost) triples; drawn as a graph  Start state and goal test  Search tree:  Nodes: represent plans for reaching states  Plans have costs (sum of action costs)  Search Algorithm:  Systematically builds a search tree  Chooses an ordering of the fringe (unexplored nodes)

6. Machinarium

7. Machinarium: Search Problem Youtube

8. A*-Search

9. General Tree Search

10. Action: flip top two Cost: 2 Action: flip all four Cost: 4 Path to reach goal: Flip four, flip three Total cost: 7

11. Uniform Cost Search  Strategy: expand lowest path cost  The good: UCS is complete and optimal!  The bad:  Explores options in every “direction”  No information about goal location Start Goal … c  3 c  2 c  1 [demo: countours UCS]

12. Best First (Greedy)  Strategy: expand a node that you think is closest to a goal state  Heuristic: estimate of distance to nearest goal for each state  A common case:  Best-first takes you straight to the (wrong) goal  Worst-case: like a badly- guided DFS … b … b [demo: countours greedy]

13. Problem with greedy?

14. Example: Heuristic Function h(x)

15. Combining UCS and Greedy  Uniform-cost orders by path cost, or backward cost g(n)  Best-first orders by goal proximity, or forward cost h(n)  A* Search orders by the sum: f(n) = g(n) + h(n) Sad b G h=5 h=6 h=2 1 5 1 1 2 h=6 h=0 c h=7 3 e h=1 1 Example: Teg Grenager

16. Three Questions  When should A* terminate?  Is A* optimal?  What heuristics are valid?

17.  Should we stop when we enqueue a goal?  No: only stop when we dequeue a goal When should A* terminate? S B A G 2 3 2 2 h = 1 h = 2 h = 0 h = 3

18. Is A* Optimal? A G S 1 3 h = 6 h = 0 5 h = 7  What went wrong?  Actual bad goal cost < estimated good goal cost  We need estimates to be less than actual costs!

19. Admissible Heuristics  A heuristic h is admissible (optimistic) if: where is the true cost to a nearest goal  Example:  Coming up with admissible heuristics is most of what’s involved in using A* in practice. 15

20. Example: Pancake Problem Cost: Number of pancakes flipped

21. Pancake: State Space Object

22. Example: Pancake Problem 3 2 4 3 3 2 2 2 4 State space graph with costs as weights 3 4 3 4 2

23. Example: Heuristic Function Heuristic: the largest pancake that is still out of place 4 3 0 2 3 3 3 4 4 3 4 4 4 h(x)

24. Example: Pancake Problem

25. Optimality of A*: Blocking … Notation:  g(n) = cost to node n  h(n) = estimated cost from n to the nearest goal (heuristic)  f(n) = g(n) + h(n) = estimated total cost via n  G*: a lowest cost goal node  G: another goal node

26. Optimality of A*: Blocking Proof:  What could go wrong?  We’d have to have to pop a suboptimal goal G off the fringe before G*  This can’t happen:  Imagine a suboptimal goal G is on the queue  Some node n which is a subpath of G* must also be on the fringe (why?)  n will be popped before G …

27. Properties of A* … b … b Uniform-CostA*

28. UCS vs A* Contours  Uniform-cost expanded in all directions  A* expands mainly toward the goal, but does hedge its bets to ensure optimality Start Goal Start Goal [demo: countours UCS / A*]

29. Creating Admissible Heuristics  Most of the work in solving hard search problems optimally is in coming up with admissible heuristics  Often, admissible heuristics are solutions to relaxed problems, where new actions are available  Inadmissible heuristics are often useful too (why?) 4 15

30. Example: 8 Puzzle  What are the states?  How many states?  What are the actions?  What states can I reach from the start state?  What should the costs be?

31. Search State

32. 8 Puzzle I  Heuristic: Number of tiles misplaced  Why is it admissible?  h(start) =  This is a relaxed- problem heuristic 8 Average nodes expanded when optimal path has length… …4 steps…8 steps…12 steps UCS1126,3003.6 x 10 6 TILES 1339227

33. 8 Puzzle II  What if we had an easier 8-puzzle where any tile could slide any direction at any time, ignoring other tiles?  Total Manhattan distance  Why admissible?  h(start) = 3 + 1 + 2 + … = 18 Average nodes expanded when optimal path has length… …4 steps…8 steps…12 steps TILES 1339227 MANHATTAN 122573 [demo: eight-puzzle]

34. 8 Puzzle III  How about using the actual cost as a heuristic?  Would it be admissible?  Would we save on nodes expanded?  What’s wrong with it?  With A*: a trade-off between quality of estimate and work per node!

35. Trivial Heuristics, Dominance  Dominance: h a ≥ h c if  Heuristics form a semi-lattice:  Max of admissible heuristics is admissible  Trivial heuristics  Bottom of lattice is the zero heuristic (what does this give us?)  Top of lattice is the exact heuristic

36. Other A* Applications  Pathing / routing problems  Resource planning problems  Robot motion planning  Language analysis  Machine translation  Speech recognition  …

37. Graph Search

38. Tree Search: Extra Work!  Failure to detect repeated states can cause exponentially more work. Why?

39. Graph Search  In BFS, for example, we shouldn’t bother expanding the circled nodes (why?) S a b d p a c e p h f r q qc G a q e p h f r q qc G a

40. Graph Search  Very simple fix: never expand a state twice

41. Graph Search  Idea: never expand a state twice  How to implement:  Tree search + list of expanded states (closed list)  Expand the search tree node-by-node, but…  Before expanding a node, check to make sure its state is new  Python trick: store the closed list as a set, not a list  Can graph search wreck completeness? Why/why not?  How about optimality?

42. Consistency A C S 1 1 h = 4 h = 1 h = 6  What went wrong?  Taking a step must not reduce f value! B h = 0 G h = 1 1 2 3

43. Consistency  Stronger than admissability  Definition:  C(A→C)+h(C) ≧ h(A)  C(A→C) ≧ h(A)-h(C)  Consequence:  The f value on a path never decreases  A* search is optimal A G 1 h = 4 h = 0 C h = 1 3

44. Consistency  Are we optimal yet?  No:  We must not pop some node n, and find its child n’ to have lower f value!  What can we require to prevent these inversions?  Consistency:  Real cost must always exceed reduction in heuristic A B G 3 h = 0 h = 10 g = 10 h = 8

45. Optimality of A* Graph Search Proof:  New possible problem: nodes on path to G* that would have been in queue aren’t, because some worse n’ for the same state as some n was dequeued and expanded first (disaster!)  Take the highest such n in tree  Let p be the ancestor which was on the queue when n’ was expanded  Assume f(p) ≦ f(n) (consistency!)  f(n) < f(n’) because n’ is suboptimal  p would have been expanded before n’  So n would have been expanded before n’, too  Contradiction!

46. Optimality  Tree search:  A* optimal if heuristic is admissible (and non- negative)  UCS is a special case (h = 0)  Graph search:  A* optimal if heuristic is consistent  UCS optimal (h = 0 is consistent)  Consistency implies admissibility  In general, natural admissible heuristics tend to be consistent

47. Summary: A*  A* uses both backward costs and (estimates of) forward costs  A* is optimal with admissible heuristics  Heuristic design is key: often use relaxed problems

48. Graph Search, Reconsidered  Idea: never expand a state twice  How to implement:  Tree search + list of expanded states (closed list)  Expand the search tree node-by-node, but…  Before expanding a node, check to make sure its state is new  Python trick: store the closed list as a set, not a list  Can graph search wreck completeness? Why/why not?  How about optimality?

49. Optimality of A* Graph Search  Consider what A* does:  Expands nodes in increasing total f value (f-contours)  Proof idea: optimal goals have lower f value, so get expanded first We’re making a stronger assumption than in the last proof… What?

50. Optimality of A* Graph Search  Consider what A* does:  Expands nodes in increasing total f value (f-contours) Reminder: f(n) = g(n) + h(n) = cost to n + heuristic  Proof idea: the optimal goal(s) have the lowest f value, so it must get expanded first … f  3 f  2 f  1 There’s a problem with this argument. What are we assuming is true?

51. Course Scheduling  From the university’s perspective:  Set of courses {c 1, c 2, … c n }  Set of room / times {r 1, r 2, … r n }  Each pairing (c k, r m ) has a cost w km  What’s the best assignment of courses to rooms?  States: list of pairings  Actions: add a legal pairing  Costs: cost of the new pairing  Admissible heuristics?