1. Problem Spaces & Search Dan Weld CSE 573

2. © Daniel S. Weld 2 Logistics Read rest of R&N chapter 4 Also read chapter 5 PS 1 will arrive by email, so be sure you are on the mailing list (and/or poll the website)

3. © Daniel S. Weld 3 Last Time Agents Problem Spaces Blind Search Heuristic Search Environment Agent percepts actions

4. © Daniel S. Weld 4 Problem Spaces Getting a PhD as a search problem? (ignore funding & don’t think too hard) Space of States Operators Initial State Goal State

5. © Daniel S. Weld 5 Blocks World Revisited Ignore crane-to-crane interactions…

6. © Daniel S. Weld 6 Search Problem spaces Blind Depth-first, breadth-first, iterative- deepening, iterative broadening Informed Best-first, Dijkstra's, A*, IDA*, SMA*, DFB&B, Beam, hill climb, limited discrep, RTDP Local search Heuristics Evaluation, construction via relaxation Pattern databases Constraint satisfaction

7. © Daniel S. Weld 7 Depth First Search a b cd e f gh Maintain stack of nodes to visit Evaluation Complete? Time Complexity? Space Complexity? Not for infinite spaces O(b^d) O(bd) d=max depth of tree

8. © Daniel S. Weld 8 Memory bottleneck of BFS DFS with limit; incrementally grow limit Complete? Time Complexity? Space Complexity? Iterative Deepening Search Yes O(b^d) O(bd) bratio ID/ DFS 23 32 51.5 101.2 251.08 1001.02

9. © Daniel S. Weld 9 A* Search Hart, Nilsson & Rafael 1968 Best first search with f(n) = g(n) + h(n) Where g(n) = sum of costs from start to n h(n) = estimate of lowest cost path n  goal h(goal) = 0 If h(n) is admissible (and monotonic) then A* is optimal Underestimates cost of any solution which can reached from node { f values increase from node to descendants (triangle inequality) {

10. © Daniel S. Weld 10 Iterative-Deepening A* Like iterative-deepening depth-first, but... Depth bound modified to be an f-limit Start with f-limit = h(start) Prune any node if f(node) > f-limit Next f-limit = min-cost of any node pruned a b c d e f FL=15 FL=21

11. © Daniel S. Weld 11 IDA* Analysis Complete & Optimal (ala A*) Space usage  depth of solution Each iteration is DFS - no priority queue! # nodes expanded relative to A* Depends on # unique values of heuristic function In 8 puzzle: few values  close to # A* expands In traveling salesman: each f value is unique  1+2+…+n = O(n 2 ) where n=nodes A* expands if n is too big for main memory, n 2 is too long to wait! Generates duplicate nodes in cyclic graphs

12. © Daniel S. Weld 12 Forgetfulness A* used exponential memory How much does IDA* use? During a run? In between runs?

13. © Daniel S. Weld 13 SMA* Use all available memory Start like A* When memory is full… Erase node with highest f-value First, backup parent with this f-value So… parent knows cost-bound on best child

14. © Daniel S. Weld 14 Alternative Approach to Finite Memory… Optimality is nice to have, but…

15. © Daniel S. Weld 15 Depth-First Branch & Bound Single DF search  uses linear space Keep track of best solution so far If f(n) = g(n)+h(n)  cost(best-soln) Then prune n Requires Finite search tree, or Good upper bound on solution cost Generates duplicate nodes in cyclic graphs Adapted from Richard Korf presentation Tradeoff: Prune space, but… Must apply test to each node

16. © Daniel S. Weld 16 No O(b^d) O(b + N) Beam Search Idea Best first but only keep N best items on priority queue Evaluation Complete? Time Complexity? Space Complexity?

17. © Daniel S. Weld 17 Hill Climbing Idea Always choose best child; no backtracking Beam search with |queue| = 1 Problems? Local maxima Plateaus Diagonal ridges “Gradient ascent”

18. © Daniel S. Weld 18 Randomizing Hill Climbing Randomly disobeying heuristic Random restarts ( heavy tailed distributions )  Local Search

19. © Daniel S. Weld 19 Simulated Annealing Objective: avoid local minima Technique: For the most part use hill climbing When no improvement possible –Choose random neighbor –Let  be the decrease in quality –Move to neighbor with probability e -  -/T Reduce “temperature” (T) over time Pros & cons Optimal? temp If T decreased slowly enough, will reach optimal state Widely used See also WalkSAT

20. © Daniel S. Weld 20 Limited Discrepancy Search Discrepancy bound indicates how often to violate heuristic Iteratively increase... a b c d e f g h Assume that heuristic says go left

21. © Daniel S. Weld 21 174611094281174629844710 Genetic Algorithms Start with random population Representation serialized States are ranked with “fitness function” Produce new generation Select random pair(s): –probability ~ fitness Randomly choose “crossover point” –Offspring mix halves Randomly mutate bits 776511094281776529844710 164611094281 776029844210 Crossover Mutation 

22. © Daniel S. Weld 22 Search Problem spaces Blind Depth-first, breadth-first, iterative- deepening, iterative broadening Informed Best-first, Dijkstra's, A*, IDA*, SMA*, DFB&B, Beam, hill climb, limited discrep, RTDP Local search Heuristics Evaluation, construction via relaxation Pattern databases Constraint satisfaction

23. Heuristics It’s what makes search actually work

24. © Daniel S. Weld 24 Importance of Heuristics D IDS A*(h1)A*(h2) 2 10 6 6 4 11213 12 6 68020 18 8 638439 25 10 4712793 39 12 364404 227 73 143473941 539 113 18 3056 363 24 391351641 h1 = number of tiles in wrong place h2 =  distances of tiles from correct loc 7 2 3 8 5 4 1 6 Decrease effective branching factor

25. © Daniel S. Weld 25 Admissable Heuristics f(x) = g(x) + h(x) g: cost so far h: underestimate of remaining costs Where do heuristics come from?

26. © Daniel S. Weld 26 Relaxed Problems Derive admissible heuristic from exact cost of a solution to a relaxed version of problem For transportation planning, relax requirement that car has to stay on road  Euclidean dist For blocks world, distance = # move operations heuristic = number of misplaced blocks What is relaxed problem? # out of place = 2, true distance to goal = 3 Cost of optimal soln to relaxed problem  cost of optimal soln for real problem

27. © Daniel S. Weld 27 Simplifying Integrals vertex = formula goal = closed form formula without integrals arcs = mathematical transformations heuristic = number of integrals still in formula what is being relaxed?

28. © Daniel S. Weld 28 Traveling Salesman Problem Problem Space Heuristic? States = partial path (not nec. connected) Operator = add an edge Start state = empty path Goal = complete path What can be Relaxed?

29. © Daniel S. Weld 29 Heuristics for eight puzzle 7 2 3 8 3 5 1 6 1 2 3 7 8 4 5 6 startgoal  What can we relax?

30. © Daniel S. Weld 30 Importance of Heuristics D IDS A*(h1)A*(h2) 2 10 6 6 4 11213 12 6 68020 18 8 638439 25 10 4712793 39 12 364404 227 73 143473941 539 113 18 3056 363 24 391351641 h1 = number of tiles in wrong place h2 =  distances of tiles from correct loc 7 2 3 8 5 4 1 6

31. © Daniel S. Weld 31 Need More Power! Performance of Manhattan Distance Heuristic 8 Puzzle< 1 second 15 Puzzle1 minute 24 Puzzle65000 years Need even better heuristics! Adapted from Richard Korf presentation

32. © Daniel S. Weld 32 Subgoal Interactions Manhattan distance assumes Each tile can be moved independently of others Underestimates because Doesn’t consider interactions between tiles Adapted from Richard Korf presentation 1 2 3 7 8 4 6 5

33. © Daniel S. Weld 33 Pattern Databases Pick any subset of tiles –E.g., 3, 7, 11, 12, 13, 14, 15 Precompute a table Optimal cost of solving just these tiles For all possible configurations –57 Million in this case Use breadth first search back from goal state –State = position of just these tiles (& blank) Adapted from Richard Korf presentation [Culberson & Schaeffer 1996]

34. © Daniel S. Weld 34 Using a Pattern Database As each state is generated Use position of chosen tiles as index into DB Use lookup value as heuristic, h(n) Admissible? Adapted from Richard Korf presentation

35. © Daniel S. Weld 35 Combining Multiple Databases Can choose another set of tiles Precompute multiple tables How combine table values? E.g. Optimal solutions to Rubik’s cube First found w/ IDA* using pattern DB heuristics Multiple DBs were used (dif cubie subsets ) Most problems solved optimally in 1 day Compare with 574,000 years for IDDFS Adapted from Richard Korf presentation

36. © Daniel S. Weld 36 Drawbacks of Standard Pattern DBs Since we can only take max Diminishing returns on additional DBs Would like to be able to add values Adapted from Richard Korf presentation

37. © Daniel S. Weld 37 Disjoint Pattern DBs Partition tiles into disjoint sets For each set, precompute table –E.g. 8 tile DB has 519 million entries –And 7 tile DB has 58 million During search Look up heuristic values for each set Can add values without overestimating! Manhattan distance is a special case of this idea where each set is a single tile Adapted from Richard Korf presentation 9 10 11 12 13 14 15 1 2 3 4 5 6 7 8

38. © Daniel S. Weld 38 Performance 15 Puzzle: 2000x speedup vs Manhattan dist IDA* with the two DBs shown previously solves 15 Puzzles optimally in 30 milliseconds 24 Puzzle: 12 million x speedup vs Manhattan IDA* can solve random instances in 2 days. Requires 4 DBs as shown –Each DB has 128 million entries Without PDBs: 65,000 years Adapted from Richard Korf presentation