1. HEURISTICS AND INTELLIGENT SEARCH CS16: Introduction to Data Structures & Algorithms Tuesday, April 12, 2016 1

2. Outline Motivation Heuristics A* Search Comparisons and caveats Tuesday, April 12, 2016 2

3. Motivation Tuesday, April 12, 2016 3 GSBC A 1 1 1 4 7 S is start node G is goal node What does Dijkstra’s algorithm do here?

4. Heuristics Heuristics can be thought of as little hints or edited guesses towards the solution to a problem Example: giving directions You don’t tell your friends how to get somewhere by using Dijkstra’s algorithm Rather, you think of what is roughly the fastest way to get somewhere and tell them to take that route. In fact, you may have in your head an idea of how long it takes to get to the destination from each point in the path This helps you in solving the overall problem Tuesday, April 12, 2016 4

5. Heuristics, formally A heuristic function takes in a state of the world related to the problem (like a location in our example of giving directions) It returns an approximation of the cost of competing that problem if you were to start at that state. Proper definition (from Wikipedia): A heuristic technique is any approach to problem solving, learning, or discovery that employs a practical methodology not guaranteed to be optimal or perfect, but sufficient for the immediate goals. Tuesday, April 12, 2016 5

6. Problem definition We want to devise an algorithm that can find the shortest path between any two nodes in the graph Tuesday, April 12, 2016 6 Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 RISD

7. Uninformed solution We have solved this problem before, in fact we have two ways to do it Dijkstra's algorithm Bellman-Ford algorithm We'll focus on Dijkstra’s for now, since we generally can‘t get between two points in negative time Suppose we are running from a meeting in Faunce to an event at the Med School Tuesday, April 12, 2016 7

8. Dijkstra’s pseudocode refresher Tuesday, April 12, 2016 8 function dijkstra(G, s, g): // Input: A graph G with vertices V, start and end vertices s & g // Output: shortest path from s to g, null if none is possible for v in V: v.cost = infinity // Initialize distance decorations v.prev = null // Initialize previous pointers to null s.cost = 0 // Set distance to start to 0 PQ = PriorityQueue(V) // Use v.cost as priorities while PQ not empty: u = PQ.removeMin() if u == g: return gen_path(u) for all edges (u, v): // all edges starting at u if v.cost > u.cost + cost(u, v): // cost() is weight v.cost = u.cost + cost(u, v) // Replace as necessary v.prev = u PQ.replaceKey(v, v.cost) return null function gen_path(n): // Input: node n with prev pointers // Output: shortest path ending with n path = [] while n != null path.append(n) n = n.prev return path.reverse()

9. Dijkstra’s run through Tuesday, April 12, 2016 9 PQ: F 0 CE ∞ WS ∞ A ∞ R ∞ MS ∞ K ∞ CIT ∞ Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 RISD

10. Dijkstra’s run through Tuesday, April 12, 2016 10 PQ: K 1 CIT 1 A 1.5 R 2 WS ∞ MS ∞ CE ∞ F 0 1) Remove Min: 2) Check if goal 3) If not, check and update neighbors RISD Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 RISD F F F F

11. Dijkstra’s run through Tuesday, April 12, 2016 11 PQ: CIT 1 A 1.5 R 2 CE 4 MS ∞ WS ∞ K 1 1) Remove Min: 2) Check if goal 3) If not, check and update neighbors Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 RISD F F F F K

12. Dijkstra’s run through Tuesday, April 12, 2016 12 PQ: CIT 1 A 1.5 R 2 CE 4 MS ∞ WS ∞ K 1 1) Remove Min: 2) Check if goal 3) If not, check and update neighbors We don’t update the path to CIT since the existing path is shorter Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 RISD F F F F K

13. Dijkstra’s run through Tuesday, April 12, 2016 13 PQ: A 1.5 R 2 CE 4 WS 5 MS ∞ CIT 1 1) Remove Min: 2) Check if goal 3) If not, check and update neighbors Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 RISD F F F K F CIT

14. Dijkstra’s run through Tuesday, April 12, 2016 14 RISD PQ: R 2 CE 4 WS 5 MS 9.5 A 1.5 1) Remove Min: 2) Check if goal 3) If not, check and update neighbors We have seen our goal. Ready to return? Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 F F F K CIT F A

15. Dijkstra’s run through Tuesday, April 12, 2016 15 RISD PQ: R 2 CE 4 WS 5 MS 9.5 A 1.5 1) Remove Min: 2) Check if goal 3) If not, check and update neighbors We have seen our goal. Ready to return? Nope! We can’t know that the path is shortest until we are ready to remove it Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 RISD We have seen our goal. Ready to return? F F F K CIT A F

16. Dijkstra’s run through Tuesday, April 12, 2016 16 RISD PQ: CE 4 WS 5 MS 5 R 2 1) Remove Min: 2) Check if goal 3) If not, check and update neighbors See, the path got updated! Its prev pointer is now R also Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 F F K R F CIT F

17. Dijkstra’s run through Tuesday, April 12, 2016 17 PQ: WS 5 MS 5 CE 4 1) Remove Min: 2) Check if goal 3) If not, check and update neighbors Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 F F K R F CIT F RISD

18. Dijkstra’s run through Tuesday, April 12, 2016 18 PQ: MS 5 WS 5 1) Remove Min: 2) Check if goal 3) If not, check and update neighbors Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 F F K R F CIT F RISD

19. Dijkstra’s run through Tuesday, April 12, 2016 19 PQ: MS 5 1) Remove Min: 2) Check if goal – yes! 3) Return path starting at MS : F→R→MS RISD Faunce Andrews Keeney Coffee Ex Med School CIT Wayland Square 3 2 3 2 4 6 1.5 2 1 1 8 F CIT F F F K R

20. Heuristic function example But what if we had a bit more information? Some sort of heuristic that could estimate the time remaining to our destination Tuesday, April 12, 2016 20 F A K CE MS CIT WS R nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9

21. Greedy search pseudocode Tuesday, April 12, 2016 21 function greedy_search(G, s, g, h): // Input: A graph G with vertices V, start and end vertices s & g, a heuristic function h // Output: list of nodes with shortest path from s to g, null if none is possible for v in V: v.cost = infinity // Initialize distance decorations v.prev = null // Initialize previous pointers to null s.cost = h(s) // Set distance to start to heuristic value PQ = PriorityQueue(V) // Use v.cost as priorities while PQ not empty: u = PQ.removeMin() if u == g: return gen_path(u) for all edges (u, v): // all edges starting at u if h(v) < v.cost: // cost() is weight – only will be updated from infinity v.cost = h(v) // Replace as necessary v.prev = u PQ.replaceKey(v, v.cost) return null

22. Greedy runthrough Tuesday, April 12, 2016 22 F A K CE MS CIT WS nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 R PQ: F 4 CE ∞ WS ∞ A ∞ R ∞ MS ∞ K ∞ CIT ∞

23. Greedy runthrough Tuesday, April 12, 2016 23 nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 R PQ: K 1 R 2.5 CIT 5 A 8 MS ∞ WS ∞ CIT ∞ F 0 Examining: F A K CE MS CIT WS F F F F

24. Greedy runthrough Tuesday, April 12, 2016 24 nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 PQ: CE 0.5 R 2.5 CIT 5 A 8 WS ∞ MS ∞ K 1 Examining: R F A K CE MS CIT WS F F F F K

25. Greedy runthrough Tuesday, April 12, 2016 25 nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 PQ: MS 0 R 2.5 CIT 5 A 8 WS ∞ CE 0.5 Examining: F A K CE MS CIT WS F F F F K R CE

26. Greedy runthrough Tuesday, April 12, 2016 26 nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 PQ: R 2.5 CIT 5 A 8 WS ∞ MS 0 Examining: F A K CE MS CIT WS This is our goal state! Return path starting here: F->K->CE->MS F F F K R CE F

27. What just happened? Did we find the same solution both times? No ☹ But what was better about the second search? How long did each algorithm take? Tuesday, April 12, 2016 27

28. Combining the approaches Do we think we can use both of these data (path cost and heuristic) to generate a correct, efficient approach? What if we combined the actual length to a given point with the estimated length to the goal! Use real length up to n to determine how far it is to get to a node Use h(n) to estimate remaining distance to goal Sum these to get an overall estimate of the path length Tuesday, April 12, 2016 28

29. A* Search Pseudocode Tuesday, April 12, 2016 29 function greedy_search(G, s, g, h): // Input: A graph G with vertices V, start and end vertices s & g, a heuristic function h // Output: int representing shortest path from s to g, infinity if none is possible for v in V: v.cost = infinity // Initialize distance decorations v.prev = null // Initialize previous pointers to null s.cost = 0 PQ = PriorityQueue(V) while PQ not empty: u = PQ.removeMin() if u == g: return gen_path(u) for all edges (u, v): // all edges starting at u if v.cost > u.cost + cost(u, v): v.cost = u.cost + cost(u, v) // *IMPORTANT* heuristic is not part of the actual cost v.prev = u PQ.replaceKey(v, v.cost + h(v)) // but the heuristic does influence our ordering return null

30. A* runthrough Tuesday, April 12, 2016 30 F A K CE MS CIT WS nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 R PQ: F 0 CE ∞ WS ∞ A ∞ R ∞ MS ∞ K ∞ CIT ∞ 3 2 3 2 4 6 1.5 2 1 8

31. A* runthrough Tuesday, April 12, 2016 31 F A K CE MS CIT WS nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 R PQ: K 2 R 4.5 CIT 6 A 9.5 MS ∞ WS ∞ CE ∞ 3 2 3 2 4 6 1.5 2 1 8 F 0 Examining: Sum of path length and heuristic R F F F F

32. A* runthrough Tuesday, April 12, 2016 32 F A K CE MS CIT WS nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 PQ: CE 4.5 R CIT 6 A 9.5 WS ∞ MS ∞ 3 2 3 2 4 6 1.5 2 1 8 K 2 Examining: Sum of path length and heuristic F F F F K R

33. A* runthrough Tuesday, April 12, 2016 33 F A K CE MS CIT WS nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 R PQ: R 4.5 CIT 6 MS 6 WS ∞ A 9.5 3 2 3 2 4 6 1.5 2 1 8 CE 4.5 Examining: 1 Sum of path length and heuristic F F F F K CE

34. A* runthrough Tuesday, April 12, 2016 34 F A K CE MS CIT WS nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 R PQ: MS 5 CIT 6 A 9.5 WS ∞ 3 2 3 2 4 6 1.5 2 1 8 R 4.5 Examining: 1 Sum of path length and heuristic 1 F F F F K R

35. A* runthrough Tuesday, April 12, 2016 35 F A K CE MS CIT WS nh(n) MS0 R2.5 CE.5 K1 CIT5 F4 A8 WS9 R PQ: CIT 6 A 9.5 WS ∞ 3 2 3 2 4 6 1.5 2 1 8 MS 5 Examining: 1 This is our goal state! Return path starting here: F->R->MS 11 F F F F K R

36. Results This approach was faster and still correct! So cool! So how do we get heuristics? Anything goes? Guess at random? Sounds sketchy… Let’s just try changing h(R) to be 1000 Tuesday, April 12, 2016 36

37. A* runthrough w/ sketchy heuristic Tuesday, April 12, 2016 37 nh(n) MS0 R1000 CE.5 K1 CIT5 F4 A8 WS9 PQ: K 2 R 4.5 CIT 6 R 1002 MS ∞ WS ∞ CE ∞ 3 2 3 2 4 6 1.5 2 1 8 F 0 Examining: We are clearly going to find MS by another path F A K CE MS CIT WS R 3 2 3 2 4 6 1.5 2 1 8 1111 F F F F

38. Results How do we get good heuristics? We need to guarantee that the estimated path cost is never going to be greater than the real path cost, or we might not look at the paths in the right order This is called admissibility Tuesday, April 12, 2016 38

39. Admissibility An admissible heuristic is one that never overestimates cost to solve the problem from a given node Why was this important? In our bad heuristic example, the heuristic made it seem like the best path had a greater cost than it actually did, even greater than a suboptimal path seemed Thus, it was never explored. If we use admissible heuristics, A* is guaranteed to be optimal Tuesday, April 12, 2016 39

40. Consistency Consistency isn’t required to give the correct answer But it is a nice property that makes our searches better Recall that in Dijkstra’s algorithm, when we visit a node, the value assigned to it is the shortest path to that node. That isn’t necessarily true in A* Our value on each node consists of the cost to that node plus the heuristic estimate from that node to the goal Tuesday, April 12, 2016 40

41. Consistency Consistency means that any transition from node to node is no greater than the actual transition that takes place This ensures that the heuristic is monotonic – strictly nondecreasing Just like Dijkstra’s, we can now draw steadily expanding circles to demonstrate the area of the graph we have already covered, and be sure that we are expanding from short -> long paths Patrick – check out the validity of this statement, may need to be rephrased Tuesday, April 12, 2016 41

42. Generating admissible heuristics So how do we generate these admissible heuristics? One easy way: h(n) = 0 Not going to be very helpful What is this the same as? A general strategy: relax problem constraints Easy to compute Simpler problems generally take less time to solve Tuesday, April 12, 2016 42

43. Relaxing constraints – Heuristic example Straight line distance If our problem’s nodes have a notion of ‘location’, we know the shortest path between them is a straight line Example: Mapping application directions Nodes may have x and y values Use these to compute the shortest possible path from any node to goal Tuesday, April 12, 2016 43 S A G B 3 4 5

44. Dominant heuristics So what if we have two perfectly admissible heuristics? What makes one heuristic better? Tuesday, April 12, 2016 44

45. Dominant heuristics Tuesday, April 12, 2016 45

46. Relaxing constraints – Heuristic examples Problem definition: 8-tile game Nodes in graph are board states (i.e layout of tiles) Edges take you from one state to the next Tuesday, April 12, 2016 46 (…) + 2 more START GOAL

47. Relaxing constraints – Heuristic examples Tuesday, April 12, 2016 47

48. GIFs! Tuesday, April 12, 2016 48

49. Heuristics everywhere! We only talked about heuristics in terms of search But heuristics come up in many CS applications Example: Virus Scanning Can’t look through the entire contents of every file Would take way too long Instead, virus scanners look for patterns with file code or activity that it knows to be associated with viruses This one potential reason why antivirus software *always* needs to update Tuesday, April 12, 2016 49