1. 1 EE 355 Unit 14 A-Star Algorithm & Heaps/Priority Queues Mark Redekopp

2. 2 ALGORITHM HIGHLIGHT A* Search Algorithm

3. 3 Search Methods Many systems require searching for goal states – Path Planning Roomba Vacuum Mapquest/Google Maps Games!! – Optimization Problems Find the optimal solution to a problem with many constraints

4. 4 Search Applied to 8-Tile Game 8-Tile Puzzle – 3x3 grid with one blank space – With a series of moves, get the tiles in sequential order – Goal state: 12 345 678 PA4 Goal State 123 456 78 Goal State for these slides

5. 5 Search Methods Brute-Force Search: When you don’t know where the answer is, just search all possibilities until you find it. Heuristic Search: A heuristic is a “rule of thumb”. An example is in a chess game, to decide which move to make, count the values of the pieces left for your opponent. Use that value to “score” the possible moves you can make. – Heuristics are not perfect measures, they are quick computations to give an approximation (e.g. may not take into account “delayed gratification” or “setting up an opponent”)

6. 6 Brute Force Search Brute Force Search Tree – Generate all possible moves – Explore each move despite its proximity to the goal node 12 483 765 12 483 765 12 483 765 182 43 765 12 483 765 12 483 765 123 485 76 123 48 765 123 48 765 123 485 76 123 45 786 WS W S E

7. 7 Heuristics Heuristics are “scores” of how close a state is to the goal (usually, lower = better) These scores must be easy to compute (i.e. simpler than solving the problem) Heuristics can usually be developed by simplifying the constraints on a problem Heuristics for 8-tile puzzle – # of tiles out of place Simplified problem: If we could just pick a tile up and put it in its correct place – Total x-, y- distance of each tile from its correct location (Manhattan distance) Simplified problem if tiles could stack on top of each other / hop over each other 183 456 27 183 456 27 # of Tiles out of Place = 3 Total x-/y- distance = 6

8. 8 Heuristic Search Heuristic Search Tree – Use total x-/y- distance (Manhattan distance) heuristic – Explore the lowest scored states 12 483 765 12 483 765 12 483 765 123 485 76 123 456 78 123 48 765 123 48 765 123 485 76 123 45 786 123 45 786 H=6 H=7 H=5 H=6 H=4 H=3 H=2 H=1 Goal 123 48 765 H=5

9. 9 Caution About Heuristics Heuristics are just estimates and thus could be wrong Sometimes pursuing lowest heuristic score leads to a less-than optimal solution or even no solution Solution – Take # of moves from start (depth) into account H=2 Start H=1 … Goal H=1

10. 10 A-star Algorithm Use a new metric to decide which state to explore/expand Define – h = heuristic score (same as always) – g = number of moves from start it took to get to current state – f = g + h As we explore states and their successors, assign each state its f-score and always explore the state with lowest f-score Heuristics should always underestimate the distance to the goal – If they do, A* guarantees optimal solutions g=1,h=2 f=3 Start g=1,h=1 f=2 g=2,h=1 f=3 g=3,h=1 f=4 Goal g=2,h=1 f=3

11. 11 A-Star Algorithm Maintain 2 lists – Open list = Nodes to be explored (chosen from) – Closed list = Nodes already explored (already chosen) Pseudocode open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. Generate successors/neighbors of s, compute their f values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value g=1,h=2 f=3 Start g=1,h=1 f=2 g=2,h=1 f=3 g=3,h=1 f=4 Goal g=2,h=1 f=3

12. 12 Data Structures & Ordering We’ve seen some abstract data structures – Queue (first-in, first-out access order) Deque works nicely There are others… – Stack (last-in, first-out access order) Vector push_back / pop_back – Trees A new one…priority queues (heaps) – Order of access depends on value – Items arrive in some arbitrary order – When removing an item, we always want the minimum or maximum valued item – To implement efficiently use heap data structure remove_min = 21 46783567 original heap 4621783567 01234 remove_min = 35 467867

13. 13 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance S G open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value Closed List Open List **If implementing this for a programming assignment, please see the slide at the end about alternate closed-list implementation

14. 14 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance S G Closed List Open List g=0, h=6, f=6 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value **If implementing this for a programming assignment, please see the slide at the end about alternate closed-list implementation

15. 15 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

16. 16 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

17. 17 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

18. 18 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 g=1, h=5, f=6 g=2, h=6, f=8 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

19. 19 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 g=2, h=6, f=8 g=3, h=7, f=10 g=3, h=5, f=8 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

20. 20 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 g=2, h=6, f=8 g=3, h=7, f=10 g=3, h=5, f=8 g=2, h=8, f=10 g=3, h=7, f=10 g=3, h=5, f=8 g=4, h=6, f=10 g=4, h=4, f=8 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

21. 21 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 g=2, h=6, f=8 g=3, h=7, f=10 g=3, h=5, f=8 g=2, h=8, f=10 g=3, h=7, f=10 g=3, h=5, f=8 g=4, h=6, f=10 g=4, h=4, f=8 g=5, h=5, f=10 g=5, h=3, f=8 g=1, h=7, f=8 g=2, h=6, f=8 g=1, h=7, f=8 g=2, h=6, f=8 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

22. 22 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 g=2, h=6, f=8 g=3, h=7, f=10 g=3, h=5, f=8 g=2, h=8, f=10 g=3, h=7, f=10 g=3, h=5, f=8 g=4, h=6, f=10 g=4, h=4, f=8 g=5, h=5, f=10 g=5, h=3, f=8 g=6, h=4, f=10 g=6, h=2, f=8 g=1, h=7, f=8 g=2, h=6, f=8 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

23. 23 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 g=2, h=6, f=8 g=3, h=7, f=10 g=3, h=5, f=8 g=2, h=8, f=10 g=3, h=7, f=10 g=3, h=5, f=8 g=4, h=6, f=10 g=4, h=4, f=8 g=5, h=5, f=10 g=5, h=3, f=8 g=6, h=4, f=10 g=6, h=2, f=8 g=7, h=3, f=10 g=7, h=1, f=8 g=1, h=7, f=8 g=2, h=6, f=8 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

24. 24 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 g=2, h=6, f=8 g=3, h=7, f=10 g=3, h=5, f=8 g=2, h=8, f=10 g=3, h=7, f=10 g=3, h=5, f=8 g=4, h=6, f=10 g=4, h=4, f=8 g=5, h=5, f=10 g=5, h=3, f=8 g=6, h=4, f=10 g=6, h=2, f=8 g=7, h=3, f=10 g=7, h=1, f=8 g=8, h=2, f=10 g=8, h=0, f=8 g=1, h=7, f=8 g=2, h=6, f=8 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

25. 25 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 g=2, h=6, f=8 g=3, h=7, f=10 g=3, h=5, f=8 g=2, h=8, f=10 g=3, h=7, f=10 g=3, h=5, f=8 g=4, h=6, f=10 g=4, h=4, f=8 g=5, h=5, f=10 g=5, h=3, f=8 g=6, h=4, f=10 g=6, h=2, f=8 g=7, h=3, f=10 g=7, h=1, f=8 g=8, h=2, f=10 g=8, h=0, f=8 g=1, h=7, f=8 g=2, h=6, f=8 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

26. 26 Path-Planning w/ A* Algorithm Find optimal path from S to G using A* – Use heuristic of Manhattan (x-/y-) distance g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=6, f=8 g=2, h=4, f=6 g=2, h=6, f=8 g=3, h=7, f=10 g=3, h=5, f=8 g=2, h=8, f=10 g=3, h=7, f=10 g=3, h=5, f=8 g=4, h=6, f=10 g=4, h=4, f=8 g=5, h=5, f=10 g=5, h=3, f=8 g=6, h=4, f=10 g=6, h=2, f=8 g=7, h=3, f=10 g=7, h=1, f=8 g=8, h=2, f=10 g=8, h=0, f=8 g=1, h=7, f=8 g=2, h=6, f=8 open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. else Generate successors/neighbors of s, compute their f-values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value

27. 27 A* and BFS BFS explores all nodes at a shorter distance from the start (i.e. g value) g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List

28. 28 A* and BFS BFS explores all nodes at a shorter distance from the start (i.e. g value) g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=8, f=10 g=2, h=6, f=8 g=2, h=4, f=6

29. 29 A* and BFS BFS is A* using just the g value to choose which item to select and expand g=1, h=7, f=8 S G g=1, h=5, f=6 g=1, h=7, f=8 Closed List Open List g=2, h=8, f=10 g=2, h=6, f=8 g=2, h=4, f=6

30. 30 Implementation Note When the distance to a node/state/successor (i.e. g value) is uniform, we can greedily add a state to the closed-list at the same time as we add it to the open-list open_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 2. Add s to closed list 3a. if s = goal node then trace path back to start; STOP! 3b. Generate successors/neighbors of s, compute their f values, and add them to open_list if they are not in the closed_list (so we don’t re-explore), or if they are already in the open list, update them if they have a smaller f value open_list.push(Start State) Closed_list.push(Start State) while(open_list is not empty) 1. s ← remove min. f-value state from open_list (if tie in f-values, select one w/ larger g-value) 3a. if s = goal node then trace path back to start; STOP! 3b. Generate successors/neighbors of s, compute their f values, and add them to open_list and closed_list if they are not in the closed_list Non-uniform g-values Uniform g-values 12 483 765 g=0,H=6 12 483 765 … g=k,H=6 The first occurrence of a board has to be on the shortest path to the solution

31. 31 DATA STRUCTURE HIGHLIGHT Heaps

32. 32 Heap Data Structure Can think of heap as a full binary tree with the property that every parent is less-than (if min-heap) or greater-than (if max- heap) both children – But no ordering property between children Minimum/Maximum value is always the top element Min-Heap 7 9 18 19 35 1410 28 39 36 43 16 25

33. 33 Heap Operations Push: Add a new item to the heap and modify heap as necessary Pop: Remove min/max item and modify heap as necessary Top: Returns min/max To create a heap from an unordered array/vector takes O(n*log 2 n) time while push/pop takes O(log 2 n)

34. 34 Push Heap Add item to first free location at bottom of tree Recursively promote it up until a parent is less than the current item 7 9 18 19 35 1410 28 39 36 43 16 25 7 8 18 19 35 14 9 28 39 36 43 16 25 8 8 10 Push_Heap(8) 1 2 3 45 6 7 8 910 11 1213 14

35. 35 Pop Heap Takes last (greatest) node puts it in the top location and then recursively swaps it for the smallest child until it is in its right place 7 9 18 19 35 1410 28 39 36 43 16 25 9 10 18 19 35 14 7 28 39 36 43 16 Original 9 25 9 18 19 35 1410 28 39 36 43 16 25 1 2 3 45 6 7 8 910 11 1213 1 2 3 45 6 7 8 910 11 12

36. 36 Practice 7 21 352624 50294336 18 19 3928 1 2 3 45 6 7 8 910 11 1213 7 9 351410 36 18 19 3928 1 2 3 45 6 7 8 910 Push(11) Pop() 4 8 352624 36 17 19 3928 1 2 3 45 6 910 Pop() 7 7 21 352624 50294336 18 19 3928 1 2 3 45 6 7 910 11 1213 Push(23)

37. 37 Array/Vector Storage for Heap Binary tree that is full (i.e. only the lowest-level contains empty locations and items added left to right) can be modeled as an array (let’s say it starts at index 1) where: – Parent(i) = i/2 – Left_child(p) = 2*p – Right_child(p) = 2*p + 1 7 9 18 19 35 1410 28 39 36 43 16 17 em718919 01234 3514102839 56789 36431617 10111213 parent(5) = 5/2 = 2 Left_child(5) = 2*5 = 10 Right_child(5) = 2*5+1 = 11

38. 38 STL Priority Queue Implements a max-heap by default Operations: – push(new_item) – pop(): removes but does not return top item – top() return top item (item at back/end of the container) – size() – empty() http://www.cplusplus.com/refere nce/stl/priority_queue/push/ // priority_queue::push/pop #include using namespace std; int main () { priority_queue mypq; mypq.push(30); mypq.push(100); mypq.push(25); mypq.push(40); cout << "Popping out elements..."; while (!mypq.empty()) { cout<< " " << mypq.top(); mypq.pop(); } cout<< endl; return 0; } Code here will print 100 40 30 25

39. 39 STL Priority Queue Template Template that allows type of element, container class, and comparison operation for ordering to be provided First template parameter should be type of element stored Second template parameter should be vector Third template parameters should be comparison function object/class that will define the order from first to last in the container // priority_queue::push/pop #include using namespace std; intmain () { priority_queue, greater > mypq; mypq.push(30); mypq.push(100); mypq.push(25); cout<< "Popping out elements..."; while (!mypq.empty()) { cout<< " " << mypq.top(); mypq.pop(); } } Code here will print 25, 30, 100 'greater' will yield a min-heap 'less' will yield a max-heap

40. 40 STL Priority Queue Template For user defined classes, must implement operator () for min- heap Code here will pop in order: – Jane – Charlie – Bill // priority_queue::push/pop #include using namespace std; class Item { public: int score; string name; Item(int s, string n) { score = s; name = n;} bool operator>(const Item &lhs, const Item &rhs) const { if(lhs.score > rhs.score) return true; else return false; } }; int main () { priority_queue, greater > mypq; Item i1(25,”Bill”); mypq.push(i1); Item i2(5,”Jane”); mypq.push(i2); Item i3(10,”Charlie”); mypq.push(i3); cout<< "Popping out elements..."; while (!mypq.empty()) { cout<< " " << mypq.top().name; mypq.pop(); }