1. 1 Theory I Algorithm Design and Analysis (9 – Priority queues: Fibonacci heaps) T. Lauer

2. 2 Priority Queues Data structures for managing elements (insert, delete) –Stack: “last in – first out” –Queue:“first in – first out” –Dictionary:“find any element” (lookup) New structure: priority queue –a priority is assigned to each inserted element –the element with highest priority can be accessed and removed –Real-world example: “to-do” list: New tasks are added The most urgent/important one is carried out first

3. 3 Priority queues: elements The key of an element (node) in a priority queue represents its assigned priority. The priority has to be of an ordered type (e.g. int, long, float, …) so the can be compared. The priority is not a unique identifier of an element (e.g. for lookup). Hence, several elements could have the same priority. We store the key as an int (a smaller value represents a higher priority). class Node { Object content; // content int key; // priority }

4. 4 Priority queues: operations Operations on a priority queue Q: Q.insert(int k) : Insert a new element with key (= priority) k. Q.accessmin() : Return the element with minimum key (= highest priority). Q.deletemin() : Remove the element with minimum key. Q.decreasekey(Node N, int k) : Decrease the key of node N to the value k. Q.delete(Node N) : Delete the given node N. Q.meld(PriorityQueue P) : Merge Q with the priority queue P. Q.isEmpty() : Return true iff Q is empty. Remark: Efficient lookup of an element with a given key is not supported in priority queues. Hence, decreasekey and delete require direct access to the respective element N (not just the key).

5. 5 Fibonacci heaps: idea List of multiway trees which are all heap-ordered. Definition: A tree is called heap-ordered if the key of each node is greater than or equal to the key of its parent (if there is a parent). The roots of the trees are organized in a doubly-connected circular list (root list). The Fibonacci heap can be accessed through a pointer to a (the) node with minimum key. 219 13458 3621 24 158352 79 117

6. 6 Representation of trees Child-sibling representation (cf. Exercise sheet 2) To be able to go upward in the tree, a pointer to the parent node is added. In order to realize deletion of a child (and the concatenation of two child lists) in O(1), we use doubly-connected circular lists. Hence, each node contains 4 pointers: child, parent, left, right 2 13458 362115 2 13458 362115 2 13458 362115

7. 7 Fibonacci heaps: node format class FibNode { Object content; // the content int key; // priority FibNode parent, child; // pointers to parent and one child FibNode left, right; // pointers to left and right sibling int rank; // number of children of this node boolean mark; // marker } rank indicates the number of children of the node (the outdegree of the node) The meaning of mark will become clear later. It is true if the node has lost any of its children since it last became the child of another node.

8. 8 Fibonacci heap: detailed figure 219 13458 3621 24 158352 79 117

9. 9 Fibonacci heap: simplified figure 219 13458 3621 24 158352 79 117

10. 10 Fibonacci heaps: operations Q.accessmin() : Return the node Q.min (or null if Q is empty). Q.insert(int k) : Create a new node N with key k and insert it in the root list of Q. If k < Q.min.key, update the minimum pointer (set Q.min = N ). 519 13458 3621 24 158352 79 117

11. Manipulation of trees in Fibonacci heaps There are only three ways how a tree can change in a Fibonacci heap: link: “growth” of trees Two trees are linked into a new tree. cut: cut out a subtree A subtree is cut out of a tree and inserted in the root list as a new tree. remove: decompose a tree at the root Removes the root of a tree from the root list and replaces it by the list of its children, i.e. the children become new roots.

12. 12 Tree manipulation: link link: Input: two root nodes of the same rank k Method: joins two trees of the same rank by making the root with larger key a child of the root with smaller key. Unmark the new child if it was marked. The total number of trees in the Fibonacci heap is reduced by 1, the number of nodes does not change. Output:one root node of rank k+1 Cost: 2 138 3621 24 8352 2 138 3621 24 8352

13. 13 Tree manipulation: cut cut: Input:one non-root node Method: separate the node (including the subtree rooted in it) from its parent and insert it as a new tree in the root list. The total number of trees is increased by 1, the number of nodes does not change. Cost: 2 13458 362115 26 2 13458 36 21 15 26

14. 14 Tree manipulation: remove remove: Input:one root node (with rank k) Method: delete the root of a tree and inserts its k children in the root list. The number of trees is increased by k-1, the number of nodes is decreased by 1. Cost: [if the parent pointers of the children are not deleted!] 2 13458 362115 26 13458 362115 26

15. 15 Further operations By combining the basic methods –link –cut –remove we can construct the missing operations –deletemin –decreasekey –delete

16. 16 Deletion of the minimum node Q.deletemin() : If Q is empty, return null. Otherwise: –Delete the minimal node (using remove). –„Consolidate“ (clean up) the root list: Join (link) two root nodes of the same rank, until only nodes with distinct rank appear in the root list. While doing this, remove all “dangling” parent pointers of root nodes. –Find the new minimum among the root nodes. –Return the deleted node.

17. 17 deletemin: example 219 13458 3621 24 158352 117 64

18. 18 deletemin: example 2 1913458 3621 24 15 8352 117 64

19. 19 deletemin: example 1913458 3621 24 15 8352 117 64

20. 20 deletemin: example 19 13 458 3621 24 158352 117 64

21. 21 deletemin: example 19 13 45 8 3621 24 158352 117 64

22. 22 deletemin: example 19 13458 3621 24 158352 117 64

23. 23 deletemin: example 1913458 3621 24 15 8352 117 64

24. 24 decreasekey Q.decreasekey(FibNode N, int k) : Set the key value of N to k. If the heap order is violated ( k < N.parent.key ): –Cut N off its parent (using cut) –If the parent is marked ( N.parent.mark == true ), also cut it from its own parent; if that parent is marked, also cut it, etc. (“cascading cuts”) –Mark the node whose child was last cut (unless it is a root node). –Update the minimum pointer (if k < min.key ).

25. 25 decreasekey: example 1 219 13458 3621 24 158352 79 117

26. 26 decreasekey: example 1 219 13458 3621 24 15831 79 117

27. 27 decreasekey: example 1 219 13458 3621 24 15831 79 117

28. 28 decreasekey: example 1 219 13458 3621 24 15 83 1 79 117

29. 29 decreasekey: example 1 219 13458 3621 24 15 83 1 79 117

30. 30 decreasekey: example 2 219 13458 3621 24 15 83 1 79 117

31. 31 decreasekey: example 2 219 13458 365 24 15 83 1 79 117

32. 32 decreasekey: example 2 219 13458 36 5 24 15 83 1 79 117

33. 33 decreasekey: example 2 219 13458 36 5 24 15 83 1 79 117

34. 34 decreasekey: example 2 2 19 1345 8 36 5 24 15 83 1 79 117

35. 35 Marking history of a node A node N is unmarked when it becomes the child of another node (via link). When N loses a child node (via cut), N is marked (unless N is a root). When a marked node loses another child, it is cut off itself and becomes a root.  Hence, if a node N is marked we know that N has lost one child since N itself was last made the child of another node.  Also, a node that is not a root cannot have lost more than one child!

36. 36 Deletion of a given node Q.delete(FibNode N) : Set the key of N to a value smaller than the current minimum, e.g. Q.decreasekey(N, -  ) Then perform Q.deletemin().

37. 37 delete: example 219 13458 3621 24 158352 79 7

38. 38 delete: example 219 13458 3621 -- 158352 79 7

39. 39 delete: example 2 19 13458 3621 -- 15 8352 79 7

40. 40 delete: example 2 19 13458 362115 8352 79 7

41. 41 delete: example 2 19 13458 362115 83 52 79 7

42. 42 delete: example 2 19 13458 362115 8352 79 7

43. 43 delete: example 2 19 13458 362115 8352 79 7

44. 44 Other operations Q.meld(FibHeap P) : Append the root list of P to the root list of Q. Then update the minimum pointer of Q : if P.min.key < Q.min.key, set Q.min = P.min. Q.isEmpty() : If Q.size == 0, return true; otherwise false.

45. 45 Analysis Complexity of deletemin(): remove:O(1) consolidate:? updatemin:O(#root nodes after consolidate) = After consolidating there is only one root with any given rank (otherwise, they would have been linked). We define maxRank(n) as the largest possible rank a root node can have in a Fibonacci heap of size n. (We will calculate maxRank(n) later.) Now we have to determine the complexity of consolidate.

46. 46 deletemin: consolidating the root list How can we implement consolidate efficiently? Observations: –Obviously, each root node has to be visited at least once –At the end, each rank may occur at most once Idea: Insert the root nodes in a temporary array. Each node is inserted at the position corresponding to its rank. If a position is occupied, we know that we have already seen another node with the same rank. We can now link the two nodes and (try to) insert the root of the resulting tree at the next higher array position.

47. 47 consolidate: example 1913458 3621 24 15 52 117 2

48. 48 consolidate: example 1913458 3621 24 15 52 117 012345 Rank array:

49. 49 consolidate: example 1913458 3621 24 15 117 012345 Rank array: 52

50. 50 consolidate: example 19 13458 3621 24 15 117 012345 Rank array: 52

51. 51 consolidate: example 19 13 458 3621 24 15 117 012345 Rank array: 52

52. consolidate: example 19 13 45 8 3621 24 15 117 012345 Rank array: 52

53. 53 consolidate: example 19 13 45 8 3621 24 15 117 52

54. 54 Analysis of consolidate rankArray = new FibNode[maxRank(n)+1]; // create the array for “each FibNode N in rootlist” { while (rankArray[N.rank] != null) { // position occupied N = link(N, rankArray[N.rank]); // link tree roots rankArray[N.rank-1] = null; // delete old pos. } rankArray[N.rank] = N; // insert in array }

55. 55 Analysis of the for-loop for “each FibNode N in rootlist” { while (rankArray[N.rank] != null) { N = link(N, rankArray[N.rank]); rankArray[N.rank-1] = null; } rankArray[N.rank] = N; } Let r before = #root nodes before consolidate and r after = #root nodes after consolidate. The total number of link operations in the loop is r before - r after. The total number of array modifications in the loop is #links + r before (because each link and each iteration of the loop ends with one modification). Since r before = r after + #links, we get 2·#links + r after array modifications. Therefore, the complexity of the for-loop is #links·O(1) + (2·#links + r after )·O(1) = O(#links) + O(maxRank(n))

56. 56 Complexity of deletemin remove:O(1) Creating the rank array:O(maxRank(n)) for-loop:O(#links) + O(maxRank(n)) Update minimum pointer:O(maxRank(n)) Total cost:O(#links) + O(maxRank(n))

57. 57 Analysis Complexity of decreasekey(): Set key to new value:O(1) cut:O(1) Cascading cuts: #cuts · O(1) Mark:O(1) Total cost: O(#cuts)

58. 58 Analysis Complexity of delete(): Sum of the costs for decreasekey and deletemin Total cost: O(#cuts) + O(#links) + O(maxRank(n))

59. 59 Amortized analysis Observations: For deletemin, the number of link operations affects the total cost. For decreasekey, it is the number of cascading cuts. Idea: Pay those costs from savings (“accounting method”)! Assumption: the cost for each link and each cut is 1€. (1)Make sure that for each root node there is always 1€ on the savings account, so we can pay for the link operation if that node becomes the child of another one. (2)Make sure that for each marked node, there are always 2€ on the savings account, so we can –pay the cut operation for that node during “cascading cuts” and –still have 1€ left as savings for the new root node

60. 60 Example 913453 3621 24 75 52 79 107 1147 36 3214 5039 61

61. Increase the savings For which operations do we have to pay extra (in order to gain savings for later)? New root nodes originate from: –insert: attach 1€ to each new node inserted in the root list –decreasekey: pay 1€ extra for the cut-off node. –deletemin: during remove, pay 1€ for each child of the removed node, i.e. up to maxRank(n) €. Marked nodes may only result at the end of decreasekey: –For each mark operation pay an extra 2€ for the marked node.

62. 62 Amortized cost of insert Create the node:O(1) Insert into root list:O(1) + O(1) Total amortized cost:O(1)

63. 63 Amortized cost of deletemin remove:O(1) + O(maxRank(n)) Create the rank array:O(maxRank(n)) link operations:#links · O(1)paid from savings! Other insertions:O(maxRank(n)) Update minimum pointer:O(maxRank(n)) Total amortized cost:O(maxRank(n))

64. 64 Example 913453 3621 24 75 52 79 107 1147 36 3214 5039 61

65. 65 Example 913453621 24 75 52 79 107 1147 36 3214 5039 61 remove: Pay 1€ extra for each child of the removed node.

66. 66 Example 91345 36 21 24 75 52 79 107 1147 36 3214 5039 61 link: Pay all link operations from the savings.

67. 67 Example 91345 36 2124 75 52 79 107 1147 36 3214 5039 61 link: Pay all link operations from the savings.

68. 68 Example 913 45 36 2124 75 52 79 107 1147 36 3214 5039 61 link: Pay all link operations from the savings.

69. 69 Example 9 1345 36 2124 75 52 79 107 1147 363214 5039 61 link: Pay all link operations from the savings.

70. 70 Example 9 13 45 36 21 24 75 52 79 10 7 1147 36 3214 5039 61 link: Pay all link operations from the savings.

71. 71 Amortized cost of decreasekey Set key to new value:O(1) cut:O(1) + O(1) Cascading cuts: #cuts · O(1)paid from savings! Mark:O(1) + 2 · O(1) Total amortized cost:O(1)

72. 72 Example 913453 3621 24 75 52 79 7 1147 36 3214 5039

73. 73 Example 913453 3621 24 20 52 79 7 1147 36 3214 5039 Pay 1€ extra for the initial cut operation.

74. 74 Example 913453 3621 24 20 52 79 7 1147 36 3214 5039 All cascading cuts are paid form savings.

75. 75 Example 913453 3621 24 20 52 79 7 1147 36 3214 50 39 All cascading cuts are paid form savings.

76. 76 Example 913453 3621 24 20 52 79 7 1147 36 3214 50 39 When marking the last parent, pay an additional 2€.

77. 77 Amortized cost of delete Sum of the costs of decreasekey and deletemin: O(1) + O(maxRank(n)) Total amortized cost: O(maxRank(n))

78. 78 Amortized Analysis Amortized costs Insert:O(1) Accessmin:O(1) Deletemin:O(maxRank(n)) Decreasekey:O(1) Delete:O(maxRank(n)) Meld:O(1) To do: show that maxRank(n) = O(log n). I.e. the largest possible rank of a node in a Fibonacci heap is logarithmic in the size n of the Fibonacci heaps.

79. 79 Calculation of maxRank(n) Lemma 1: Let N be a node in a Fibonacci heap and k = N.rank. Consider the children C 1,..., C k of N in the order, in which they have beed added to N (with link). Then: (1)C 1.rank ≥ 0 (2) C i.rank ≥ i - 2for i = 2,..., k Proof: (1) obvious (2)When S i was made achild of N, C 1,..., C i-1 had already been children of N, i.e. we had N.rank ≥ i-1. Since link always links two nodes of the same rank, at the time of linking C i.rank ≥ i-1. Since then, C i cannot have lost more than one of its children (otherwise it would have been cut off too), hence: C i.rank ≥ i - 2

80. 80 Calculation of maxRank(n) Lemma 2: Let N be a node in a Fibonacci heap and k = N.rank. Let size(N) be the number of nodes in the subtree rooted in N. Then:size(N) ≥ F k+2 ≥ 1.618 k (F k = k-th Fibonacci number) i.e. a node with k children is root of a subtree with at least F k+2 nodes. Proof: Let S(k) = min {size(N) | N with N.rank = k}, i.e. the smallest possible size of a tree whose root has rank k. (Obviously, S(0) = 1.) Again, let C 1,..., C k be the children of N in the order of their linking to N. Then (Lemma 1)

81. 81 Calculation of maxRank(n) Theorem: The maximum rank maxRank(n) of any node in a Fibonacci heap with n nodes is bounded by O(log n). Proof: Let N be a node in a Fibonacci heap with n nodes and let k = N.rank. Thenn ≥ size(N) ≥ 1.618 k (cf. Lemma 2) Hencek ≤ log 1.618 (n) = O(log n)

82. 82 Conclusion Linear list(Min-)heap Fibonacci heap insert: O(1)O(log n) O(1) accessmin: O(1)O(1)O(1) deletemin: O(n) O(log n)O(log n)* decreasekey: O(1)O(log n) O(1)* delete: O(n) O(log n)O(log n)* meld:O(1) O(m log(n+m)) O(1) *Amortized cost