Binomial heaps, Fibonacci heaps, and applications

Slides:



Advertisements
Similar presentations
Fibonacci Heaps Especially desirable when the number of calls to Extract-Min & Delete is small (note that all other operations run in O(1) This arises.
Advertisements

Single Source Shortest Paths
Priority Queues  MakeQueuecreate new empty queue  Insert(Q,k,p)insert key k with priority p  Delete(Q,k)delete key k (given a pointer)  DeleteMin(Q)delete.
Advanced Data structure
Chapter 23 Minimum Spanning Trees
Discussion #36 Spanning Trees
Greedy Algorithms Reading Material: Chapter 8 (Except Section 8.5)
1 Minimize average access time Items have weights: Item i has weight w i Let W =  w i be the total weight of the items Want the search to heavy items.
Fibonacci Heaps. Single Source All Destinations Shortest Paths
1 Maximum flow: The preflow/push method of Goldberg and Tarjan (87)
Greedy Algorithms Like dynamic programming algorithms, greedy algorithms are usually designed to solve optimization problems Unlike dynamic programming.
Princeton University COS 423 Theory of Algorithms Spring 2002 Kevin Wayne Fibonacci Heaps These lecture slides are adapted from CLRS, Chapter 20.
Minimal Spanning Trees What is a minimal spanning tree (MST) and how to find one.
1 Binomial heaps, Fibonacci heaps, and applications.
Algorithms 2005 Ramesh Hariharan. Amortization in Dynamic Algorithms A single insertion/deletion might take say O(log n) time Does a sequence of n insertions.
9/10/10 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne Adam Smith Algorithm Design and Analysis L ECTURE 8 Greedy Graph.
1 Fibonacci heaps, and applications. 2 Yet a better MST algorithm (Fredman and Tarjan) Iteration i: We grow a forest, tree by tree, as follows. Start.
Binomial heaps, Fibonacci heaps, and applications
2IL05 Data Structures Fall 2007 Lecture 13: Minimum Spanning Trees.
Spring 2015 Lecture 11: Minimum Spanning Trees
Four different data structures, each one best in a different setting. Simple Heap Balanced Heap Fibonacci Heap Incremental Heap Our results.
1 Fat heaps (K & Tarjan 96). 2 Goal Want to achieve the performance of Fibonnaci heaps but on the worst case. Why ? Theoretical curiosity and some applications.
CMSC 341 Binomial Queues and Fibonacci Heaps. Basic Heap Operations OpBinary Heap Leftist Heap Binomial Queue Fibonacci Heap insertO(lgN) deleteMinO(lgN)
1 22c:31 Algorithms Minimum-cost Spanning Tree (MST)
Fibonacci Heaps. Analysis FibonacciAnalysis.ppt Video  iew/cop5536sahni
1 Fibonacci heaps: idea List of multiway trees which are all heap-ordered. Definition: A tree is called heap-ordered if the key of each node is greater.
Fibonacci Heaps. Fibonacci Binary insert O(1) O(log(n)) find O(1) N/A union O(1) N/A minimum O(1) O(1) decrease key O(1) O(log(n)) delete O(log(n) O(log(n))
Leftist Trees Linked binary tree.
Lecture ? The Algorithms of Kruskal and Prim
Introduction to Algorithms
Chapter 5 : Trees.
ליאור שפירא, חיים קפלן וחברים
Data Structures Binomial Heaps Fibonacci Heaps Haim Kaplan & Uri Zwick
Minimum Spanning Trees
Binomial heaps, Fibonacci heaps, and applications
12. Graphs and Trees 2 Summary
Heaps Binomial Heaps Lazy Binomial Heaps 1.
Lecture 12 Algorithm Analysis
Introduction to Graphs
Greedy Algorithms / Minimum Spanning Tree Yin Tat Lee
Priority Queues MakeQueue create new empty queue
Fibonacci Heaps Remove arbitrary is useful in (for example) correspondence structures and may also be used to do an increase key in a min structure (remove.
Pairing Heaps Actual Complexity.
Minimum-Cost Spanning Tree
Maximum flow: The preflow/push method of Goldberg and Tarjan (87)
A simpler implementation and analysis of Chazelle’s
ערמות בינומיות ופיבונצ'י
§3 Worst-Case vs. Amortized
Strict Fibonacci Heaps
Lecture 12 Algorithm Analysis
CS 583 Analysis of Algorithms
Fibonacci Heaps Remove arbitrary is useful in (for example) correspondence structures.
Minimum Spanning Tree Algorithms
Binary and Binomial Heaps
Minimum Spanning Tree.
Algorithms Searching in a Graph.
Binomial heaps, Fibonacci heaps, and applications
Binomial heaps, Fibonacci heaps, and applications
Fibonacci Heaps.
Binomial heaps, Fibonacci heaps, and applications
Lecture 12 Algorithm Analysis
Priority Queues Supports the following operations. Insert element x.
Important Problem Types and Fundamental Data Structures
Maximum flow: The preflow/push method of Goldberg and Tarjan (87)
Chapter 14 Graphs © 2011 Pearson Addison-Wesley. All rights reserved.
Lecture 14 Minimum Spanning Tree (cont’d)
Chapter 24: Single-Source Shortest-Path
Minimum-Cost Spanning Tree
More Graphs Lecture 19 CS2110 – Fall 2009.
Presentation transcript:

Binomial heaps, Fibonacci heaps, and applications

Binomial trees B0 B1 Bi B(i-1) B(i-1)

Binomial trees Bi . . . . . . B0 B(i-1) B(i-2) B1

Properties of binomial trees 1) | Bk | = 2k 2) degree(root(Bk)) = k 3) depth(Bk) = k ==> The degree and depth of a binomial tree with at most n nodes is at most log(n). Define the rank of Bk to be k

Binomial heaps (def) A collection of binomial trees at most one of every rank. Items at the nodes, heap ordered. 5 6 1 8 2 9 10 Possible rep: Doubly link roots and children of every node. Parent pointers needed for delete.

Binomial heaps (operations) Operations are defined via a basic operation, called linking, of binomial trees: Produce a Bk from two Bk-1, keep heap order. 1 6 4 11 6 9 5 10 2 9 5 8 5 10

Binomial heaps (ops cont.) Basic operation is meld(h1,h2): Like addition of binary numbers. B5 B4 B2 B1 h1: B4 B3 B1 B0 + h2: B4 B3 B0 B5 B4 B2

Binomial heaps (ops cont.) Findmin(h): obvious Insert(x,h) : meld a new heap with a single B0 containing x, with h deletemin(h) : Chop off the minimal root. Meld the subtrees with h. Update minimum pointer if needed. delete(x,h) : Bubble up and continue like delete-min decrease-key(x,h,) : Bubble up, update min ptr if needed All operations take O(log n) time on the worst case, except find-min(h) that takes O(1) time.

Amortized analysis We are interested in the worst case running time of a sequence of operations. Example: binary counter single operation -- increment 00000 00001 00010 00011 00100 00101

Amortized analysis (Cont.) On the worst case increment takes O(k). k = #digits What is the complexity of a sequence of increments (on the worst case) ? Define a potential of the counter:  (c) = ? Amortized(increment) = actual(increment) + 

Amortized analysis (Cont.) Amortized(increment1) = actual(increment1) + 1- 0 Amortized(increment2) = actual(increment2) + 2- 1 + … Amortized(incrementn) = actual(incrementn) + n- (n-1) iAmortized(incrementi) = iactual(incrementi) + n- 0 iAmortized(incrementi)  iactual(incrementi) if n- 0  0

Amortized analysis (Cont.) Define a potential of the counter:  (c) = #(ones) Amortized(increment) = actual(increment) +  Amortized(increment) = 1+ #(1 => 0) + 1 - #(1 => 0) = O(1) ==> Sequence of n increments takes O(n) time

Binomial heaps - amortized ana.  (collection of heaps) = #(trees) Amortized cost of insert O(1) Amortized cost of other operations still O(log n)

Binomial heaps + lazy meld Allow more than one tree of each rank. Meld (h1,h2) : Concatenate the lists of binomial trees. Update the minimum pointer to be the smaller of the minimums O(1) worst case and amortized.

Binomial heaps + lazy meld As long as we do not do a delete-min our heaps are just doubly linked lists: 9 11 4 6 9 5 Delete-min : Chop off the minimum root, add its children to the list of trees. Successive linking: Traverse the forest keep linking trees of the same rank, maintain a pointer to the minimum root.

Binomial heaps + lazy meld Possible implementation of delete-min is using an array indexed by rank to keep at most one binomial tree of each rank that we already traversed. Once we encounter a second tree of some rank we link them and keep linking until we do not have two trees of the same rank. We record the resulting tree in the array Amortized(delete-min) = = (#links + max-rank) - #links = O(log(n))

Fibonacci heaps (Fredman & Tarjan 84) Want to do decrease-key(x,h,) faster than delete+insert. Ideally in O(1) time. Why ?

Dijkstra’s shortest path algorithm Let G = (V,E) be a weighted (weights are non-negative) undirected graph, let s  V. Want to find the distance (length of the shortest path), d(s,v) from s to every other vertex. 3 3 s 2 1 3 2

Dijkstra’s shortest path algorithm Dijkstra: Maintain an upper bound d(v) on d(s,v). Every vertex is either scanned, labeled, or unlabeled. Initially: d(s) = 0 and d(v) =  for every v  s. s is labeled and all others are unlabeled. Pick a labeled vertex with d(v) minimum. Make v scanned. For every edge (v,w) if d(v) + w(v,w) < d(w) then 1) d(w) := d(v) + w(v,w) 2) label w if it is not labeled already

Dijkstra’s shortest path algorithm (implementation) Maintain the labeled vertices in a heap, using d(v) as the key of v. We perform n delete-min operations and n insert operations on the heap. O(n log(n)) For each edge we may perform a decrease-key. With regular heaps O(m log (n)). But if you can do decrease-key in O(1) time then you can implement Dijkstra’s algorithm to run in O(n log(n) + m) time !

Back to Fibonacci heaps Suggested implementation of decrease-key(x,h,): If x with its new key is smaller than its parent, cut the subtree rooted at x and add it to the forest. Update the minimum pointer if necessary.

5 6 2 3 8 9 10 5 6 2 1 8 9 10

Decrease-key (cont.) Does it work ? Obs1: Trees need not be binomial trees any more.. Do we need the trees to be binomial ? Where have we used it ? In the analysis of delete-min we used the fact that at most log(n) new trees are added to the forest. This was obvious since trees were binomial and contained at most n nodes.

Decrease-key (cont.) 5 6 2 3 9 Such trees are now legitimate. So our analysis breaks down.

Fibonacci heaps (cont.) We shall allow non-binomial trees, but will keep the degrees logarithmic in the number of nodes. Rank of a tree = degree of the root. Delete-min: do successive linking of trees of the same rank and update the minimum pointer as before. Insert and meld also work as before.

Fibonacci heaps (cont.) Decrease-key (x,h,): indeed cuts the subtree rooted by x if necessary as we showed. in addition we maintain a mark bit for every node. When we cut the subtree rooted by x we check the mark bit of p(x). If it is set then we cut p(x) too. We continue this way until either we reach an unmarked node in which case we mark it, or we reach the root. This mechanism is called cascading cuts.

2 4 20 5 8 11 9 6 14 10 16 12 15 9 16 5 6 14 7 12 15 4 8 11 2 20

Fibonacci heaps (delete) Delete(x,h) : Cut the subtree rooted at x and then proceed with cascading cuts as for decrease key. Chop off x from being the root of its subtree and add the subtrees rooted by its children to the forest If x is the minimum node do successive linking

Fibonacci heaps (analysis) Want everything to be O(1) time except for delete and delete-min. ==> cascading cuts should pay for themselves  (collection of heaps) = #(trees) + 2#(marked nodes) Actual(decrease-key) = O(1) + #(cascading cuts) (decrease-key) = O(1) - #(cascading cuts) ==> amortized(decrease-key) = O(1) !

Fibonacci heaps (analysis) What about delete and delete-min ? Cascading cuts and successive linking will pay for themselves. The only question is what is the maximum degree of a node ? How many trees are being added into the forest when we chop off a root ?

Fibonacci heaps (analysis) Lemma 1 : Let x be any node in an F-heap. Arrange the children of x in the order they were linked to x, from earliest to latest. Then the i-th child of x has rank at least i-2. x 2 1 Proof: When the i-th node was linked it must have had at least i-1 children. Since then it could have lost at most one.

Fibonacci heaps (analysis) Corollary1 : A node x of rank k in a F-heap has at least k descendants, where  = (1 + 5)/2 is the golden ratio. Proof: Let sk be the minimum number of descendants of a node of rank k in a F-heap. By Lemma 1 sk  i=0si + 2 k-2 x s0=1, s1= 2

Fibonacci heaps (analysis) Proof (cont): Fibonnaci numbers satisfy Fk+2 = i=2Fi + 2, for k  2, and F2=1 so by induction sk  Fk+2 It is well known that Fk+2  k k It follows that the maximum degree k in a F-heap with n nodes is such that k  n so k  log(n) / log() = 1.4404 log(n)

Application #2 : Prim’s algorithm for MST Start with T a singleton vertex. Grow a tree by repeating the following step: Add the minimum cost edge connecting a vertex in T to a vertex out of T.

Application #2 : Prim’s algorithm for MST Maintain the vertices out of T but adjacent to T in a heap. The key of a vertex v is the weight of the lightest edge (v,w) where w is in the tree. Iteration: Do a delete-min. Let v be the minimum vertex and (v,w) the lightest edge as above. Add (v,w) to T. For each edge (w,u) where uT, if key(u) =  insert u into the heap with key(u) = w(w,u) if w(w,u) < key(u) decrease the key of u to be w(w,u). With regular heaps O(m log(n)). With F-heaps O(n log(n) + m).

Thin heaps (K, Tarjan 97) A variation of Fibonacci heaps where trees are “almost” binomial. In particular they have logarithmic depth. You also save a pointer and a bit per node. So they should be more efficient in practice. A thin binomial tree is a binomial tree where each nonroot and nonleaf node may have lost its leftmost child

Thin binomial trees A thin binomial tree is a binomial tree where each nonroot and nonleaf node may have lost its leftmost child Bi . . . . . . B0 . . B(i-2) B1 B(i-3) B(i-1)

Thin binomial trees (cont) So either rank(x) = degree(x) or rank(x) = degree(x) + 1 In the latter case we say that the node is marked Bi . . . . . . B0 . . B(i-2) B1 B(i-3) B(i-1)

Thin heaps (K, Tarjan -1998) Thin heaps maintain the tree to be thin binomial tree by changing the way we do cascading cuts.

Cascading cuts Bi . . . . . . B0 . . B(i-2) B1 B(i-1) We may get an illegal “hole”

Cascading cuts Bi . . . . . . B0 . . B(i-2) B1 B(i-1) Or a “rank violation”

How do we fix a “hole” ? Two cases: depends upon whether the left sibling is marked or not Bi . . . . . . B0 . . B(i-2) B1 B(i-3) B(i-1)

How do we fix a “hole” ? If it is marked then unmark it Bi . . . . . . This moves the “hole” to the left or creates a “rank violation” at the parent B(i-1) B(i-2)

How do we fix a “hole” ? If it is unmarked Bi . . . . . . B0 . . And we are done

How do we fix a “rank violation” ? Cut the node in which the violation occurred. Bi . . . . . . B0 . . B(i-2) B1 B(i-3) B(i-1) You may create a violation at the parent

Yet a better MST algorithm Iteration i: We grow a forest, tree by tree, as follows. Start with a singleton vertex and continue as in Prim’s algorithm until either 1) The size of the heap is larger than ki 2) Next edge picked is connected to an already grown tree 3) Heap is empty (if the graph is connected this will happen only at the very end)

Contract each tree into a single vertex and start iteration i+1. How do we contract ? Do a DFS on the tree, marking for each vertex the # of the tree which contains it. Each edge e gets two numbers l(e), h(e) of the trees at its endpoints. If h(e) = l(e) remove e (self loop). (stable) Bucket sort by h(e) and by l(e), parallel edge then become consecutive so we can easily remove them. O(m) time overall.

Analysis: each iteration takes linear time Let ni be the number of vertices in the i-th iteration. O(m) inserts, O(m) decrease-key, O(ni) delete-min total : O(nilog(ki) + m) Set ki = 2(2m/ni) so the work per phase is O(m).

How many iterations do we have ? Every tree in iteration i is incident with at least ki edges. So ni+1 ki  2mi  2m ==> ni+1  2mi / ki  2m / ki ==> ki+1 = 2(2m/ni+1)  2ki  2m/n 2

This runs in O(m(m,n)) Once ki  n we stop. So the number of iterations is bounded by the minimum i such that 2 2m/n  n i j = min{i | 2m/n  logi(n) } = (m,n)

Summary The overall complexity of the algorithm is O(m (m,n) ) Where (m,n) = min{i | logi(n)  2m/n} for every m  n (m,n)  log*(n) For m > n log(n) the algorithm degenerates to Prim’s. One can prove that O(m (m,n) ) = O(nlogn + m).

So our record is O(m (m,n) ), can we do better ? Where is the bottleneck now ? We may scan an edge (m,n) times. When we abandon a heap we will rescan an edge per vertex in the heap. Delay scanning of edges (heavy edges)

Packets (Gabow, Galil, Spencer, Tarjan) Group the edges incident to each vertex into packets of size p each Sort each packet Treat each packet as a single edge (the first edge in the packet)

Working with packets When you extract the min from the heap it is associated with a packet whose top edge is (u,v). You add (u,v) to the tree, delete (u,v) from its packet, and relax this packet of u Traverse the packets of v and relax each of them How do you relax a packet ?

Relaxing packet p of vertex v Check the smallest edge (v,u) in p If u is already in the tree, discard (v,u), and recur If u is not in the heap: insert it into the heap with weight w(v,u) If u is in the heap: If the weight of u is larger than the weight of (v,u) then decrease its key Let p be the packet with larger weight among the current and the previous packet associated with u, discard its first edge and recur on p

Analysis Initialization: O(m) to partition into packets, O(mlog(p)) to sort the packets An iteration: O(nilog(ki)) for extract-mins, and O(m/p) work for the packets Additional work per packet we can charge to an edge which we discard…… Total O(m) Summing up O( m + mlog(p) + Σ(nilog(ki) + m/p) ) iterations

Analysis (Cont) O(mlog(p) + Σ(nilog(ki) + m/p) ) iterations Set ki = 2(2m/pni) so the work per phase is O(m/p). Set k1 = 2(2m/n) ; the work in the first phase is O(m) Every tree in iteration i is incident with at least ki packets So ni+1 ki  2m/p ==> ni+1  2m / pki  We have ≤ (m,n) iterations 2m/n 2 ==> ki+1 = 2(2m/pni+1)  2ki 

Analysis (Cont) The running time is O(mlog(p) + (m,n) m/p ) Choose p= (m,n) so we get O(mlog((m,n)))

But we cheated… If we want the running time per iteration to be m/p, how do we do contractions ? Use union/find (conract by uniting vertices and concatenating their adjacency lists) You get an (m,n) overhead when you relax a packet and overall O(m (m,n) + m log((m,n)) + Σni ) = O(m log((m,n)))

Furthermore, it wasn’t our only cheat.. We cannot partition the edges incident to a vertex into packets each of size p exactly ? So there can be at most one undersized packet per vertex When you contract merge undersized packets How do you merge undersized packets ?

Use a little F-heap to represent each packet This gives log(p) overhead to relax a packet But still the overall time is O(m log((m,n)))

Further research Obtain similar bounds on the worst case. Other generalizations/variations. If the heap pointer is not given with the decrease-key and delete what then can be done ? Better bound if the keys are integers in the RAM model.