1. Amortized Algorithm Analysis COP3503 July 25, 2007 Andy Schwartz

2. Contents Indirect Solutions Amortized Analysis Introduction Extendable Array Example Binomial Queue Binomial Queue Example

3. Indirect Solution (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java. 10 yards / minute 100 yards / minute

4. Indirect Solution (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java. 10 yards / minute 100 yards / minute Geometric Series? The easy solution is indirect. It takes a kitten 5 minutes to go 50 yards, how far can the mother go in 5 minutes?.... 500 yards!

5. Amortized Analysis Introduction The worst-case running time is not always the same as the worst possible average running time. Example: –Worst case-time is O(n) –Amortized worst-case is O(1) –This could be from a series of table inserts and clears (Source: Arup Guha. CS2 Notes – Summer 2007.

6. Amortization Techniques Two Techniques: –Potential Function –Accounting Method Start with $X dollars for n operations. Each simple operation costs $1, but larger operations are more expensive. If $X dollars is enough for n operations, then the amortized worst- case is $x / n. Example: School Semester costs (worst-case most expensive in Fall, but amortized worst-case including spring and summer is cheaper). Compute the amortized worst-case running time of n consecutive variable-popping stack operations. (details on board) (Source: Arup Guha. CS2 Notes – Summer 2007.

7. Extendable Array Example ArrayList (or Vector) in Java Can we maintain an extendable array and keep the amortized worst-case running time down to O(1)? –Yes! (or this would be a bad example) –How?.... –Simple case: add an element and space is left over. –Worst case: ? (Source: Arup Guha. CS2 Notes – Summer 2007.

8. Extendable Array Example Simple Case: Add an element and space is left over. Worst Case: Add an element, and there is no more space after  we need to make space  allocate new array and copy elements –The time complexity of this worst-case depends on n, the size of the original array. How big should we make the new array? –There were n-1 simple operations in order to fill the array to this point, so let’s double it. That way a series of n-1 simple insertions, followed by one worst case will result in about n-1 + n = ~2n total operations. –2n operations / n add operations = 2 = O(2) = O(1) *There were really more than n-1 operations if this was not the first extension, but we know a series of those operations were O(1) and O(1) + O(2) = O(1). O(1) O(k) (Source: Arup Guha. CS2 Notes – Summer 2007.

9. Binomial Queue Binomial Trees: B 0 B 1 B 2 B 3 B 4 Each tree “doubles” the previous.

10. Binomial Queue A queue of binomial trees. A “Forest”. Each tree is essentially a heap constrained to the format of a binary tree. Example: B 0, B 2, B 3 Insertion: Create a B 0 and merge Deletion: remove minimum( the root) from tree B k. This leaves a queue of trees: B 0, B 1, …, B k-1. Merge this queue with the rest of the original queue. (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

11. Binomial Queue Example The Most important step is Merge Merge Rules: (for two binomial queues) –0 or 1 B k trees  just leave merged –2 B k trees  meld into 1 B k+1 tree. –3 B k trees  meld two into 1 B k+1, and leave the third. Insertion Runtime: –M+1 steps, where M is smallest tree not present. Worst-case is k+2, when B k+1 is the smallest tree not present. How does k relate to the total number of nodes in the tree? k = lg n, thus (nonamortized) worst-case time is O(lg n). (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

12. Binomial Queue Example MakeBinQ Problem: Build a binomial queue of N elements. (Like makeHeap). How long should this take? Insertion worst-case runtime: –Worst-case O(lg n) for 1 insert  O(n lg n) n inserts, but we want O(n) – like makeHeap Try amortized analysis directly: –Considering each linking step of the merge. The 1 st, 3 rd, 5 th, ettc… odd steps require no linking step because there will be no B 0. So ½ of all insertions require no linking, similarly ¼ require 1 linking steps. –We could continue down this path, but the it’ll be come especially difficult for deletion (we should learn an indirect analysis). (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

13. Binomial Queue Example Indirect Analysis (time = M + 1) –If no B 0  cost is 1 (M is 0) Results in 1 B 0 tree added to the forest –If B 0 but no B 1  cost is 2 (M is 1) Results in same # of trees (new B 1 but B 0 is gone) –When cost is 3 (M is 2) Results in 1 less tree (new B 2 but remove B 0 and B 1 ) ….etc… –When cost is c (M is c – 1) Results in increase of 2 – c trees (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

14. Binomial Queue Example increase of 2 – c trees How can we use this? T i = # of trees after i th iteration T 0 = 0 = # of trees initially C i = Cost of i th iteration Then, T i = T i-1 + (2 – C i )  C i + (T i – T i-1 ) = 2 This is only the i th iteration (Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.

15. Binomial Queue Example C i + (T i – T i-1 ) = 2 To get all iterations: C 1 + (T 1 – T 0 ) = 2 C 2 + (T 2 – T 1 ) = 2 … C n-1 + (T n-1 – T n-2 ) = 2 C n + (T n – T n-1 ) = 2 n Σ C i + (T n – T 0 ) = 2n (T 1.. T n-1 cancel out) i=1

16. Binomial Queue Example n Σ C i + (T n – T 0 ) = 2n i=1 T 0 = 0 and T n is definitely not negative, so T n – T 0 is not negative. n  Σ C i <= 2n i=1 Thus, the total cost < 2n  makeBinQ = O(n) Since, makeBinQ consists of O(n) inserts, then the amortized worst- case of each insert is O(1).